Project

General

Profile

Actions

Bug #48998

closed

FAL files get reindexed in storage null when used with tt_content

Added by Klaus Hinum about 11 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
Start date:
2012-09-06
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
6.1
PHP Version:
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

As described in the fixed bug 40669, the FAL files are reindexed in storage 0 again when a page is shown.
http://forge.typo3.org/issues/40669#change-168691
E.g. one image indexed in storage 1 (fileadmin) as
/test/test.jpg
and as soon as it is added to a tt_content and previewed in the frontend
/fileadmin/test/test.jpg is also added to the sys_file storage (with storage 0) and not referenced in sys_file_reference.

The bug breaks replication all the time (if the page was not previewed on the backend server first) and of course bloats the database with nearly twice the amount of sys_file entries than needed.

Tested with Typo3 6.1.1, and may be a follow up since the bug was fixed that prevented the adding of processed files to the sys_file table.


Related issues 3 (0 open3 closed)

Related to TYPO3 Core - Bug #40669: FAL files get reindexed in storage null when used with tt_contentClosedSteffen Ritter2012-09-06

Actions
Related to TYPO3 Core - Bug #47211: ProcessedFile is persisted in sys_file as wellClosedOliver Hader2013-04-14

Actions
Related to TYPO3 Core - Bug #49212: File links from tt_content bodytext are not/wrong indexedClosed2013-06-18

Actions
Actions #1

Updated by Klaus Hinum about 11 years ago

  • % Done changed from 100 to 0
Actions #2

Updated by Ernesto Baschny almost 11 years ago

  • Status changed from New to Accepted
  • Target version set to next-patchlevel

According to Steffen this was indeed fixed in 6.0 and appears again in 6.1 (and thus 6.2). Remains unclear still what (re-)introduced this bug.

Actions #3

Updated by Ernesto Baschny almost 11 years ago

  • Status changed from Accepted to Needs Feedback

I was unable to reproduce that on 6.2 "introduction package" standard installation. Could you please

a) test with 6.2 / master
b) check if there aren't any extensions influencing this?

Actions #4

Updated by Nico de Haen almost 11 years ago

I just tested it with 6.1.1 and 6.2 and can confirm that it is somehow fixed in 6.2.
Although I have no idea where, since the involved files RteHtmlParser and ResourceFactory did not change in relevant parts.

Maybe the change happened somewhere else in FAL? (Perhaps a case for git-bisect)

Actions #5

Updated by Nico de Haen almost 11 years ago

I was wrong, the bug is not yet fixed!

The problem is:
The parser calls ResourceFactory->retrieveFileOrFolderObject($input) where input is the filepath.
The comment says: input can be "uploads/myfile.png" (backwards-compatibility, storage "0")

Thus the file is added with storage 0 and "lost". No idea how storage with uid 0 should be handled...

Actions #6

Updated by Alexander Opitz almost 11 years ago

  • Status changed from Needs Feedback to New
Actions #7

Updated by Steffen Ritter almost 11 years ago

  • Status changed from New to Needs Feedback

Puh, I tried to reproduce as I think I had this some weeks ago, too.
Currently I cannot reproduce.

Please give more information about, when this happens...

Actions #8

Updated by Nico de Haen almost 11 years ago

It happens when opening a content element containing a file link with a not converted reference like
<link fileadmin/download/myfile.txt >Download</link> in the backend.

Then the RteHtmlParser->TS_links_rte converts the link into a FAL reference by calling ResourceFactory->retrieveFileOrFolderObject as described above and a sys_file entry with storage 0 is added.

With the RteFileLinksUpdateWizard Francois pushed yesterday to gerrit https://review.typo3.org/#/c/22002/ such links should not occur anymore after migration, but it should be fixed anyway. Perhaps with the same routine as in the Wizards.

Actions #9

Updated by Klaus Hinum almost 11 years ago

I can reproduce it with a 6.1.1 installation.
Just add a new content element text w. image to a page, add a new image (best upload it in the window). As soon as you do a preview of the page, the image is added again to the sys_file table without any storage (already in it with storage 1). As you can see of the DB dump below, two identical entries are in the sys_file table, one with storage 1 (fileadmin) and the other with storage 0 and fileadmin prepended.

INSERT INTO `sys_file` (`uid`, `pid`, `tstamp`, `crdate`, `cruser_id`, `deleted`, `t3ver_oid`, `t3ver_id`, `t3ver_wsid`, `t3ver_label`, `t3ver_state`, `t3ver_stage`, `t3ver_count`, `t3ver_tstamp`, `t3ver_move_id`, `t3_origuid`, `type`, `storage`, `identifier`, `extension`, `mime_type`, `name`, `title`, `sha1`, `size`, `creation_date`, `modification_date`, `width`, `height`, `description`, `alternative`) VALUES
(1805961, 0, 1373027842, 1373027842, 1, 0, 0, 0, 0, '', 0, 0, 0, 0, 0, 0, '2', 1, '/user_upload/amd_richland_a6.jpg', 'jpg', 'image/jpeg', 'amd_richland_a6.jpg', NULL, '5e08dfb45f9dc4aca9ce71cbe0c3562604938cdf', 159303, 1373183468, 1373183468, 803, 1080, NULL, NULL),
(1806193, 0, 1373183551, 1373183551, 0, 0, 0, 0, 0, '', 0, 0, 0, 0, 0, 0, '2', 0, '/fileadmin/user_upload/amd_richland_a6.jpg', 'jpg', 'image/jpeg', 'amd_richland_a6.jpg', NULL, '5e08dfb45f9dc4aca9ce71cbe0c3562604938cdf', 159303, 1373183468, 1373183468, 803, 1080, NULL, NULL);

Btw it also happens for old extensions that still use the uploads directory, the sys file is added when previewed.

Actions #10

Updated by Klaus Hinum almost 11 years ago

still happening with 6.1.5 all the time (and therefore breaking our replication every day)

Actions #11

Updated by Alexander Opitz almost 11 years ago

  • Status changed from Needs Feedback to New
  • Is Regression set to No
Actions #12

Updated by Klaus Hinum over 10 years ago

looked into it closer now, I cant reproduce it with 6.1.5 (therefore the above mentioned case wont happen anymore), but still the replication broke a couple of times due to sys_file items created on the frontend server. Will look into it and update this as soon as it happens again and I find out more.

Actions #13

Updated by Ernesto Baschny over 10 years ago

  • Status changed from New to Needs Feedback

Thanks for your help, Klaus, please keep us updated on what exactly is causing these duplicate sys_file entries to be generated in the FE.

Actions #14

Updated by Steffen Ritter over 10 years ago

  • Status changed from Needs Feedback to Resolved

the double indexation should be fixed with the "findBestMatchingSTorage"

Actions #15

Updated by Benni Mack over 5 years ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF