Bug #62400
closedLot of entries in sys_file_processed with name=NULL and identifier empty
0%
Description
We have a fairly large site with an enormous amount of entries in sys_file_processed that have name = NULL, identifier = ''. A substantional subset of those has configuration = a:9:{s:5:"width";N;s:6:"height";N;s:13:"fileExtension";N;s:8:"maxWidth";i:0;s:9:"maxHeight";i:0;s:8:"minWidth";i:0;s:9:"minHeight";i:0;s:7:"noScale";N;s:20:"additionalParameters";N;}
A related issue can be found here: https://forge.typo3.org/issues/50392#note-15
This leads to all kinds of problems, for example images not being displayed in the FE according to dimensions specified in a content element, but in the original size (no processed file is being generated). If the entry in sys_file_processed in manually removed and the FE page reloaded, the image is correctly displayed.
It also breaks the 'replace image' in Media (current 0.6.0-dev from the repo): as soon as a FE page is generated with an image a correct entry is added to sys_file_processed, but also an extra entry with name = NULL and configuration = a:9:{s:5:"width";N;s:6:"height";N;s:13:"fileExtension";N;s:8:"maxWidth";i:0;s:9:"maxHeight";i:0;s:8:"minWidth";i:0;s:9:"minHeight";i:0;s:7:"noScale";N;s:20:"additionalParameters";N;}
If you then use 'replace image' through Media on that image, the new image is not handled correctly: you don't get a new thumb and no output on FE. This can be traced back to the function flushProcessedFiles in media/Classes/Cache/CacheService.php which deletes old processed files. Because the entry with name=NULL exists, the newly uploaded (original, not processed) file is immediately deleted again. Just using 'replace image' again and again (without generating the FE page) works fine.
Changing $processedFile->delete(TRUE) to $processedFile->delete(FALSE) in that same function 'fixes' the problem, but I presume these NULL entries shouldn't be generated in the first place?
Anyone have an idea why these entries are generated?
Updated by Albert van der Veen about 10 years ago
Looking into the class TYPO3.CMS.Core.Resource.ProcessedFile, I now understand that entries with name=NULL are valid: 'A file may also meet the expectations set in the configuration without any processing. In that case, the ProcessedFile object still exists, but there is no physical file directly linked to it. Instead, it then redirects most method calls to the original file object. The data of these objects are also stored in the database, to indicate that no processing is required. With such files, the identifier and name fields in the database are empty to show this.'
But that still doesn't explain why these entries are generated for processed files. However, I suspect something is wrong with the config of this particular site, as I tested the same bahaviour on another site (far less complex as far as extensions etc is considered), and there no 'wrong' entries are generated. I'll investigate further and report back on my findings.
Updated by Mathias Schreiber almost 10 years ago
- Target version set to 7.1 (Cleanup)
- Sprint Focus set to On Location Sprint
Updated by Frans Saris almost 10 years ago
- Complexity set to easy
possible solution
In typo3/sysext/core/Classes/Resource/Service/FileProcessingService::process() extend the check
if ($processedFile->isProcessed()) {
That also checks if the identifier is set
Updated by Benni Mack over 9 years ago
- Target version changed from 7.1 (Cleanup) to 7.4 (Backend)
Updated by Jonathan IROULIN over 9 years ago
- Assignee set to Andreas Wolf
After discussion with Andreas Wolf, this is not a bug it is a feature.
This is the normal behaviour when the original file can be used,
to avoid to run the process file again and again.
Updated by Mathias Schreiber over 9 years ago
- Sprint Focus deleted (
On Location Sprint)
Updated by Martin Kutschker over 8 years ago
If name is null and the identifier empty, what sould be the values of width and height? Shoudl they both be 0? I have seen this data and it broke the FE.
And there is also a fix concerning such entries:
Updated by Andreas Wolf over 8 years ago
Martin Kutschker wrote:
If name is null and the identifier empty, what sould be the values of width and height? Shoudl they both be 0? I have seen this data and it broke the FE.
Of course the width and height of the original file should be used, as it can be used without processing (and the record is merely a flag to show that no processing is necessary and the processing attempt does not have to be repeated over and over again).
Updated by Jonas Eberle over 8 years ago
We had this happen to a high extent (up to 20% of invalid records) on a very image-heavy site. It shows in the frontend as not rescaled images where the src-output of a <f:image>-Viewhelper has the original file instead of the processed file.
It can be seen in all Typo3-7.6 - instances that we have running so this is a bug with massive impact. Some instances use ImageMagick, some GraphicsMagick.
After 'clean up' -> 'clear processed files' in the InstallTool it would randomly happen again.
Here is a patch for typo3-7.6.4 (I am checking for the existence of 'identifier'):
diff -r typo3_src-7.6.4/typo3/sysext/core/Classes/Resource/ProcessedFileRepository.php typo3_src-7.6.4-processedFilePatched/typo3/sysext/core/Classes/Resource/ProcessedFileRepository.php 136,138c136,140 < $this->databaseConnection->exec_INSERTquery($this->table, $insertFields); < $uid = $this->databaseConnection->sql_insert_id(); < $processedFile->updateProperties(array('uid' => $uid)); --- > if (isset($insertFields['identifier'])) { > $this->databaseConnection->exec_INSERTquery($this->table, $insertFields); > $uid = $this->databaseConnection->sql_insert_id(); > $processedFile->updateProperties(array('uid' => $uid)); > } 154c156,158 < $this->databaseConnection->exec_UPDATEquery($this->table, 'uid=' . (int)$uid, $updateFields); --- > if (isset($updateFields['identifier'])) { > $this->databaseConnection->exec_UPDATEquery($this->table, 'uid=' . (int)$uid, $updateFields); > }
Sorry I did not check further. This is a workaround only. Finding the real culprit would be helpful of course.
Updated by Florian Schuhmann about 5 years ago
- Related to Bug #89716: identifier not null in sys_file_processedfile added