Bug #90600
openFrontend cache inconsistency with processed images
0%
Description
We discovered a cache inconsistency where processed images can be removed from the filesystem, even though they are still part of a cached page. We have tested this with TYPO3 9 LTS, but it is very likely also an issue with TYPO3 10 LTS.
Steps to reproduce the issue
- Create two pages pageA and pageB
- Add an images content element to pageA and pageB and choose the same image demo_image.png
- Apply image processing to the image (e.g. max width 50px) on both pages
- Open both pages in the frontend and note the generated unique hash (e.g. /fileadmin/_processed_/d/2/csm_demo_2cb965e686.png -> 2cb965e686). The hash should be the same on both pages.
- Change the column modification_date in sys_file for demo_image.png, this will lead to a new hash for the processed image
- Clear the page cache for pageA only
- Open pageA in the frontend, note the new unique hash e.g. /fileadmin/_processed_/d/2/csm_demo_bf52a6057a.png -> bf52a6057a)
- Note that the previous processed image csm_demo_2cb965e686.png has been removed from the file system
- Open pageB (make sure caching is not disabled, e.g. use inkognito mode) and note that the image is missing on the page because it is still referencing csm_demo_2cb965e686.png, which has been deleted.
Proposed fix
Add page cache tags for all processed images. Whenever a processed image is removed, clear the page cache of all pages including that image using the cache tags.
Additional info
At the moment the deletion is done in TYPO3\CMS\Core\Resource\ProcessedFile in a function called needsReprocessing. Probably processed images should not be deleted in a seemingly innocent method called needsReprocessing. See https://github.com/TYPO3/TYPO3.CMS/blob/9bb994f492e75efcfe139c3d6a64ef42dbef9bd1/typo3/sysext/core/Classes/Resource/ProcessedFile.php#L513-L515
Updated by Friedemann Altrock about 1 year ago
This bug very much still exists.
The phenomenon occurred for a customer of mine, some images vanishing after an arbitrary number of days or weeks of no changes in the system (or so it seemed).
It took a while to find out the underlying cause. As it turns out some external process updates the images in an export/import.
Even if the file is the same, the modification_date column is updated when the metadata extraction or FAL indexing scheduler task runs.
This leads to a different checksum, and as soon as the page containing the image is regenerated, the "old" processed file is deleted.
If another cache_pages entry containing the old URL has not expired, it shows a 404 image. In our case multiple cache entries existed
because of some TypoScript conditions.
Just adding cache tags for the processed file DB entry would not work, as the FileDeletionAspect removes the row via QueryBuilder, not DataHandler.
As an aside: delete()
is also called in the method ProcessedFile::setName
, another "innocent" name hiding a serious side effect.
Updated by Friedemann Altrock about 1 year ago
I've made an automated reproduction here https://github.com/fwg/typo3-bugs/tree/forge/90600.