ImageMagick - Prevent Concurrent Processing of Image Files
- Apache 2.4.23
- PHP 7.0.20 Nonthread safe
- TYPO3 7.6.20
- ImageMagick 6.7.8-9
Things to note:
- A website uploads pictures in extremely high resolution, meaning that the average filesize of each file is about 30-40MB.
- There are a number of custom content elements, each with its own DataProcessor. For instance, one creates 8 different processed versions of an image for various viewports.
- Switching to GraphicsMagick did not solve the problem.
ImageMagick fails to process all images for all viewports and thus results in creating corrupted files when multiple requests to the server are made that causes the same image files to be processed multiple times simultaneously.
Previously attempted strategies, etc.:
- Create a database locking mechanism that prevents a second conversion if a flag is set signaling that processing is already in progress.
- Create smaller versions of the file to then convert instead of the original
- Analyzing the errors thrown by ImageMagick. No recognizable pattern was noticable. The conversion failed at multiple points with different pictures with different resolutions.
Under the assumption that ImageMagick is working properly AND/OR cannot be reconfigured to prevent this problem:
- Implement preprocessing by utilizing appropriate hooks in DataHandler. For this to be reliable, we would like to limit requests to those made by the TYPO3 backend, thus, essentially turning off "on the fly" conversion.
- The overall amount of traffic seems to be irrelevant. This has been tested in development and production environments. It is sufficient that two processes are started (at least more or less) simultaneously to process an image that causes ImageMagick to fail, at least partially.
- The filesize seems to be a large contributing factor. If it is smaller than 5MB, this problem rarely (if at all) occurs.
Nice to have
- We would like to be able to use the original file with no intermediary reduction of the image quality.
Concurrent processing of large images using ImageMagick fails.
#2 Updated by Joseph Linden about 2 years ago
I agree that database locking could be too slow. Yes, file locking has been considered, but we were slightly wary of that approach since access exceptions/warnings could be thrown which could result in problems for the request that "lost", since access to the file itself would be denied. Are these concerns justified?
By API, I assume you mean the class \TYPO3\CMS\Core\Locking\Locker?
#6 Updated by Joseph Linden about 2 years ago
OK, we extended the LocalImageProcessor and forced images to be processed after saving. Both use the locking mechanism. This seems to work, but we still need to test in a production environment to test for locking failures. I have also posted the issue here:
Please do not yet close this ticket.
#7 Updated by Webadmin no-lastname-given 12 months ago
We discovered a similar problem. On Ubuntu 18.04., PHP 7.2, with GraphicsMagick even with "normal" image sizes around 1 to 2 MB.
Requestion a Typo3 page renderes all the images and will store them under fileadmin/_processed_. So far this works fine. But as soon as there are more requests to different pages at the same time which contains the same images, we got corrupted JPGs. Mostly the images are grey. One example is attached. Sometimes the image isn't processed in any way and delivered in the original size to the client.
We think that typo3 starts the processing for the image multiple times if it is not already processed. Then both jobs will store their result in the same file?
For testing we now switched from GM to IM.
How did you solve the problem in the meantime?
#9 Updated by Webadmin no-lastname-given 10 months ago
In the meantime we could collect some more details:
- Seems to happen especially for pictures which are includes on several pages. So a parallel processing will start as soon as two pages are requested at the same time.
- We have two different effects: The image is rendered in a corrupt way or it is outputted in it's original size.
- If the picture is outputted in it's original size we are seeing a new entry in the sys_file_processedfile table which contains an empty identifier and NULL in name, width and heigt.
Especially the last point is very bad for the user experience. Often we deliver 7MB JPGs for an 50x50 teaser image.
Typo3 should implement some locking within the image processing which should prevent the system from processing images with the same processing instructions at the same time.
We also tested IM, GM, PHP 7.0, PHP 7.2: Same results. Other PHP based image processing tools are working fine. So it's an typo3 related problem.