Bug #44518
closedCaching problem with Gifbuilder
100%
Description
I had a cachingproblem with the Gifbuilder. Everytime I cleared the cache, a new image was generated although the TS-setup wasn't changed.
Here my the TS-Code: file = GIFBUILDER
file{
quality = 100
XY = [10.w],[10.h]
format = png
10 = IMAGE
10{
file.import.data = register:ORIG_FILENAME
file.maxH = 760m
file.maxW = 940m
}
20 = IMAGE
20{
file = fileadmin/img/wasserT.png
offset = [10.w]/2 - 25, [10.h] - 45
}
}
(Without the second layer I hadn't the problem (without 20).)
In typo3/sysext/frontend/Classes/Imaging/GifBuilder.php
I figured out that the generated filename was different every run.
@ public function fileName($pre) {
...
// WARNING: In PHP5 I discovered that rendering with freetype of Japanese letters was totally corrupt.
// Not only the wrong glyphs are printed but also some memory stack overflow resulted in strange additional
// chars - and finally the reason for this investigation: The Bounding box data was changing all the time
// resulting in new images being generated all the time. With PHP4 it works fine.
return $this->tempPath . $pre . $meaningfulPrefix . \TYPO3\CMS\Core\Utility\GeneralUtility::shortMD5(serialize($this->setup)) . '.' . $this->extension();
}@
(the comment shows that the problem already appeared in a different way)
The problem is the md5hash of $this->setup. $this->setup is an array witch contains 2 objects ['BBOX']['originalFile'] and ['BBOX']['originalFile']. These objects are different after the cache was cleared.
My solution: copy $this->setup to $this->setupLight and remove these objects from setupLight. I've done this in:
@
public function start($conf, $data) {
...
// BBOX (the fileinfo) is to big and different each run: $this->fileName() generates a new file every run, because it uses a md5hash of this array
// ['BBOX']['originalFile'] and ['BBOX']['originalFile'] are Objects (to big and different each run)
// $this-setupLight is used now in $this->fileName()
//
$this->setupLight = $this->setup;
foreach ($sKeyArray as $theKey) {
$this->setupLight[$theKey . '.']['BBOX']['originalFile']='';
$this->setupLight[$theKey . '.']['BBOX']['processedFile']='';
}
//
// Get trivial data
$XY = \TYPO3\CMS\Core\Utility\GeneralUtility::intExplode(',', $this->setup['XY']);
$maxWidth = isset($this->setup['maxWidth.']) ? intval($this->cObj->stdWrap($this->setup['maxWidth'], $this->setup['maxWidth.'])) : intval($this->setup['maxWidth']);
$maxHeight = isset($this->setup['maxHeight.']) ? intval($this->cObj->stdWrap($this->setup['maxHeight'], $this->setup['maxHeight.'])) : intval($this->setup['maxHeight']);
$XY[0] = \TYPO3\CMS\Core\Utility\MathUtility::forceIntegerInRange($XY[0], 1, $maxWidth ? $maxWidth : 2000);
$XY[1] = \TYPO3\CMS\Core\Utility\MathUtility::forceIntegerInRange($XY[1], 1, $maxHeight ? $maxHeight : 2000);
$this->XY = $XY;
$this->w = $XY[0];
$this->h = $XY[1];
$this->OFFSET = \TYPO3\CMS\Core\Utility\GeneralUtility::intExplode(',', $this->setup['offset']);
// this sets the workArea
$this->setWorkArea($this->setup['workArea']);
// this sets the default to the current;
$this->defaultWorkArea = $this->workArea;
}
}
@
then I changed in function fileName():
return $this->tempPath . $pre . $meaningfulPrefix . \TYPO3\CMS\Core\Utility\GeneralUtility::shortMD5(serialize($this->setup)) . '.' . $this->extension();
to
return $this->tempPath . $pre . $meaningfulPrefix . \TYPO3\CMS\Core\Utility\GeneralUtility::shortMD5(serialize($this->setupLight)) . '.' . $this->extension();
Updated by Mathias Schreiber almost 10 years ago
- Target version set to 7.2 (Frontend)
- Is Regression set to No
Updated by Benni Mack over 9 years ago
- Target version changed from 7.2 (Frontend) to 7.4 (Backend)
Updated by Susanne Moog over 9 years ago
- Target version changed from 7.4 (Backend) to 7.5
Updated by Benni Mack about 9 years ago
- Target version changed from 7.5 to 7 LTS
Updated by Mathias Schreiber about 9 years ago
- Category changed from Caching to Image Cropping
- Target version deleted (
7 LTS)
Updated by Susanne Moog about 7 years ago
- Category changed from Image Cropping to Image Generation / GIFBUILDER
Updated by Thomas Oliver Moll almost 6 years ago
- Priority changed from Should have to Must have
- Target version set to Candidate for patchlevel
- TYPO3 Version changed from 6.0 to 8
- PHP Version changed from 5.3 to 7.1
Hi,
this is a very old Bug, but it was never really solved or it is back with TYPO 3 8.7.
We recently updated our huge Installations from 7.6 to 8.7 and we now run into issues with the Gifbulder.
We use Gifbuilder to render metadata into images. This data doesn't change often, but Gifbulder creates the same Image over and over with different hashes.
Like Martin, I traced it back to that [BBOX] part in the setup array that is sent through md5 to get the hash.
BBOX changes constantly and is HUGE! I wrote it via print_r into Text-Files and my disk filled up instantly because these text files had 32MB each and there were thousands of them.
The BBOX part is included by the Gifbuilder layers, that load Images. The patch above seemed not to be working for me, but what I did was very similar:
$setupLight=this.setup
unset($setupLight[10.][BBOX]['originalFile'])
unset($setupLight[10.][BBOX]['prosessedFile'])
unset($setupLight[10.][BBOX]['fileHash'])
and then I changed the code to use $setupLight instead of the original to create the Hash.
Our Images have the included Image always on Layer 10, so I did not have to iterate over the whole array.
Setting the entries to '' did not work for me, but unset worked fine.
I don't have the code at hand, so I had to write it here by memory, I hope there are not too many syntax errors after all, I'm not a php coder.
I'm not sure why the FileHash within the BBOX changes as well, I did not find any differences that (I thought) mattered. But this is hard to say when you compare two 32MB text files with scrambled order and many RECURSION in it.
Getting rid of the 2 subtrees and the filehash resulted in much smaller setup arrays (the text file size went down to 1800 Bytes and such like)
Please inform me, If I should open a new issue instead opening this 6 year old one.
I can provide a patch that works on the same iterating principle like Martins does, but I'm not sure if there is a smarter way not to use too much data for hashing, after all I firstly create a copy of that huge array, and delete the unnecessary bits afterwards
Cheers
TOM
Updated by Thomas Oliver Moll almost 6 years ago
- Related to Bug #86947: Gifbuilder: combine images generates duplicate images of the same picture. added
Updated by Gerrit Code Review almost 6 years ago
- Status changed from New to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59644
Updated by Gerrit Code Review almost 6 years ago
Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59644
Updated by Gerrit Code Review over 5 years ago
Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59644
Updated by Gerrit Code Review over 5 years ago
Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59956
Updated by Markus Klösges over 5 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset 7d5e5944e3a044d4e993028ad3bdd45a1f96d9e6.
Updated by Gerrit Code Review over 5 years ago
- Status changed from Resolved to Under Review
Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59959
Updated by Markus Klösges over 5 years ago
- Status changed from Under Review to Resolved
Applied in changeset 8d584f7436b33950908b26e1aa3ffcb387d817c9.
Updated by Gerrit Code Review over 5 years ago
- Status changed from Resolved to Under Review
Patch set 2 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59959
Updated by Markus Klösges over 5 years ago
- Status changed from Under Review to Resolved
Applied in changeset 5bb260cb0170355d23ff696c1bc3c2d2caffa37b.