TYPO3 Core

Hi,

we recently updated all our installations from 7.6 to 8.7. and we experienced a problem:

/typo3temp/assets/images overflowed with images.

Our filesystem can handle only ~22000 files per folder, and our Installations are huge. But that was not the main problem. Some files were in there many hundered times, in one case over 7000 times.
Each time with a different hash.

I'm not a PHP developer and not too familiar with the innards of the TYPO3 core, but my analysis showed, that the hash is generated in GifBuilder.php by putting $setup through md5. Not a problem per se.
But in our case $setup was up to 32MB big (when written als Text to disk) - and even worse differed from call to call.
The culprit seemed to be a sub-array of $setup ['BBOX'] containing huge amounts of data, recursions and a changing order of its elements.

The code is nearly unchanged since TYPO3 6.2 (oldest copy I had) and I even found a open issue going back some 6 years:
https://forge.typo3.org/issues/44518#change-391424

I did not receive any answer when I updated that old issue, so I do it here. I hope double-posting this is not too bad.

I now have a working work-around in place, but I don't think it's the best solution:

in function fileName in typo3/sysext/frontend/Classes/Imaging/GifBuilder.php I changed the line 708

        return 'typo3temp/' . $pre . $filePrefix . '_' . GeneralUtility::shortMD5(serialize($this->setup)) . '.' . $this->extension();

to

        $setupLight=$this->setup;        
        foreach ($setupLight as $theKey =>$theValue) {          
            unset($setupLight[$theKey.'.']['BBOX']['originalFile']);
            unset($setupLight[$theKey.'.']['BBOX']['processedFile']);
            unset($setupLight[$theKey.'.']['BBOX']['fileCacheHash']);          
        }
        return 'typo3temp/' . $pre . $filePrefix . '_' . GeneralUtility::shortMD5(serialize($setupLight)) . '.' . $this->extension();

This is roughly the same approach of the mentioned issue 44518.

Since the problem seemingly to arrived for us with 8.7 (or at least worsened to be noticeable) and since the code in GifBuilder.php hasn't really changed in a long time, I think the problem lies more in having too much data in that BBOX array.
To my understanding it is for the "BoundingBox" of an image layer, but it contains so much more 'stuff'.

I can not rule out that this array gets this big because of something we do somewhere else, but I have no idea, what this could be.

I have to admit, I struggled with getting GifBuilder to work with FAL when we switched to TYPO3 7.6 so my quirky typoscript does some 'things' to the image ID to produce a version of it with text layered on top:

lib.imageplus=FILES
lib.imageplus{
  # The Fluid Template gives an array of "fields", using currentValueKey we set the current object,
  # which contains the id of the file we want
  files.current=1

  # The String we get starts with "file:" in front of the id, so we cut it away using substing
  files.stdWrap.substring=5

  # But sometimes this code is called in a way, that gives us the id without "file:" 
  # So we check if the given String already is a number. There seems to be no isNumber() function though
  # so we use this abomination that gives us the same result 
  # This was written to be compatible with data stored in databases written with TYPO3 6.2, 7.6 and 8.7
  # and was not re-written for a long time so we had to cope with old obsolete data formats
  # The version for 8.7 now uses a ViewHelper to get the ID from strings containing t3://
  # So this code may be not needed anymore.
  files.stdWrap.substring.if.isLessThan=100000000
  files.stdWrap.substring.if.value={field:image}
  files.stdWrap.substring.if.value.insertData=1

  renderObj = COA
  renderObj{
    # After entering the scope of GIFBUILDER we seem to be unable to access the file 
    # object so we store everything we need in the register
    10 = LOAD_REGISTER
    10 {
      fileid.data = file:current:uid
      creator.data = file:current:creator
      status.data = file:current:status
      visible.data = file:current:visible

      storage.data = file:current:storage

      identifier.data = file:current:identifier
      identifier_hash.data = file:current:identifier_hash
      width.data= file:current:width
      height.data= file:current:height
      filecaption.data = file:current:caption
      desc.data = file:current:description
      title.data = file:current:title
      alternative.data = file:current:alternative
      link{
    typolink.parameter.data = file:current:link
    typolink.returnLast = url

      }
    }
    20=IMG_RESOURCE
    20{
      file = GIFBUILDER
      file {
    # The size of the to-be-created images is equal to the image we load in layer 10:
    XY=[10.w],[10.h]
    XY.override.if.isFalse.data = register:visible
    XY.override=[15.w],[15.h]
    format = jpg
    quality=90
    10 = IMAGE
    10 {
      if.isTrue.data = register:visible
      file {
        #We get the ID from the register and use it as the source of the image:
        import.data = register:fileid
        # we get the wanted width from the ViewHelper in the field array
        maxW={field:width}
        maxW.insertData=1
      }
    }

...