TYPO3 Core

I really can't tell why the the sys_file_metadata records are missing. The problem occurred in an updated TYPO3 version (6.1 -> 6.2). The problem only occurred in this instance. The records were not missing but the width and height was empty. The files where not added recently and there is no custom code at all involved in processing the files.

Of cause it would be easy to set "UPDATE sys_file SET missing=1 - but this is exactly the problem. If you do and start the indexer the meta data is regenerated with width and height = 0 and not with the correct values. So this does not help in this situation.

We also had this problem with width and height=0 in sys_file_metadata. This was on an installation which started with an early version of 6.0 or 6.1 and had been updated. Files had been added continuously on the server and the FAL indexer run regularly.

Also, there were duplicate entries (2 entries with the same identifier and the same storage id in sys_file).

One reason sys_file_metadata records could be missing is an upgrade from 6.1 where there only was sys_file.

Our instance was also an updated one (6.1 to 6.2)

Did you run all the upgrade wizards correct when upgrading to 6.2?

Of cause it would be easy to set "UPDATE sys_file SET missing=1 - but this is exactly the problem. If you do and start the indexer the meta data is regenerated with width and height = 0 and not with the correct values. So this does not help in this situation.

Could it be that the width and height columns of the sys_file table are still present? These need to be removed. The width and height columns are moved to sys_file_metadata since 6.2.

My instance was updated from 6.1 to 6.2 too.
Same problem here.
All workarounds in this issue and #46020 were applied without success.
I built a script based on #46020#note-81 that executes sql updates over sys_file_metadata using "identify" command, but not all images were covered.

Any solution for this?

You build a script that looks in all folders that are indexed?

Do you see a common difference between the sys_file_metadata records that are correct now and the once that aren't? Same folder or storage uid?

Does your sys_file table still have the width and height column? Because that would result in some issues.

Frans Saris wrote:

You build a script that looks in all folders that are indexed?

Correct. I have three "storages": uploads/, fileadmin/ and "the rest". And my scripts goes for all pictures (png,jpg,gif), identifies the width and the height for every image and builds a sql update for every image in the sys_file_metadata table.

Do you see a common difference between the sys_file_metadata records that are correct now and the once that aren't? Same folder or storage uid?

No. No pattern as far as I can see.

Does your sys_file table still have the width and height column? Because that would result in some issues.

No. After applying the upgrade wizard in the backend, I applied the compare tool and deleted these fields.
I would like to recreate them again but in the sys_file_metadata table.
The scheduler tasks referring to FAL didn't work for me.
Width and height are setup to 0 again.

• what filesystem are you using? *nix, mac, windows
• Are your filestorages set to case-sensitive
• You got a storage 0, 1 and 2? 0 = the rest, 1 = uploads, 2 = fileadmin?
• If you fetch a file that doesn't get a width and height, rename it and upload it in BE filelist the width and height are set correct?
• If your script identifies the width and height of every png, jpg and gif image. Why are the width and height still 0 than? Is the file identifier somehow incorrect?

gr. Frans

Frans Saris wrote:

• what filesystem are you using? *nix, mac, windows

Unix.

• Are your filestorages set to case-sensitive

Yes.

• You got a storage 0, 1 and 2? 0 = the rest, 1 = uploads, 2 = fileadmin?

Yes.

• If you fetch a file that doesn't get a width and height, rename it and upload it in BE filelist the width and height are set correct?

Yes.

• If your script identifies the width and height of every png, jpg and gif image. Why are the width and height still 0 than? Is the file identifier somehow incorrect?

The file identifier is correct. Width and height are setup properly with my script but I can't get all images to have proper width and height, as "identify" command fails for some that are corrupted.

What I am worried is that, even when all are setup properly, refreshing the cache makes them back to 0 width and height again.

Refreshing the cache will not touch the sys_file_metadata record.

The "required" metadata will only be re-set when the sys_file record is reindex because of the missing flag or filesize change.

Could you maybe do some further debugging and have a look at the result of list($metaData['width'], $metaData['height']) = getimagesize($rawFileLocation); in TYPO3\CMS\Core\Resource\Index\Indexer::extractRequiredMetaData().

Flag a sys_file missing and run the FAL indexer task.

As you recommended, I made some debugging and I could see that the scheduler task fills properly the dimensions. That allowed me to observe something: images outside of folders "fileadmin" and "uploads" are never indexed and filled with the proper dimensions.

I created two scheduler tasks for every storage in my system ("fileadmin" and "uploads") and ran them.
Everything went ok.
But what about images that are used in my extensions under typo3conf/ ?
My workaround was creating a new "storage" on PID=0 and create a new scheduler task for indexing this storage.
Now the image is twice in sys_file_metadata table, but one row is for storage 0 (all), which has width=height=0, and the other is for my new storage, that sets up the proper dimensions. The frontend takes this one.

I confirm the original bug commented still exists:

1. If sys_file_metadata table is truncated for some reason (or the upgrade tool doesn't work well), the first time the frontend is refreshed, the images are shown. They don't have width and height parameters at all.

2. New sys_file_metadata rows for these images are created with width=height=0.

3. The user refreshes the cache and then the frontend displays no images as they take the information from sys_file_metadata.

Running this new scheduler task with a "storage" for typo3conf/ solves the problem for me.

Thank you very much for the feedback, Frans.

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/33990

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/36454

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/36457

Thank you, the hint with "UPDATE sys_file SET missing=1" helped.

This also worked for me:

UPDATE sys_file_metadata fm, sys_file f SET fm.width = f.width WHERE fm.file = f.uid;
UPDATE sys_file_metadata fm, sys_file f SET fm.height = f.height WHERE fm.file = f.uid;

The width and height columns of sys_file should be removed (see install tool => DB compare). Else you keep running into this issue.