Bug #100746
closedBetter handling of images with wrong file ending
0%
Description
If files with wrong file ending (e.g. png image with .jpg or vice versa) are uploaded there are some problem which are difficult to detect if one is not aware of it.
This may vary, depending on the Webserver configuration and graphics tool used (Image Magick or Graphics Magick).
Note: Behaviour may vary depending on TYPO3 version, most of my tests were with v11, should be verified with latest main.
I describe what happens on my system.
- for processed files, a processed file with size 0 is created (Graphics Magick does not convert it and shows an error message, but this is not displayed in BE)
- in the BE, no image is displayed as preview
- in the FE, the image is not scaled
- no error messages are displayed and no errors are logged (severity >= ERROR)
Unfortunately, I have a number of these files, noticed this when searching for files with size 0 (_processed files are created with size=0).
Suggested changes¶
- if possible show an error on upload if the wrong file ending is used or refuse to upload the file (requires 100% correct detection of wrong file ending, at least for images, which may be difficult, should probably be configurable)
- or use correct file ending when creating the processed file (e.g. as detected by graphics tool)
- or do not create processed files with file ending 0, possibly fallback to creating a placeholder image if the created file has size 0
- show errors / log messages
System¶
- TYPO3 11.5.26
- Graphics Magick (gm)
Reproduce¶
(1)
1. Upload a file with wrong file ending (e.g. previously mv, do not convert 1.jpg 1.png)
2. Insert file in "Text & Media" as "asset", specify smaller height / width, view in FE (check if scaled), compare with file which is ok
3. activate preview images in file list
- error message or fail on trying to upload image with wrong file extension or error message when creation of scaled image fails (actual: there is no flash message and no log message)
- image is scaled in FE or error message is diplayed. (Actual: is not scaled)
- preview image is displayed in filelist. (Actual: is not)
How to detect affected processed files¶
On command line:
find htdocs/fileadmin/_processed -type f -size 0
The file name of actual file can be derived from the processed file name or looked up in DB sys_file_processedfile.original => sys_file.uid.
Resources / Tools¶
gm This is what was used by TYPO3 to create the thumbnail image in the filelist
'/usr/bin/gm' 'convert' -interlace None -auto-orient +profile '*' -sample '64'x'64' 'png:/var/www/site-uol11/htdocs/fileadmin/test/problems/Koala.png[0]' '/var/www/site-uol11/var/transient/preview_6908622721787002005.png'
output:
/usr/bin/gm identify: Improper image header (/var/www/site-uol11/var/transient/preview_6908622721787002005.png) [No such file or directory].
file
$ file htdocs/fileadmin/test/problems/Koala.png
output:
JPEG image data, JFIF standard 1.02, resolution (DPI), density 96x96, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=7, datetime=2009:03:12 13:48:28], baseline, precision 8, 1024x768, components 3
file can be used to show the mime type:
$ file -b --mime-type htdocs/fileadmin/test/problems/Koala.png
shows:
image/jpeg
or the (correct) extension(s):
$ file --extension htdocs/fileadmin/test/problems/Koala.png
shows:
jpeg/jpg/jpe/jfif
PHP exif_imagetype
https://www.php.net/manual/en/function.exif-imagetype.php
also determines real image type.
Updated by Sybille Peters over 1 year ago
In my tests, the internal information seems to be inconsistent, e.g. with above example which is a jpg but poses as a .png file:
$mimetype = $currentFileObject->getMimeType();
$extension = $currentFileObject->getExtension();
$absoluteFilePath = Environment::getPublicPath() . '/' . $currentFileObject->getPublicUrl(false);
$type = $currentFileObject->getType();
$isImage = $currentFileObject->isImage();
if ($isImage) {
$realImageType = \exif_imagetype($absoluteFilePath);
$this->io->writeln(sprintf('file: extension=%s mimetype=%s, type=%d, real type=%d', extension, mimetype, type, realImageType));
}
result:
file: extension=png, mimetype=image/png, type=2, real type=2
So, mimetype contains the incorrect mimetype ("image/png"), but type contains the correct type (IMAGETYPE_JPEG = 2), see https://www.php.net/manual/en/function.exif-imagetype.php
Updated by Sybille Peters over 1 year ago
The mimetype is derived from the file extension in FileInfo::getMimeType:
public function getMimeType()
{
$mimeType = false;
if ($this->isFile()) {
$fileExtensionToMimeTypeMapping = $GLOBALS['TYPO3_CONF_VARS']['SYS']['FileInfo']['fileExtensionToMimeType'];
$lowercaseFileExtension = strtolower($this->getExtension());
if (!empty($fileExtensionToMimeTypeMapping[$lowercaseFileExtension])) {
$mimeType = $fileExtensionToMimeTypeMapping[$lowercaseFileExtension];
https://github.com/TYPO3/typo3/blob/main/typo3/sysext/core/Classes/Type/File/FileInfo.php
Updated by Claude Unterleitner over 1 year ago
in addition: Today I detected in a TYPO3 V11 that it is possible to upload files without filename extension, e.g. jpg without .jpg or .jpeg ending
When this (e.g. image) files are used on pages, the affected pages will be broken (Error 503).
Updated by Andreas Kießling 4 months ago
My client just stumbled up on this one as well after upgrading from 10.4 to 12.4
And since the mime type is wrongly set, we can't easily spot the error and you have to inspect/download the image etc. Would be great to have this fixed
Updated by Sybille Peters 4 months ago · Edited
We use the extension secure_downloads . I did my above tests with the extension installed (I think, not 100% sure). It makes sense to repeat the tests without it and check if / how the result is different.
My gut feeling is what the extension does is not good and we might get different results without it.
Would be great if someone could double-check this or repeat my tests.
secure_downloads overrides the setting:
$GLOBALS['TYPO3_CONF_VARS']['SYS']['FileInfo']['fileExtensionToMimeType'] += \Leuchtfeuer\SecureDownloads\MimeTypes::ADDITIONAL_MIME_TYPES;
TYPO3 explictly does not recommend this, see
typo3/sysext/core/Configuration/DefaultConfiguration.php:
'FileInfo' => [ // Static mapping for file extensions to mime types. // In special cases the mime type is not detected correctly. // Use this array only if the automatic detection does not work correct! 'fileExtensionToMimeType' => [ 'svg' => 'image/svg+xml', 'youtube' => 'video/youtube', 'vimeo' => 'video/vimeo', ], ],
The question is if TYPO3 does "real" detection of mimetime or detects mimetype by using the file extension.
Updated by Andreas Kießling 4 months ago
Oh, that is a VERY good hint. secure_downloads is installed as well...
I'll put it on my todo list to check without, but it might take some time :/
Updated by Sybille Peters 4 months ago
I confirm this problem does not occur with TYPO3 v11 without extra extensions and is most likely related to secure_downloads.
Strictly speaking, we should close because it only occurs with not recommended configuration. If someone disagrees and can make a good case, please feel free to reopen.
I performed my tests above (see "Reproduce") by uploading image with "wrong" file ending in TYPO3 system without additional extensions, latest 11.5 branch.
This now is ok:
- preview displayed in file list
- image scaled correctly.
So, it looks like this is related to secure_downloads and specifically overriding the fileExtensionToMimeType as I wrote in my previous comment.
You can also look in the database, this is for my "t3coredev11" - without extensions: (file has .jpg but is png)
select uid,type,extension,mime_type,identifier from sys_file where identifier like '%WRONG_FILE_ENDING%'; +-----+------+-----------+-----------+------------------------------------------------------------------------------+ | uid | type | extension | mime_type | identifier | +-----+------+-----------+-----------+------------------------------------------------------------------------------+ | 253 | 2 | jpg | image/png | /test/wrong_file_extension/edit_redirect_target_WRONG_FILE_ENDING_IS_PNG.jpg | +-----+------+-----------+-----------+------------------------------------------------------------------------------+
This is on same machine but different installation v11 with extensions, including secure_downloads:
select uid,type,extension,mime_type,identifier from sys_file where identifier like '%WRONG_FILE_ENDING%'; +---------+------+-----------+------------+-------------------------------------------------------------------------------------+ | uid | type | extension | mime_type | identifier | +---------+------+-----------+------------+-------------------------------------------------------------------------------------+ | 1010289 | 2 | jpg | image/jpeg | /dateien_mit_falscher_dateiendung/edit_redirect_target_WRONG_FILE_ENDING_IS_PNG.jpg | +---------+------+-----------+------------+-------------------------------------------------------------------------------------+
Updated by Sybille Peters 4 months ago
I added issue for secure_downloads: https://github.com/Leuchtfeuer/typo3-secure-downloads/issues/205
Updated by Andreas Kießling 4 months ago
Hi @Sybille Peters ,
confirmed --> the behavior in v12 is exactly the same with and without secure_downloads. Thanks for your investigation!