Project

General

Profile

Actions

Bug #100746

closed

Better handling of images with wrong file ending

Added by Sybille Peters about 1 year ago. Updated 24 minutes ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
-
Start date:
2023-04-25
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
11
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:
Needs Decision

Description

If files with wrong file ending (e.g. png image with .jpg or vice versa) are uploaded there are some problem which are difficult to detect if one is not aware of it.

This may vary, depending on the Webserver configuration and graphics tool used (Image Magick or Graphics Magick).

Note: Behaviour may vary depending on TYPO3 version, most of my tests were with v11, should be verified with latest main.

I describe what happens on my system.

  • for processed files, a processed file with size 0 is created (Graphics Magick does not convert it and shows an error message, but this is not displayed in BE)
  • in the BE, no image is displayed as preview
  • in the FE, the image is not scaled
  • no error messages are displayed and no errors are logged (severity >= ERROR)

Unfortunately, I have a number of these files, noticed this when searching for files with size 0 (_processed files are created with size=0).

Suggested changes

  • if possible show an error on upload if the wrong file ending is used or refuse to upload the file (requires 100% correct detection of wrong file ending, at least for images, which may be difficult, should probably be configurable)
  • or use correct file ending when creating the processed file (e.g. as detected by graphics tool)
  • or do not create processed files with file ending 0, possibly fallback to creating a placeholder image if the created file has size 0
  • show errors / log messages

System

  • TYPO3 11.5.26
  • Graphics Magick (gm)

Reproduce

(1)

1. Upload a file with wrong file ending (e.g. previously mv, do not convert 1.jpg 1.png)
2. Insert file in "Text & Media" as "asset", specify smaller height / width, view in FE (check if scaled), compare with file which is ok
3. activate preview images in file list

Expected:
  • error message or fail on trying to upload image with wrong file extension or error message when creation of scaled image fails (actual: there is no flash message and no log message)
  • image is scaled in FE or error message is diplayed. (Actual: is not scaled)
  • preview image is displayed in filelist. (Actual: is not)

How to detect affected processed files

On command line:

find htdocs/fileadmin/_processed -type f -size 0

The file name of actual file can be derived from the processed file name or looked up in DB sys_file_processedfile.original => sys_file.uid.

Resources / Tools

gm This is what was used by TYPO3 to create the thumbnail image in the filelist

'/usr/bin/gm' 'convert' -interlace None -auto-orient +profile '*' -sample '64'x'64' 'png:/var/www/site-uol11/htdocs/fileadmin/test/problems/Koala.png[0]' '/var/www/site-uol11/var/transient/preview_6908622721787002005.png'

output:

/usr/bin/gm identify: Improper image header (/var/www/site-uol11/var/transient/preview_6908622721787002005.png) [No such file or directory].

file

$ file htdocs/fileadmin/test/problems/Koala.png 

output:

JPEG image data, JFIF standard 1.02, resolution (DPI), density 96x96, segment length 16, Exif Standard: [TIFF image data, big-endian, direntries=7, datetime=2009:03:12 13:48:28], baseline, precision 8, 1024x768, components 3

file can be used to show the mime type:

$ file -b --mime-type htdocs/fileadmin/test/problems/Koala.png 

shows:

image/jpeg

or the (correct) extension(s):

$ file  --extension htdocs/fileadmin/test/problems/Koala.png 

shows:

jpeg/jpg/jpe/jfif


PHP exif_imagetype

https://www.php.net/manual/en/function.exif-imagetype.php

also determines real image type.

Actions #1

Updated by Sybille Peters about 1 year ago

  • Description updated (diff)
Actions #2

Updated by Sybille Peters about 1 year ago

In my tests, the internal information seems to be inconsistent, e.g. with above example which is a jpg but poses as a .png file:

$mimetype = $currentFileObject->getMimeType();
$extension = $currentFileObject->getExtension();
$absoluteFilePath = Environment::getPublicPath() . '/' . $currentFileObject->getPublicUrl(false);
$type = $currentFileObject->getType();
$isImage = $currentFileObject->isImage();
if ($isImage) {
    $realImageType = \exif_imagetype($absoluteFilePath);
    $this->io->writeln(sprintf('file: extension=%s mimetype=%s, type=%d, real type=%d', extension, mimetype, type, realImageType));
}

result:

file: extension=png, mimetype=image/png, type=2, real type=2


So, mimetype contains the incorrect mimetype ("image/png"), but type contains the correct type (IMAGETYPE_JPEG = 2), see https://www.php.net/manual/en/function.exif-imagetype.php

Actions #3

Updated by Sybille Peters about 1 year ago

The mimetype is derived from the file extension in FileInfo::getMimeType:

public function getMimeType()
{
    $mimeType = false;
    if ($this->isFile()) {
        $fileExtensionToMimeTypeMapping = $GLOBALS['TYPO3_CONF_VARS']['SYS']['FileInfo']['fileExtensionToMimeType'];
        $lowercaseFileExtension = strtolower($this->getExtension());
        if (!empty($fileExtensionToMimeTypeMapping[$lowercaseFileExtension])) {
            $mimeType = $fileExtensionToMimeTypeMapping[$lowercaseFileExtension];

https://github.com/TYPO3/typo3/blob/main/typo3/sysext/core/Classes/Type/File/FileInfo.php

Actions #4

Updated by Claude Unterleitner about 1 year ago

in addition: Today I detected in a TYPO3 V11 that it is possible to upload files without filename extension, e.g. jpg without .jpg or .jpeg ending

When this (e.g. image) files are used on pages, the affected pages will be broken (Error 503).

Actions #5

Updated by Sybille Peters about 1 year ago

  • Description updated (diff)
Actions #6

Updated by Georg Ringer 20 days ago

  • Sprint Focus set to Needs Decision
Actions #7

Updated by Andreas Kießling about 11 hours ago

My client just stumbled up on this one as well after upgrading from 10.4 to 12.4
And since the mime type is wrongly set, we can't easily spot the error and you have to inspect/download the image etc. Would be great to have this fixed

Actions #8

Updated by Sybille Peters about 5 hours ago · Edited

We use the extension secure_downloads . I did my above tests with the extension installed (I think, not 100% sure). It makes sense to repeat the tests without it and check if / how the result is different.

My gut feeling is what the extension does is not good and we might get different results without it.

Would be great if someone could double-check this or repeat my tests.

secure_downloads overrides the setting:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['FileInfo']['fileExtensionToMimeType'] += \Leuchtfeuer\SecureDownloads\MimeTypes::ADDITIONAL_MIME_TYPES;

TYPO3 explictly does not recommend this, see

typo3/sysext/core/Configuration/DefaultConfiguration.php:

'FileInfo' => [
            // Static mapping for file extensions to mime types.
            // In special cases the mime type is not detected correctly.
            // Use this array only if the automatic detection does not work correct!
            'fileExtensionToMimeType' => [
                'svg' => 'image/svg+xml',
                'youtube' => 'video/youtube',
                'vimeo' => 'video/vimeo',
            ],
        ],

The question is if TYPO3 does "real" detection of mimetime or detects mimetype by using the file extension.

Actions #9

Updated by Andreas Kießling about 5 hours ago

Oh, that is a VERY good hint. secure_downloads is installed as well...

I'll put it on my todo list to check without, but it might take some time :/

Actions #10

Updated by Sybille Peters about 3 hours ago

I confirm this problem does not occur with TYPO3 v11 without extra extensions and is most likely related to secure_downloads.

Strictly speaking, we should close because it only occurs with not recommended configuration. If someone disagrees and can make a good case, please feel free to reopen.


I performed my tests above (see "Reproduce") by uploading image with "wrong" file ending in TYPO3 system without additional extensions, latest 11.5 branch.

This now is ok:

  • preview displayed in file list
  • image scaled correctly.

So, it looks like this is related to secure_downloads and specifically overriding the fileExtensionToMimeType as I wrote in my previous comment.

You can also look in the database, this is for my "t3coredev11" - without extensions: (file has .jpg but is png)

select uid,type,extension,mime_type,identifier from sys_file where identifier like '%WRONG_FILE_ENDING%';
+-----+------+-----------+-----------+------------------------------------------------------------------------------+
| uid | type | extension | mime_type | identifier                                                                   |
+-----+------+-----------+-----------+------------------------------------------------------------------------------+
| 253 | 2    | jpg       | image/png | /test/wrong_file_extension/edit_redirect_target_WRONG_FILE_ENDING_IS_PNG.jpg |
+-----+------+-----------+-----------+------------------------------------------------------------------------------+

This is on same machine but different installation v11 with extensions, including secure_downloads:

select uid,type,extension,mime_type,identifier from sys_file where identifier like '%WRONG_FILE_ENDING%';
+---------+------+-----------+------------+-------------------------------------------------------------------------------------+
| uid     | type | extension | mime_type  | identifier                                                                          |
+---------+------+-----------+------------+-------------------------------------------------------------------------------------+
| 1010289 | 2    | jpg       | image/jpeg | /dateien_mit_falscher_dateiendung/edit_redirect_target_WRONG_FILE_ENDING_IS_PNG.jpg |
+---------+------+-----------+------------+-------------------------------------------------------------------------------------+

Actions #12

Updated by Sybille Peters about 2 hours ago

  • Status changed from New to Closed
Actions #13

Updated by Andreas Kießling 24 minutes ago

Hi @Sybille Peters ,
confirmed --> the behavior in v12 is exactly the same with and without secure_downloads. Thanks for your investigation!

Actions

Also available in: Atom PDF