Bug #63979
closed
FAL fails with "special characters" and whitespaces in filenames
0%
Description
This issue occurs for ages now and still exists in TYPO3 6.2 LTS
Steps to reproduce
Upload a file into filelist with some "special" character in it - for example i used this here:
http://commons.wikimedia.org/wiki/File:Pollença_-_Ma-2210_-_Cap_de_Formentor_-_Ephedra_fragilis_05_ies.jpg
the ç is not allowed in URL context and needs to be escaped - this is not done and therefore the thumbnail is not shown in the filelist
same goes for the relation, if you insert this in a content element, the thumbnail is not shown either
this also applys to the frontend
This behaves similar if you use a space-Character in the filename - for example "foo bar.jpg", the space is not allowed in URL context and needs to be escaped
In this case, the thumbnail is shown in the filelist
It is not shown in the relation (at the content element)
and it is not shown in frontend
also this applys, if a folder contains any of those characters
Updated by Mathias Schreiber over 9 years ago
- Category set to File Abstraction Layer (FAL)
- Status changed from New to Accepted
Updated by Frans Saris over 9 years ago
- Status changed from Accepted to Needs Feedback
Hi Sigfried,
because TYPO3 isn't always responsible for putting the files in the filestorage we can not be 100% sure all files have names that are URL prove.
Thats why we now allow these files also while uploading files in the backend. You can disable this by setting [SYS][UTF8filesystem] = 0 in install tool.
When disabled the mentioned special chars (ç) will get replaced (c).
I just tested with your test file and for me the thumbnail is shown in BE. Are you on latest version of 6.2 because there were some fixed in the first releases (that allowed spaces in filename/folder when uploading in BE).
If you use the correct API the images will show correct and all urls are rawurlencoded.
Could be that some typoscript config give you troubles with rendering the images in frontend.
gr. Frans
Updated by Sigfried Arnold over 9 years ago
your first statement "we can not be 100% sure all files have names that are URL prove." is a bit odd - it is a basic rule in software developement: never trust any input at all, anything should be masked/escaped - so no matter what filesystem you use - characters that are not allowed in url context need to be urlencoded
i used a plain 6.2.9 installation to verify this issue and it exists - the files were not uploaded through the backend, the were put there via ssh/sftp
if you use the upload function in the backend, those problematic characters are not escaped either, they get simple replaced to "safe" characters before they are put in the filesystem - for example a space simply gets replaced by an underscore - thats why it works - also the "ç" is replaced with a "c" like you mentioned.
since TYPO3 does not know how the files where put in the file system, this part should be considered "broken" since there is no proper escaping/encoding done, where it is actually needed
btw: the default value for ['SYS']['UTF8filesystem'] is already 0 - so you have to enable this feature by setting it to 1 - since almost any filesystem nowadays is UTF8, the default behaviour should be the other way around.
Updated by Frans Saris over 9 years ago
Sigfried Arnold wrote:
your first statement "we can not be 100% sure all files have names that are URL prove." is a bit odd - it is a basic rule in software developement: never trust any input at all, anything should be masked/escaped - so no matter what filesystem you use - characters that are not allowed in url context need to be urlencoded
My statement isn't odd, only wanted to point out that we can not assure that there are no files present with non URL prove characters in there names.
The filenames are properly excaped by the core if you use the correct API. Could be that there are still some parts of the core that do not handle the encoding correct, we should try to find these locations. What do you use to render the images/file urls in frontend. Fluid, typoscript with or without [FE][activateContentAdapter] enabled?
i used a plain 6.2.9 installation to verify this issue and it exists - the files were not uploaded through the backend, the were put there via ssh/sftp
if you use the upload function in the backend, those problematic characters are not escaped either, they get simple replaced to "safe" characters before they are put in the filesystem - for example a space simply gets replaced by an underscore - thats why it works - also the "ç" is replaced with a "c" like you mentioned.
since TYPO3 does not know how the files where put in the file system, this part should be considered "broken" since there is no proper escaping/encoding done, where it is actually needed
For me the thumb is shown if I put http://upload.wikimedia.org/wikipedia/commons/f/f0/Pollen%C3%A7a_-_Ma-2210_-_Cap_de_Formentor_-_Ephedra_fragilis_05_ies.jpg in my fileadmin folder. Could it be that you run into a UTF8 difference problem? See https://forge.typo3.org/issues/57695
btw: the default value for ['SYS']['UTF8filesystem'] is already 0 - so you have to enable this feature by setting it to 1 - since almost any filesystem nowadays is UTF8, the default behaviour should be the other way around.
If you set ['SYS']['UTF8filesystem'] = 1 then the ç isn't replaced on upload in BE. Maybe it would be better to set this to 1 by default. Not sure if we can detect (by code) if current file system supports UTF8.
gr. Frans
Updated by Christoph RUnkel about 9 years ago
- typo3/sysext/core/classes/utility/file/basicfileutility.php
const UNSAFE_FILENAME_CHARACTER_EXPRESSION = '\\x00-\\x2F\\/\\x3A-\\x3F\\x5B-\\x60\\x7B-\\xBF';
this function replaces spaces no matter if UTF8 or not - we can leave \\x20 out - or even better make that constant configurable in the install tool.
Regards
Christoph Runkel
Updated by Alexander Opitz almost 9 years ago
Hi,
does the problem still exists within newer versions of TYPO3 CMS (6.2.14)?
Updated by Alexander Opitz over 8 years ago
- Status changed from Needs Feedback to Closed
No feedback within the last 90 days => closing this issue.
If you think that this is the wrong decision or experience this issue again, then please write to the mailing list typo3.teams.bugs with issue number and an explanation or open a new ticket and add a relation to this ticket number.