TYPO3 Core

your first statement "we can not be 100% sure all files have names that are URL prove." is a bit odd - it is a basic rule in software developement: never trust any input at all, anything should be masked/escaped - so no matter what filesystem you use - characters that are not allowed in url context need to be urlencoded

i used a plain 6.2.9 installation to verify this issue and it exists - the files were not uploaded through the backend, the were put there via ssh/sftp

if you use the upload function in the backend, those problematic characters are not escaped either, they get simple replaced to "safe" characters before they are put in the filesystem - for example a space simply gets replaced by an underscore - thats why it works - also the "ç" is replaced with a "c" like you mentioned.

since TYPO3 does not know how the files where put in the file system, this part should be considered "broken" since there is no proper escaping/encoding done, where it is actually needed

btw: the default value for ['SYS']['UTF8filesystem'] is already 0 - so you have to enable this feature by setting it to 1 - since almost any filesystem nowadays is UTF8, the default behaviour should be the other way around.

Sigfried Arnold wrote:

your first statement "we can not be 100% sure all files have names that are URL prove." is a bit odd - it is a basic rule in software developement: never trust any input at all, anything should be masked/escaped - so no matter what filesystem you use - characters that are not allowed in url context need to be urlencoded

My statement isn't odd, only wanted to point out that we can not assure that there are no files present with non URL prove characters in there names.
The filenames are properly excaped by the core if you use the correct API. Could be that there are still some parts of the core that do not handle the encoding correct, we should try to find these locations. What do you use to render the images/file urls in frontend. Fluid, typoscript with or without [FE][activateContentAdapter] enabled?

i used a plain 6.2.9 installation to verify this issue and it exists - the files were not uploaded through the backend, the were put there via ssh/sftp

if you use the upload function in the backend, those problematic characters are not escaped either, they get simple replaced to "safe" characters before they are put in the filesystem - for example a space simply gets replaced by an underscore - thats why it works - also the "ç" is replaced with a "c" like you mentioned.

since TYPO3 does not know how the files where put in the file system, this part should be considered "broken" since there is no proper escaping/encoding done, where it is actually needed

For me the thumb is shown if I put http://upload.wikimedia.org/wikipedia/commons/f/f0/Pollen%C3%A7a_-_Ma-2210_-_Cap_de_Formentor_-_Ephedra_fragilis_05_ies.jpg in my fileadmin folder. Could it be that you run into a UTF8 difference problem? See https://forge.typo3.org/issues/57695

btw: the default value for ['SYS']['UTF8filesystem'] is already 0 - so you have to enable this feature by setting it to 1 - since almost any filesystem nowadays is UTF8, the default behaviour should be the other way around.

If you set ['SYS']['UTF8filesystem'] = 1 then the ç isn't replaced on upload in BE. Maybe it would be better to set this to 1 by default. Not sure if we can detect (by code) if current file system supports UTF8.

gr. Frans

1. typo3/sysext/core/classes/utility/file/basicfileutility.php
   const UNSAFE_FILENAME_CHARACTER_EXPRESSION = '\\x00-\\x2F\\/\\x3A-\\x3F\\x5B-\\x60\\x7B-\\xBF';

this function replaces spaces no matter if UTF8 or not - we can leave \\x20 out - or even better make that constant configurable in the install tool.

Regards

Christoph Runkel

Hi,

does the problem still exists within newer versions of TYPO3 CMS (6.2.14)?