TYPO3 Core

Is it possible to create files with umlauts in the filename using the TYPO3 API?

It is possible to use umlauts in the filelist. There's a setting in the install tool "UTF8filesystem" that allows to store UTF8 characters in the filesystem. I tested this issue and could not reproducte with folders names like "schärfen" or "Überprüfung" - Thumbnails rendered fine in my case.

Can you check, whether the files are stored correctly in the filesystem? If your filesystem converts umlauts to something else/strange, that might be the reason.

Besides that: Do you have PHP safe_mode enabled on your server?

UTF8 Filesystem is enabled and works. I created once this folder and looked on console, everything is fine. now i upload some files (01.jpg,02.jpg and so on) in this folder. No thumbnails are generated, the typo3temp folder becomes no new files. i looked in my error lo9gs but nothing found...

safe_mode is off.

the t3 system is not produvtive yet, can i test more or other things? my server uses gm 1.1.11. IM6 is also availible.

Manfred

console screen is Bild 7.png...

Do you have the same problem with IM6?

Test with im6:

on the first test i was after uploading the files when t3 refreshes the file list logged out from system... on the second test it's the same as the image processing with gm...

the problem is found in t3lib/thumbs.php -> SC_t3lib_thumbs::wrapFileName

in typo3 4.3 this method uses escapeshellarg for security reasons. but because of a bug in php 5.2.6 this removes special chars like umlauts. more info on that: http://bugs.php.net/bug.php?id=44564

a temporary solution is to take the method out of typo3 4.2.x.

The problem is also present in t3lib/class.t3lib_stdgraphic.php -> t3lib_stdGraphic::wrapFileName()

Furthermore in 4.2.12 wrapFileName() also uses escapeshellarg()...

Just read:
http://bugs.php.net/bug.php?id=44945

Adding
setlocale(LC_CTYPE, 'UTF8', 'en_US.UTF-8');
to wrapFileName() made escapeshellarg() work...

@Jigal van Hemert, a hack like this works for us.

This problem exists in 5.2.6, especially the version that is delivered with debian lenny! So every unicode sign in a filename will be affected here, if php is runs as mod_php, the bug does not exist in cli mode ... and as this is no security issue debian won't port a fix to stable ....

I had the same issue when installing TYPO3 4.3.3 on my development machine which runs Debian Lenny.

I tried to tweak the code using setlocale(LC_CTYPE, 'UTF8', 'en_US.UTF-8'); directive as described above, but with no luck. Upgraded to PHP 5.3.2 and still was experiencing the same issue. After hours of Googling I was finally able to track the issue down to the /etc/init.d/apache2 file which sets its own environmental variables:

ENV="env -i LANG=C PATH=/usr/local/bin:/usr/bin:/bin"

I changed this to

ENV="env -i LANG=en_GB.UTF-8 PATH=/usr/local/bin:/usr/bin:/bin"

Restarted apache2 service and my issue was fixed.

Same issue on RHEL 5.4.
Had to set HTTPD_LANG=en_US.UTF8 in /etc/sysconfig/httpd to fix this

I attached a patch file that allows one to define a locale setting if required.
It's difficult to use default settings here, since on Linux it's 'UTF-8' or 'en_US@utf-8' and on Windows it's somehow different... 'english_english@65001'...

What's the status here?

@ Oliver: Jigal posted a similar patch on Core List (no other patches in that thread till now). Could you have a look there?

DAM doesn't create thumbnail of file, which has umlaut in filename. I tried this and works alright. It's TYPO3 4.4.4 and IM6.

Committed to trunk rev: 9811
(waiting for RMs for okay on other branches)

Hi,

it would be great to have this option better documented. I can't see an entry in Pending Documentation, some more comments, in config_default.php would be great, as there's no starting point, where to search for.

I've tried several xx_XX.UTF-8 or .ISO8859-1 combinations, but none of them worked for thumbnail generations with umlauts.

This is fixed in TYPO3 4.5.
Setting $TYPO3_CONF_VARS['SYS']['systemLocale'] = 'de_DE.utf8';
solves the issue in my case (using a linux server with German locales).

@ Steffen: Yes the explanation text in the Install Tool is not really helpful.
Adding something like the following would be good:

Set this option to a locale which is installed on your system, if thumbnails of image files with special characters in the file name are not generated.

It's not meant fore image related function only. Some PHP functions are locale aware where this is not immediately obvious: dirname(), basename(), escapeshellarg(), pathinfo(). For these functions we need to set a locale (not necessarily the BE or FE locale).
Can someone come up with a description which is understandable and not limited to image generation?

Ran into the exact same problem myself a while ago. Setting a proper locale fixed it. So I think this is nothing we can solve on the TYPO3-side (PHP-related "maybe"). Only thing left to do is imho improve the error-message here.

@Stefan Exactly. Perhaps you could come up with a better description and that will be sufficient.

BTW, thank you very much for your tremendous review work!

This reminds me of #24807, which makes it easier to spot, which file is responsible for the problem (I guess, people then could at least think about the special characters in the file name..).

Okay, as #24807 is in, what to do now? Maybe the message there isn't really sufficient and we should just add an additional hint that, if the file was not found that it might be related to the locale?

Patch set 1 of change I866dd29c941652f4f47492e7b256ae5bf7fed832 has been pushed to the review server.
It is available at http://review.typo3.org/3714

Patch set 1 of change I30b355604a1052f452ab4c15ae57724e92472a1b has been pushed to the review server.
It is available at http://review.typo3.org/3715