Bug #21609
closedProblem with german "Umlaute" in BE file list, no thumbs generated
100%
Description
Do the following:
On the left go to file list. Create a new folder named something about "schärfen-exif" or so, selct the folder, click on file upload icon, upload jpg files and when t3 reloads the file list no thumbnails are generated.
create a new folder without a german "Umlaut" and upload the same files already is ok.
(issue imported from #M12661)
Files
Updated by Chris topher over 14 years ago
Is it possible to create files with umlauts in the filename using the TYPO3 API?
Updated by Oliver Hader over 14 years ago
It is possible to use umlauts in the filelist. There's a setting in the install tool "UTF8filesystem" that allows to store UTF8 characters in the filesystem. I tested this issue and could not reproducte with folders names like "schärfen" or "Überprüfung" - Thumbnails rendered fine in my case.
Can you check, whether the files are stored correctly in the filesystem? If your filesystem converts umlauts to something else/strange, that might be the reason.
Besides that: Do you have PHP safe_mode enabled on your server?
Updated by Manfred Rutschmann over 14 years ago
UTF8 Filesystem is enabled and works. I created once this folder and looked on console, everything is fine. now i upload some files (01.jpg,02.jpg and so on) in this folder. No thumbnails are generated, the typo3temp folder becomes no new files. i looked in my error lo9gs but nothing found...
safe_mode is off.
the t3 system is not produvtive yet, can i test more or other things? my server uses gm 1.1.11. IM6 is also availible.
Manfred
Updated by Chris topher over 14 years ago
Do you have the same problem with IM6?
Updated by Manfred Rutschmann over 14 years ago
Test with im6:
on the first test i was after uploading the files when t3 refreshes the file list logged out from system... on the second test it's the same as the image processing with gm...
Updated by Dimitri Koenig over 14 years ago
the problem is found in t3lib/thumbs.php -> SC_t3lib_thumbs::wrapFileName
in typo3 4.3 this method uses escapeshellarg for security reasons. but because of a bug in php 5.2.6 this removes special chars like umlauts. more info on that: http://bugs.php.net/bug.php?id=44564
a temporary solution is to take the method out of typo3 4.2.x.
Updated by Jigal van Hemert over 14 years ago
The problem is also present in t3lib/class.t3lib_stdgraphic.php -> t3lib_stdGraphic::wrapFileName()
Furthermore in 4.2.12 wrapFileName() also uses escapeshellarg()...
Updated by Jigal van Hemert over 14 years ago
Just read:
http://bugs.php.net/bug.php?id=44945
Adding
setlocale(LC_CTYPE, 'UTF8', 'en_US.UTF-8');
to wrapFileName() made escapeshellarg() work...
Updated by Christian Kuhn over 14 years ago
@Jigal van Hemert, a hack like this works for us.
This problem exists in 5.2.6, especially the version that is delivered with debian lenny! So every unicode sign in a filename will be affected here, if php is runs as mod_php, the bug does not exist in cli mode ... and as this is no security issue debian won't port a fix to stable ....
Updated by Artis Lismanis over 14 years ago
I had the same issue when installing TYPO3 4.3.3 on my development machine which runs Debian Lenny.
I tried to tweak the code using setlocale(LC_CTYPE, 'UTF8', 'en_US.UTF-8'); directive as described above, but with no luck. Upgraded to PHP 5.3.2 and still was experiencing the same issue. After hours of Googling I was finally able to track the issue down to the /etc/init.d/apache2 file which sets its own environmental variables:
ENV="env -i LANG=C PATH=/usr/local/bin:/usr/bin:/bin"
I changed this to
ENV="env -i LANG=en_GB.UTF-8 PATH=/usr/local/bin:/usr/bin:/bin"
Restarted apache2 service and my issue was fixed.
Updated by Marc Bastian Heinrichs about 14 years ago
Same issue on RHEL 5.4.
Had to set HTTPD_LANG=en_US.UTF8 in /etc/sysconfig/httpd to fix this
Updated by Oliver Hader almost 14 years ago
I attached a patch file that allows one to define a locale setting if required.
It's difficult to use default settings here, since on Linux it's 'UTF-8' or 'en_US@utf-8' and on Windows it's somehow different... 'english_english@65001'...
Updated by Chris topher almost 14 years ago
What's the status here?
@ Oliver: Jigal posted a similar patch on Core List (no other patches in that thread till now). Could you have a look there?
Updated by Ivan Dharma Kartolo over 13 years ago
DAM doesn't create thumbnail of file, which has umlaut in filename. I tried this and works alright. It's TYPO3 4.4.4 and IM6.
Updated by Jigal van Hemert over 13 years ago
Committed to trunk rev: 9811
(waiting for RMs for okay on other branches)
Updated by Steffen Gebert over 13 years ago
Hi,
it would be great to have this option better documented. I can't see an entry in Pending Documentation, some more comments, in config_default.php would be great, as there's no starting point, where to search for.
I've tried several xx_XX.UTF-8 or .ISO8859-1 combinations, but none of them worked for thumbnail generations with umlauts.
Updated by Chris topher over 13 years ago
This is fixed in TYPO3 4.5.
Setting $TYPO3_CONF_VARS['SYS']['systemLocale'] = 'de_DE.utf8';
solves the issue in my case (using a linux server with German locales).
@ Steffen: Yes the explanation text in the Install Tool is not really helpful.
Adding something like the following would be good:
Set this option to a locale which is installed on your system, if thumbnails of image files with special characters in the file name are not generated.
This text would have helped me, but I don't know if that is the only usecase of this variable.
Updated by Jigal van Hemert over 13 years ago
It's not meant fore image related function only. Some PHP functions are locale aware where this is not immediately obvious: dirname(), basename(), escapeshellarg(), pathinfo(). For these functions we need to set a locale (not necessarily the BE or FE locale).
Can someone come up with a description which is understandable and not limited to image generation?
Updated by Oliver Hader about 13 years ago
- Target version changed from 1238 to 1281
Updated by Xavier Perseguers about 13 years ago
- Target version changed from 1281 to 4.6.0-beta1
Updated by Stefan Neufeind about 13 years ago
Ran into the exact same problem myself a while ago. Setting a proper locale fixed it. So I think this is nothing we can solve on the TYPO3-side (PHP-related "maybe"). Only thing left to do is imho improve the error-message here.
Updated by Xavier Perseguers about 13 years ago
@Stefan Exactly. Perhaps you could come up with a better description and that will be sufficient.
BTW, thank you very much for your tremendous review work!
Updated by Steffen Gebert about 13 years ago
This reminds me of #24807, which makes it easier to spot, which file is responsible for the problem (I guess, people then could at least think about the special characters in the file name..).
Updated by Steffen Gebert about 13 years ago
Okay, as #24807 is in, what to do now? Maybe the message there isn't really sufficient and we should just add an additional hint that, if the file was not found that it might be related to the locale?
Updated by Mr. Hudson about 13 years ago
Patch set 1 of change I866dd29c941652f4f47492e7b256ae5bf7fed832 has been pushed to the review server.
It is available at http://review.typo3.org/3714
Updated by Mr. Hudson about 13 years ago
Patch set 1 of change I30b355604a1052f452ab4c15ae57724e92472a1b has been pushed to the review server.
It is available at http://review.typo3.org/3715
Updated by Steffen Gebert about 13 years ago
- Status changed from Needs Feedback to Resolved
- % Done changed from 0 to 100
Applied in changeset 61bafcfab80e7fb837f53dfcf35990d4c696eca9.
Updated by Steffen Gebert about 13 years ago
- Category set to Image Cropping
- Target version changed from 4.6.0-beta1 to 4.5.4
Updated by Riccardo De Contardi almost 7 years ago
- Status changed from Resolved to Closed