Bug #55721

wrong filenames when UTF8filesystem is set and 'systemLocale' is empty

Added by Tymoteusz Motylewski almost 6 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
-
Target version:
-
Start date:
2014-02-06
Due date:
% Done:

100%

TYPO3 Version:
6.2
PHP Version:
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

While checking out issue http://forge.typo3.org/issues/32683 on 6.2 I stumbled upon the same issue.
I found out that if you have UTF8Filesystem set to 1, but $GLOBALS['TYPO3_CONF_VARS']['SYS']['systemLocale'] is empty, then TYPO3 generate wrong file names, resulting e.g. in thumbnails not being generated in the backend. The problem disappear if I set '['SYS']['systemLocale']' to e.g. 'C.UTF-8' or 'en_US.UTF-8' or similar.

But the issue is not limited to thumbnails ony. It also occurs in many other places. There are plenty of related issues in Forge.

The common pattern all over TYPO3 is sth like this:

    if ($GLOBALS['TYPO3_CONF_VARS']['SYS']['UTF8filesystem']) {
        $currentLocale = setlocale(LC_CTYPE, 0);
        setlocale(LC_CTYPE, $GLOBALS['TYPO3_CONF_VARS']['SYS']['systemLocale']);
    }
//do sth here e.g. basename, escapeshellarg, dirname, pathinfo
    $escapedInputName = escapeshellarg($inputName);
    if ($GLOBALS['TYPO3_CONF_VARS']['SYS']['UTF8filesystem']) {
        setlocale(LC_CTYPE, $currentLocale);
    }

Btw, there is already a ticket to refactor this and put the logic in the central place. http://forge.typo3.org/issues/39928

I see several solutions here:
1. Use fallback when setting locale.
Instead of
setlocale(LC_CTYPE, $currentLocale);
setlocale(LC_CTYPE, $currentLocale, 'C.UTF-8', 'en_US.utf8');

2. Set default value 'C.UTF-8' to $GLOBALS['TYPO3_CONF_VARS']['SYS']['systemLocale'], so it should be never empty.
3. Add install tool warning when systemLocale is not set
4. BTW, why can not we set setlocale() globally in bootstrap instead of setting it for each function separately?


Related issues

Related to TYPO3 Core - Task #39928: Central function to correctly (locale-aware) escape filenames and shell-args Closed 2012-08-18
Related to TYPO3 Core - Bug #32683: showpic.php does not show pics with utf8 in filenames Closed 2011-12-19

Associated revisions

Revision df2ebf28 (diff)
Added by Zbigniew Jacko almost 6 years ago

[TASK] Install Tool checks for system locale

Add two new functions to check if 'systemLocale'
setting is correct and if configured 'systemLocale'
supports UTF-8 when 'UTF8filesystem' is set.

Resolves: #55721
Releases: 6.2
Change-Id: Id9edfc896a87d5025a5170690a6b7d6eb25950e6
Reviewed-on: https://review.typo3.org/27510
Reviewed-by: Markus Klein
Reviewed-by: Tymoteusz Motylewski
Tested-by: Tymoteusz Motylewski
Reviewed-by: Jigal van Hemert
Tested-by: Jigal van Hemert

History

#1 Updated by Markus Klein almost 6 years ago

For 6.2 I'd go for 2 and 3 (in case somebody set it to empty again)

4: good question, I'd like to know as well.

#2 Updated by Jigal van Hemert almost 6 years ago

1. Which locale as fallback? There are about a zillion locales with different names on different OSs. There no reliable, cross platform way to detect available locales.

2. Same story: which locale as default?

3. Good idea to add a check, but also check if the locale can in fact be used (= is available on the system)

4. In frontend context the locale for LC_TYPE is set according to the TS value in config.locale_all. This is not necessarily one that can be used for file system operations that need a locale. That's the reason for the locale swapping.

#3 Updated by Gerrit Code Review almost 6 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27510

#4 Updated by Gerrit Code Review almost 6 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27510

#5 Updated by Gerrit Code Review almost 6 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27510

#6 Updated by Gerrit Code Review almost 6 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27510

#7 Updated by Gerrit Code Review almost 6 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27510

#8 Updated by Zbigniew Jacko almost 6 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#9 Updated by Benni Mack about 1 year ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF