Project

General

Profile

Actions

Bug #46118

closed

Epic #72333: Clean up charset conversion and language service

Overriding _LOCAL_LANG in TS broken for fluid templates

Added by Dmitriy Simakov over 11 years ago. Updated almost 8 years ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
Localization
Target version:
-
Start date:
2013-03-08
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
6.2
PHP Version:
5.3
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

I'm using news extension, and overriding several labels in TS via plugin.tx_news._LOCAL_LANG.ru.*
That worked in 4.6 with forceCharset, since there was a code in sysext/extbase/Classes/Utility/Localization.php:

// For labels coming from the TypoScript (database) the charset is assumed to be "forceCharset" and if that is not set, assumed to be that of the individual system languages
if (isset($GLOBALS['TYPO3_CONF_VARS']['BE']['forceCharset']) && strlen($GLOBALS['TYPO3_CONF_VARS']['BE']['forceCharset']) > 0) {
        self::$LOCAL_LANG_charset[$extensionName][$languageKey][$labelKey] = $GLOBALS['TYPO3_CONF_VARS']['BE']['forceCharset'];
}

I.e. encoding of _LOCAL_LANG labels set from TypoScript was always assumed as utf-8 (if forceCharset=utf-8)
However, that code disappeared in 4.7. And now this code has an effect:
self::$LOCAL_LANG_charset[$extensionName][$languageKey][$labelKey] = $GLOBALS['LANG']->csConvObj->charSetArray[$languageKey];

But csConvObj, which is t3lib_cs in t3lib/class.t3lib_cs.php, defines charset encoding for russian language as "windows-1251"
var $charSetArray = array(
        'ru' => 'windows-1251',

So, labels from TypoScript templates, which are in utf-8 (since BE encoding is utf-8) are encoded as if they had windows-1251 encoding in sysext/extbase/Classes/Utility/Localization.php:
$value = self::convertCharset($value, self::$LOCAL_LANG_charset[$extensionName][self::$languageKey][$key]);

I guess, it's a good time to change charset to utf-8 in t3lib_cs, for 'ru' at least.

Also, extensions not using extbase, like sr_freecap, are working ok with overriden _LOCAL_LANG labels

Actions #1

Updated by Ralf Merz over 10 years ago

Hi,

I use a TYPO3 v6.0.6 for a multi-language site with languages DE, EN, FR, PL, CS and ES. We do also overwrite e.g. the back-link label of tx_news via Typoscript.
Using FR it works fine: "Retour à l'aperçu" looks ok
But CS (CZ) fails: "Zpět na přehled" becomes " ZpÄ›t na pĹ™ehled "
PL fails, too: "Powrót do przeglądu" becomes " PowrĂłt do przeglÄ?du "

Maybe this is due to what Dmitry already mentioned in his description?
Can anybody confirm this?

Thanks a lot,
Ralf

Edit:
I´ve looked up the languages in TYPO3\CMS\Core\Charset\CharsetConverter:
'cs' => 'windows-1250',
'cz' => 'windows-1250',
'pl' => 'iso-8859-2',

Are those values correct? - I guess yes.

Edit 2:
If Typoscript does not work, you can use an XML (or XLIFF) file to override, e.g.:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['locallangXMLOverride']['EXT:news/Resources/Private/Language/locallang.xml'][]= 'EXT:tx_my_ext/Resources/Private/Language/locallang_news.xml';

Thanks to Xavier Persequers: http://xavier.perseguers.ch/tutoriels/typo3/articles/managing-localization-files.html

Actions #2

Updated by Xavier Perseguers over 10 years ago

  • Status changed from New to Needs Feedback
  • Target version set to next-patchlevel
  • Is Regression set to No

Trying to reproduce on TYPO3 6.2 with an Extbase-based extension of mine:

config.language= pl

plugin.tx_clcourses_picourses._LOCAL_LANG.pl {
    tx_clcourses_domain_model_course.number = Powrót do przeglądu
}

I have no problem, everything is properly overridden.

Current activity is on 6.2, 4.7 only get security fixes and 6.0 is out of maintenance.

Actions #3

Updated by Ralf Merz over 10 years ago

Hi Xavier,

ok, about the versions and maintenances... you´re right. I´ve not tried it in a newer version yet. I was just "hanging" on that single project with that issue.

Thank´s a lot for testing. I´ll keep in mind that it should work with a current version.

Regards,
Ralf

Actions #4

Updated by Xavier Perseguers over 10 years ago

Your problem may be related to a website not using UTF8 everywhere, as it was easily possible (and common) in TYPO3 4.x. Maybe this is simply related to that.

Actions #5

Updated by Alexander Opitz about 10 years ago

  • Status changed from Needs Feedback to Closed
  • Target version deleted (next-patchlevel)

No feedback within the last 90 days => closing this issue.

If you think that this is the wrong decision or experience this issue again, then please write to the mailing list typo3.teams.bugs with issue number and an explanation or open a new ticket and add a relation to this ticket number.

Actions #6

Updated by Thomas Schwarz over 9 years ago

The Problem is still remaining in 6.2.10.

I use this language settings:
config {
language = pl
locale_all = pl_PL.UTF-8
htmlTag_langKey = pl
}

I Insert this text in the TypoScript of my Extbase Extension:
_LOCAL_LANG {
pl {
product_list_nutritionalValue_title = <b>Zawartość & wartości odżywcze</b><br />100g zawiera składniki odżywcze zalecanej dziennej dawki (GDA) przeciętnej osoby dorosłej.
product_list_recipe_header = Pomysły na przepisy
}
}

The Output via Translate Viewhelper (f:translate) is:
ZawartoĹÄ & wartoĹci odĹźywcze 100g zawiera skĹadniki odĹźywcze zalecanej dziennej dawki (GDA) przeciÄtnej osoby dorosĹej.

and

PomysĹy na przepisy

The thext was converted from iso-8859-2 to utf-8 but the Backend is already utf-8.

Actions #7

Updated by Jigal van Hemert over 9 years ago

  • Status changed from Closed to New
  • TYPO3 Version changed from 4.7 to 6.2
Actions #8

Updated by Online Now! GmbH over 9 years ago

I experienced the same behavior when i tried to add a custom label to an extension ( Powermail 2.20 which is an Extbase extension ) via TypoScript. The russian translation of the label is not displayed correctly, but only if the language of the site is set tu russian ( config.language = ru ). Russian text in default language are working fine. Russian labels directly from the original extension language file are working as expected, as well. The translate view helper is used to output the text in a fluid template. I also tried all available charset converting options in install tool ( mbstring, iconv etc ). The site is using utf-8 and all database tables are using utf-8 as charset.

After i changed the entry in the class TYPO3\CMS\Core\Charset\CharsetConverter ( $charSetArray ) from windows-1251 to utf-8 the russian label set in TypoScript is displayed correctly.

I never experienced this behavior in the past with a pibase extensions.

Edit: The TYPO3 version i am using is 6.2.12.

Actions #9

Updated by Stephan Großberndt over 9 years ago

  • Subject changed from overriding _LOCAL_LANG in TS got broken in 4.7 for fluid templates to Overriding _LOCAL_LANG in TS broken for fluid templates
Actions #10

Updated by Anonymous over 9 years ago

Can confirm this issue for powermail 2.3.2. We're trying to override two labels for Czech:
  • validationerror_mandatory = Povinné pole
  • validationerror_validation.1 = Prosím vložte platnou emialovou adresu ()

For some reason only the first one is having trouble with special chars. When var_dumping self::$LOCAL_LANG_charset only the first one is present among others. Maybe this dot containing key is a Powermail speciality, maybe it's another core bug.

Actions #11

Updated by Sinisa Mitrovic over 9 years ago

Is this fixed in 6.2.14?

Actions #12

Updated by Sven Juergens over 9 years ago

same Problem here

i tried to set

plugin.xxx_local_lang.hu.downloadPDF = Letöltés PDF fájlként

Result was this--> LetĂśltĂŠs PDF fĂĄjlkĂŠnt

as Workaround

[globalVar = GP:L= idOfLanguage]
plugin.xxx_local_lang.default.downloadPDF = Letöltés PDF fájlként
[global]

Actions #13

Updated by Rafal Brzeski over 9 years ago

I got the same issue during translation tx_news, any progress with that ? (6.2.14)

Actions #14

Updated by Hannes Gesmann about 9 years ago

Sven Juergens wrote:

same Problem here

i tried to set

plugin.xxx_local_lang.hu.downloadPDF = Letöltés PDF fájlként

Result was this--> LetĂśltĂŠs PDF fĂĄjlkĂŠnt

as Workaround

[globalVar = GP:L= idOfLanguage]
plugin.xxx_local_lang.default.downloadPDF = Letöltés PDF fájlként
[global]

Having the same issue with _LOCAL_LANG for sr_freecap (cs, ru, zh, cn) in TYPO3 6.2.14, I just wanted to confirm Sven Juergens' workaround.

Actions #15

Updated by Willi Martens about 9 years ago

The problem is still remaining in 6.2.15. Sven Juergens' workaround works.

plugin.tx_news._LOCAL_LANG.ja.more-link = 詳細を見る
Actions #16

Updated by Riccardo De Contardi over 8 years ago

Still present on latest master, I guess. See typo3/sysext/core/Classes/Charset/CharsetConverter.php

Actions #17

Updated by Riccardo De Contardi about 8 years ago

  • Parent task set to #72333
Actions #18

Updated by Riccardo De Contardi about 8 years ago

  • Status changed from New to Needs Feedback

I tried to reproduce the problem on 8.5.0-dev with my MAMP environment, and I was not able to reproduce it.

My TS Setup configuration

config.defaultGetVars.L = 0
config.linkVars = L(0-1)
config.uniqueLinkVars = 1
config.sys_language_overlay = 0
config.sys_language_mode = content_fallback
config.language = it
config.locale_all = it_IT.utf8
config.htmlTag_langKey = it-IT
config.sys_language_uid = 0
config.htmlTag_langKey = it

[globalVar = GP:L = 1]
config.language = cs
config.locale_all = cs_CS
config.htmlTag_langKey = cs-CS
config.sys_language_uid = 1
config.htmlTag_langKey = cs
[global]

2) installed the "news" extension
2) downloaded the "Czech" language pack, also for the "news" extension

3) created 2 pages for news (list+detail) in both Italian and Czech
4) I used the "translate" option to create the translated CE
5) I created a couple of news items, in both Italian and Czech

Result:
- the "more" link tranlated to "Čtěte více"
- the "back" link translated to Zpět

- I even tried to change the "More" link via typoscript

plugin.tx_news._LOCAL_LANG.cs.more-link = úůýžáčďéěíňóřšť

and the result was: úůýžáčďéěíňóřšť

Do you think I should perform different tests? Thank you!

P.s. I will try soon with 7.6 too

Actions #19

Updated by Riccardo De Contardi about 8 years ago

I repeated the same test of my comment https://forge.typo3.org/issues/46118#note-18 with TYPO3 7.6.12 (XAMPP Windows environment) with the same result.

Actions #20

Updated by Riccardo De Contardi almost 8 years ago

  • Status changed from Needs Feedback to Closed

No feedback since 90 days > Closing.

If you think that this is the wrong decision or experience the issue again, or have more information about how to reproduce it on the latest TYPO3 versions, please reopen it or open a new issue with a reference to this one. Thank you.

Actions

Also available in: Atom PDF