Project

General

Profile

Actions

Feature #15748

closed

Incorrect caching of locallang

Added by old_hkdennis2k about 18 years ago. Updated over 13 years ago.

Status:
Closed
Priority:
Should have
Category:
Communication
Target version:
-
Start date:
2006-03-01
Due date:
% Done:

0%

Estimated time:
PHP Version:
5
Tags:
Complexity:
Sprint Focus:

Description

locallang.xml are converted into native charset before cache into typo3temp, even the xml is stored in unicode.

And if the page is render in UTF-8 or any other charset.
It will do 2nd convertion form native charset to utf-8, and do the 3rd convertion to the renderCharset.

As a result.
It clause datalost if the default charset of the locale do not contains the character. Also, there is a huge performance hit when you using alot of localized label in the page.

Solution suggested:
All locallang labels should cache in RENDER CHARSET.
locallang*.php will be cached like an locallang*.xml (If the php does not contains logic)

Because it can print the label directly from cache. The csConv can skip loading the charset mapping files. It will also save some memory.

In my test,
read EXT:chc_forum/pi/locallang.php and convert all labels into UTF8.

Typo3 takes 550ms if not cached.
Typo3 takes less then 10ms if cached(I still do include the source php and md5 the $LOCALLANG for doing checksum).
Typo3 takes less then 2ms if I convert it into xml by extdeveval.

--
Easten language is a problem, they are usualy have 3 charset encoding in current use.
e.g.
ch(Simpified Chinese) is using GB2312 in typo3 by default.
But, some character only exists in GBK (a newer version) charset
The result clause the final display may not same as expected from XML.

Also, there is another question. Typo3 default hk(Tradinal Chinese) is using Big5.
But big5 is never a standardized by any offical organization at all.
I wonder which version of big5 typo3 is using.
FYI: http://my.opera.com/community/forums/topic.dml?id=104187
I suggest typo3 update it to BIG5-HKSCS, it is already widely supported.

PS.
If you ask me for patch, I need some time to do that.
I have to extract that logic from my local version, it is mixed with my others customization.

(issue imported from #M2731)


Files

bug_2731-v2.diff (22.8 KB) bug_2731-v2.diff Administrator Admin, 2008-02-04 16:55
Actions #1

Updated by Sebastian Kurfuerst about 18 years ago

Hi Martin,
I think this issue might be interesting for you, I hope it's ok I inform you about it this way.

@hkdennis2k: I suppose a patch is welcome, but I am no expert of charsets, so you might wait what Masi says.

Greets, Sebastian

Actions #2

Updated by Martin Kutschker about 18 years ago

I'll investigate it, but I don't think there is time to make it into 4.0. The l10n matters are complicated and I don't want to bust anything right now.

Actions

Also available in: Atom PDF