TYPO3 Core

Can't confirm this for Typo3 4.1 RC2 with PHP 5.2. All umlauts are rendered correct. I'm using different true type fonts from a windows xp machine on a LAMP system. Maybe there is another reason for this behavior on your machine?

I changed the font to one I used for other websites (running php4), but it didn't resolve the problem. After checking all possible reasons in the configuration, I tried to switch back to php4 which was the only way to solve the problem. It's a big commercial hoster running FreeBSD. Could you think of a php-misconfiguration? I try to get the php4 and php5 configurations as soon as I can connect to servers again.

I already had this problem together with an incorrect working horizontal offset for GMENU!
My system (SuSE 10.0 64Bit) with php5 (bundled gdlib).
And this was the problem, the bundled gdlib in php (even in the newest version)!

Workaround: Compiling php form the source an even gdlib and freetype.

Now it works!!!

Hint (my configure-string):

./configure --quiet --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/php --sysconfdir=/etc --libdir=/usr/share --includedir=/usr/include --mandir=/usr/share/man --with-_lib=lib64 --with-apxs2=/usr/sbin/apxs2-prefork --disable-cli --disable-debug --with-config-file-path=/etc --enable-safe-mode --with-exec-dir=/usr/lib64/php/bin --enable-sigchild --enable-magic-quotes --enable-bcmath --with-bz2--enable-calendar --disable-ctype --with-curl=/usr/local --enable-dba --enable-exif --enable-ftp --with-gettext --with-gmp --with-mhash --with-iconv --enable-mbstring --with-mcrypt --with-mime-magic --enable-sockets --enable-memory-limit --enable-inline-optimization --enable-mbstring --enable-shmop --enable-wddx --enable-yp --with-libxml --with-regex=php --with-config-file-path=/etc/php5 --with-xsl --with-ttf --with-freetype-dir --with-cpdflib --with-mysql=/usr/lib/mysql --without-mysqli --without-pear --with-zlib=/usr --with-gd=/usr/local/ x86_64-suse-linux

It will be a little bit different for 32Bit-systems!

I have the problem solved with a HACK in class.t3lib_stdgraphic.php.

In the function ImageTTFTextWrapper(), I convert each char > 127 to the html "&#<ord>" representation.

Line 1032:
if (is_readable($fontFile)) {
// Umlaute HACK
for ($in_text = $strCfg['str'],
$out_text = "",
$tp = 0,
$tl = strlen($in_text);
$tp < $tl;
$tp++) {
if (ord($in_text[$tp]) > 127) {
$out_text .= "&#" . ord($in_text[$tp]) . ";";
} else {
$out_text .= $in_text[$tp];
}
}
// Render part:
ImageTTFText($im, t3lib_div::freetypeDpiComp($sF*$strCfg['fontSize']), $angle, $x, $y, $colorIndex, $fontFile, $out_text);

Dirty hack, I know but the bug is in ImageTTFText(), so hack it or wait for a fixed PHP version (from SuSE).

If you want to apply this 'dirty patch' also to 'manual' produced ttf gifs, you'll have to include it in the function ImageTTFBBoxWrapper() too!!

For me it worx fine. thxal! :-)

But probably there should be thought about an install tool setting, because this bugs resides within all default php5 installations on suse linux since over a year, AFAIR. ...

There has to be one more place where this hack has to be integrated, as it does not work with graphical headers, except I write their unicode entity to the input field.
Anoying!

Here is a dirty but working solution:

First of all: the codesnippet above only works in latin charsets (at least at iso-8859-1). But it won't work, if your Backend and Database is set to utf-8.
I'm also astonished, that it actually works, 'cause your delivering ords (ordinal numbers) which is different to unicode numbers. (and the unicode numbers are the supported ones from php5-gd).

What we need is a function which delivers us the correct numbers depending on the charset we use in database.

This can be done with this one:
< ? php
function ricEncode($content) {
$convmap = array(0x80, 0xff, 0, 0xff);
return mb_encode_numericentity($content,$convmap, "utf-8");
}
? >

I call it ricEncode 'cause maybe there's already a function encode ...

Please make sure that the string "utf-8" is set to the encoding of backend and database!! For me it's utf-8.
But you may detect encoding of the string with the detection function: like this:

mb_detect_encoding($content, "auto");

The function ricEncode has to be put to class.t3lib_stdgraphic.php somewhere around line 950 just before function ImageTTFBBoxWrapper(...)

After that we have to change TWO functions in this class:

f. ImageTTFBBoxWrapper()
f. ImageTTFTextWrapper ()

Within both functions go to: if (is_readable($fontFile)) {
and insert just after the {
$out_text = $this->ricEncode($strCfg['str']);

in f. ImageTTFBBoxWrapper() you have to change now the next line (beginning with $calc) to:

$calc = ImageTTFBBox(t3lib_div::freetypeDpiComp($sF*$strCfg['fontSize']), $angle, $fontFile, $out_text);

in f. ImageTTFTextWrapper () next line beginning with ImageTTFText

you have to change to:

ImageTTFText($im, t3lib_div::freetypeDpiComp($sF*$strCfg['fontSize']), $angle, $x, $y, $colorIndex, $fontFile, $out_text);

What you save with this solution?

Recompiling php5 on running suselinux 10.1 ++ liveservers!! ;)

Working example:

http://www.klosterbraeu.com/ (startpage right side: 'Gästebuch')

See also: http://lists.netfielders.de/pipermail/typo3-german/2007-July/035700.html

@ Clemens Riccabona

works fine. !! well done.. . but where is the hook to implement that for tv fce´s header ?
can´t find it.

any suggestions ?

my fault... ;-) wrong ttf in FCE ... *sorry

What worked for me a second ago: I used FontForge to re-encode the font from Unicode MBP to Full Unicode...

Thanks Clemens, works fine for me on Suse 10.x

Thanks Clemens,

I wrote little extension to solve this bug without the need to edit the core files.It is called cr_bugfix_5078 and should be availible in TER soon.

Please write some feedback if the bugfix-extension works.

Here you can find a working example
http://www.dokpro.eu/leistungen/uebersetzungsmanagement/uebersetzung/

Hi Nico!

I had the time now, for testing your ext. But I needed to bugfix your bugfix ;)

You missunderstood my suggestion above.
If you want to auto-detect the encoding, you need first to detect, and then to remap with numeric entity.

e.g.: I wrote:

< ? php
function ricEncode($content) {
$convmap = array(0x80, 0xff, 0, 0xff);
return mb_encode_numericentity($content,$convmap, "utf-8");
}
? >

your wrote in your ext:
< ? php
function ricEncode($content) {
$convmap = array(0x80, 0xff, 0, 0xff);
return mb_encode_numericentity($content,$convmap, "auto");
}
? >

But "auto" is no encoding AFAIK.

so you need to do it like that:

< ? php
function ricEncode($content) {
$convmap = array(0x80, 0xff, 0, 0xff);
return mb_encode_numericentity($content,$convmap, mb_detect_encoding($content, "auto"));
}
? >

as mb_detect_encoding() returns the encoding string.

I thought, that this would have been clear. ;)

greets, clemens

Hi Clemens.

I tried to update the extension with the mb_detect_encoding function.
But with the this function the Umlauts won't get rendered anymore.

I consulted the function help on php.net and the charset parameter for mb_encode_numericentity and mb_detect_encoding is optional.
I think that this is the reason why it ignores the parameter if it is set to "auto".

I deleted the "auto" parameter for the mb_encode_numericentity function and everything works as it should. I Uploaded a new version to the TER wich contains this changes.

I'm not quite sure if autodetection is done when no charset parameter is given since the manual is a bit short on this topic.

Maybe someone can enlighten me with his knowlege upon this topic.

greets Nico

What worked fine for me:

function ricEncode($content) {
$enc = mb_detect_encoding($content, "auto");
$convmap = array(0x80, 0xff, 0, 0xff);
return mb_encode_numericentity($content,$convmap, $enc);
}

But remember, in my case, everything is utf-8 and not latin!

I Have Change the Files,
but it don't work.

PHP Version: 5.1.2
I have change the file with the last post from Clemens, but the UMLAUTES don't show with GIFBUILDER...

my localconf:
///
$TYPO3_CONF_VARS['SYS']['compat_version'] = '4.1';
$TYPO3_CONF_VARS['BE']['disable_exec_function'] = '0';
$TYPO3_CONF_VARS['GFX']["im"] = '1';
$TYPO3_CONF_VARS['GFX']["im_path"] = '/usr/bin/';
$TYPO3_CONF_VARS['GFX']["im_path_lzw"] = '/usr/bin/';
$TYPO3_CONF_VARS['GFX']['TTFdpi'] = '96';
$TYPO3_CONF_VARS['GFX']['im_combine_filename'] = 'composite';
$TYPO3_CONF_VARS['GFX']['im_version_5'] = 'im5';
$TYPO3_CONF_VARS['GFX']["gdlib_png"] = '1';
$TYPO3_CONF_VARS['GFX']['thumbnails_png'] = '2';
$TYPO3_CONF_VARS['GFX']['png_truecolor'] = '1';
$TYPO3_CONF_VARS['GFX']['gdlib_png'] = '1';
$TYPO3_CONF_VARS['GFX']['gdlib_2'] = '1';
$TYPO3_CONF_VARS['GFX']['jpg_quality'] = '100';
$TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
///

IM Version 6.2.5

sorry my english isn't good :(

Try to change everything to the original code, and after that, try this extension:

cr_bugfix_5078

which is available in TER and seems to work for lots of people who experience this problem (related to a thread on typo3.net)

kind regards, clemens

btw: If you have im version 6 you probably should also set this to im_version_5 property (not really straight that setting, isn't it).
Even look, if you really have the gdlib 1, i think it could be version 2. ... (phpinfo in installtool for lookup).
And another thing: jpg_quality 100? hough.

i have made a backup and have do an new T3 installtion...

Install your extension... and it works perfectly (but not on UTF-8 only on ISO)
i'm have installed this versions off GM and IM from:
http://typo3.sunsite.dk/software/linux/

jpeg_quality 100 shows on this server better than 12...

@Clemens Riccabona

In the following szenario the described solution solved the Problem:

1. Step: install the cr_bugfix_5078,
2. Step: im_version_5 property set to im5
3. Step (very important) the varible nicetext must set to zero.

But I can't understand the dependencies to Step 3. If I set nicetext to 1 the menu text won't be render. Please could you explain this?

mkind regards, mario

@Clemens Riccabona

I tested this extension...
but the gifbuilder rendered the Font in "Ãber Uns"...

I have change this line in the "class.ux_tslib_gifbuilder.php"
from: return mb_encode_numericentity($content,$convmap);
to: return mb_encode_numericentity($content,$convmap,mb_detect_encoding($content, "auto"));

and it works

more information: http://www.typo3.net/forum/list/list_post//48329/?page=5#pid298887

greetz
Mathis

What about that issue? Umlaute in GIFBUILDER with TTF fonts seem to be working with TYPO3 4.2/4.3 and PHP5+. With OTF fonts umlaute are not displayed.

@ Lorenz: Thanks for the note. There were no further reports of problems here for two years. Same for the linked topic from typo3.net. Seems like this problem is solved, right?

Just the OTF font problem remains open. We should deal with that seperately in #4603.

Just for Info: The Problem still exists (TYPO3 4.4.4) but cr_bugfix_5078 and cr_bugfix_5078_add work both for me - both with nicetext enabled.

Why is this not fixed in the Core?

It seems to appear with some fonts when PHP is compiled with --enable-gd-jis-conv .
In my phpinfo() output in the 'gd' section I see:
JIS-mapped Japanese Font Support enabled
and with the commercial font ChaparralPro-Semibold.otf (versie OTF 1.008;PS 001.000;Core 1.0.27;makeotf.lib1.3.1) I can see the problem too. The suggested code to use mb_encode_numericentity() does not work with this font.

Can anybody confirm this?

References:
[1] http://nl.php.net/manual/en/function.imagettftext.php (comment by Endeer
12-Jun-2010 11:22)
[2] https://bugzilla.novell.com/show_bug.cgi?id=236680

I can confirm the bug with JIS-mapped Japanese Font Support enabled with ttf-fonts
On my local machine all works fine with the following php setup.

GD Support enabled
GD Version 2.0 or higher
FreeType Support enabled
FreeType Linkage with freetype
FreeType Version 2.3.5
T1Lib Support enabled
GIF Read Support enabled
GIF Create Support enabled
JPG Support enabled
PNG Support enabled
WBMP Support enabled

On the live machine it only works with "cr_bugfix_5078" and "mk_bugfix_5078_add" installed with this setup.

GD Support enabled
GD Version bundled (2.0.34 compatible)
FreeType Support enabled
FreeType Linkage with freetype
FreeType Version 2.3.9
T1Lib Support enabled
GIF Read Support enabled
GIF Create Support enabled
JPG Support enabled
PNG Support enabled
WBMP Support enabled
XBM Support enabled
JIS-mapped Japanese Font Support enabled

So maybe the problem is really "JIS-mapped Japanese Font Support"

Looks like the problem is still here. Does anybody detected exactly the reason of the bug with encoding when we are using otf fonts ?

I've just migrated from one server to an another with similar configuration, TYPO3 is exactly the same, and I've bugs in programmatically generated banners with accents.

If somebody has some ideas, please post.

Thanks.

For the moment, the best solution for us was the converting of the font into TTF.

convertFont.sh :

#!/usr/bin/fontforge
Print("Opening "+$1);
Open($1);
Print("Saving "+$1:r+".ttf");
Generate($1:r+".ttf");
Quit(0);

Shell :

fontforge -script convertFont.sh TOTO.otf

• http://fontforge.org/
• http://www.stuermer.ch/blog/convert-otf-to-ttf-font-on-ubuntu.html