TYPO3 Core

Confirming this error which is still alive in 4.1.0RC2 where I just ran into it (might not happen to often; actually I don't know what I typed into my template to produce it, but it indeed may harm the sql insert query).

The previous attached file class.t3lib_tstemplate.php does decoding/unserializing in the wrong order and therefore is no good either - correctly it should state:
- $this->setup = base64_decode(unserialize($setupData));
+ $this->setup = unserialize(base64_decode($setupData));

However, I would suggest to patch functions getHash and storeHash in class pageselect instead... attached patch 4.1.0RC2.patch against class.t3lib_page.php of 4.1.0RC2

Did you enable sqlDebug in your TYPO3 installation. IMO the error occurs while saving data and is based on an error in fullQuoteStr. So, did you get any MySQL warnings/errors on writing the data to the cache?

Sorry for wrong order.
Yes, this patch works as well.

To Oliver: I checked the length of data before saving and after saving. The saved data is less in several symbols, so I think MySQL 5.0.26 incorrectly removes some quotes.

Hm... that sounds plausible, Oliver.

Having thought about it again, I'm quite sure that the error isn't caused by a MySQL query error, but has to do something with (un)serializing (and might of course has arised by fullQuoteString changing the insert statement so that the queried data isn't unserializable after being retrieved).

In my case GLOBALS['TYPO3_DB']->debugOutput = 1 and errors displayErrors=On was set while the problem hit me - but nothing showed up.

Unfortunately I can't reproduce the error anymore (it did arise reproducibly but on a system I'm just working on the templates; I might have changed the string within the template that caused it - now it's gone), I'm very sorry.

MySQL version is 4.1.11 , database utf-8 with dbinit set to SET NAMES utf8 SET CHARACTER SET utf8

same problem in my installation after updaten to 4.1

With Version 4.0.5 everything works fine.

I also have debugOutput = 1 and errors displayErrors=On, but nothing is shown.

When shutting down the index_search Ext this error is shown:

caller t3lib_DB::exec_INSERTquery
ERROR Duplicate entry '2-222419149' for key 1
lastBuiltQuery INSERT INTO cache_pagesection
(
content,
tstamp,
page_id,
mpvar_hash
) VALUES (
..............)
debug_backtrace require // tslib_fe->getconfigarray // t3lib_tstemplate->start // t3lib_db->exec_insertquery // t3lib_db->debug

Kai, I don't think your problem is the same as the original bug report.

Any chance you are using frames? See http://bugs.typo3.org/view.php?id=5152

Hi dAniel hAhler, thanks for the reply. No I'm not working with frames. Today I will go through the template switching of some lines. Maybe i will find the one causing this error.

Hi!

This is related to 0005157?

@Thomas Esders: maybe. But I think it is a caching Prolbem. I its like you desrcibe it. After clearing the cache the page is show, but after reloading it says "no template found"

Hello, I'm experiencing this on an upgrade to 4.1 final.

Can a core dev please look into this issue? It's a block, a major bug, and shipped with 4.1 final - maybe it's worth releasing a patch?

I'm a bit confused how this was forgotten when shipping 4.1

Hi! As I have written in bug report 5157 we have the same problem here with our page (updated from 4.0.5 to 4.1).
As this might be related to an installed extension, I went through all of them on my server and made a list of all that I have installed and loaded here. Maybe it helps if you can go through yours and write down which ones you have too. So we might be able to reduce the extension list to a view entries where one of them hopefully is the one that causes sleepless nights here ;)
Greetings
Andreas

beuser_ip_lock
context_help
date2cal
swg_tca_ext_10mb
extra_page_cm_options
skin_grey_2
rtehtmlarea
impexp
kj_becalendar
stfl_saveandnew
cron_setdefaultauthor
tm_shared_lib
lang
tsconfig_help
cms
version
zed_more_columns
gb_bedraganddrop
content_uneraser
imagelist
w4x_backup
aboutmodules
dam
dam_index
dam_cron
mwimagemap
lowlevel
install
belog
phpmyadmin
beuser
t3quixplorer
setup
taskcenter
sys_action
sys_messages
sys_notepad
taskcenter_recent
taskcenter_rootlist
cms_plaintext_import
func_wizards
wizard_crpages
wizard_sortpages
info_pagetsconfig
plugin_mgm
tstemplate
tstemplate_ceditor
tstemplate_styler
tstemplate_info
tstemplate_objbrowser
tstemplate_analyzer
viewpage
captcha
css_styled_content
cwt_feedit
rlmp_dateselectlib
cron_feuserscase
mc_googlesitemap
sg_zfelib
metatags
page_php_content
prototypejs
de_extcontentcomments
ts_language_de
tt_address
vjchat
cal
chc_forum
rf_content_comments
cwt_community
flvplayer
feuser_admin
indexed_search
newloginbox
tt_news
jk_poll
pmk_rssnewsexport
ltg_phpadsnew
cwt_community_user
api_macmade
gabriel
pdf_generator2
sr_static_info
static_info_tables
bf_xml_for_flash
kyakcc_metaexif_metaaudio
cc_metaexec
cc_meta_xmp
cc_txtextphp
sv
ch_handbuch

UPDATE: I have installed a second server and switched off all extensions (i mean really all via install tool and then remove all caches and delete temp files..) The bug is still there.

Hi again ;)
I wanted to mention that I am willing to send a virtual server with a copy of our webserver to a Core Developer by mail (within Europe), so that he or she is able to reproduce the error or see in which circumstances it's occuring.

Anyone who is able to help and wants a copy : Please send an informal email containing your address, telephonenumber and a link to a page that approves that you're a Core dev ( ;) ) to support@ess-erfurt.de.

You'll get a free CD containing a virtual server with parts of our website via mail for free if your address is located within Europe and you are willing to recieve the mail via standard mail service. If you want to get it via express (arriving within afaik 12 hours after sending it away), you'll have to pay for the mail service (afaik 40-60 euros within europe).

So if you're interested: support@ess-erfurt.de

I really do hope that this bug is fixed as fast as possible.

Andreas

P.S: We are able to send you the 5 GB server image via internet connection (for free) too.

I can confirm this bug. I don't know the reason, but I can reproduce the behaviour in black-boxing. I'm going to look what's going on.

• the bug occurs if UTF-8 is used
• serializing an array with UTF-8 chars seems not to work always (e.g. two-byte-char is handled as single-byte-char)
• If "wrong" data (with incorrect UTF-8 stuff) is written to the database, the string is cutted at the first wrong UTF-8 character, what also destroys the serialized array (with the TypoScript setup)

Thus, the problem is based on character encoding. If you use e.g. UTF-8 and have unicode characters in your TypoScripts, this might break the whole site and show the "No template found" error.
However, encapsulation the (still wrong) serialized arrays in a base64 container, might help but is not the solution of the wrong character encoding.

OK, I just spent some time to find an easy how-to-reproduce scenario.

Here is a list of prerequisites:
- The backend must run in latin1 (no forceCharset=utf8 set!)
- setDBinit=SET NAMES utf8;
- cache_hash.content = utf8_general_ci
- Your template contains two page types (typeNum)
- Your template contains special characters

Changing a single one of these requirements makes the bug go away.

"- The backend must run in latin1 (no forceCharset=utf8 set!)
Changing a single one of these requirements makes the bug go away."
false. Our backend runs as utf-8 and the bug occurs.

I finally solved the problem! The fix is easy: cache_hash.content must be changed back to "mediumblob". This was changed to mediumtext during 4.1 development.

I was wondering why the problem does not occur for the sys_template.config field which contains the TS setup with the same special chars. The reason seems to be that so far, the field type was not changed here (luckily we did not change any of the system extensions from *blob to *text so far)...

So to prevent the problem, the following things must be considered:
- TS setup + constants must remain as binary fields
- cache_hash.content must be changed back to mediumblob
- If we want to get rid of those blob fields, we need to introduce character conversion on TS parser level, which is probably not worth such a hassle...

@Andreas Otto †: Interesting that you even encounter the problem with UTF8 in the backend. However, this is countless now since the problem is solved :-)

Thanks Michael for providing the patch. I confirm, that this solves the problem.

when can we expect T3 4.1.1? :D

Honestly, I don't consider the problem as urgent, since the prerequisites seem to be quite special. You can quickly fix the issue by changing the field type with phpmyadmin for example.

But that's just my impression. What do you think?

@Andreas Otto †: I was trying to understand why your configuration breaks although you claim to have UTF-8 in the backend. At which character is it breaking (last character of the failing cache_hash record)? Could it be that your template contains a ISO-8859-1 character which you didn't convert after migrating to UTF-8?

Thank you Michael, for taking care of and solving this issue.

I can confirm your prerequisites, I think one might be hit by this one (at least that was the problem in my case) when migrating one or more special chars containing TS templates from LATIN1 to utf8.

Good work!

No, the problem is not solved in this way. I have version 4.0.4 as you can see, cache_hash.content is mediumblob, but error persists. Only base64 functions fix it.

That is simply not possible! If cache_hash.content was mediumblob, everything could be stored correctly. Please check again.

It is mediumblob. I did not changed it at all.
I use UTF-8 everywhere, including typoscripts.

Just an idea (it is neither tested nor do I know if it has anything to do with this one): Might there be interdependencies with $GLOBALS['TYPO3_CONF_VARS']['SYS']['t3lib_cs_convMethod'] ?

Since I've seen e.g. iconv chopping conversions after invalid chars found in the string to convert for the desired conversion.

I'm sorry that my post might be misleading; I'm just guessing since it sounds familar to me.

Michael Stucki wrote:

Honestly, I don't consider the problem as urgent, since the prerequisites seem to be quite special. You can quickly fix the issue by changing the field >type with phpmyadmin for example.

Yes, true. I only thought about that many 'normal' users don't know, where to find such bug patches. But it's true: A new version for such a thing was really a bad idea ;) However: I have a new one, which is quite friendly for usuability too. What do you think about another part of the extension manager or install tool, which is able to import all the new bug patches into a live installation (after admin confirmation)? Would it be a good idea to discuss about in mailinglists or somewhere else?

@Andreas Otto †: I was trying to understand why your configuration breaks although you claim to have UTF-8 in the backend. At which character is it breaking (last >character of the failing cache_hash record)? Could it be that your template contains a ISO-8859-1 character which you didn't convert after migrating to >UTF-8?

Yes, our template contains some invalid characters in some T3 comments.. like "#text some bad §$% here".. ;)
I didn't look into the cache_hash records but if you want me to do so, I will. :)

Greetings
Andreas

Just to let you know that i do experience this bug too.

my case :
- typo3 4.1
- backend in latin1 (which shall not be under-considered since this is the default installation setting)
- typoscripts stored in external iso-8859-1 files and included thanks to the <INCLUDE_TYPOSCRIPT:...> syntax (this is the tricky part in my setting)

I'd appreciate to have this bug fix in the next release since its resolution is trivial and that it seems to have been introduced because of a mistake in the developpement process (which is not very reassuring i'm afraid)

greetings
duch

Solved.

Make sure you update your database tables in the Install Tool when you have this problem...