cache is not saved properly
After clearing all cache the first generation of a page shows it correctly (because it is not loaded from cache). After first update of a page, it is loaded from cache and cannot be unserialized, that's why all pages are not shown, only the error message "No template found".
Solution: use base64_encode/base64_decode functions on serialized strings, so that double and single quotes are not present in values of SQL query at all.
Initially reported for TYPO3 4.0.4
Confirmed and reproduced for TYPO3 4.1.0
(issue imported from #M5088)
#1 Updated by Ralf Hettinger over 13 years ago
Confirming this error which is still alive in 4.1.0RC2 where I just ran into it (might not happen to often; actually I don't know what I typed into my template to produce it, but it indeed may harm the sql insert query).
The previous attached file class.t3lib_tstemplate.php does decoding/unserializing in the wrong order and therefore is no good either - correctly it should state:
- $this->setup = base64_decode(unserialize($setupData));
+ $this->setup = unserialize(base64_decode($setupData));
However, I would suggest to patch functions getHash and storeHash in class pageselect instead... attached patch 4.1.0RC2.patch against class.t3lib_page.php of 4.1.0RC2
#4 Updated by Ralf Hettinger over 13 years ago
Hm... that sounds plausible, Oliver.
Having thought about it again, I'm quite sure that the error isn't caused by a MySQL query error, but has to do something with (un)serializing (and might of course has arised by fullQuoteString changing the insert statement so that the queried data isn't unserializable after being retrieved).
In my case GLOBALS['TYPO3_DB']->debugOutput = 1 and errors displayErrors=On was set while the problem hit me - but nothing showed up.
Unfortunately I can't reproduce the error anymore (it did arise reproducibly but on a system I'm just working on the templates; I might have changed the string within the template that caused it - now it's gone), I'm very sorry.
MySQL version is 4.1.11 , database utf-8 with dbinit set to SET NAMES utf8 SET CHARACTER SET utf8
#5 Updated by Kai Rudolph over 13 years ago
same problem in my installation after updaten to 4.1
With Version 4.0.5 everything works fine.
I also have debugOutput = 1 and errors displayErrors=On, but nothing is shown.
When shutting down the index_search Ext this error is shown:
ERROR Duplicate entry '2-222419149' for key 1
lastBuiltQuery INSERT INTO cache_pagesection
) VALUES (
debug_backtrace require // tslib_fe->getconfigarray // t3lib_tstemplate->start // t3lib_db->exec_insertquery // t3lib_db->debug
#10 Updated by Sudara Williams - over 13 years ago
Hello, I'm experiencing this on an upgrade to 4.1 final.
Can a core dev please look into this issue? It's a block, a major bug, and shipped with 4.1 final - maybe it's worth releasing a patch?
I'm a bit confused how this was forgotten when shipping 4.1
#11 Updated by Andreas Balzer over 13 years ago
Hi! As I have written in bug report 5157 we have the same problem here with our page (updated from 4.0.5 to 4.1).
As this might be related to an installed extension, I went through all of them on my server and made a list of all that I have installed and loaded here. Maybe it helps if you can go through yours and write down which ones you have too. So we might be able to reduce the extension list to a view entries where one of them hopefully is the one that causes sleepless nights here ;)
UPDATE: I have installed a second server and switched off all extensions (i mean really all via install tool and then remove all caches and delete temp files..) The bug is still there.
#12 Updated by Andreas Balzer over 13 years ago
Hi again ;)
I wanted to mention that I am willing to send a virtual server with a copy of our webserver to a Core Developer by mail (within Europe), so that he or she is able to reproduce the error or see in which circumstances it's occuring.
Anyone who is able to help and wants a copy : Please send an informal email containing your address, telephonenumber and a link to a page that approves that you're a Core dev ( ;) ) to firstname.lastname@example.org.
You'll get a free CD containing a virtual server with parts of our website via mail for free if your address is located within Europe and you are willing to recieve the mail via standard mail service. If you want to get it via express (arriving within afaik 12 hours after sending it away), you'll have to pay for the mail service (afaik 40-60 euros within europe).
So if you're interested: email@example.com
I really do hope that this bug is fixed as fast as possible.
P.S: We are able to send you the 5 GB server image via internet connection (for free) too.
#14 Updated by Oliver Hader over 13 years ago
- the bug occurs if UTF-8 is used
- serializing an array with UTF-8 chars seems not to work always (e.g. two-byte-char is handled as single-byte-char)
- If "wrong" data (with incorrect UTF-8 stuff) is written to the database, the string is cutted at the first wrong UTF-8 character, what also destroys the serialized array (with the TypoScript setup)
Thus, the problem is based on character encoding. If you use e.g. UTF-8 and have unicode characters in your TypoScripts, this might break the whole site and show the "No template found" error.
However, encapsulation the (still wrong) serialized arrays in a base64 container, might help but is not the solution of the wrong character encoding.
#15 Updated by Michael Stucki over 13 years ago
OK, I just spent some time to find an easy how-to-reproduce scenario.
Here is a list of prerequisites:
- The backend must run in latin1 (no forceCharset=utf8 set!)
- setDBinit=SET NAMES utf8;
- cache_hash.content = utf8_general_ci
- Your template contains two page types (typeNum)
- Your template contains special characters
Changing a single one of these requirements makes the bug go away.
#17 Updated by Michael Stucki over 13 years ago
I finally solved the problem! The fix is easy: cache_hash.content must be changed back to "mediumblob". This was changed to mediumtext during 4.1 development.
I was wondering why the problem does not occur for the sys_template.config field which contains the TS setup with the same special chars. The reason seems to be that so far, the field type was not changed here (luckily we did not change any of the system extensions from *blob to *text so far)...
So to prevent the problem, the following things must be considered:
- TS setup + constants must remain as binary fields
- cache_hash.content must be changed back to mediumblob
- If we want to get rid of those blob fields, we need to introduce character conversion on TS parser level, which is probably not worth such a hassle...
#21 Updated by Michael Stucki over 13 years ago
Honestly, I don't consider the problem as urgent, since the prerequisites seem to be quite special. You can quickly fix the issue by changing the field type with phpmyadmin for example.
But that's just my impression. What do you think?
@Andreas: I was trying to understand why your configuration breaks although you claim to have UTF-8 in the backend. At which character is it breaking (last character of the failing cache_hash record)? Could it be that your template contains a ISO-8859-1 character which you didn't convert after migrating to UTF-8?
#22 Updated by Ralf Hettinger over 13 years ago
Thank you Michael, for taking care of and solving this issue.
I can confirm your prerequisites, I think one might be hit by this one (at least that was the problem in my case) when migrating one or more special chars containing TS templates from LATIN1 to utf8.
#26 Updated by Ralf Hettinger over 13 years ago
Just an idea (it is neither tested nor do I know if it has anything to do with this one): Might there be interdependencies with $GLOBALS['TYPO3_CONF_VARS']['SYS']['t3lib_cs_convMethod'] ?
Since I've seen e.g. iconv chopping conversions after invalid chars found in the string to convert for the desired conversion.
I'm sorry that my post might be misleading; I'm just guessing since it sounds familar to me.
#27 Updated by Andreas Balzer over 13 years ago
Michael Stucki wrote:
Honestly, I don't consider the problem as urgent, since the prerequisites seem to be quite special. You can quickly fix the issue by changing the field >type with phpmyadmin for example.
Yes, true. I only thought about that many 'normal' users don't know, where to find such bug patches. But it's true: A new version for such a thing was really a bad idea ;) However: I have a new one, which is quite friendly for usuability too. What do you think about another part of the extension manager or install tool, which is able to import all the new bug patches into a live installation (after admin confirmation)? Would it be a good idea to discuss about in mailinglists or somewhere else?
@Andreas: I was trying to understand why your configuration breaks although you claim to have UTF-8 in the backend. At which character is it breaking (last >character of the failing cache_hash record)? Could it be that your template contains a ISO-8859-1 character which you didn't convert after migrating to >UTF-8?
Yes, our template contains some invalid characters in some T3 comments.. like "#text some bad §$% here".. ;)
I didn't look into the cache_hash records but if you want me to do so, I will. :)
#28 Updated by gregory over 13 years ago
Just to let you know that i do experience this bug too.
my case :
- typo3 4.1
- backend in latin1 (which shall not be under-considered since this is the default installation setting)
- typoscripts stored in external iso-8859-1 files and included thanks to the <INCLUDE_TYPOSCRIPT:...> syntax (this is the tricky part in my setting)
I'd appreciate to have this bug fix in the next release since its resolution is trivial and that it seems to have been introduced because of a mistake in the developpement process (which is not very reassuring i'm afraid)