Bug #98126
closedPage caching does not work as expected on multi language sites
100%
Description
I would expect that as soon as a page is cached it's cached content is served to all clients. But if you have a multi language site and any client changes to a different language, the cached version isn't served anymore.
Steps to reproduce:- Create a new TYPO3 installation and install the introduction package
- Enable HTTP debug headers via
config.debug = 1
- Set
config.cache_period = 86400
- On the "home page" (HP) "Congratulations" create a new page translation
- Open the HP - cache is created
- Press F5 - cached page is served
- change to page "Features" - cache is created
- change to HP - cached page is served
- switch to translated page - cache is created
- Press F5 - cached page is served
- switch back to HP in default language - cache is created instead of serving the cached page
- switch to translated page - cache is created instead of serving the cached page
Here's an example curl output (all caches were cleared before, a translation for german was created for the HP):
# Setting base domain without leading slash
$ DOMAIN="https://www.example.com"
# Do some calls to default language
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:52. Expires 13-08-22 08:52
$ curl -v "${BASE}/features" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/features" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:56. Expires 13-08-22 08:56
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:55. Expires 13-08-22 08:55
# switch to language de
$ curl -v "${BASE}/de/" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/de/" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:58. Expires 13-08-22 08:58
# switch back to default language
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
# empty result but why? Shouldn't the page be in the cache?
Do I get something wrong?
Files
Updated by Manfred Egger over 2 years ago
Manfred Egger wrote:
I would expect that as soon as a page is cached it's cached content is served to all clients. But if you have a multi language site and any client changes to a different language, the cached version isn't served anymore.
I've tested this with TYPO3 10.4 and it is working as expected: even if I switch between languages, the cached content is served. So it seems it's a problem introduced with TYPO3 11.
Updated by Matthias Schneider about 2 years ago
We observed the exact same behavior. The cache on a large multi-language site with 30 languages is constantly cleared because every request one language deletes the cache for all other languages.
Our test case, with curl
and time
to track the time the request takes (everthing larger than 200ms is not cached):
# first request, all caches are cleared. cache is being built. time curl -I https://example.de HTTP/2 200 content-language: de expires: Tue, 13 Sep 2022 22:00:00 GMT cache-control: max-age=23784 pragma: public x-typo3-parsetime: 0ms etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" x-content-type-options: nosniff x-ua-compatible: IE=edge content-type: text/html; charset=utf-8 date: Tue, 13 Sep 2022 15:23:36 GMT server: Apache 0,00s user 0,00s system 0% cpu 2,312 total # second request, same language, cached result is used. time curl -I https://example.de HTTP/2 200 content-language: de x-typo3-debug-cache: Cached page generated 13-09-22 17:26. Expires 14-09-22 00:00 expires: Tue, 13 Sep 2022 22:00:00 GMT cache-control: max-age=23630 pragma: public x-typo3-parsetime: 0ms etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" x-content-type-options: nosniff x-ua-compatible: IE=edge content-type: text/html; charset=utf-8 date: Tue, 13 Sep 2022 15:26:10 GMT server: Apache 0,00s user 0,00s system 0% cpu 0,125 total # third request, DIFFERENT language, cache is built. time curl -I https://example.com HTTP/2 200 content-language: hr expires: Tue, 13 Sep 2022 22:00:00 GMT cache-control: max-age=23550 pragma: public x-typo3-parsetime: 0ms etag: "ceefa8cb5dc836b370c8e9212e580573" x-content-type-options: nosniff x-ua-compatible: IE=edge content-type: text/html; charset=utf-8 date: Tue, 13 Sep 2022 15:27:30 GMT server: Apache 0,00s user 0,00s system 0% cpu 1,242 total # fourth request, back to first language, cache is NOT used. time curl -I https://example.de HTTP/2 200 content-language: de expires: Tue, 13 Sep 2022 22:00:00 GMT cache-control: max-age=23119 pragma: public x-typo3-parsetime: 0ms etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" x-content-type-options: nosniff x-ua-compatible: IE=edge content-type: text/html; charset=utf-8 date: Tue, 13 Sep 2022 15:34:41 GMT server: Apache 0,00s user 0,00s system 0% cpu 1,132 total
This is reproducible on two independent Typo3 11.5 instances.
This StackOverflow seems to have has the same problem:
https://stackoverflow.com/questions/72444255/typo3-11-5-clears-cache-after-user-navigates-to-other-language
Updated by Matthias Schneider about 2 years ago
One observation:
Using `TypoScriptFrontendController->createHashBase()` with a forced argument of true (`createHashBase(true)`) will NOT add the "all" parameters to the hash function (hash parameters) and fixes the caching issue.
We are not sure of this has any side effects, but for the time being we could not observe any.
Updated by Reiner Kempkes about 2 years ago
- Is Regression set to Yes
We can confirm this issue, we have two affected installations.
This is critical in my opinion, as it causes huge load on systems which have a lot configured translations.
Furthermore systems are affected that are monitored via an uptime ping, especially when the ping will run very frequently.
Each call to a page or an translated page seems to flush the caches of other translated pages.
We also confirm that this worked in v10 before without any issues, so this is a regression.
What we test at the moment as a temporary workaround until this is fixed:
hooking into $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['createHashBase']
and unset($_params['hashParameters']['all']);
First results seem to be promising.
Updated by Stefan Bürk about 2 years ago
- Status changed from New to Accepted
- Assignee set to Stefan Bürk
Confirmations:
- 11.5-dev
Not tested yet - may follow
- 12.1-dev
- 10.4-dev
First raport / summerize:
- verified using the described curl technique in openening issue, but with ext:styleguide instead of introduction package
- it's not the full hashParamater['all'] array which is the devil for this - it seems to be the rootline subinformation, which is mixed between languages, thus generating a new hash
- lanuage caches have full page records for rootline entries, default language only a narrowed down subset (which we already observed, not relating it to this kind of cache isssu's)
We will investigate this further.
Updated by Michael Voehringer about 2 years ago
- PHP Version set to 8.1
I can also confirm this problem on TYPO3 11.5.17
Updated by Ina Beckmann about 2 years ago
We recently updated a customer project to TYPO3 V11 and also ran across the issue. This is a major problem and real performance killer for multi language installations.
We tested different configurations, language versions with different domains vs. one domain and language prefix, but to no avail.
Following Reiner Kempkes example, we used the "createHashBase" hook, but instead of unsetting the whole $_params['hashParameters']['all'], assigning an empty array to the sub entry "rootline" also does the trick ($params['hashParameters']['all']['rootLine'] = []). Stefan Bürks observation seems to be correct, there is something weird going on with the rootlines.
For now we will use this "quickfix" to keep the project working, but hopefully there will be a real fix coming soon.
Updated by Stefan Bürk almost 2 years ago
- Related to Bug #65044: Relative paths not respected when using ExtensionManagementUtility::addPageTSConfig added
Updated by Stefan Bürk almost 2 years ago
Still no solution, but after additionally couple of hours of debuging,
comparing and reviewing git history ... it seems that this is tightly
related to a bugfix, which moved loading the language based rootline
before TypoScript parsing to allow language based matchings. Thus the
already detected jumping behaviour with the rootlines in the pagesection
cache now can verified.
I reverted that change locally, and it worked like in 10. Will try to
find a solution than in the next days.
Related issue #23736
Related patch: https://review.typo3.org/c/Packages/TYPO3.CMS/+/65044
Updated by Stefan Bürk almost 2 years ago
- Related to deleted (Bug #65044: Relative paths not respected when using ExtensionManagementUtility::addPageTSConfig)
Updated by Stefan Bürk almost 2 years ago
- Related to Bug #23736: Condition with globalVar TSFE does not care about page language overlay added
Updated by Stefan Bürk almost 2 years ago
- File verify-curl-calls.sh verify-curl-calls.sh added
- Status changed from Accepted to In Progress
UPDATE
First, I have attached a small bash script which helps with executing a bunch
of curl calls and with making a result state / overview. This maked testing it
a multitude easier. At least if you have to do it a LOT again and again.
For that you can configure the main uri of your page, defining the slug for
a defult language and two translations. To make it usefull, the installation
should be set to debug mode to get the corresponding `x-typo3-debug-cache`
header.
Secondly, besides already having added the "source" of the issue as related
issue (patch) I now have a more detailed insight - and at least a more suitable
solution. Will discuss this with some of the other core team member first,
as this a really delicate area. Maybe they have some other suggestions.
But stay tuned - I think a bugfix will show up the next days.
And it has really re-verified that this only occures in v11. v10 is not hit by
this bug, as the patch kicking this issue off has been included in v11 only.
v12 healed itself through the massive overwork/rework of the TypoScript parser
and the connected places, like the TypoScript parsing. Kudso to lolli on here.
So this bug is only targeting v11 installations .... sadly thats more than enough.
Updated by Gerrit Code Review almost 2 years ago
- Status changed from In Progress to Under Review
Patch set 10 for branch 11.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302
Updated by Gerrit Code Review almost 2 years ago
Patch set 11 for branch 11.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302
Updated by Stefan Bürk almost 2 years ago
Maybe some of the people which having that issue can verify/test the bugfix (as composer patch or similiar). Would be nice to have some verifications here.
Updated by Joey Bouten almost 2 years ago
Hi Stefan,
I also encountered a caching issue, however this was in V10. What happened in my situation was te following.
In constants.typoscript i had the following:
1. Tracking is disabled by default (constants)
forms.microsoftDynamics.tracking =
2. For a specific language i enable the tracking (constants)
[siteLanguage("languageId") == 1] forms.microsoftDynamics.tracking = 1 [global]
3. I load in tracking scripts with the following (typoscript)
[{$forms.microsoftDynamics.scriptsEnabled} == '1'] page.headerData.850187 { ... scripts ... } [global]
- In TypoScriptFrontendController.php the `$this->getHash()` is called twice.
- However when called in the function generatePage_preProcessing() the $hashParameters['all'] do not seem to have the constants loaded from languageUid 1
- This causes a different MD5 hash compared to the `$this->getHash()` called in function `getFromCache()`
- Which then causes the regeneration of cache on all other languages when the page is called.
Your script helped me alot with debugging.
I'm not sure how related these issues are.
What i can do is the following.Thursday we have a TYPO3 development day planned, i could test my situation and yours in:
- TYPO3 V11 - with your fix
- TYPO3 V11 - without your fix
- TYPO3 V12
This however might also be another specific bug in V10/V11
Updated by Stefan Bürk almost 2 years ago
I would say, you describe another thing. I would say, that that's more related to the issue which has been fixed in v11, and is the source of the problem in this issue.
In v10 the language loading has been processed later. Rootline in PageSection cache is there always the default language rootline. Condition are matched later. Thus see related issue #23736 Condition with globalVar TSFE does not care about page language overlay.
We may can have a talk/call if you contact me on slack ...
Updated by Meelis Karulin almost 2 years ago
Stefan Bürk wrote in #note-15:
Maybe some of the people which having that issue can verify/test the bugfix (as composer patch or similiar). Would be nice to have some verifications here.
Tested, the patch works for us.
Updated by Sybille Peters almost 2 years ago
Many thanks for people above for tracking this down. This might explain some of the performance problems on my site. (despite the pages being "warmed up").
Additional tip for reproducing problem:¶
In my case, reproducing this issue was more simple:
1. use v11 local test system, add config.debug = 1 (TypoScript)
2. create a page with translation
3. alternate loading default language page and translated language page
(this reproduces problem without having to flush cache several times)
- load translation: loaded from cache (ok)
- load translation: loaded from cache (ok)
- load default: not loaded from cache (ERROR)
- load default: loaded from cache (ok)
- load translation: not loaded from cache (ERROR)
- ... repeat
Note: X-TYPO3-Debug-Cache is only sent if page is loaded from cache.
In my script, 3 and 5 (after switch) were always not loaded from cache.
Updated by Stefan Bürk almost 2 years ago
@Meelis Karulin May it be possible that you can vote at least plus 1 verified on the patch itself ? -> https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302
Updated by Stefan Bürk almost 2 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset e537029c0e6aabc77b4ab8b479d78b65dfe1c7e0.
Updated by Benni Mack almost 2 years ago
- Status changed from Resolved to Closed