Project

General

Profile

Actions

Bug #98126

closed

Page caching does not work as expected on multi language sites

Added by Manfred Egger over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
Caching
Target version:
-
Start date:
2022-08-12
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
11
PHP Version:
8.1
Tags:
performance
Complexity:
Is Regression:
Yes
Sprint Focus:

Description

I would expect that as soon as a page is cached it's cached content is served to all clients. But if you have a multi language site and any client changes to a different language, the cached version isn't served anymore.

Steps to reproduce:
  • Create a new TYPO3 installation and install the introduction package
  • Enable HTTP debug headers via config.debug = 1
  • Set config.cache_period = 86400
  • On the "home page" (HP) "Congratulations" create a new page translation
  • Open the HP - cache is created
  • Press F5 - cached page is served
  • change to page "Features" - cache is created
  • change to HP - cached page is served
  • switch to translated page - cache is created
  • Press F5 - cached page is served
  • switch back to HP in default language - cache is created instead of serving the cached page
  • switch to translated page - cache is created instead of serving the cached page

Here's an example curl output (all caches were cleared before, a translation for german was created for the HP):

# Setting base domain without leading slash
$ DOMAIN="https://www.example.com" 

# Do some calls to default language
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:52. Expires 13-08-22 08:52

$ curl -v "${BASE}/features" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/features" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:56. Expires 13-08-22 08:56

$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache        
< x-typo3-debug-cache: Cached page generated 12-08-22 08:55. Expires 13-08-22 08:55

# switch to language de
$ curl -v "${BASE}/de/" 2>&1 | grep x-typo3-debug-cache
# empty result as page is not cached
$ curl -v "${BASE}/de/" 2>&1 | grep x-typo3-debug-cache
< x-typo3-debug-cache: Cached page generated 12-08-22 08:58. Expires 13-08-22 08:58

# switch back to default language
$ curl -v "${BASE}/" 2>&1 | grep x-typo3-debug-cache
# empty result but why? Shouldn't the page be in the cache?

Do I get something wrong?


Files

verify-curl-calls.sh (7.21 KB) verify-curl-calls.sh Stefan Bürk, 2022-11-28 21:23
test_typo3_page_cache_issue_98126.sh (967 Bytes) test_typo3_page_cache_issue_98126.sh Sybille Peters, 2022-12-05 11:05

Related issues 1 (0 open1 closed)

Related to TYPO3 Core - Bug #23736: Condition with globalVar TSFE does not care about page language overlayClosedBenni Mack2010-10-15

Actions
Actions #1

Updated by Manfred Egger over 1 year ago

Manfred Egger wrote:

I would expect that as soon as a page is cached it's cached content is served to all clients. But if you have a multi language site and any client changes to a different language, the cached version isn't served anymore.

I've tested this with TYPO3 10.4 and it is working as expected: even if I switch between languages, the cached content is served. So it seems it's a problem introduced with TYPO3 11.

Actions #2

Updated by Matthias Schneider over 1 year ago

We observed the exact same behavior. The cache on a large multi-language site with 30 languages is constantly cleared because every request one language deletes the cache for all other languages.

Our test case, with curl and time to track the time the request takes (everthing larger than 200ms is not cached):

# first request, all caches are cleared. cache is being built.

time curl -I https://example.de 

HTTP/2 200 
content-language: de
expires: Tue, 13 Sep 2022 22:00:00 GMT
cache-control: max-age=23784
pragma: public
x-typo3-parsetime: 0ms
etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" 
x-content-type-options: nosniff
x-ua-compatible: IE=edge
content-type: text/html; charset=utf-8
date: Tue, 13 Sep 2022 15:23:36 GMT
server: Apache

0,00s user 0,00s system 0% cpu 2,312 total

# second request, same language, cached result is used.

time curl -I https://example.de 

HTTP/2 200 
content-language: de
x-typo3-debug-cache: Cached page generated 13-09-22 17:26. Expires 14-09-22 00:00
expires: Tue, 13 Sep 2022 22:00:00 GMT
cache-control: max-age=23630
pragma: public
x-typo3-parsetime: 0ms
etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" 
x-content-type-options: nosniff
x-ua-compatible: IE=edge
content-type: text/html; charset=utf-8
date: Tue, 13 Sep 2022 15:26:10 GMT
server: Apache

0,00s user 0,00s system 0% cpu 0,125 total

# third request, DIFFERENT language, cache is built.

time curl -I https://example.com

HTTP/2 200 
content-language: hr
expires: Tue, 13 Sep 2022 22:00:00 GMT
cache-control: max-age=23550
pragma: public
x-typo3-parsetime: 0ms
etag: "ceefa8cb5dc836b370c8e9212e580573" 
x-content-type-options: nosniff
x-ua-compatible: IE=edge
content-type: text/html; charset=utf-8
date: Tue, 13 Sep 2022 15:27:30 GMT
server: Apache

0,00s user 0,00s system 0% cpu 1,242 total

# fourth request, back to first language, cache is NOT used.

time curl -I https://example.de

HTTP/2 200 
content-language: de
expires: Tue, 13 Sep 2022 22:00:00 GMT
cache-control: max-age=23119
pragma: public
x-typo3-parsetime: 0ms
etag: "f8b6bfb4b4e88b138b72eea81bfb3a99" 
x-content-type-options: nosniff
x-ua-compatible: IE=edge
content-type: text/html; charset=utf-8
date: Tue, 13 Sep 2022 15:34:41 GMT
server: Apache

0,00s user 0,00s system 0% cpu 1,132 total

This is reproducible on two independent Typo3 11.5 instances.

This StackOverflow seems to have has the same problem:
https://stackoverflow.com/questions/72444255/typo3-11-5-clears-cache-after-user-navigates-to-other-language

Actions #3

Updated by Matthias Schneider over 1 year ago

One observation:

Using `TypoScriptFrontendController->createHashBase()` with a forced argument of true (`createHashBase(true)`) will NOT add the "all" parameters to the hash function (hash parameters) and fixes the caching issue.

We are not sure of this has any side effects, but for the time being we could not observe any.

Actions #4

Updated by Reiner Kempkes over 1 year ago

  • Is Regression set to Yes

We can confirm this issue, we have two affected installations.
This is critical in my opinion, as it causes huge load on systems which have a lot configured translations.
Furthermore systems are affected that are monitored via an uptime ping, especially when the ping will run very frequently.
Each call to a page or an translated page seems to flush the caches of other translated pages.
We also confirm that this worked in v10 before without any issues, so this is a regression.

What we test at the moment as a temporary workaround until this is fixed:
hooking into $GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['tslib/class.tslib_fe.php']['createHashBase']
and unset($_params['hashParameters']['all']);
First results seem to be promising.

Actions #5

Updated by Stefan Bürk over 1 year ago

  • Status changed from New to Accepted
  • Assignee set to Stefan Bürk

Confirmations:
- 11.5-dev

Not tested yet - may follow
- 12.1-dev
- 10.4-dev

First raport / summerize:

- verified using the described curl technique in openening issue, but with ext:styleguide instead of introduction package
- it's not the full hashParamater['all'] array which is the devil for this - it seems to be the rootline subinformation, which is mixed between languages, thus generating a new hash
- lanuage caches have full page records for rootline entries, default language only a narrowed down subset (which we already observed, not relating it to this kind of cache isssu's)

We will investigate this further.

Actions #6

Updated by Michael Voehringer over 1 year ago

  • PHP Version set to 8.1

I can also confirm this problem on TYPO3 11.5.17

Actions #7

Updated by Ina Beckmann over 1 year ago

We recently updated a customer project to TYPO3 V11 and also ran across the issue. This is a major problem and real performance killer for multi language installations.
We tested different configurations, language versions with different domains vs. one domain and language prefix, but to no avail.

Following Reiner Kempkes example, we used the "createHashBase" hook, but instead of unsetting the whole $_params['hashParameters']['all'], assigning an empty array to the sub entry "rootline" also does the trick ($params['hashParameters']['all']['rootLine'] = []). Stefan Bürks observation seems to be correct, there is something weird going on with the rootlines.

For now we will use this "quickfix" to keep the project working, but hopefully there will be a real fix coming soon.

Actions #8

Updated by Stefan Bürk over 1 year ago

  • Related to Bug #65044: Relative paths not respected when using ExtensionManagementUtility::addPageTSConfig added
Actions #9

Updated by Stefan Bürk over 1 year ago

Still no solution, but after additionally couple of hours of debuging,
comparing and reviewing git history ... it seems that this is tightly
related to a bugfix, which moved loading the language based rootline
before TypoScript parsing to allow language based matchings. Thus the
already detected jumping behaviour with the rootlines in the pagesection
cache now can verified.

I reverted that change locally, and it worked like in 10. Will try to
find a solution than in the next days.

Related issue #23736
Related patch: https://review.typo3.org/c/Packages/TYPO3.CMS/+/65044

Actions #10

Updated by Stefan Bürk over 1 year ago

  • Related to deleted (Bug #65044: Relative paths not respected when using ExtensionManagementUtility::addPageTSConfig)
Actions #11

Updated by Stefan Bürk over 1 year ago

  • Related to Bug #23736: Condition with globalVar TSFE does not care about page language overlay added
Actions #12

Updated by Stefan Bürk over 1 year ago

UPDATE

First, I have attached a small bash script which helps with executing a bunch
of curl calls and with making a result state / overview. This maked testing it
a multitude easier. At least if you have to do it a LOT again and again.

For that you can configure the main uri of your page, defining the slug for
a defult language and two translations. To make it usefull, the installation
should be set to debug mode to get the corresponding `x-typo3-debug-cache`
header.

Secondly, besides already having added the "source" of the issue as related
issue (patch) I now have a more detailed insight - and at least a more suitable
solution. Will discuss this with some of the other core team member first,
as this a really delicate area. Maybe they have some other suggestions.

But stay tuned - I think a bugfix will show up the next days.

And it has really re-verified that this only occures in v11. v10 is not hit by
this bug, as the patch kicking this issue off has been included in v11 only.
v12 healed itself through the massive overwork/rework of the TypoScript parser
and the connected places, like the TypoScript parsing. Kudso to lolli on here.

So this bug is only targeting v11 installations .... sadly thats more than enough.

Actions #13

Updated by Gerrit Code Review over 1 year ago

  • Status changed from In Progress to Under Review

Patch set 10 for branch 11.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302

Actions #14

Updated by Gerrit Code Review over 1 year ago

Patch set 11 for branch 11.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302

Actions #15

Updated by Stefan Bürk over 1 year ago

Maybe some of the people which having that issue can verify/test the bugfix (as composer patch or similiar). Would be nice to have some verifications here.

Actions #16

Updated by Joey Bouten over 1 year ago

Hi Stefan,

I also encountered a caching issue, however this was in V10. What happened in my situation was te following.

In constants.typoscript i had the following:

1. Tracking is disabled by default (constants)

forms.microsoftDynamics.tracking =

2. For a specific language i enable the tracking (constants)

[siteLanguage("languageId") == 1]
forms.microsoftDynamics.tracking = 1
[global]

3. I load in tracking scripts with the following (typoscript)

[{$forms.microsoftDynamics.scriptsEnabled} == '1']
 page.headerData.850187 { 
 ... scripts ...
}
[global]

  • In TypoScriptFrontendController.php the `$this->getHash()` is called twice.
  • However when called in the function generatePage_preProcessing() the $hashParameters['all'] do not seem to have the constants loaded from languageUid 1
  • This causes a different MD5 hash compared to the `$this->getHash()` called in function `getFromCache()`
  • Which then causes the regeneration of cache on all other languages when the page is called.

Your script helped me alot with debugging.

I'm not sure how related these issues are.

What i can do is the following.
Thursday we have a TYPO3 development day planned, i could test my situation and yours in:
  • TYPO3 V11 - with your fix
  • TYPO3 V11 - without your fix
  • TYPO3 V12

This however might also be another specific bug in V10/V11

Actions #17

Updated by Stefan Bürk over 1 year ago

@Joey Bouten @Joey Bouten

I would say, you describe another thing. I would say, that that's more related to the issue which has been fixed in v11, and is the source of the problem in this issue.

In v10 the language loading has been processed later. Rootline in PageSection cache is there always the default language rootline. Condition are matched later. Thus see related issue #23736 Condition with globalVar TSFE does not care about page language overlay.

We may can have a talk/call if you contact me on slack ...

Actions #18

Updated by Sybille Peters over 1 year ago

  • Tags set to performance
Actions #19

Updated by Meelis Karulin over 1 year ago

Stefan Bürk wrote in #note-15:

Maybe some of the people which having that issue can verify/test the bugfix (as composer patch or similiar). Would be nice to have some verifications here.

Tested, the patch works for us.

Actions #20

Updated by Sybille Peters over 1 year ago

Many thanks for people above for tracking this down. This might explain some of the performance problems on my site. (despite the pages being "warmed up").

Additional tip for reproducing problem:

In my case, reproducing this issue was more simple:

1. use v11 local test system, add config.debug = 1 (TypoScript)
2. create a page with translation
3. alternate loading default language page and translated language page
(this reproduces problem without having to flush cache several times)

e.g.
  • load translation: loaded from cache (ok)
  • load translation: loaded from cache (ok)
  • load default: not loaded from cache (ERROR)
  • load default: loaded from cache (ok)
  • load translation: not loaded from cache (ERROR)
  • ... repeat

Note: X-TYPO3-Debug-Cache is only sent if page is loaded from cache.

In my script, 3 and 5 (after switch) were always not loaded from cache.

Actions #21

Updated by Stefan Bürk over 1 year ago

@Meelis Karulin May it be possible that you can vote at least plus 1 verified on the patch itself ? -> https://review.typo3.org/c/Packages/TYPO3.CMS/+/76302

Actions #22

Updated by Stefan Bürk over 1 year ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
Actions #23

Updated by Benni Mack over 1 year ago

  • Status changed from Resolved to Closed
Actions

Also available in: Atom PDF