Bug #105555
openTypoScript caching locks other page calls exklusive
0%
Description
The TypoScript frontend controller locks complete website on each call exclusivly.
We have the following setup:
- Load Balancer + Varnish
- 3 Webserver with TYPO3 on each Node
- NFS Share for shared files
- Redis Server
Due to NFS share we cannot use file locking strategy - we use Redis locking strategy (https://github.com/b13/distributed-locks).
We have approximately 7-10 Requests per second and 1.3 million pages in TYPO3. Many of them are not cached, so the page cache, especially the TypoScript cache has to be generated new on different calls.
The aquire lock is on 3 different places in the code:
- https://github.com/TYPO3/typo3/blob/v12.4.22/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php#L1402
- https://github.com/TYPO3/typo3/blob/v12.4.22/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php#L1428
- https://github.com/TYPO3/typo3/blob/v12.4.22/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php#L1432
The locking here causes the complete T3 to wait for the lock to be release (the lock name is 'pages'). So if there are 7 calls per second and for 10 seconds -> 70 calls has to be handled. The locking part takes for example 1 second to lock/unlock -> after 10 seconds only 10 requests of 70 are handled and the PHP FPM process starts to spawn until the limit is reached.
In the ResourcesMutex an exclusive and non-block lock is done for the key 'pages' (https://github.com/TYPO3/typo3/blob/main/typo3/sysext/core/Classes/Locking/ResourceMutex.php#L87-L89) which causes all other pages to wait until a random page is finished.
I would expect that only this specific page is locked instead of all pages are locked (with the hash included).
What if the the code is adapted to:
$this->lock->acquireLock('pages_' . $this->newHash, $this->newHash);
Then the lock is acquired only per specific page instead that everything is locked.
The same ResourceMutex is used in T3 v13 - so the problem should be the same.
How to reproduce?¶
1. Add a sleep(40); before line 90 (https://github.com/TYPO3/typo3/blob/main/typo3/sysext/core/Classes/Locking/ResourceMutex.php#L90)
2. Call one random page without cache.
3. Now call another page without cache and the second call will wait until the first call ist finished :)
If you need further information let me now!
Thanks Andi
Files
Updated by Marcel Macasso 27 days ago · Edited
Updated by Sybille Peters 27 days ago
Hello, just my 2c.
In my experience, this might get more attention if you:
- create a patch and upload it to the review server, see Contribution guide
- or raise question in Slack channel to discuss possible solution. (see https://typo3.org/community/meet/chat-slack, use channel #typo3-cms-coredev)
Updated by Gerrit Code Review 26 days ago
- Status changed from New to Under Review
Patch set 1 for branch 12.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/86981