Task #104038
open
Typoscript cache can become quite large
0%
Description
In our setup the typoscript cache gets quite large. We're seeing ~10MB cache size.
This is not a huge issue if you're running a few sites, but we're running a setup with ~3000 TYPO3 sites, having 10MB data that needs to be read for each site affects disk caching rates, which is untenable for us.
I've been examining the typoscript cache and it's not that we're really storing that much, it's mostly that the design of the typoscript parser combined with the ineffeciencies of serialize, causes the file to explode in size. I've tried replacing it with igbinary_serialize and that cuts the size to ~1/8.
Ofcourse, adding a dependency on igbinary to TYPO3 is probably not a good decision, so that's not the right fix.
However, Looking through the typoscript cache, I don't really understand why it's a code cache, it just defines a PHP file that returns the contents of the serialized objects, it doesn't use any capability that shouldn't work with a plain data cache.
I've created a patch for this and hope for some feedback.
Updated by Georg Ringer about 1 month ago
thanks for your issue, just a FYI: no patch yet added
Updated by Gerrit Code Review about 1 month ago
- Status changed from New to Under Review
Patch set 1 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Gerrit Code Review about 1 month ago
Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Gerrit Code Review about 1 month ago
Patch set 3 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Gerrit Code Review about 1 month ago
Patch set 4 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Gerrit Code Review about 1 month ago
Patch set 5 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Gerrit Code Review about 1 month ago
Patch set 6 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/84603
Updated by Rasmus Sallling about 1 month ago
Patch is up now.
The patch just contains the raw changes to convert it from a code cache to a data cache.
But I believe some compatibility changes are probably also needed before this could potentially be merged. I guess that's a premature discussion, but I figure either some code to override the class to VariableFrontend if it's defined as a PhpFrontend in user configuration. Alternatively the cache could be renamed to avoid that issue completely.
Updated by Christian Kuhn about 1 month ago
· Edited
We'll probably switch away from serialize/unserialize towards a direct creation of data objects using new() in cache files.
This promises higher performance since opcache can kick in. It will require not switching away from PhpFrontend.
I'm currently working within the parser code to allow this (among other things), so we can see if that works out.
We'll end up with smaller files along the way.
Updated by Rasmus Sallling about 1 month ago
Makes sense, not sure I completely share the your apparent comfort with dynamically creating php code, but I can certainly see it performing better than the current serialize/unserialize approach.
Will track your progress instead then. Thanks for the review.