Project

General

Profile

Actions

Bug #86952

closed

flushByTags() is using a all in one , slow mysql command

Added by Jörg Velletti about 6 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Caching
Target version:
-
Start date:
2018-11-20
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
8
PHP Version:
7.1
Tags:
flushByTags, 8.7. zap, gremlin
Complexity:
Is Regression:
Sprint Focus:

Description

on a big TYPO3 instance with 16 languages, working in the backend is a "pain" :

When I press save button on a simple text media element, it takes up to 20 seconds. Changing the status "Hidden" status to visible of some elements in the list mode often runs into timeouts.

reason is in my eyes this uncahchable MySQL command inside of flushByTags()

DELETE tags2, cache1 FROM cf_cache_hash_tags AS tags1 JOIN cf_cache_hash_tags AS tags2 ON tags1.identifier = tags2.identifier JOIN cf_cache_hash AS cache1 ON tags1.identifier = cache1.identifier WHERE tags1.tag IN ('pageId_5529','tt_content','tt_content_102996','pageId_5531','pageId_5535','pageId_5532','pageId_5541','pageId_5538','pageId_5539','pageId_5540','pageId_5528','pages','pages_5529')

This is genereted for each translated language of that page for each changes content element :

If i do tests directly it takes 2-5 seconds each. But if this text is referenced many times it runs sometimes into a timeout.

Solution (for us):
if i change it from ONE Call to many single Calls in a for Loop ( like it was 10 Years ago in 4.5 ) :

 foreach ($tags as $tag ) {
                $quotedTag = '\'' . $tag . '\'';
                $connection->executeQuery(
                    'DELETE tags2, cache1'
                    . ' FROM ' . $this->tagsTable . ' AS tags1'
                    . ' JOIN ' . $this->tagsTable . ' AS tags2 ON tags1.identifier = tags2.identifier'
                    . ' JOIN ' . $this->cacheTable . ' AS cache1 ON tags1.identifier = cache1.identifier'
                    . ' WHERE tags1.tag = ' . $quotedTag
                );
            }

The total needed time 13 * 0.016 sec = means < than 0.2 seconds in total

See the attached patch, i am using actually with 8.7 LTS.
NOW working in backend is again acceptable. I will try to put the patch also into Github as a pull request

Hiding or activationg a lot of Content elements or database entries in the list mode is now again working FAST as expeted.


Some technical background:

- the MySQL Server is dedicated for this TYPO3 instance (no shared hoster) and attached with 10 GB network ..
- cf_cache_hash table has actually 50.000 rows and 500 MB
- cf_cache_hash_tags table has actually 250.000 rows and 46 MB
- 200.000 registered frontend user with many different usergroups and usergroup combinations.


Files

speed_up_backend_flushByTags.diff (2.9 KB) speed_up_backend_flushByTags.diff Jörg Velletti, 2018-11-20 10:45

Related issues 2 (0 open2 closed)

Related to TYPO3 Core - Task #86953: #86952 - flushByTags() in backend is using a much faster php LoopClosed2018-11-20

Actions
Related to TYPO3 Core - Bug #78596: Deleting a page tree (and causing cache flush by tags) hammers the cacheClosed2016-11-07

Actions
Actions

Also available in: Atom PDF