Project

General

Profile

Actions
Copy link

Bug #86028

closed

getTreeList inserts duplicate keys in cache_treelist

Added by Olaf Döring over 5 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Must have
Assignee:
Alexander Schnitzler
Category:
Caching
Target version:
Candidate for patchlevel
Start date:
2018-08-29
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
8
PHP Version:
7.1
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

In my installation the getTreeList() function from TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer tries to insert a duplicate entry into cache_treelist.
I think this is possible on large sites with many getTreeList usage.

I think it will help to encapsulate the following block (line 6978-6986) into a try/catch construct:

GeneralUtility::makeInstance(ConnectionPool::class)->getConnectionForTable('cache_treelist')->insert(
                'cache_treelist',
                [
                    'md5hash' => $requestHash,
                    'pid' => $id,
                    'treelist' => implode(',', $theList),
                    'tstamp' => $GLOBALS['EXEC_TIME']
                ]
            );

If it fails inserting, there is an entry inserted just microseconds before, so we can ignore that.
The problem occures every 2-3 days.

Related issues 2 (0 open — 2 closed)

Related to TYPO3 Core - Bug #87139: Regression: getTreeList inserts duplicate keys in cache_treelistClosedAlexander Schnitzler2018-12-12

Actions
Has duplicate TYPO3 Core - Bug #86491: Duplicate entry for PRIMARY key in cache_treelistClosedAlexander Schnitzler2018-10-01

Actions
Actions
Copy link
 #1

Updated by Olaf Döring over 5 years ago

Some additions:
I upgraded my TYPO3 from 6.2 to 8.7, there was no problem with 6.2. Maybe the insertion of a duplicate entry into a key field in 6.2 did not throw such an exception.

Our TYPO3 intallation is a web- and intranetsite, used by over 1000 visitors daily (website) and over 1000 employees (intranet). Our pagetree is very large, the site is online since 2001, and so we have hundreds of new pages every year. The large pagetree is a problem since many years, it slows down the menu-creation and the handling in backend.

Actions
Copy link
 #2

Updated by Rens Admiraal over 5 years ago

Quick FYI without digging deep: we run into the same issue, but not with a lot of pages (there's 262 pages in the list of the failing query)

Actions
Copy link
 #3

Updated by Alexander Schnitzler over 5 years ago

  • Status changed from New to Accepted
  • Assignee set to Alexander Schnitzler
  • Target version set to Candidate for patchlevel
Actions
Copy link
 #4

Updated by Alexander Schnitzler over 5 years ago

  • Has duplicate Bug #86491: Duplicate entry for PRIMARY key in cache_treelist added
Actions
Copy link
 #5

Updated by Gerrit Code Review over 5 years ago

  • Status changed from Accepted to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/58951

Actions
Copy link
 #6

Updated by Alexander Schnitzler over 5 years ago

Short explanation why this issue occurres:

By default all new cache_treelist entries do not expire. This means, that if there is a cache entry for a given hash, you will get that entry.

There is just one exception though. If the endtime of a page gets updated, the cache entry gets said endtime as expiry timestamp. The problem with that is, that the method \TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer::getTreeList didn't check for expired entries and also didn't delete them.
The method "thought", there is no entry at all and that it is fine to create a new one, which obviously failes.

Actions
Copy link
 #7

Updated by Gerrit Code Review over 5 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/58951

Actions
Copy link
 #8

Updated by Gerrit Code Review over 5 years ago

Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59031

Actions
Copy link
 #9

Updated by Anonymous over 5 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
Actions
Copy link
 #10

Updated by Gerrit Code Review over 5 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_8-7 of project Teams/Security/TYPO3v4-Core has been pushed to the review server.
It is available at https://review.typo3.org/59040

Actions
Copy link
 #11

Updated by Stefan P over 5 years ago

We also have this issue. According to the changelog this patch is in 8.7.21 -- but we still experience it with this version. Completly random the Frontend is populated with occassional "Ooops, an error occured" and the log always says "Duplicate entry 'hash' for key 'PRIMARY' in..."

Actions
Copy link
 #12

Updated by Guido Schmechel over 5 years ago

Same here with 8.7.21

Actions
Copy link
 #13

Updated by Christian Toffolo over 5 years ago

Same here after yesterday I upgraded from 8.7.20 to 8.7.21, in the various "typo3_xxxxxxxxxxx.log" files of my installations I have many errors related to `MysqliException: Duplicate entry` in table `cache_treelist`.

I have a script that crawl my websites and this errors happen completely randomly.

I reverted https://review.typo3.org/#/c/58951/ and after many runs of my crawler the errors didn't show up again.

Actions
Copy link
 #14

Updated by Alexander Schnitzler over 5 years ago

  • Related to Bug #87139: Regression: getTreeList inserts duplicate keys in cache_treelist added
Actions
Copy link
 #15

Updated by Alexander Schnitzler over 5 years ago

  • Status changed from Under Review to Closed
Actions
Copy link

Also available in: Atom PDF