getTreeList inserts duplicate keys in cache_treelist
In my installation the getTreeList() function from TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer tries to insert a duplicate entry into cache_treelist.
I think this is possible on large sites with many getTreeList usage.
I think it will help to encapsulate the following block (line 6978-6986) into a try/catch construct:
GeneralUtility::makeInstance(ConnectionPool::class)->getConnectionForTable('cache_treelist')->insert( 'cache_treelist', [ 'md5hash' => $requestHash, 'pid' => $id, 'treelist' => implode(',', $theList), 'tstamp' => $GLOBALS['EXEC_TIME'] ] );
If it fails inserting, there is an entry inserted just microseconds before, so we can ignore that.
The problem occures every 2-3 days.
Updated by Olaf Döring about 3 years ago
I upgraded my TYPO3 from 6.2 to 8.7, there was no problem with 6.2. Maybe the insertion of a duplicate entry into a key field in 6.2 did not throw such an exception.
Our TYPO3 intallation is a web- and intranetsite, used by over 1000 visitors daily (website) and over 1000 employees (intranet). Our pagetree is very large, the site is online since 2001, and so we have hundreds of new pages every year. The large pagetree is a problem since many years, it slows down the menu-creation and the handling in backend.
Updated by Alexander Schnitzler about 3 years ago
Short explanation why this issue occurres:
By default all new cache_treelist entries do not expire. This means, that if there is a cache entry for a given hash, you will get that entry.
There is just one exception though. If the endtime of a page gets updated, the cache entry gets said endtime as expiry timestamp. The problem with that is, that the method \TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer::getTreeList didn't check for expired entries and also didn't delete them.
The method "thought", there is no entry at all and that it is fine to create a new one, which obviously failes.
Updated by Stefan P almost 3 years ago
We also have this issue. According to the changelog this patch is in 8.7.21 -- but we still experience it with this version. Completly random the Frontend is populated with occassional "Ooops, an error occured" and the log always says "Duplicate entry 'hash' for key 'PRIMARY' in..."
Updated by Christian Toffolo almost 3 years ago
Same here after yesterday I upgraded from 8.7.20 to 8.7.21, in the various "typo3_xxxxxxxxxxx.log" files of my installations I have many errors related to `MysqliException: Duplicate entry` in table `cache_treelist`.
I have a script that crawl my websites and this errors happen completely randomly.
I reverted https://review.typo3.org/#/c/58951/ and after many runs of my crawler the errors didn't show up again.