Bug #84541

Updated by Sybille Peters over 1 year ago

I get an Exception with HTTP status code 503 on loading a page with the Exception message:

Oops, an error occurred!
An exception occurred while executing 'INSERT INTO `index_words` (`wid`, `baseword`, `metaphone`) VALUES (?, ?, ?)' with params [55602717, "images\/botschaften", "66411504"]: Duplicate entry '55602717' for key 'PRIMARY'


h2. System

* TYPO3 version 8.7.17
* current master

h2. Severity of error

Even if Configuration preset "Live" is selected, an error is still displayed in the frontend on first load of page. The page is not rendered. Doesn't happen for all pages, only for pages where the hash collision occurs. Still, it's pretty ugly.


h2. Reproduce

# Activate the extension indexed_search
# Add static includes for indexed_search
# Add the following words to a new or existing content element: graf gettogethers abfluss erworbener
# Load the page in the frontend (you may have to do that twice)

h2. Expected result

Page is displayed

h2. Actual result

Exception message:

Uncaught TYPO3 Exception
An exception occurred while executing 'INSERT INTO `index_words` (`wid`, `baseword`, `metaphone`) VALUES (?, ?, ?)' with params [186135449, "gettogethers", "199699927"]: Duplicate entry '186135449' for key 'PRIMARY'


h2. Additional information

If you need to reproduce this several times, you can delete the index for the page by selecting the "Indexing" module, select the page with the content you inserted earlier, then select "Detailed statistics" and press the "Delete" garbage can.

You can also check the sys_log for similar errors:

<pre><code class="sql">
SELECT uid,FROM_UNIXTIME(tstamp),details FROM sys_log WHERE details LIKE '%index_word%' and error = 2 ORDER BY uid DESC LIMIT 10;

h2. Cause of error

possibly: https://github.com/TYPO3/TYPO3.CMS/blob/8124407655ae73656bf6c21f6bc8841b8e1d2023/typo3/sysext/indexed_search/Classes/Indexer.php#L2134

(the link goes to a specific commit which may no longer reflect the current codebase)

hash collision?

h2. Affected fields

These are the fields in the DB, that get filled using the current hash algorithm:

(all affected fields by looking at DB schema and content and guessing):

* index_debug.phash

* index_fulltext.phash
* index_grlist.phash
* index_grlist.phash_x
* index_grlist.hash_gr_list
* index_phash.phash
* index_phash.phash_grouping
* index_phash.contentHash
* index_rel.phash
* index_rel.wid
* *index_section.phash* (see https://forge.typo3.org/issues/79802)
* index_section.phash_t3
* *index_words.wid* (see this issue)
* *index_words.metaphone*

They should be changed from int to char(32), and the hash algorithm changed to simple PHP md5() (without truncation).