Bug #81555

Bug #80875: Multiple problems with relation handler if too many records exist on a page

Do not use prepared statements for reference index queries

Added by Sascha Egerer over 2 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
Database API (Doctrine DBAL)
Target version:
Start date:
2017-06-12
Due date:
% Done:

100%

TYPO3 Version:
8
PHP Version:
Tags:
Complexity:
medium
Is Regression:
Sprint Focus:
Remote Sprint

Description

The queries to update the reference index could be very huge and contain a very long list of uids.
When using prepared statements this can end up in a fatal error as "only" 65.536 placeholders are supported in prepared statements.

Core: Exception handler (WEB): Uncaught TYPO3 Exception: An exception occurred while executing 'SELECT `uid`, `t3ver_oid`, `t3ver_state` FROM `tt_address` WHERE (`pid` = ?) AND (`t3ver_oid` IN (?, ?, ?, ?, ### HERE ARE AROUND 70.000 PLACEHOLDERS!### , ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)) AND (`t3ver_wsid` <> ?) ORDER BY `t3ver_state` DESC' with params [-1, 6877, ... ###HERE ARE AROUND 70.000 IDS!###, 0]:

Related issues

Related to TYPO3 Core - Bug #81993: 'Uncaught TYPO3 Exception' pops up when news records are localized Closed 2017-07-28

Associated revisions

Revision dfffc87d (diff)
Added by Sascha Egerer over 1 year ago

[BUGFIX] Process large uid lists in chunks in RelationHandler

Uid lists in the relation handler can be very big. To avoid exceeding
query limits like maximum number of placeholder per query or the max
allowed statement length these large lists are split into chunks of
safe length before processing.

Change-Id: I176acb85feb91c6162a77016c1918cf5a992625c
Resolves: #81555
References: #80875
Releases: master, 8.7
Reviewed-on: https://review.typo3.org/53193
Tested-by: TYPO3com <>
Reviewed-by: Manuel Selbach <>
Tested-by: Manuel Selbach <>
Reviewed-by: Christian Kuhn <>
Tested-by: Christian Kuhn <>

Revision e3a4bd77 (diff)
Added by Sascha Egerer over 1 year ago

[BUGFIX] Process large uid lists in chunks in RelationHandler

Uid lists in the relation handler can be very big. To avoid exceeding
query limits like maximum number of placeholder per query or the max
allowed statement length these large lists are split into chunks of
safe length before processing.

Change-Id: I176acb85feb91c6162a77016c1918cf5a992625c
Resolves: #81555
References: #80875
Releases: master, 8.7
Reviewed-on: https://review.typo3.org/55439
Tested-by: TYPO3com <>
Reviewed-by: Christian Kuhn <>
Tested-by: Christian Kuhn <>

History

#1 Updated by Gerrit Code Review over 2 years ago

  • Status changed from Accepted to Under Review

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#2 Updated by Gerrit Code Review over 2 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#3 Updated by Gerrit Code Review over 2 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#4 Updated by Gerrit Code Review over 2 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#5 Updated by Gerrit Code Review over 2 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#6 Updated by Gerrit Code Review over 2 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#7 Updated by Gerrit Code Review over 2 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#8 Updated by Gerrit Code Review over 2 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#9 Updated by Gerrit Code Review over 2 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#10 Updated by Gerrit Code Review over 2 years ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#11 Updated by Gerrit Code Review over 2 years ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#12 Updated by Gerrit Code Review over 2 years ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#13 Updated by Gerrit Code Review about 2 years ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#14 Updated by Gerrit Code Review about 2 years ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#15 Updated by Anu Bhuvanendran Nair about 2 years ago

  • Related to Bug #81993: 'Uncaught TYPO3 Exception' pops up when news records are localized added

#16 Updated by Gerrit Code Review almost 2 years ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#17 Updated by Gerrit Code Review almost 2 years ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#18 Updated by Gerrit Code Review over 1 year ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53193

#19 Updated by Gerrit Code Review over 1 year ago

Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/55439

#20 Updated by Sascha Egerer over 1 year ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#21 Updated by Gleb Levitin about 1 year ago

  • Related to Bug #85795: Too many records on reference index updating break TYPO3 added

#22 Updated by Gleb Levitin about 1 year ago

  • Related to deleted (Bug #85795: Too many records on reference index updating break TYPO3)

#23 Updated by Benni Mack about 1 year ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF