Bug #96042

Row update wizard may consume too much memory for tables with many records and content

Added by Sybille Peters 2 months ago. Updated about 2 months ago.

Status:
Under Review
Priority:
Should have
Assignee:
-
Category:
Install Tool
Target version:
-
Start date:
2021-11-20
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
12
PHP Version:
Tags:
memory hog, memory, out of memory, large-site
Complexity:
Is Regression:
Sprint Focus:

Description

Reproduce

I did not reproduce it exactly like this but it should be able to be reproducible:

1. Use a row updater
2. Apply this on a table with lots of rows and fields with lots of content (e.g. tt_content).

When the QueryBuilder Statement is constructed, it will out of memory.

See also patch set 1 where a test was used to create a number of rows: https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243/1

The test has now been removed because not really necessary for patch, just to check that out of memory is possible with current code.

Possible todos

1. Fix this. Can be done with chunking, but how to define the size of chunks?
2. Think about how to fix these things in general and find other places


Related issues

Related to TYPO3 Core - Epic #93547: Collection of problems with large sitesAccepted2021-02-19

Actions
#1

Updated by Sybille Peters 2 months ago

  • Related to Epic #93547: Collection of problems with large sites added
#2

Updated by Gerrit Code Review 2 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#3

Updated by Sybille Peters about 2 months ago

  • Tags changed from memory hog, memory, out of memory, slurping to memory hog, memory, out of memory, large-site
#4

Updated by Sybille Peters about 2 months ago

  • Description updated (diff)
#5

Updated by Gerrit Code Review about 2 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#6

Updated by Gerrit Code Review about 2 months ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#7

Updated by Gerrit Code Review about 2 months ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#8

Updated by Sybille Peters about 2 months ago

  • Description updated (diff)
#9

Updated by Gerrit Code Review about 2 months ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#10

Updated by Gerrit Code Review about 2 months ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

#11

Updated by Gerrit Code Review about 2 months ago

Patch set 7 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/72243

Also available in: Atom PDF