Bug #63675

Bug #63692: Memory consumption while bulk inserting

High memory consumption for softRefs in ReferenceIndex->getRelations  while bulk inserting

Added by Stephan Großberndt over 7 years ago. Updated over 7 years ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Performance
Target version:
Start date:
2014-12-08
Due date:
% Done:

80%

Estimated time:
TYPO3 Version:
6.2
PHP Version:
Tags:
Complexity:
Is Regression:
No
Sprint Focus:
Stabilization Sprint

Description

For every field of every record ReferenceIndex->getRelations executes:

                // Soft References:
                if (strlen($value) && ($softRefs = BackendUtility::explodeSoftRefParserList($conf['softref']))) {

The return value of explodeSoftRefParserList is not dependent on the record itself, but only table and field. Introducing a cache for softRefs

public function getRelations($table, $row, $onlyField = '') {
    static $softRef_cache = array();

    [...]

                // Soft References:
                if (isset($softRef_cache[$table . 'f' . $field])) {
                    $softRefs = $softRef_cache[$table . 'f' . $field];
                } else {
                    $softRefs = BackendUtility::explodeSoftRefParserList($conf['softref']);
                    $softRef_cache[$table . 'f' . $field] = $softRefs;
                }
                if (strlen($value) && $softRefs) {

reduces when inserting 6500 records:

  • 8 instead of 51,024 calls to explodeSoftRefParserList
  • 0.52 seconds faster
  • 4kb used instead of 2,5MB
#1

Updated by Philipp Gampe over 7 years ago

  • Parent task set to #63692
#2

Updated by Gerrit Code Review over 7 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#3

Updated by Philipp Gampe over 7 years ago

  • Subject changed from High memory consumption in ReferenceIndex->getRelations while bulk inserting to High memory consumption for softRefs in ReferenceIndex->getRelations  while bulk inserting
#4

Updated by Gerrit Code Review over 7 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#5

Updated by Gerrit Code Review over 7 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#6

Updated by Gerrit Code Review over 7 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#7

Updated by Gerrit Code Review over 7 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#8

Updated by Gerrit Code Review over 7 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#9

Updated by Gerrit Code Review over 7 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#10

Updated by Gerrit Code Review over 7 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#11

Updated by Gerrit Code Review over 7 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#12

Updated by Gerrit Code Review over 7 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#13

Updated by Gerrit Code Review over 7 years ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35190

#14

Updated by Stephan Großberndt over 7 years ago

  • Target version changed from next-patchlevel to 6.2.12
  • Sprint Focus set to Stabilization Sprint
#15

Updated by Gerrit Code Review over 7 years ago

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/38662

#16

Updated by Stephan Großberndt over 7 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
#17

Updated by Xavier Perseguers over 7 years ago

  • Status changed from Resolved to Under Review
  • % Done changed from 100 to 80

Still not ready for 6.2

#18

Updated by Gerrit Code Review over 7 years ago

Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/38662

#19

Updated by Andreas Fernandez over 7 years ago

  • Status changed from Under Review to Closed

Merged

Also available in: Atom PDF