Bug #63666
closedBug #63692: Memory consumption while bulk inserting
High memory consumption in BackendUtility->getTCAtypes while bulk inserting
100%
Description
DataHandler->fillInFieldArray calls
$types_fieldConfig = BackendUtility::getTCAtypes($table, $currentRecord);for each record to insert. In each call the type of the current record is checked and the complete types configuration is parsed again.
static public function getTCAtypes($table, $rec, $useFieldNameAsKey = 0) { if ($GLOBALS['TCA'][$table]) { // Get type value: $fieldValue = self::getTCAtypeValue($table, $rec); // Get typesConf $typesConf = $GLOBALS['TCA'][$table]['types'][$fieldValue]; // Get fields list and traverse it $fieldList = explode(',', $typesConf['showitem']); $altFieldList = array(); // Traverse fields in types config and parse the configuration into a nice array: foreach ($fieldList as $k => $v) { [...] } if ($useFieldNameAsKey) { $fieldList = $altFieldList; } // Return array: return $fieldList; }
A new static cache using tablename, type and parameter as key improves performance and memory consumption:
static public function getTCAtypes($table, $rec, $useFieldNameAsKey = 0) { static $tcaTableTypeConf_cache = array(); if ($GLOBALS['TCA'][$table]) { // Get type value: $fieldValue = self::getTCAtypeValue($table, $rec); if ($tcaTableTypeConf_cache[$table . 'type' . $fieldValue . 'fnk' . $useFieldNameAsKey]) { return $tcaTableTypeConf_cache[$table . 'type' . $fieldValue . 'fnk' . $useFieldNameAsKey]; } // Get typesConf $typesConf = $GLOBALS['TCA'][$table]['types'][$fieldValue]; // Get fields list and traverse it $fieldList = explode(',', $typesConf['showitem']); $altFieldList = array(); // Traverse fields in types config and parse the configuration into a nice array: foreach ($fieldList as $k => $v) { [...] } if ($useFieldNameAsKey) { $fieldList = $altFieldList; } $tcaTableTypeConf_cache[$table . 'type' . $fieldValue . 'fnk' . $useFieldNameAsKey] = $fieldList; // Return array: return $fieldList; }For bulk insert of 6500 typeless (only type '1') records:
- 465000 calls less
- 3.3 seconds faster
- memory usage of explode()-Function down from 128MB to 58MB
Updated by Gerrit Code Review almost 10 years ago
- Status changed from New to Under Review
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35185
Updated by Gerrit Code Review almost 10 years ago
Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35521
Updated by Stephan Großberndt almost 10 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset e0467960abe3e269c8bbc38135ff0c7245c2b5b3.