Task #54251

Epic #55070: Workpackages

Epic #55065: WP: Overall System Performance (Backend and Frontend)

Bug #52949: Speed decrease since 4.5

GeneralUtility::array_merge_recursive_overrule has bad performance

Added by Markus Klein almost 6 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
Performance
Target version:
Start date:
2013-12-06
Due date:
% Done:

100%

TYPO3 Version:
6.2
PHP Version:
5.5
Tags:
Complexity:
easy
Sprint Focus:

Description

According to my profiling activities, GeneralUtility::array_merge_recursive_overrule has really bad performance. (~ 111 ms own time for a BE call, that ranks number 3 in the list of the most expensive calls based on own time sorting.)
Some tests show that this is due to many array copy actions during the recursive calls (foreach constructs with $val being a copy of possibly large subarrays).

I propose to optimize this method by:
  1. making the first parameter by reference thus working on the original array
  2. move the function from GeneralUtility to ArrayUtility
  3. deprecate the function in GeneralUtility, but still work on a copy of the first array there to preserve backwards compatibility
  4. change all calls in core to use the new method

This brings the own time of the function down to 42 ms on my test system, getting the function out of the expensive calls in TYPO3 CMS.


Related issues

Related to TYPO3 Core - Task #57499: Improve deprecation message of GeneralUtitlity::array_merge_recursive_overrule() Closed 2014-04-01
Related to TYPO3 Core - Bug #63812: Lower memory use in ArrayUtil->mergeRecursiveWithOverrule Closed 2014-12-12

Associated revisions

Revision 2917b074 (diff)
Added by Markus Klein almost 6 years ago

[TASK] Improve performance of array_merge_recursive_overrule

The method GeneralUtility::array_merge_recursive_overrule()
always works on a copy of the given array(s). This is highly
inefficient when it comes to really big arrays and recursion.

This patches moves the functionality into the class ArrayUtility
and changes the behaviour to use a reference to the
original array.
All calls in the core are adjusted accordingly.

Furthermore we deprecate the method in GeneralUtility and
preserve backward compatibility.

Resolves: #54251
Releases: 6.2
Change-Id: I5499905593c2124897de5998be985e546a3d05ee
Reviewed-on: https://review.typo3.org/25986
Reviewed-by: Michiel Roos
Tested-by: Michiel Roos
Reviewed-by: Wouter Wolters
Tested-by: Wouter Wolters
Reviewed-by: Ernesto Baschny
Tested-by: Ernesto Baschny

History

#1 Updated by Markus Klein almost 6 years ago

  • Parent task set to #52949

#2 Updated by Gerrit Code Review almost 6 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#3 Updated by Gerrit Code Review almost 6 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#4 Updated by Gerrit Code Review almost 6 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#5 Updated by Gerrit Code Review almost 6 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#6 Updated by Gerrit Code Review almost 6 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#7 Updated by Gerrit Code Review almost 6 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#8 Updated by Gerrit Code Review almost 6 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#9 Updated by Gerrit Code Review almost 6 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#10 Updated by Gerrit Code Review almost 6 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#11 Updated by Gerrit Code Review almost 6 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/25986

#12 Updated by Markus Klein almost 6 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#13 Updated by Riccardo De Contardi about 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF