Bug #32756

Massive Memory Leak in 4.5.8+ / 4.6

Added by Ronny Vorpahl almost 10 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
Performance
Target version:
-
Start date:
2011-12-21
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
4.5
PHP Version:
Tags:
Complexity:
hard
Is Regression:
Sprint Focus:

Description

This Bug is caused by the fix for #15984

... due to patched tslib_fe::checkPageGroupAccess used by tslib_cObj::getTreeList

When a treelist is built by tslib_cObj::getTreeList, the access rights are checked using tslib_fe::checkPageGroupAccess.
Due to the Patch #15984 the rootline is now loaded for EVERY PAGE IN THIS TREELIST, just to check the rights of the parent pages.
It seems the PHP Garbage collection fails in this case causing a memory limit exception.
The implementation of t3lib_pageSelect::getRootLine itself seems to be alright.

An example (limited test case):
Creating a full treelist of 537 pages in 4 levels (1+14+138+384 pages) resulted in over 10330 calls of t3lib_DB::execSelectQuery and 10330 t3lib_DB::sql_fetch_assoc just for t3lib_pageSelect::getRootLine. The memory limit had to be raised to 1GB or the call failed with an exception.

This case was experienced in multiple PHP 5.3 environments.

xhprof PHP-Profiler including compared callgraphs are available.
Two solution(s) are also on their way, but are still work in progress.

Please contact us for further details ...


Files

patch_32756.diff (2.25 KB) patch_32756.diff Ronny Vorpahl, 2011-12-22 15:20
xhprof_PHP-Profiler_getTreeList_4.5.7.htm (34.7 KB) xhprof_PHP-Profiler_getTreeList_4.5.7.htm Ronny Vorpahl, 2011-12-22 15:20
xhprof_PHP-Profiler_getTreeList_4.5.9.htm (40.5 KB) xhprof_PHP-Profiler_getTreeList_4.5.9.htm Ronny Vorpahl, 2011-12-22 15:20

Related issues

Related to TYPO3 Core - Bug #34800: Page tree not respondingClosedMathias Schreiber2012-03-13

Actions
Follows TYPO3 Core - Bug #15984: menu.showAccessRestrictedPages doesn't replace link for "include subpages"Accepted2006-04-05

Actions
#1

Updated by Ronny Vorpahl almost 10 years ago

  • Assignee set to Steffen Kamper
#2

Updated by Steffen Gebert almost 10 years ago

  • Assignee deleted (Steffen Kamper)
  • Priority changed from Must have to Should have
  • Target version deleted (4.5.11)

Thanks for your report, Ronny. Please do not assign issues to anybody - especially not to inactive people!

#3

Updated by Markus Klein almost 10 years ago

And can you please simply post the "further details" here, so that everybody in the community has access to these details?

#4

Updated by Ronny Vorpahl almost 10 years ago

Some further Details:

The correct call order is:
- tslib_cObj::getTreeList
- tslib_fe::checkEnableFields
- tslib_fe::checkPageGroupAccess
- t3lib_pageSelect::getRootLine
- t3lib_DB::execSelectQuery + t3lib_DB::sql_fetch_assoc

The appended patch bypasses the getRootLine-functionality for getTreeList by using own functionality - it behaves now like 4.5.7.

The appended xhprof PHP-Profiler show the behavior of getTreeList using the unpatched Typo3-sources 4.5.7 and 4.5.9

#5

Updated by Helmut Hummel almost 10 years ago

  • Category changed from Pagetree to Backend API
#6

Updated by Helmut Hummel almost 10 years ago

  • Category changed from Backend API to Performance
  • Priority changed from Should have to Must have

#15984 introduced a performance nightmare. I would rather revert this "fix" until there is a solution which does not have such a huge impact.

#7

Updated by Jigal van Hemert almost 10 years ago

The implementation of getTreeList is a bit illogical: it gets the child elements recursively, so it goes from top to bottom. Yet for each element it checks the rootline for access properties (thus it goes back from bottom to top!).
If the recursion is removed the loop for the sublevels can check the access from its own data. Only the starting page needs to check the rootline for inherited properties.

#8

Updated by Andreas Otto † over 9 years ago

Helmut Hummel wrote:

#15984 introduced a performance nightmare. I would rather revert this "fix" until there is a solution which does not have such a huge impact.

Today 4.5.12 was released and the performance nightmare still exists.

What else than reverting #15984 can be done?

And please remember that 4.5 is a LTS release and should be a bit more stable than it currently is.

#9

Updated by Gerrit Code Review over 9 years ago

  • Status changed from New to Under Review

Patch set 2 for branch TYPO3_4-5 has been pushed to the review server.
It is available at http://review.typo3.org/9552

#10

Updated by Gerrit Code Review over 9 years ago

Patch set 1 for branch TYPO3_4-6 has been pushed to the review server.
It is available at http://review.typo3.org/9730

#11

Updated by Gerrit Code Review over 9 years ago

Patch set 1 for branch TYPO3_4-7 has been pushed to the review server.
It is available at http://review.typo3.org/9731

#12

Updated by Gerrit Code Review over 9 years ago

Patch set 1 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/9732

#13

Updated by Christian Kuhn over 9 years ago

The patch of #15984 is reverted now in all branches.

#14

Updated by Helmut Hummel over 9 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
#15

Updated by Benni Mack about 3 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF