Project

General

Profile

Actions
Copy link

Bug #78915

closed

Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects submits of Extbase-Forms)

Added by Juergen Kussmann almost 8 years ago. Updated about 6 years ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
Caching
Target version:
Candidate for patchlevel
Start date:
2016-12-08
Due date:
% Done:

100%

Estimated time:
TYPO3 Version:
8
PHP Version:
Tags:
critical performance bug extbase reflection cache
Complexity:
Is Regression:
No
Sprint Focus:

Description

Hi,
we at AOE.com have found a critical performance-bug in Extbase-Reflection-Cache in all TYPO3-versions since 6.0.

The bug is:
The cached data of Extbase-Reflection-Cache (in DB-table cf_extbase_reflection) will not be stored completly/correct in cache-record. The data/part of property 'methodReflections' is NOT stored in the cache. The result is, that the Extbase-Reflection-Cache will be deleted/inserted each time, when the Extbase-Reflection will be used. This caused a major problem in one of our big TYPO3-installations (the DB-server crashed).

Our bugfix (see attachment) is very simple and should be used to patch all TYPO3-versions since 6.0.

Best regards,
Jürgen Kußmann

------------------------------------------------------------
German description of the bug:
Problem ist der Extbase-Reflection-Cache (in DB-Tabelle cf_extbase_reflection), welcher pro Extbase-basierter Extension ein Datensatz anlegt mit allen möglichen Informationen zu den PHP-Klassen, welche sich in einer Extension befinden. Dieser Reflection-Cache wird bei JEDEM Seitenaufruf (wo ein extbase-plugin verwendet wird) gelöscht und neu geschrieben. Grund dafür war, dass der Teil des Caches nicht korrekt in die DB gespeichert wurde, TYPO3 daher diesen „Teil des Caches“ immer wieder neu erstellt hat.

Viele Grüße,
Jürgen Kußmann

Files

bugfix-add_property_methodReflections_to_reflection_cache.patch (798 Bytes) bugfix-add_property_methodReflections_to_reflection_cache.patch Juergen Kussmann, 2016-12-08 09:43

Related issues 1 (0 open — 1 closed)

Precedes TYPO3 Core - Bug #78977: Fatal error: Failed to retrieve the reflection object in Extbase Reflection ServiceClosed2016-12-13

Actions
Actions
Copy link
 #1

Updated by Tymoteusz Motylewski almost 8 years ago

  • Status changed from New to Needs Feedback

little notes from the slack discussion on #typo3-cms-coredev
Claus could not reproduce the issue on master.

I can confirm that it doesn't contain methodReflections, but apart from that, cannot confirm your exact symptom that cache records are constantly rewritten (which is what I see as the core of your reported performance problem)

[11:05]  
plenty ViewHelpers involved in rendering and parsing those absolutely calls on method reflections (when detecting render method arguments)

[11:07]  
also asserted that reflection cache expiration time in DB is not changed by browsing through multiple modules and fe plugins (just to exclude that the page I profiled is an exception)

[11:08]  
could you track down whether this only happens in specific cases, e.g. when calling a controller a certain way or bootstrapping plugins a certain way?

Jürgen or Valentin, can you reproduce the issue on master?
and provide more information like a stacktrace ?

Actions
Copy link
 #2

Updated by Valentin Despa almost 8 years ago

  • Subject changed from Critical performance-bug in Extbase-Reflection-Cache in all TYPO3-versions since 6.0 to Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects non-cacheable controller actions
Actions
Copy link
 #3

Updated by Juergen Kussmann almost 8 years ago

  • Subject changed from Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects non-cacheable controller actions to Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects submits of Extbase-Forms)

Hi Tymoteusz Motylewski,

the bug occurs, when you submit an Extbase-Form and the Form is represented by an Extbase-Domain-Model. Than, Extbase checks if the Extbase-Domain-Model has a setter-method to set the values from the form into the Domain-Model.

Take a look at this stacktrace, which seems to help a lot to understand the logic:
0 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/TypeConverter/ObjectConverter.php(113): TYPO3\CMS\Extbase\Reflection\ReflectionService->hasMethod('Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', 'setDataStoring')
1 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/PropertyMapper.php(169): TYPO3\CMS\Extbase\Property\TypeConverter\ObjectConverter->getTypeOfChildProperty('Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', 'dataStoring', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration))
2 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/PropertyMapper.php(103): TYPO3\CMS\Extbase\Property\PropertyMapper->doMapping(Array, 'Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration), Array)
3 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/Argument.php(378): TYPO3\CMS\Extbase\Property\PropertyMapper->convert(Array, 'Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration))
4 /var/www/devbox/htdocs/typo3conf/ext/extbase_filter/Classes/MVC/Controller/Argument.php(61): TYPO3\CMS\Extbase\Mvc\Controller\Argument->setValue(Array)
5 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/AbstractController.php(423): Tx_ExtbaseFilter_MVC_Controller_Argument->setValue(Array)
6 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/ActionController.php(151): TYPO3\CMS\Extbase\Mvc\Controller\AbstractController->mapRequestArgumentsToControllerArguments()
7 /var/www/devbox/htdocs/typo3conf/ext/eft_shop_checkout/Classes/Controller/AbstractController.php(85): TYPO3\CMS\Extbase\Mvc\Controller\ActionController->processRequest(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
8 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Dispatcher.php(69): Aoe\EftShopCheckout\Controller\AbstractController->processRequest(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
9 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Web/FrontendRequestHandler.php(56): TYPO3\CMS\Extbase\Mvc\Dispatcher->dispatch(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
10 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Core/Bootstrap.php(195): TYPO3\CMS\Extbase\Mvc\Web\FrontendRequestHandler->handleRequest()
11 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Core/Bootstrap.php(184): TYPO3\CMS\Extbase\Core\Bootstrap->handleRequest()
12 [internal function]: TYPO3\CMS\Extbase\Core\Bootstrap->run('', Array)
13 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/ContentObjectRenderer.php(6706): call_user_func_array(Array, Array)
14 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/UserContentObject.php(41): TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer->callUserFunction('TYPO3\\CMS\\Extba...', Array, '')
15 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/ContentObjectRenderer.php(904): TYPO3\CMS\Frontend\ContentObject\UserContentObject->render(Array)
16 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3733): TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer->USER
17 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3681): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->INTincScript_process(Array)
18 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3648): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->recursivelyReplaceIntPlaceholdersInContent()
19 /var/www/devbox/htdocs/typo3_src/typo3/sysext/cms/tslib/index_ts.php(232): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->INTincScript()
20 /var/www/devbox/htdocs/typo3_src/index.php(28): require('/var/www/congst...')
21 {main}

We use TYPO3 6.2.26...so the bug is occured in TYPO3 6.2.26. But i have seen, that the bug theoretically also must exist in all TYPO3-versions since 6.0.

Best Regards,
Jürgen Kußmann

Actions
Copy link
 #4

Updated by Valentin Despa almost 8 years ago

This most likely affects only extensions having non-cacheable controller actions and forms which you can POST.

Steps to reproduce in FE:

- install an Extbase extension
- insert and configure the plugin in FE
- call the page in FE (without having a BE session)
- after the page is rendered, you will see a new entry in the table cf_extbase_reflection

mysql> select id,identifier,expires from cf_extbase_reflection WHERE identifier = 'ReflectionData_XXXX';

id identifier expires
10 ReflectionData_XXXX 2145909600

- Delete the page cache (for the page where the plugin is located) => The reflection cache will still be in the database. This step is probably not necessary if the extension has non-cacheable controller actions
- Reload the FE page
- You will get a new entry in the reflection cache, which was actually not expected

mysql> select id,identifier,expires from cf_extbase_reflection WHERE identifier = 'ReflectionData_XXXX';

id identifier expires
11 ReflectionData_XXXX 2145909600
Actions
Copy link
 #5

Updated by Tymoteusz Motylewski almost 8 years ago

  • Status changed from Needs Feedback to New
Actions
Copy link
 #6

Updated by Gerrit Code Review almost 8 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50913

Actions
Copy link
 #7

Updated by Gerrit Code Review almost 8 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50913

Actions
Copy link
 #8

Updated by Tymoteusz Motylewski almost 8 years ago

just found an easy way to reproduce the issue:
1. Go to the "Log" module
2. check table cf_extbase_reflection for a record with "ReflectionData_Belog" identifier
3. remember the id of the record, remember the content of the "content" column
4. change number of items per page in the Log module (e.g. from 25 to 100)
5. check the DB again,
6. the id of the ReflectionData_Belog record is now by one bigger than before, and the content column has the same value as one from step 3.

--
6. apply the patch
7. clear the cache
8. redo steps 1. to 5.
9. the id of the ReflectionData_Belog record will stay the same

Actions
Copy link
 #9

Updated by Gerrit Code Review almost 8 years ago

Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50929

Actions
Copy link
 #10

Updated by Anonymous almost 8 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
Actions
Copy link
 #11

Updated by Gerrit Code Review almost 8 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50930

Actions
Copy link
 #12

Updated by Anonymous almost 8 years ago

  • Status changed from Under Review to Resolved
Actions
Copy link
 #13

Updated by Gerrit Code Review almost 8 years ago

  • Status changed from Resolved to Under Review

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953

Actions
Copy link
 #14

Updated by Helmut Hummel almost 8 years ago

  • % Done changed from 100 to 90
  • Complexity deleted (no-brainer)
Actions
Copy link
 #15

Updated by Gerrit Code Review almost 8 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953

Actions
Copy link
 #16

Updated by Gerrit Code Review almost 8 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953

Actions
Copy link
 #17

Updated by Gerrit Code Review almost 8 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953

Actions
Copy link
 #18

Updated by Gerrit Code Review almost 8 years ago

Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50980

Actions
Copy link
 #19

Updated by Helmut Hummel almost 8 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 90 to 100
Actions
Copy link
 #20

Updated by Gerrit Code Review almost 8 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984

Actions
Copy link
 #21

Updated by Gerrit Code Review almost 8 years ago

Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984

Actions
Copy link
 #22

Updated by Helmut Hummel almost 8 years ago

  • Status changed from Under Review to Resolved
Actions
Copy link
 #23

Updated by Gerrit Code Review almost 8 years ago

  • Status changed from Resolved to Under Review

Patch set 3 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984

Actions
Copy link
 #24

Updated by Helmut Hummel almost 8 years ago

  • Status changed from Under Review to Resolved
Actions
Copy link
 #25

Updated by Benni Mack about 6 years ago

  • Status changed from Resolved to Closed
Actions
Copy link

Also available in: Atom PDF