Bug #78915
closed
Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects submits of Extbase-Forms)
Added by Juergen Kussmann over 7 years ago. Updated over 5 years ago.
100%
Description
Hi,
we at AOE.com have found a critical performance-bug in Extbase-Reflection-Cache in all TYPO3-versions since 6.0.
The bug is:
The cached data of Extbase-Reflection-Cache (in DB-table cf_extbase_reflection) will not be stored completly/correct in cache-record. The data/part of property 'methodReflections' is NOT stored in the cache. The result is, that the Extbase-Reflection-Cache will be deleted/inserted each time, when the Extbase-Reflection will be used. This caused a major problem in one of our big TYPO3-installations (the DB-server crashed).
Our bugfix (see attachment) is very simple and should be used to patch all TYPO3-versions since 6.0.
Best regards,
Jürgen Kußmann
------------------------------------------------------------
German description of the bug:
Problem ist der Extbase-Reflection-Cache (in DB-Tabelle cf_extbase_reflection), welcher pro Extbase-basierter Extension ein Datensatz anlegt mit allen möglichen Informationen zu den PHP-Klassen, welche sich in einer Extension befinden. Dieser Reflection-Cache wird bei JEDEM Seitenaufruf (wo ein extbase-plugin verwendet wird) gelöscht und neu geschrieben. Grund dafür war, dass der Teil des Caches nicht korrekt in die DB gespeichert wurde, TYPO3 daher diesen „Teil des Caches“ immer wieder neu erstellt hat.
Viele Grüße,
Jürgen Kußmann
Files
| bugfix-add_property_methodReflections_to_reflection_cache.patch (798 Bytes) bugfix-add_property_methodReflections_to_reflection_cache.patch | Juergen Kussmann, 2016-12-08 09:43 |
Updated by Tymoteusz Motylewski over 7 years ago
- Status changed from New to Needs Feedback
little notes from the slack discussion on #typo3-cms-coredev
Claus could not reproduce the issue on master.
I can confirm that it doesn't contain methodReflections, but apart from that, cannot confirm your exact symptom that cache records are constantly rewritten (which is what I see as the core of your reported performance problem) [11:05] plenty ViewHelpers involved in rendering and parsing those absolutely calls on method reflections (when detecting render method arguments) [11:07] also asserted that reflection cache expiration time in DB is not changed by browsing through multiple modules and fe plugins (just to exclude that the page I profiled is an exception) [11:08] could you track down whether this only happens in specific cases, e.g. when calling a controller a certain way or bootstrapping plugins a certain way?
Jürgen or Valentin, can you reproduce the issue on master?
and provide more information like a stacktrace ?
Updated by Valentin Despa over 7 years ago
- Subject changed from Critical performance-bug in Extbase-Reflection-Cache in all TYPO3-versions since 6.0 to Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects non-cacheable controller actions
Updated by Juergen Kussmann over 7 years ago
- Subject changed from Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects non-cacheable controller actions to Critical performance bug in Extbase Reflection Cache in all TYPO3 versions since 6.0 (affects submits of Extbase-Forms)
Hi Tymoteusz Motylewski,
the bug occurs, when you submit an Extbase-Form and the Form is represented by an Extbase-Domain-Model. Than, Extbase checks if the Extbase-Domain-Model has a setter-method to set the values from the form into the Domain-Model.
Take a look at this stacktrace, which seems to help a lot to understand the logic:
0 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/TypeConverter/ObjectConverter.php(113): TYPO3\CMS\Extbase\Reflection\ReflectionService->hasMethod('Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', 'setDataStoring')
1 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/PropertyMapper.php(169): TYPO3\CMS\Extbase\Property\TypeConverter\ObjectConverter->getTypeOfChildProperty('Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', 'dataStoring', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration))
2 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Property/PropertyMapper.php(103): TYPO3\CMS\Extbase\Property\PropertyMapper->doMapping(Array, 'Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration), Array)
3 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/Argument.php(378): TYPO3\CMS\Extbase\Property\PropertyMapper->convert(Array, 'Aoe\EftShopCheckout\Domain\Model\Order\Item\ItemizedBill', Object(TYPO3\CMS\Extbase\Mvc\Controller\MvcPropertyMappingConfiguration))
4 /var/www/devbox/htdocs/typo3conf/ext/extbase_filter/Classes/MVC/Controller/Argument.php(61): TYPO3\CMS\Extbase\Mvc\Controller\Argument->setValue(Array)
5 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/AbstractController.php(423): Tx_ExtbaseFilter_MVC_Controller_Argument->setValue(Array)
6 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Controller/ActionController.php(151): TYPO3\CMS\Extbase\Mvc\Controller\AbstractController->mapRequestArgumentsToControllerArguments()
7 /var/www/devbox/htdocs/typo3conf/ext/eft_shop_checkout/Classes/Controller/AbstractController.php(85): TYPO3\CMS\Extbase\Mvc\Controller\ActionController->processRequest(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
8 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Dispatcher.php(69): Aoe\EftShopCheckout\Controller\AbstractController->processRequest(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
9 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Mvc/Web/FrontendRequestHandler.php(56): TYPO3\CMS\Extbase\Mvc\Dispatcher->dispatch(Object(TYPO3\CMS\Extbase\Mvc\Web\Request), Object(TYPO3\CMS\Extbase\Mvc\Web\Response))
10 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Core/Bootstrap.php(195): TYPO3\CMS\Extbase\Mvc\Web\FrontendRequestHandler->handleRequest()
11 /var/www/devbox/htdocs/typo3_src/typo3/sysext/extbase/Classes/Core/Bootstrap.php(184): TYPO3\CMS\Extbase\Core\Bootstrap->handleRequest()
12 [internal function]: TYPO3\CMS\Extbase\Core\Bootstrap->run('', Array)
13 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/ContentObjectRenderer.php(6706): call_user_func_array(Array, Array)
14 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/UserContentObject.php(41): TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer->callUserFunction('TYPO3\\CMS\\Extba...', Array, '')
15 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/ContentObject/ContentObjectRenderer.php(904): TYPO3\CMS\Frontend\ContentObject\UserContentObject->render(Array)
16 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3733): TYPO3\CMS\Frontend\ContentObject\ContentObjectRenderer->USER
17 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3681): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->INTincScript_process(Array)
18 /var/www/devbox/htdocs/typo3_src/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php(3648): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->recursivelyReplaceIntPlaceholdersInContent()
19 /var/www/devbox/htdocs/typo3_src/typo3/sysext/cms/tslib/index_ts.php(232): TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->INTincScript()
20 /var/www/devbox/htdocs/typo3_src/index.php(28): require('/var/www/congst...')
21 {main}
We use TYPO3 6.2.26...so the bug is occured in TYPO3 6.2.26. But i have seen, that the bug theoretically also must exist in all TYPO3-versions since 6.0.
Best Regards,
Jürgen Kußmann
Updated by Valentin Despa over 7 years ago
This most likely affects only extensions having non-cacheable controller actions and forms which you can POST.
Steps to reproduce in FE:
- install an Extbase extension
- insert and configure the plugin in FE
- call the page in FE (without having a BE session)
- after the page is rendered, you will see a new entry in the table cf_extbase_reflection
mysql> select id,identifier,expires from cf_extbase_reflection WHERE identifier = 'ReflectionData_XXXX';
| id | identifier | expires |
| 10 | ReflectionData_XXXX | 2145909600 |
- Delete the page cache (for the page where the plugin is located) => The reflection cache will still be in the database. This step is probably not necessary if the extension has non-cacheable controller actions
- Reload the FE page
- You will get a new entry in the reflection cache, which was actually not expected
mysql> select id,identifier,expires from cf_extbase_reflection WHERE identifier = 'ReflectionData_XXXX';
| id | identifier | expires |
| 11 | ReflectionData_XXXX | 2145909600 |
Updated by Tymoteusz Motylewski over 7 years ago
- Status changed from Needs Feedback to New
Updated by Gerrit Code Review over 7 years ago
- Status changed from New to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50913
Updated by Gerrit Code Review over 7 years ago
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50913
Updated by Tymoteusz Motylewski over 7 years ago
just found an easy way to reproduce the issue:
1. Go to the "Log" module
2. check table cf_extbase_reflection for a record with "ReflectionData_Belog" identifier
3. remember the id of the record, remember the content of the "content" column
4. change number of items per page in the Log module (e.g. from 25 to 100)
5. check the DB again,
6. the id of the ReflectionData_Belog record is now by one bigger than before, and the content column has the same value as one from step 3.
--
6. apply the patch
7. clear the cache
8. redo steps 1. to 5.
9. the id of the ReflectionData_Belog record will stay the same
Updated by Gerrit Code Review over 7 years ago
Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50929
Updated by Anonymous over 7 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset a5c1e222bf9858f5a44af1007cd9e1677b9a534d.
Updated by Gerrit Code Review over 7 years ago
- Status changed from Resolved to Under Review
Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50930
Updated by Anonymous over 7 years ago
- Status changed from Under Review to Resolved
Applied in changeset b4f2dc41392ad56a9e663e48f0cff1f77a3a1192.
Updated by Gerrit Code Review over 7 years ago
- Status changed from Resolved to Under Review
Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953
Updated by Helmut Hummel over 7 years ago
- % Done changed from 100 to 90
- Complexity deleted (
no-brainer)
Updated by Gerrit Code Review over 7 years ago
Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953
Updated by Gerrit Code Review over 7 years ago
Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953
Updated by Gerrit Code Review over 7 years ago
Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50953
Updated by Gerrit Code Review over 7 years ago
Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50980
Updated by Helmut Hummel over 7 years ago
- Status changed from Under Review to Resolved
- % Done changed from 90 to 100
Applied in changeset 5b7c3f6f9abeb04a0e8803b82dea5a002fd69df0.
Updated by Gerrit Code Review over 7 years ago
- Status changed from Resolved to Under Review
Patch set 1 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984
Updated by Gerrit Code Review over 7 years ago
Patch set 2 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984
Updated by Helmut Hummel over 7 years ago
- Status changed from Under Review to Resolved
Applied in changeset 4983935849938f12ba9b5782b3121b0edc8d4550.
Updated by Gerrit Code Review over 7 years ago
- Status changed from Resolved to Under Review
Patch set 3 for branch TYPO3_6-2 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/50984
Updated by Helmut Hummel over 7 years ago
- Status changed from Under Review to Resolved
Applied in changeset 5124e88a7f4319b593fbf2dbfac767afb7dc0435.
Updated by