TYPO3 Forge: Issueshttp://forge.typo3.org/http://forge.typo3.org/themes/typo3_forge/favicon/favicon.png?17058661692023-07-27T13:40:23ZTYPO3 Forge
Redmine TYPO3 Core - Task #101461 (Under Review): Use php cache frontend for ClassSchema cache entrieshttp://forge.typo3.org/issues/1014612023-07-27T13:40:23ZAlexander SchnitzlerTYPO3 Core - Task #101459 (Under Review): Introduce type declarations in AbstractFilehttp://forge.typo3.org/issues/1014592023-07-27T10:47:46ZAlexander SchnitzlerTYPO3 Core - Task #101392 (Under Review): Remove (get/set)Identifier() from AbstractFilehttp://forge.typo3.org/issues/1013922023-07-19T18:26:15ZAlexander SchnitzlerTYPO3 Core - Task #101292 (New): Introduce type declarations in PropertyMappingConfigurationInter...http://forge.typo3.org/issues/1012922023-07-07T16:27:14ZAlexander SchnitzlerTYPO3 Core - Task #101267 (Under Review): Introduce parts of new ClassSchemaFactoryhttp://forge.typo3.org/issues/1012672023-07-06T15:37:43ZAlexander SchnitzlerTYPO3 Core - Story #100973 (New): VariableProcessor performancehttp://forge.typo3.org/issues/1009732023-06-08T06:43:14ZAlexander Schnitzler
<a name="Introduction"></a>
<h2 >Introduction<a href="#Introduction" class="wiki-anchor">¶</a></h2>
<p>Well, these days I faced a performance issue in one of my sites and I'd like to share my findings as a foundation for performance improvements.</p>
<p>First of all a short description of my use case:<br />A customer of mine has a site (basically an app) which was split into around 30 plugins with switchable controller actions. The navigation was built with actual pages in the backend, each site containing one of those plugins.</p>
<p>During the upgrade to 12, we decided to get rid of all pages, integrate all plugins into one and solve the removal of switchable controller actions with routing.<br />The routing configuration for that plugin contains 263 routes. So this is not the everyday use case but it's what out there in the wild.</p>
<p>However, after the upgrade we realized the site was horribly slow compared to the 8.7 instance although switching from PHP 7.3 to 8.2. So I knew something was off and a digged into it with blackfire.</p>
<p>I quickly realized the culprit: <code>\TYPO3\CMS\Extbase\Routing\ExtbasePluginEnhancer::getVariant()</code></p>
<hr />
<a name="Findings"></a>
<h2 >Findings<a href="#Findings" class="wiki-anchor">¶</a></h2>
<p>To understand the issue, we need to understand how the routing works. First, we have a rootline like <code>/page/controller/action</code>. In this case the rootline is <code>/page/</code> containing of both <code>/page/</code> and <code>/</code>. Now, <strong>without</strong> plugin enhancer option <code>limitToPages</code> set to <code>/page/</code>, the enhancer does the following during route matching (a process that cannot be cached):<br />Take all pages of the rootline (2 in this case) and multiply those by the amount of configured routes (263 in this case). That means, 526 route variants aka candidates are created.</p>
<p>Now, the amount of variants is not the main issue. I was able to half the number by using <code>limitToPages</code> and have route variants only be calculated for <code>/page/</code> but the problem was still huge. Now, what I describe here is just the url matching, not the generation. In my case, the generation is the much bigger issue but the matching is a process that isn't cachable. Whenever TYPO3 receives a request to <code>/page/controller/action</code>, it needs to match it against the configured rules. Let me be clear on this. Part of this is cachable but at the moment it is not.</p>
<p>Link generation can often be cached. Or let me rephrase: That's often user land code and the user needs to take care of this, not the framework. But also here, parts are cachable.</p>
<p>So let's concentrate on actual numbers. I have <strong>26.126</strong> calls to <code>\TYPO3\CMS\Extbase\Routing\ExtbasePluginEnhancer::getVariant()</code>, the majority (96.8%) coming from url generation, not matching. That's 92% of all wall time. The most problematic path is <code>getVariant()</code> calling <code>applyRouteAspects()</code> which calls <code>deflateKeys()</code> and <code>getVariant()</code> calling <code>deflateKeys()</code> directly. So there are 51.565 calls on that single method. Now things are getting wild. Those 51.565 calls, result in 810.379 calls to <code>deflateValues()</code>.</p>
<p>Now I had two options. Should I try to find issues on upper or lower levels? Checking the upper level means, going back to <code>getVariant()</code> and understand why it's called 26.126 times. I do generate a lot of url's on my site and given the amount of configured routes I still thought something was off there but I decided to check the lower level and seek room for optimisation there.</p>
<a name="deflateValues"></a>
<h3 >deflateValues()<a href="#deflateValues" class="wiki-anchor">¶</a></h3>
<p>So, what did I do to check that method for optimisations? My main questions are:</p>
<p>- Is the set of method arguments small enough to create a cache identifier from it?<br />- Is further execution side effect free? This means, is the method deterministic and only dependent on the set of incoming parameters?</p>
<p>Both is the case!<br />I hashed the set of arguments and counted the amount of calls to that method with the same set of arguments. And boom. Didn't expect this. There were fewer than 100 different argument variants. I quickly built a runtime cache on the class (static properties) and the result are striking. -810.379 calls to <code>addNestedValue</code>, -810.379 calls to <code>addHash</code>. Around the same amount of calls less to <code>str_contains</code> and <code>preg_match</code>. That's -40% in CPU and wall time.</p>
<hr />
<p>to be continued</p> TYPO3 Core - Task #100376 (Under Review): Introduce type declarations in ClassSchema related classeshttp://forge.typo3.org/issues/1003762023-04-01T08:34:54ZAlexander SchnitzlerTYPO3 Core - Task #100277 (Under Review): Introduce DatabaseType enumhttp://forge.typo3.org/issues/1002772023-03-23T18:52:27ZAlexander SchnitzlerTYPO3 Core - Task #100136 (New): ObjectConverter should infer property types from properties, not...http://forge.typo3.org/issues/1001362023-03-10T08:34:04ZAlexander SchnitzlerTYPO3 Core - Task #98143 (Under Review): Simplify several elseif blocks with early returnshttp://forge.typo3.org/issues/981432022-08-14T09:09:22ZAlexander SchnitzlerTYPO3 Core - Task #98140 (Under Review): Add property type declarations with rectorhttp://forge.typo3.org/issues/981402022-08-13T13:33:02ZAlexander SchnitzlerTYPO3 Core - Feature #92976 (Under Review): Introduce new uri builder classeshttp://forge.typo3.org/issues/929762020-12-02T14:31:42ZAlexander SchnitzlerTYPO3 Core - Bug #90214 (New): TCA maxitems is ignoredhttp://forge.typo3.org/issues/902142020-01-27T12:59:15ZFlorian Rivalflorian.typo3@oktopuce.fr
<p>Hi,</p>
<p>When setting TCA maxitems to 1 for INLINE relations, and update the record from FE form, maxitems is ignored, many relations are created in sys_file_reference where there should be only one.</p>
<p>New relation is inserted but old relation is not deleted.</p>
<p>Sample TCA :</p>
<pre>
'cv' => [
'exclude' => false,
'label' => 'cv',
'config' => \TYPO3\CMS\Core\Utility\ExtensionManagementUtility::getFileFieldTCAConfig(
'cv',
[
'minitems' => 0,
'maxitems' => 1,
'foreign_match_fields' => [
'fieldname' => 'cv',
'tablenames' => 'fe_users',
'table_local' => 'sys_file',
]
],
'pdf'
),
]
</pre>
<p>And update with :</p>
<pre><code class="php syntaxhl" data-language="php"><span class="nv">$this</span><span class="o">-></span><span class="n">userRepository</span><span class="o">-></span><span class="nf">update</span><span class="p">(</span><span class="nv">$user</span><span class="p">);</span>
</code></pre>
<p>The bug appeared in Typo3 9.5.13 but wasn't present in previous V9 versions (don't know exactly which one).</p>
<p>Florian</p> TYPO3 Core - Task #89932 (Under Review): Harden QOM objects and interfaceshttp://forge.typo3.org/issues/899322019-12-13T09:35:15ZAlexander SchnitzlerTYPO3 Core - Feature #89581 (Under Review): Introduce new system_log tablehttp://forge.typo3.org/issues/895812019-11-04T17:05:29ZAlexander Schnitzler