Bug #82730
open
Opening record in backend painfully slow with foreign table and large main table
0%
Description
Opening a record in the backend can become painfully slow under the following circumstances:
The main table (10.000 records) has got a 1:n relation to a foreign table (3 records). When trying to open the record of the main table in the TYPO3 backend, the following SQL statement is executed that often that it takes an exceptionally long amount of time to actually open the record.
SELECT * FROM `tx_myext_domain_model_foreigntable` WHERE (`uid` = ?) AND (`tx_myext_domain_model_foreigntable`.`deleted` = 0)
TCA of main table:
'attribute' => [
'exclude' => true,
'label' => 'attribute',
'config' => [
'type' => 'select',
'renderType' => 'selectSingle',
'foreign_table' => 'tx_myext_domain_model_foreigntable',
],
],
Updated by Susanne Moog almost 7 years ago
- Category set to FormEngine aka TCEforms
Updated by SiteGefühl no-lastname-given almost 7 years ago
I can confirm the Problem in TYPO3 7 and PHP 7.1
We have something about 80.000 SQL queries after we clicked the edit button of the main record. It takes 3 minutes to open the record.
One identical query is executed something about 5000 times.
// Edit: Maybe this is going to fix the Problem? https://github.com/TYPO3/TYPO3.CMS/commit/f3e7a034c129aafc8b081e0125c50330207fae82#diff-ddfa1112aff05d23dcd52f1e19e04ba9
// If yes, I would be happy about a fix for Version 7
Updated by Stefan Franke almost 7 years ago
Unfortunately, commit f3e7a034c129aafc8b081e0125c50330207fae82 doesn't fix the issue for me.
It seems that it has got something to do with the localization. In my case I don't need the localization feature, so I got rid of it and that "solved" the problem.
Updated by Andre Koller almost 7 years ago
- Related to Bug #81036: TCA l18n_parent is processed for sys_language_uid 0 added
Updated by Andre Koller almost 7 years ago
Typo3 v8.7.8 PHP 7.1
When "label_alt" and "label_alt_force" ist used in the ctrl-Section in TCA of the foreign model, 10000 sql queries executed.
Without label and label_alt_force 500 sql queries executed.
Can anyone confirm this?
Updated by Jakob Berlin over 6 years ago
Not really confirming it but after disabling label_alt my edit view get back to work in a few seconds wich just CPU usage and not minutes with have DB usage. So it might be the cause of the problem.
Btw: It even happened when label is set to a foreign attribute of the foreign attribute itself. So, to have now troubles label should be a static value from the foreign table by now.
Updated by S P over 6 years ago
We are having this issue as well (with both, TYPO3 7 and 8). Our data structure is like this: products are the main table, products contain attributes via 1:n (irre, inline, so the product uid with index is inside the attributes table), And the attributes contain n:1 an attributelabel (attribute contains a field attributelabel holding a uid):
product ---1:n--- (field w/ index product) attribute (field label) --- n:1 --- attributelabel
The ctrl => label in attributes is set to the field attributelabel - and this constellation triggers this behaviour. So, loading the inline label (or alt label) from a third table is responsible for this.
Updated by Webadmin no-lastname-given over 6 years ago
- Related to Bug #84247: TCA label_userfunc called for every record in current page/folder added
Updated by S P over 6 years ago
What is the state of this bug? We have two customers where this is a major problem, because they are not able to effectivley edit their datasets due to this behaviour.
Updated by Michael Ecker over 6 years ago
Andre Koller wrote:
Typo3 v8.7.8 PHP 7.1
When "label_alt" and "label_alt_force" ist used in the ctrl-Section in TCA of the foreign model, 10000 sql queries executed.
Without label and label_alt_force 500 sql queries executed.Can anyone confirm this?
I can definitely confirm that disabling 'label_alt_force' reduces the execution time by magnitudes, especially with an object containing more than about 10 inline children. No other configuration variation had a similar effect.
This bug does gravely hamper the use of inline elements and should be resolved immediately. In my experience it was a smaller issue even with versions before 7, but with the migration to Doctrine DBAL in 8 the execution time multiplied again. I got regularly 'max execution time exceeded' errors even with significantly raised max_execution_time values.
Updated by Oliver Beck about 6 years ago
I have to push this issue. We have several typo3 instances, which have inline fields (irre) in tca. As soon as the fields contains more than 10 or 20 child-records, it takes about 30 seconds until the whole record can be edited.
Updated by Bernhard Berger almost 6 years ago
I can also co firm this issue. In our case we actually need localization as we have child records attached to parent records and unfortubately we really need that relation.
We can't even change the design to use the list module as it would require re-schooling about 300 editors..
Updated by Stephan Bauer over 5 years ago
Same problem here with 8.7.
'label' => 'title',
'label_alt' => 'labels',
'label_alt_force' => 1,
'labels' is a selectMultipleSideBySide
As described in the related bug #84247 changing
'foreign_table_where' => 'AND xxx.pid=###CURRENT_PID### AND ...
to
'foreign_table_where' => 'AND xxx.pid=-1 AND
it is fast again.
Updated by Jan Kornblum over 5 years ago
Same problem still in 9.5.
Removing 'label_alt' (pointing to child or parent records) and 'label_alt_force' brought great performance back! In addition, i tried to remove the 'label_alt' for the IRRE appearance only using overrideChildTca() (https://forge.typo3.org/issues/87426) to perserve them in normal list-view, but that is not possible (missing feature).
The other way to get even better performance: Removed localization feature at all from my extension, which made the usage of 'label_alt' and 'label_alt_force' possible again without performance loss.
Updated by M. Ecker almost 5 years ago
Hint:
To build an options list based on a foreign_table TYPO3 will first get all records according to foreign_table_where from this table and then in a second step build the labels for the options. If label_alt points e.g. to a parent table or localizations are needed there will at least one additional db request for every option. That's why removing label_alt can boost performance.
If you use IRRE elements where you defined a parent selector within TCA you should use overrideChildTca within the parent element to set this field to passthrough when displayed as child element.
These performance problems could only be avoided using a (complex) JOIN statement to get the already localized labels within the first statement - but that wouldn't fit all cases, I think.
Updated by Christian Eßl over 4 years ago
- Related to Bug #82100: Backend Form Dataprovider load every foreign Entity, not only the related one added
Updated by Markus Klein over 4 years ago
- Related to Bug #87426: label_alt + label_alt_force for IRRE childs cannot be overwritten using OverrideChildTca() added
Updated by Markus Klein over 4 years ago
- Status changed from New to Closed
- Priority changed from Must have to Won't have this time
M. Ecker already outlined the behaviour and his conclusion.
This is not fixable on a general scale as far as we can tell.
So performance optimization is possible only by: * implementing a custom label function * changing the relation type/rendering type (e.g.g group instead of select) * adjusting the TCA in an inline relation (overrideChildTCA)
Updated by Gerrit Code Review 10 months ago
- Status changed from Closed to Under Review
Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 10 months ago
Patch set 3 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 10 months ago
Patch set 4 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 4 months ago
Patch set 5 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 4 months ago
Patch set 6 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 2 months ago
Patch set 7 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871
Updated by Gerrit Code Review 2 months ago
Patch set 8 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/81871