Bug #78845
closedStory #69712: Further FormEngine development
Editing a sys_file_metadata record (eg. through filelist) fetches all sys_file records
100%
Description
After some discussion on Slack with lolli, he wrote the following:
Analysis:- field "file" in "sys_file_metadata" is configured as select field to sys_file. it is used as: the "inline" counterpart field of field "sys_file" "metadata" since the fields holds the uid of the sys_file record, this sys_file_metadata is for.
- during opening a sys_file_metadata record (eg. through filelist), the TcaSelectItems data provider fetches all "possible" select targets (which basically means "all" sys_file records) with one single query. this can become "a lot" - 10MB for 10k records
- the fun part is, the file field itself is never displayed (not rendered), since it is a) set to readonly, b) shut down per display condition for sys_language_uid!=0, c) it is in NO showitem configuration
- however, it is important to submit the current field value to the data handler, since otherwise the relation to sys_file would get lost upon editing a sys_file_metadata record
- it "may" indeed be possible to switch the field to a group field to suppress fetching these records all the time
- we need to verify that the inline "from sys_file to sys_file_metadata" direction still works if the counter side is switched from select to group
- we still should mark the field "readOnly" for group, is that possible?
- does the "placeholder" / "override" stuff still work if switched group, for instance in tt_content->sys_file->sys_file_metadata ? it is very well possible warnings are raised here since the internal select and group data handling is slightly different (select=array, group=comma mess) ... i remember the placeholder logic is also funny and recursive and this needs an additional check then.
- currently, if doing tt_content editings ... i "think" the calculation of that "sys_file_metadata" "file" field item calculation is probably suppressed already somehow (we had massive performance issues in fal with new FormEngine, and I somehow solved that by suppressing some calculation - question is "how"?). this should have a detailed look again.
- does stuff like the fal indexer and uploader work successfully with group and fills / updates metadata correctly and sets correct relations? (edited)
- Ignore fetching records in the TcaSelectItems for readOnly fields (which may not work if the field is used in relations)
Updated by Markus Klein over 7 years ago
This is the same issue we had with normal records and translations. The l10n_parent by default is a select field as well. Very naughty if you have 25k records on the same page, which all might be the a potential language parent.
We changed TCA to group here to get rid of the issue.
Updated by Wladimir Hofmann about 7 years ago
The column "file" in TCA"sys_file_metadata" has a display condition:
'displayCond' => 'FIELD:sys_language_uid:=:0'
Doesn't it make sense to add an additional
'foreign_table_where' => 'AND sys_file.uid=###REC_FIELD_file###'
condition as well?
worked for me, without side effects so far.
Updated by Rémy DANIEL about 7 years ago
Hello
Adding the foreign_table_where clause worked for me too.
The query array before:
SELECT => 'sys_file.uid,sys_file.name,sys_file.type' FROM => 'sys_file' WHERE => 'sys_file.pid<>-1 ' GROUPBY => '' ORDERBY => '' LIMIT => ''
The query array after:
SELECT => 'sys_file.uid,sys_file.name,sys_file.type' FROM => 'sys_file' WHERE => 'sys_file.pid<>-1 AND sys_file.uid='11499'' GROUPBY => '' ORDERBY => '' LIMIT => ''
I cannot see either any side effects with this fix.
Updated by Torben Hansen about 7 years ago
In my case, the FAL contains 300.000+ records. Memory usage of the edit file metadata request raised to 500 MB and rendering took 10-30 seconds depending on the webserver.
The foreign_table_where
seems to be a good solution for this scenario.
Updated by Gerrit Code Review over 6 years ago
- Status changed from New to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/55125
Updated by Gerrit Code Review over 6 years ago
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/55125
Updated by Stephan Großberndt over 6 years ago
- Has duplicate Bug #83308: Allowed memory size exhausted when editing file meta data added
Updated by Gerrit Code Review over 6 years ago
Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/55167
Updated by Gerrit Code Review over 6 years ago
Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/55168
Updated by Stephan Großberndt over 6 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset 7050335d9d5eeb2befe3f182fc01d795ff9a8fa3.
Updated by Benni Mack almost 6 years ago
- Status changed from Resolved to Closed