Bug #104718
open
Filelist extremly slow having a lot of folders with subfolders on Rootlevel
0%
Description
Story¶
Loading the file tree takes ages on a storage which has many Folders on rootlevel (122 folder ins my Case) when the folders themselves are containing a lot of subfolders.
This might even lead to max_memory exceed errors or maximum_execution time exceed
Cause¶
The FileStorageTreeProvides determines the presence of subfolder in a very inefficent and redundant way. Due to recursive nature of the used fuction the performance impact is extremly high.
Solution¶
Step 1
To detect if a folder has a subfolder or not it is not neccessary to fetch all subfolder one is enought so change:
https://github.com/TYPO3/typo3/blob/11.5/typo3/sysext/backend/Classes/Tree/FileStorageTreeProvider.php#L64
$hasSubfolders = !empty($folder->getSubfolders());
To:
$hasSubfolders = !empty($folder->getSubfolders(0, 1));
Step 2
The prepareFolderInformation already holds the information if there is a subfolder or not. So no need to determine it again.
So remove this
https://github.com/TYPO3/typo3/blob/11.5/typo3/sysext/backend/Classes/Tree/FileStorageTreeProvider.php#L207
And referre to "prepareFolderInformation['hasChildren']" instead and reaad the check for InaccessibleFolder here:
https://github.com/TYPO3/typo3/blob/11.5/typo3/sysext/backend/Classes/Tree/FileStorageTreeProvider.php#L223
I applied the above changes to a my testystem and the loading time dropped from 30-55s to 2-3s.
Affected Systems:¶
- Confirmed on TYPO3 11
- Code looks the same in TYPO3 12 and 13 (only tested on v11)
Updated by Chris Müller 3 months ago
- Category set to File Abstraction Layer (FAL)
Updated by Garvin Hicking 3 months ago
- Subject changed from Fileliste extremly slow having a lot of folders with subfolders on Rootlevel to Filelist extremly slow having a lot of folders with subfolders on Rootlevel
- Category changed from File Abstraction Layer (FAL) to Performance
Updated by Garvin Hicking 2 months ago
- Status changed from New to Needs Feedback
Have you checked out #100230 ? A patch was implemented there which already optimizes this...?
Updated by Garvin Hicking 2 months ago
- Related to Task #100230: Improve performance of FileStorageTreeProvider added
Updated by Nikolas Hagelstein 2 months ago
Garvin Hicking wrote in #note-3:
Have you checked out #100230 ? A patch was implemented there which already optimizes this...?
Even though the usage of Folder::getSubfolders() is reduced, it is still used in an inefficent way:
(line 64 of the patch)
$hasSubfolders = is_array($children) ? $children !== [] : !empty($folder->getSubfolders());
It is not neccessary to fecht all subfolders of a Folder to determine if a folder has Subfolders, just check for one:
$folder->getSubfolders(0, 1)
Otherwise you will run into the same issue if just one level deeper.
Updated by Garvin Hicking 2 months ago
Would you be interested to try to provide a patch for further improvements of (following https://docs.typo3.org/m/typo3/guide-contributionworkflow/main/en-us/Quickstart/Index.html#quickstart)?
Updated by Thomas Hohn 2 months ago
· Edited
If not I can provide a patch @Garvin Hicking / @Nikolas Hagelstein ? If you have a diff it would ease it