Bug #91372

Filelist module stops working if a lot of files in one directory

Added by Alexander Grein over 1 year ago. Updated 12 months ago.

Status:
New
Priority:
Should have
Assignee:
-
Category:
Performance
Target version:
-
Start date:
2020-05-12
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
10
PHP Version:
7.4
Tags:
performance,large-site, many files
Complexity:
Is Regression:
Sprint Focus:

Description

After updating a website from TYPO3 9.5 to 10.4 i got huge problems with the performance of the filelist module.

Just clicking on the filelist module button makes the whole backend unusable.

Also deactivation several extensions which could be the reason for the problem (EXT:filefill, EXT:autoresize_images, EXT:extractor) had no effect.

The website has a huge download section with around 100.000 files organized in ~ 26.000 directories.
But this was not a problem under the previous TYPO3 9.5 installation.

Several other tries (restart my boot2docker vm, update, rebuild, restart docker containers) did not changed anything.

So I started to xdebug the cause...

I set one breakpoint at the first line of the TYPO3\CMS\Backend\Tree\View\FolderTreeView::getNumberOfSubfolders method, and watched the incomming $folderObject and another in the next line (return):

    public function getNumberOfSubfolders(Folder $folderObject)
    {
        $subFolders = $folderObject->getSubfolders();
        return count($subFolders);
    }


The first incomming calls where proccessed quite fast, until the recycler-Folder comes in ...
This time, the getSubfolders() call needs endless long, so that I decided after some minutes to kill all php processes using the console.

After a look into this directory, I realized that it contains over 30.000 files!

I really have no idea how this could happend, but after deleting all this files manually and also deleting the corresponding sys_file records (DELETE * FROM sys_file WHERE identifier LIKE "/_recycler_%") the filelist module starts working again.

Conclusion:
The filelist module under 10.4 seams to have problems, with directories containing several 1000 files.


Related issues

Related to TYPO3 Core - Epic #93547: Collection of problems with large sitesAccepted2021-02-19

Actions
#1

Updated by Riccardo De Contardi 12 months ago

  • Category set to Performance
#2

Updated by Sybille Peters 12 months ago

  • Tags set to performance,large-site, many files
#3

Updated by Sybille Peters 10 months ago

  • Related to Epic #93547: Collection of problems with large sites added

Also available in: Atom PDF