Project

General

Profile

Actions

Bug #91372

closed

Filelist module stops working if a lot of files in one directory

Added by Alexander Grein about 4 years ago. Updated about 1 month ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Performance
Target version:
-
Start date:
2020-05-12
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
10
PHP Version:
7.4
Tags:
performance,large-site, many files
Complexity:
Is Regression:
Sprint Focus:

Description

After updating a website from TYPO3 9.5 to 10.4 i got huge problems with the performance of the filelist module.

Just clicking on the filelist module button makes the whole backend unusable.

Also deactivation several extensions which could be the reason for the problem (EXT:filefill, EXT:autoresize_images, EXT:extractor) had no effect.

The website has a huge download section with around 100.000 files organized in ~ 26.000 directories.
But this was not a problem under the previous TYPO3 9.5 installation.

Several other tries (restart my boot2docker vm, update, rebuild, restart docker containers) did not changed anything.

So I started to xdebug the cause...

I set one breakpoint at the first line of the TYPO3\CMS\Backend\Tree\View\FolderTreeView::getNumberOfSubfolders method, and watched the incomming $folderObject and another in the next line (return):

    public function getNumberOfSubfolders(Folder $folderObject)
    {
        $subFolders = $folderObject->getSubfolders();
        return count($subFolders);
    }


The first incomming calls where proccessed quite fast, until the recycler-Folder comes in ...
This time, the getSubfolders() call needs endless long, so that I decided after some minutes to kill all php processes using the console.

After a look into this directory, I realized that it contains over 30.000 files!

I really have no idea how this could happend, but after deleting all this files manually and also deleting the corresponding sys_file records (DELETE * FROM sys_file WHERE identifier LIKE "/_recycler_%") the filelist module starts working again.

Conclusion:
The filelist module under 10.4 seams to have problems, with directories containing several 1000 files.


Related issues 1 (1 open0 closed)

Related to TYPO3 Core - Epic #93547: Collection of problems with large sitesAccepted2021-02-19

Actions
Actions #1

Updated by Riccardo De Contardi over 3 years ago

  • Category set to Performance
Actions #2

Updated by Sybille Peters over 3 years ago

  • Tags set to performance,large-site, many files
Actions #3

Updated by Sybille Peters over 3 years ago

  • Related to Epic #93547: Collection of problems with large sites added
Actions #4

Updated by Benni Mack about 1 month ago

  • Status changed from New to Closed

Hey Alexander,

I ran into a similar issue with v10 + v11. With TYPO3 v12 we now have real pagination in the file list module, making this problem go away (I actually had this in one of my installations with 20.000 files in user_upload). I will close this ticket now, if you still have problems in v12, let me know!

All the best,
Benni.

Actions

Also available in: Atom PDF