Bug #52374
closedEditing a sys_filemount is very slow
0%
Description
When editing a filemount as popup window from the edit form of a be_group, the form takes about 9 sec to show up.
xhprof shows that this method is responsible for this amount of time:
TYPO3\CMS\Core\Resource\Service\UserFileMountService::renderTceformsSelectDropdown
Info¶
- My list of sys_filemounts contains 2600 entries
- I see 20 calls to TYPO3\CMS\Core\Resource\Folder::getSubfolders taking 4,728,752 ms
- After 2nd recursion, still 298 calls to TYPO3\CMS\Core\Resource\Service\UserFileMountService::getSubfoldersForOptionList taking 4,505,079 ms
Actually there is much recursion but it's always TYPO3\CMS\Core\Resource\Folder::getSubfolders that eats so much time
Updated by Markus Klein over 10 years ago
- Parent task changed from #52304 to #52949
Updated by Mathias Schreiber over 9 years ago
- Status changed from New to Needs Feedback
- Assignee set to Mathias Schreiber
Hi Xavier,
I just have 600 folders, but for me it's fine on master.
Is this done?
Updated by Xavier Perseguers over 9 years ago
- TYPO3 Version changed from 6.1 to 6.2
- PHP Version changed from 5.3 to 5.5
Updated the PHP and TYPO3 version.
Using a real life server (that is, not with SSD but quite powerful anyway):
- 3522 directories (8 at the 1st level)
- about 27 sec to show up the edit form (edit pencil for a filemount record)
This is still not usable.
The same using SSD (development environment):
- 2250 directories (8 at the 1st level)
- about 3 sec to show up the edit form
This time it's totally OK of course but users are usually not using a development machine with local MySQL and SSD.
Didn't test with master, 6.2 is less than 1 year old and is a LTS with big projects so I'd like to see improvements for it if possible (we shouldn't forget that 6.2 is even not yet used by many bigger websites).
Updated by Alexander Opitz about 9 years ago
- Status changed from Needs Feedback to New
Updated by Xavier Perseguers about 9 years ago
Did not debug again but this huge amount of time just appeared again this morning while configuring a Dropbox storage. Since every directory is searched for recursively, it took minutes (!) to fetch the structure of directories in my Dropbox account (total of about 50 directories?). So locally this may be slow but with remote storages this is even worse.
Updated by Tymoteusz Motylewski over 4 years ago
- Related to Bug #51402: Filemount record UI issues added
Updated by Sybille Peters over 4 years ago
Creating a new filemount or editing a filemount creates a select box of all (!) directories in the storage (e.g. fileadmin).
We had this problem too. It can make managing filemounts very sluggish if you have lots of directories. Takes several minutes.
It would be nice if you could have the choice of filtering before TYPO3 starts scanning everything. Also, the usability of the long select list is not so great.
btw. we decided to split up the fileadmin into several storages on our site (mostly because of this problem). It takes the load off this problem. This also made sense for organizational reasons and has so far not had had any negative effect. It's made a bit difficult by the fact that you currently can't move entire directories through the GUI across storages (see #77988. #66514, #87662), but it is possible.
Updated by Tymoteusz Motylewski over 4 years ago
I think the workaround would be to change the field type to simple input text input field.
The issue then is that there is no validation and user needs to put correct path.
Another workaround is to limit the number of levels for fetched folders.
In my case I've just commented the foreach in "getSubfoldersForOptionList()", as I just need one level of folders.
Another solution would be to utilize file browser with filed type group
'config' => [ 'type' => 'group', 'internal_type' => 'folder', 'maxitems' => 1, ]
The problem with this approach is that it saves data with slightly different format in db (path is prepended with storage id) e.g "2:/foldername/",
and there is no build in functionality right now to limit element browser to just one storage.
On the other hand maybe the whole UX could be then simplified:
- the "base" field being removed
- only one group field with element browser being there in which you select both storage and path in one go?
Updated by Sybille Peters over 4 years ago
- About the input field: I think that would not be so good from usability view: User has to type in (or copy/paste) the complete path name
- Limit the recursion depth: I think this might work if it's possible to do this optionally (at the beginning you have to select the storage anyway and you get a reload.). But it's not so elegant. It seems technical and is not intuitive for everyone.
- with the group + internal_type=folder: This might be a less better solution than current for high recursion depth but few directories (because choosing them from the select list is 1 click). But for a lot of directories in the storage, it is a better solution than current.
About selecting both storage and path in one go: I think that would be nice for usability (because the current choose storage, reload is a bit clunky anyway).
All considered, I would probably prefer option 3 (group) and the combining of storage + folder selection.
Updated by Sybille Peters almost 4 years ago
- Related to Feature #89762: Add pagination to forms list added
Updated by Sybille Peters almost 4 years ago
- Tags set to performance, large sites, many directories, many
Updated by Sybille Peters over 3 years ago
- Tags changed from performance, large sites, many directories, many to performance, large-site
Updated by Richard Haeser over 3 years ago
- Related to deleted (Bug #51402: Filemount record UI issues)
Updated by Richard Haeser over 3 years ago
- Has duplicate Bug #51402: Filemount record UI issues added
Updated by Richard Haeser over 3 years ago
- Assignee changed from Mathias Schreiber to Richard Haeser
- TYPO3 Version changed from 6.2 to 10
Updated by Sybille Peters over 3 years ago
- Related to Epic #93547: Collection of problems with large sites added
Updated by Frank Nägler about 2 years ago
- Related to Feature #97922: Use combined field for base and path in sys_filemounts added
Updated by Sybille Peters about 2 years ago
- Status changed from New to Closed
Is resolved with #97922