Epic #93547
openCollection of problems with large sites
Added by Sybille Peters over 3 years ago. Updated over 2 years ago.
0%
Description
I used the following tags in sub-issues - where appropriate.
tags : "large-site", "memory hog", "prepared statement", "placeholderlimit", "memory", "performance", "throttle", "usability" (or UI related things)
All issues related to large site are tagged with large-site.
Can be many pages, many files, many directories, many categories, etc. or a combination.
The problems are mostly related to:
- performance and resource (memory) problems.
- exceptions, e.g. database query - “Prepared statement contains too many placeholders”
- usability: e.g. long, unusable lists (see sys_filemount: long list of directories + performance issue, unable to filter for specific pages in pagetree filter etc.)
Can the question of large sites be addressed in general?
e.g.
- consider large sites when creating concepts for new features
- create a reference "large site"
- test patches on “large sites”
- define hard limits / soft limits (if possible) - e.g. are there specific known limitations, can we say TYPO3 easily runs with x number of pages (redirects, sites, files, be user groups etc.) in a typical installation (e.g. with a reference site)
Updated by Sybille Peters over 3 years ago
content grouped by category in https://notes.typo3.org/xpZwXmWrRVi5bRCJ392p6Q here.
Updated by Sybille Peters over 3 years ago
- Related to Bug #57953: Rendering time of HMENU is really bad, maybe bug? added
Updated by Sybille Peters over 3 years ago
- Related to Bug #76581: Backend Performance Flaws added
Updated by Sybille Peters over 3 years ago
- Related to Bug #78258: List view CSV export goes out of memory added
Updated by Sybille Peters over 3 years ago
- Related to Bug #92493: linkvalidator: scheduler task + list of broken links dies if too many pages ("Prepared statement contains too many placeholders") added
Updated by Sybille Peters over 3 years ago
- Related to Bug #36744: Workspaces are unusable on larger installations added
Updated by Sybille Peters over 3 years ago
- Related to Feature #89015: Lazy Loading for TCA / Backend View to improve performance added
Updated by Sybille Peters over 3 years ago
- Related to Bug #81996: Read ony select field 'renderType' => 'selectSingle' renders all available items added
Updated by Sybille Peters over 3 years ago
- Related to Epic #89759: Performance improvements in Forms module added
Updated by Sybille Peters over 3 years ago
- Related to Bug #84558: TYPO3 sys_category problem with performance/slow opening category in BE added
Updated by Sybille Peters over 3 years ago
- Related to Task #90143: Redirects: Poor performance of redirect matching for large redirects table added
Updated by Sybille Peters over 3 years ago
- Related to Bug #92033: "Page tree error: Got unexpected reponse" with filter - allowed memory size exhausted added
Updated by Sybille Peters over 3 years ago
- Related to Bug #92036: New behaviour of page tree filter might more easily submit "monster queries" or too many queries added
Updated by Sybille Peters over 3 years ago
- Related to Feature #78760: Make pagetree panel resizable/expandable on large screens added
Updated by Sybille Peters over 3 years ago
- Related to Feature #92576: Page Tree filter: make it possible to explicitly filter by uid - to go to a specific page in the pagetree added
Updated by Sybille Peters over 3 years ago
- Related to Feature #89762: Add pagination to forms list added
Updated by Sybille Peters over 3 years ago
- Related to Bug #52374: Editing a sys_filemount is very slow added
Updated by Sybille Peters over 3 years ago
- Related to Bug #88873: ext: form hangs (run into 500 error) when a lot of files are in the fileadmin added
Updated by Sybille Peters over 3 years ago
- Related to Bug #91372: Filelist module stops working if a lot of files in one directory added
Updated by Riccardo De Contardi over 3 years ago
- Related to Epic #88474: Page tree performance in 9.5 added
Updated by Riccardo De Contardi over 3 years ago
- Related to Bug #88134: Lots a database queries to sys_refindex cause slow page saving added
Updated by Riccardo De Contardi over 3 years ago
- Related to Bug #84051: SelectTree (Category Tree) - add horizontal scrolling added
Updated by S P over 3 years ago
- Related to Bug #88672: SlugHelper->isUniqueInSite() slow for installations with many sites and similar URL structures added
Updated by S P over 3 years ago
A very important change in the heads of core devs or in the "mentality" in the future must be that new features are not developed simply "parallel" to the core, but so that it respect the already existing architecture. For example did the redirects module that came with v9 totally ignore user permissions (if you are allowed to see the module you just see all redirects from all domains and not just the ones you are allowed to see - no simple "bug", just not implemented at all).
Or #88672 just assumed totally arbitrary (with hardcoded magic number) that there are a maximum of 100 pages with the same name.
Or #91995 just assumed that never an unconfigured domain (in sys_domain or Site YAML) can (accidentally!) point to a TYPO3 system, leading to arbitrary output.
This shows that many new features are implmented from a single POV: single/few domain setups with mostly admin BE users. This mentaility automatically also leads to the problems in this epic: performance problems.
Updated by Sybille Peters over 3 years ago
@Stefan P I don't think it is intentionally. Some new features / changes are already pretty complex in themselves and considering user permissions, languages, workspaces etc. makes it even more complex. Really thinking everything through for multiple users, multiple sites, large sites - not an easy task.
But I agree with you that these things are not always considered, which is the reason I started this issue.
I think finding a method for testing with large sites (also with multiple users, multiple domains etc.) would at least make potential problems detected early.
What I also find missing is a definition of the limits. I realize, this is not easy to do. In some cases it also depends on the memory allocated and on other factors. But if you have a reference site and say, ok TYPO3 11 was tested with this reference site with 100 000 pages, 2000 backend users, 300 backend groups, 5 sites, 2000 redirects etc. Don't know if this is realistic, it might require some looking into.
And maybe additionally post some numbers, where problems may be expected, e.g. are there currently any hard limits?
And also finding best practices for how to solve things in general in the code might also help.
From my practical experience with a "large site" not everything is a huge problem and it depends. But I did expect TYPO3 to work well with large sites and as an enterprise solution, I think it should. Most of the problems are just a nuisance or do not affect us but some have been a big problem and have made time - and cost-intensive workarounds necessary.
Updated by Sybille Peters over 3 years ago
- Related to Task #93305: Long lists of parameters in QueryBuilder handled differently ("too many placeholders") added
Updated by Torben Hansen over 3 years ago
- Related to Bug #93937: Live search really slow for non admin users in large TYPO3 websites added
Updated by Sybille Peters almost 3 years ago
- Related to Epic #95690: Performance issues when hosting a large amount of websites, and optimizations propositions added
Updated by Sybille Peters over 2 years ago
- Related to Bug #86859: Search with indexed_search plugin throws exception: Prepared statement contains too many placeholders added
Updated by Sybille Peters over 2 years ago
- Related to Bug #96042: Row update wizard may consume too much memory for tables with many records and content added
Updated by Sybille Peters over 2 years ago
- Related to Task #89287: Make linkvalidator crawling polite added
Updated by S P over 2 years ago
- Related to Feature #97017: Allow LiveSearch to be disabled via configuration added
Updated by Guillaume Germain over 1 year ago
- Related to Bug #99326: DataHandler - Process too many records added
Updated by Sybille Peters over 1 year ago
- Related to Bug #90518: DB problems with rootline cache: DELETE cf_cache_rootline double-JOIN with cf_cache_rootline_tags take excessively long added
Updated by Jan Kornblum 3 months ago
- Related to Feature #75017: Search for relations in backend added