Bug #57953

Rendering time of HMENU is really bad, maybe bug?

Added by Florian Schwaiger about 5 years ago. Updated 12 months ago.

Status:
New
Priority:
Must have
Assignee:
-
Category:
Performance
Target version:
-
Start date:
2014-04-15
Due date:
% Done:

0%

TYPO3 Version:
6.2
PHP Version:
5.5
Tags:
Complexity:
Is Regression:
No
Sprint Focus:

Description

We use the following menu on every page.

Because of expAll = 1 on the first two levels, the menu is by far the slowest element on the pages. This Bug has been adressed in Ticket (http://forge.typo3.org/issues/23486) so far, but was closed due to missing feedback.
Also I am unsure whether this is actually the problem, but in combination with two other menus on the page it is responsible for 90% of the processing time.

60 = HMENU
    60 {
        entryLevel = 1

        1 = TMENU
        1 {
            expAll = 1

            NO.allWrap = <li>|</li>
            NO.ATagTitle.field = abstract // description // title

            ACT = 1
            ACT.wrapItemAndSub = <li class="active">|</li>
            ACT.ATagTitle.field = abstract // description // title

            IFSUB = 1
            IFSUB.ATagTitle.field = abstract // description // title
            IFSUB.stdWrap.wrap = |&nbsp;<span class="text-muted">/ ...</span>
            IFSUB.stdWrap2.wrap = |<ul class="nav collapse">
            IFSUB.wrapItemAndSub (
                <li class="sub">
                    <a href="#" title="Untermenü aufklappen" class="sidebar-dropdown-btn" 
                                data-toggle="collapse" data-parent="#sidebar .panel-body.nav">
                        <span class="caret"></span>
                    </a>
                    |</ul>
                </li>
            )

            ACTIFSUB < .IFSUB
            ACTIFSUB.stdWrap2.wrap = |<ul class="nav collapse in">
            ACTIFSUB.wrapItemAndSub (
                <li class="active sub">
                    <a href="#" title="Untermenü aufklappen" class="sidebar-dropdown-btn" 
                                data-toggle="collapse" data-parent="#sidebar .panel-body.nav">
                        <span class="caret"></span>
                    </a>
                    |</ul>
                </li>
            )

            CURIFSUB < .ACTIFSUB

            wrap = <ul id="sidebar-menu" class="panel-body nav">|</ul>
        }

        2 < .1
        2.wrap >

        3 = TMENU
        3 {
            NO.allWrap = <li>|</li>
            NO.ATagTitle.field = abstract // description // title

            ACT = 1
            ACT.wrapItemAndSub = <li class="active">|</li>
            ACT.ATagTitle.field = abstract // description // title

            ACTIFSUB < .ACT
            CURIFSUB < .ACT
        }

        wrap = <div id="sidebar-menu" class="panel-collapse collapse in">|</div>
    }

PHPinfo_Localhost.html View - phpinfo local (99.4 KB) Florian Schwaiger, 2014-04-22 09:45

PHPinfo_Staging.html View - phpinfo remote (slow) (97.7 KB) Florian Schwaiger, 2014-04-22 09:45

Status_Localhost.txt View - mysql status local (28.6 KB) Florian Schwaiger, 2014-04-22 09:45

Status_Staging.txt View - mysql status remote (slow) (25.2 KB) Florian Schwaiger, 2014-04-22 09:45

Cattura_a_1500pages.PNG View - around 1500 pages added (30.8 KB) Riccardo De Contardi, 2017-09-11 18:00

Cattura_a_6000pages.PNG View - around 6000 pages added (29.2 KB) Riccardo De Contardi, 2017-09-11 18:00

Cattura_a_9000pages.PNG View - around 9000 pages added (24.1 KB) Riccardo De Contardi, 2017-09-11 18:00

Cattura_a_3000pages.PNG View - around 3000 pages added (29 KB) Riccardo De Contardi, 2017-09-11 18:00

profiling-hmenu.png View (368 KB) Jonas Ulrich, 2018-06-07 12:13


Related issues

Related to TYPO3 Core - Bug #23486: nested menu generates huge number of mysql queries Closed 2010-08-31
Related to TYPO3 Core - Bug #63550: Menu configuration caching disabled by ineffective type checks Closed 2014-12-03
Related to TYPO3 Core - Bug #87729: MenuProcessor can't cache New 2019-02-18

History

#1 Updated by Alexander Stehlik about 5 years ago

Thanks for bringing this up. I also have the impression that menu rendering takes quite long. I also started looking around a bit but I did not find a bottleneck. The process simply seems to be complex (access checking, translation checking etc.).

But it would be really cool if someone has an idea. Especially for rendering sitemaps the slow menu rendering is a real pain.

#2 Updated by Florian Schwaiger about 5 years ago

I compared my two servers and did not find a clue to the difference in performance yet (that is, the local machine takes 40ms to render, the remote one 4000ms). Find attached the server configs and mysql table status, perhaps you can spot a difference.

The only other thing I can imagine, that the mysql server on my local machine is a lot faster due to the SSD disk.

#3 Updated by Florian Schwaiger about 5 years ago

In case someone has more experience analyzing profiler data, I conducted 3 test cases using the xdebug profiler:

0) without profiler, cleared FE caches
0) without profiler, reloaded news list to fill general caches
1) with profiler, loaded single news object
2) with profiler, loaded other single news object
3) with profiler, loaded calendar page


1) https://dl.dropboxusercontent.com/u/70926418/xdebug.local.out.1
2) https://dl.dropboxusercontent.com/u/70926418/xdebug.local.out.2
3) https://dl.dropboxusercontent.com/u/70926418/xdebug.local.out.3

1) https://dl.dropboxusercontent.com/u/70926418/xdebug.staging.out.1
2) https://dl.dropboxusercontent.com/u/70926418/xdebug.staging.out.2
3) https://dl.dropboxusercontent.com/u/70926418/xdebug.staging.out.3

#4 Updated by Florian Schwaiger about 5 years ago

So, we found out that indeed the performance problem has to do with too much database access. Finally, after searching for hours, I almost accidentally looked into the configuration module and found out, that even though apc was enabled in the install tool, it was not used anywhere.

Adding the following lines to typo3conf/AdditionalConfiguration.php disabled most cache to the database and now page rendering times dropped to a 10th! This is now acceptable.

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pages']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pages']['options'] = array();

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_hash']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_hash']['options'] = array();

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pagesection']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_pagesection']['options'] = array();

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_rootline']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['cache_rootline']['options'] = array();

$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_reflection']['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_reflection']['options'] = array();

The question remains though, why does enabling the APC backend in the install tool not affect TYPO3 default caching behavior?

#5 Updated by Alexander Stehlik about 5 years ago

Hi Florian,

thank you very much for your research.

Where do you set the APC cache in the install tool? I know one setting under "Configuration Presets" but this only changes the cache type for the extbase_object cache.

I'm not sure which of your caches improves the menu rendering performance. Or does it only improve the performance for cached pages?

As far as I can see most of the menu rendering is done in the \TYPO3\CMS\Frontend\Page\PageRepository (e.g. ->getMenu()) which only uses direct database queries or its local caches. So I can not see how changing the configuration for all the other caches can improve the rendering times for menus.

Cheers,
Alex

#6 Updated by Alexander Stehlik about 5 years ago

Another thought: I'm wondering if the performance could be improved using MySQL indexes for the fields queried by the PageRepository, e.g. deleted, hidden, starttime, endtime, fe_group and optionally the versioning fields.

#7 Updated by Florian Schwaiger about 5 years ago

Where do you set the APC cache in the install tool? I know one setting under "Configuration Presets" but this only changes the cache type for the extbase_object cache.

I set it in the install tool only, it only changed extbase_object cache. Now I have this file in AdditionalConfiguration as I see no benefit at all for storing anything in database:

if (!function_exists('apc_for_cache_backend')) {
    function apc_for_cache_backend($cacheName)
    {
        $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations'][$cacheName]['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\ApcBackend';
        $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations'][$cacheName]['options'] = array();
    }
}

apc_for_cache_backend('cache_pages');
apc_for_cache_backend('cache_hash');
apc_for_cache_backend('cache_pagesection');
apc_for_cache_backend('cache_rootline');
apc_for_cache_backend('extbase_reflection');
apc_for_cache_backend('cache_news_category');

I'm not sure which of your caches improves the menu rendering performance. Or does it only improve the performance for cached pages?

I checked on cache performance once more by resetting the "cache_hash" table to database backend.
The rendering times increased to 4 seconds once more, so I believe the bottleneck is with this cache table.

#8 Updated by Jörg Wagner almost 5 years ago

I can fully confirm this report on TYPO3 CMS 6.2.4.

Some more details to reproduce:
  • The effect is NOT visible if your DB is running from an SSD, so you need a classical HD based MySQL server to reproduce
  • expAll = 1 must be used to force the rendering of the complete menu tree
  • you don't need a very big tree but you need some nesting, and ... very important ...
  • the MenuItem depth must be reasonably big (e.g. 2 < .1, 3 < .1, 4 < .1 - see example below).

Here is a quick sketch of a minimal HMENU to get the effect.

page.10 = HMENU
page.10 {
    1 = TMENU
    1 {
        expAll = 1
        wrap = <ul>|</ul>
        NO = 1
        NO {
            wrapItemAndSub = <li>|</li>
        }
    }
    2 < .1
    3 < .1
    4 < .1
}

With a small page tree of some 30 pages (total page count over all levels) nested three levels deep, I get rendering times of 10 to 15 seconds per page (with pages that contain nothing but the above menu)! After a page has been rendered once and has been stored in cache, the next call serves the page reasonably fast. But with only 30 pages you can easily calculate that the total rendering time for all pages on first hit sums up to a total of 5 minutes. This is beyond any reasonable figure, especially considering that the pages are practically empty (only a menu contained).
I have tested all of this on two server systems with reasonable performance, one of them being a DomainFactory (Dedicated Server, nearly empty) which has always been a great and speedy hoster for us.
My testing CMS is a fresh and empty TYPO3 installation. On DomainFactory I have even used their TYPO3 default installation to test.

My conclusion is that there is some major problem in the back-ported caching framework. As a side note, I also have rendering times of about 15 seconds for a simple Extbase phone list extension in another project. That phone list has only one person table with 600 records and two 1:n tables for person attributes (company and department, about 20 records each). My profiling trials lead to the same conclusion: The caching framework eats up unbelievable resources.

Considering the ramifications of both effects (menu performance problems and extbase extension performance problems) and both issues obviously originating from the caching framework, I have to conclude, that 6.2 is simply not yet production ready. Unfortunately the problem is buried so deep in the caching system that I feel unable to debug it myself and present a solution (as I have done so many times before for TYPO3 over the last 10 years).

#9 Updated by Nikola Vignjevic over 4 years ago

Hi everyone.

I have here the same problem.

I have the 6.2.4 version with the demo pages from the bootstrap_package. I watch the rendering time via admintool, and the results are this:

I take this page (not cashed) as example:

/about/history.html

The HMENU is called 6 times, (that seems OK) but this times:

10 HMENU 1364 1493 +12 =1505
10 HMENU 2870 653 +17 =670
10 HMENU 3541 1174
10 HMENU 4779 891
20 HMENU 5704 3381
10 HMENU 9089 148

Only the HMENU costed here 7.611 sec.

The times are every time a different, but nearly every time the HMENU costs between 50% and 75% of the whole rendering time.

I was now over one week looking for the bottleneck, and fond two points where the speed is going away in typo3.

The second was in the view, if you have a loops or similar elements, which you use to build for example tables, edit forms or something like that, for example:

<f:for each="{stores}" as="store">

... here some needful table rows ...

</f:for>

If you have both things HMENU and a plugin, which use f:for you get loading times up to:

-->> 15 sec.

maybe on servers with a better setup, SSHDs and with workarounds You get some better results. but every page with a simple navigation and a table with 20 rows which need longer then 0.3 - 0.5 sec is in my opinion google-dead; and that is for me a K.O. criterion ,... :-(

please help.

best regards

Nik

#10 Updated by Jan Kiesewetter over 4 years ago

I can confirm this problem.
I have a website with about 120 pages on 4 levels.
The rendering of an uncached page takes 5000ms, the HMENU takes 3500ms of them, so about 70%.

TYPO3 6.2.6

#11 Updated by Elmar Putz over 4 years ago

I have the same problem. On Typo3 6.2.9, a website with about 150 pages on 4 levels is very slow rendering the uncached pages.
When deactivating the menus, page rendering time improves about 3000%. When page is cached, performance is ok, but the initial loading takes endlessly.

For testing purposes i created a special page type, which returns the menu (4 levels) only - it takes about 5 sec. to render on a hetzner managed server q90, which is a quite powerful machine. Other Typo3 projects on the same machine with a similar number of pages, but less menu levels are delievered in a blink of an eye

#12 Updated by Matthew Colton over 4 years ago

I can confirm this problem.
On Typo3 6.2.9, on a site with about 150 pages and a three level Menu HMENU rendering takes about 12 seconds according to the profiler extension. The configuration is simuliar to the other examples on this page.
Once the page is cached the performance is ok. The server is a root server with a very low load running PHP 5.5.19 with memory_limit set to 256M.

#13 Updated by Alexander Opitz over 4 years ago

  • Category set to Performance

Would like to take a look into this over the christmas days.

Does someone have a demonstrational setup and can send me the data so I only need to import all into my testing system?

#14 Updated by Jörg Wagner over 4 years ago

Hi Alexander,

all you need is:
– a page tree with some 30 empty pages nested 4 to 5 levels deep (quick to build with the Functions module and some page dragging to nest the pages)
– a HMENU of this form:

page.10 = HMENU
page.10 {
    1 = TMENU
    1 {
        expAll = 1
        wrap = <ul>|</ul>
        NO = 1
        NO {
            wrapItemAndSub = <li>|</li>
        }
    }
    2 < .1
    3 < .1
    4 < .1
    5 < .1
}

Make sure that you do NOT host your MySQL database on an SSD, as this will increase the db access speed so much that you will not witness the problem.
If you use a Mac Book Pro with an SSD system drive, you must temporarily move your MySQL storage to an externally attached conventional HD.

Thanks for looking into this!
Cheers, Jörg.

#15 Updated by Alexander Opitz over 4 years ago

SSD or not is not the question. I will take a look into all SQL Queries that will be executed.

#16 Updated by Andre Michels over 4 years ago

I upgraded one installation of Typo3 6.2.7 to Typo3 7 because of this problem.
It looks like the bug was fixed. Probably we only need a back port of the chaching changes?

#17 Updated by Florian no-lastname-given over 4 years ago

I have the same problem with Typo3 6.2.9. Rendering of my main menu based on HMENU takes more than 10 seconds.
APC saved my day (see above) and rendering performance is ok now.

If I can help anyone fixing this problem feel free to contact me.

Merry Christmas
Florian

#18 Updated by Arjen Hoekema over 4 years ago

Could be related to: https://forge.typo3.org/issues/63550

Ran into some performance issues with a TMENU using expAll = 1 where empty menu objects were stored in cache (cache_hash) as NULL. The cache was checked using 'is_array' so the menu object was regenerated.
On some pages generating the page took 5 sec, now dropped to < 1s.

#19 Updated by Arjen Hoekema over 4 years ago

Also check if $GLOBALS['TYPO3_CONF_VARS']['FE']['activateContentAdapter'] is still activated which results in providing a fallback for the 'media' field from the 'pages' table and generating numerous additional queries for menu items.

#20 Updated by Jörg Wagner over 4 years ago

@Arjen:
A) #63550 sounds very promising. This could very well be the source of the issue here!
B) ['FE']['activateContentAdapter'] is FALSE in all of our environments. This was not the reason for the problem in our tests.

@Florian:
This has nothing to do with APC or any other OPCache system. We use OPCaches in all our live systems and still have the problem.

#21 Updated by Sebastian Lemke over 4 years ago

Hello,

is it sure that caching helps out here? Sure - when the cache works correcty (#63550) it will be much faster.
But I do not understand why generating the MENU itself takes such a long time, regardless of the cache?

Sebastian

#22 Updated by Stefan Neufeind over 4 years ago

Depending on the number of pages a huge number of SQL-queries are fired. Last time I looked there was a query (or two?) for every page being displayed in the menu. That might be related to checks for visibility or so.

#23 Updated by Andre Michels over 4 years ago

I've applied the #63550 patch but clearly it doesn't fix the problem at all. This bug is very annoying in our bigger Typo3 instances. Looked serveral hours, debugging, writing emails to customers... I don't understand why this bug doesn't affect so much people. In Typo3 7 the bug was fixed, but where?

#24 Updated by Alexander Stehlik over 4 years ago

I think not everyone is affected by this bug because it heavily depends on your database / IO performance.

When your storage performance is not optimal the impact of this problem seems to be much bigger.

When you have fast harddisks (SSDs) you will not notice this problem so much.

At least these are my experiences.

#25 Updated by Andre Michels over 4 years ago

We run MariaDB on huge DB servers, they arent slow... But the slowquery log shows very slow querys (1-5s) for INSERT statements in the cf_cache_hash.

#26 Updated by Mario Hirschfeld over 4 years ago

confirmed

Actually we´re updating round about 30 Typo3-Services from 4.5 to 6.2 LTS but can´t explain these performance problems to our customers.

A pulldownmenu with 250 entries in 3 menulevel will slow down the service to 20 seconds response time without any content on the site.
This is a Root-Server with SSDs.

A sulution is needed fast as possible, we can´t work with typo3 6 since it´s fixed.

#27 Updated by Alexander Stehlik over 4 years ago

We are using a redis cache as a workaround. This configuration improved the speed for us quite a bit:

if (
    isset($GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['myext']['enableRedisCaches'])
    && $GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['myext']['enableRedisCaches']
    && extension_loaded('redis')
) {

    if (!function_exists('configureRedisAsCacheBackend')) {
        function configureRedisAsCacheBackend($cacheName)
        {
            $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations'][$cacheName]['backend'] = 'TYPO3\\CMS\\Core\\Cache\\Backend\\RedisBackend';
            $GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations'][$cacheName]['options'] = array();
        }
    }

    configureRedisAsCacheBackend('cache_pages');
    configureRedisAsCacheBackend('cache_hash');
    configureRedisAsCacheBackend('cache_pagesection');
    configureRedisAsCacheBackend('cache_rootline');
    configureRedisAsCacheBackend('extbase_reflection');
    configureRedisAsCacheBackend('cache_news_category');
}

#28 Updated by Clemens Tilgner over 4 years ago

We skippt the Menu generation with Typoscript and used a small Fluid Template to generate the Menu. This helped speeding up the rendering process a lot.
It is a workaround, but no direct solution for this problem.

#29 Updated by Jörg Wagner over 4 years ago

Clemens Tilgner wrote:

We skippt the Menu generation with Typoscript and used a small Fluid Template to generate the Menu. This helped speeding up the rendering process a lot.
It is a workaround, but no direct solution for this problem.

How do you do that? VHS viewhelper?

#30 Updated by Stefan Neufeind over 4 years ago

If the menu is the same on all pages, you can cache it so it will only be rendererd on the first accessed page (taking the performance-penalty) and later be fetched from cache for all other pages that are generated.

One possibility for caching is for example stdWrap.cache: http://docs.typo3.org/typo3cms/TyposcriptReference/Functions/Cache/Index.html

#31 Updated by Clemens Tilgner over 4 years ago

Jörg Wagner wrote:

How do you do that? VHS viewhelper?

Correct. This is the template we used:
page.10.marks.NAVI = FLUIDTEMPLATE
page.10.marks.NAVI {
file = fileadmin/templates/fluid-templates/mainmenu.html
}

mainmenu.html: {namespace v=FluidTYPO3\Vhs\ViewHelpers}

<v:page.menu pageUid="1" expandAll="1" useShortcutUid="0">
<ul id="nav">
<f:for each="{menu}" as="menuPage">
...
</f:for>
</ul>
</v:page.menu>

#32 Updated by Faktor E GmbH about 4 years ago

Any update for that issue? It really is a pain right now for some of our installations with 2 or more levels in the HMENU. We have 6.2.11 running.

#33 Updated by taywa gmbh about 4 years ago

We can confirm this issue with 6.2.11 which includes #63550. on a large site with hundreds of pages the generation of an empty page with just the menu takes over a minute.

it is considerably better with 6.2.12 but it is still slow, about ~12 seconds. 4.5 LTS was able to do that in ~3 seconds. (tested with the same page, one upgraded one old, on the same server on two different ports)

#34 Updated by Tao Nguyen almost 4 years ago

Hi, we have copied our Typo3 4.5 to a test environment to Typo3 6.2.13. We use a nested HMenu (3 levels) and about 120 Pages. Before 6.2.12 it took about 20-30 seconds minimum to load a page without cache, with 6.2.13 it is now much better and takes about 7-12 seconds to load completely. We tried to optimize the TS as much as possible, removing conditions (except language conditions) and using less extensions. I even removed the ACT-state from the HMENU (so the menu basically always stays the same and is supposed to load very quick).

Everything else loads quick if I take a look at the adminpanel, only the language HMENU and the navigation HMENU take 1-2 seconds and 5-11 seconds respectively. The more or less same installation on 4.5 loads without cache in about 3 seconds.

I initially tried to convince my client to update to 6.2 because it is the new LTS with all its benefits, but the initial loading time is not acceptible for him (at least it is still quick with static cache after the first loading). Is the only solution putting the database to a SSD?

#35 Updated by Jörg Wagner almost 4 years ago

Is the only solution putting the database to a SSD?

As mentioned above there is also the option to use VHS viewhelper.

But our solution for now is that we have moved all database storage files to SSDs on our public servers. Luckily this is possible with our hoster at reasonable cost. This completely irons out the performance issues of TYPO3 6.2 for all our hosted customers and cured a lot of headaches.

Anyway, it is quite embarrassing that such measures are still necessary 15 months after launch of 6.2.

#36 Updated by Tao Nguyen almost 4 years ago

Jörg Wagner wrote:

Is the only solution putting the database to a SSD?

As mentioned above there is also the option to use VHS viewhelper.

Hi Jörg, thanks for the tip, It took me awhile to be able to create a 3 level columnar dropdown menu. Here is my code, if it might help someone. I had to adapt the css since I couldn't find out how to adapt Typoscript's wrapItemAndSub correctly with fluid. The Website loads faster now, 6-8 seconds and without spikes (with HMENU it sometimes took anything between 12 and 50 seconds).

{namespace v=FluidTYPO3\vhs\ViewHelpers}
<v:page.menu useShortcutData="TRUE" pageUid="264" expandAll="1" levels="3">
<ul class="menu">
  <f:for each="{menu}" as="item">
  <li>
    <f:link.page pageUid="{item.uid}" title="{item.linktext}">{item.linktext}</f:link.page>
    <f:if condition="{item.hasSubPages}">

      <v:page.menu pageUid="{item.uid}">
      <ul class="second">
        <f:for each="{menu}" as="subItem">
        <li>
          <f:link.page pageUid="{subItem.uid}" title="{subItem.linktext}">{subItem.linktext}</f:link.page>
            <f:if condition="{item.hasSubPages}">

              <v:page.menu pageUid="{subItem.uid}">
              <ul class="third">
                <f:for each="{menu}" as="subsubItem">
                <li>
                  <f:link.page pageUid="{subsubItem.uid}" title="{subsubItem.linktext}">{subsubItem.linktext}</f:link.page>
                </li>
                </f:for>
              </ul>
              </v:page.menu>

            </f:if>
        </li>         
        </f:for>
      </ul>
      </v:page.menu>

    </f:if>
  </li>
  </f:for>
</ul>
</v:page.menu>

#37 Updated by Simon Schaufelberger almost 4 years ago

Instead of nesting all the levels, a main template with section would be helpful in this case which calls itself. Something like this (from Neos):

https://git.typo3.org/Packages/TYPO3.Neos.NodeTypes.git/blob/HEAD:/Resources/Private/Templates/NodeTypes/Menu.html

#38 Updated by Philipp Kerling over 3 years ago

You don't need to repeat the menu for every level, just use <v:page.menu.sub>

#39 Updated by Riccardo De Contardi over 1 year ago

I've performed a test on a fresh 8.7.6 with the introduction package under the following conditions:

1) Environment: Xampp server on my Windows7 laptop (Intel Core I5 with 6GB ram, NO SSD disk); php 7.0 with:

max_input_time    60    
max_input_vars    1500
memory_limit    1024M    

2) Typoscript setup: the page contains just a 4-level menu and cache is disabled

config.no_cache=1

page >

page=PAGE

page.10 = HMENU
page.10 {
    1 = TMENU
    1 {
        expAll = 1
        wrap = <ul>|</ul>
        NO = 1
        NO {
            wrapItemAndSub = <li>|</li>
        }
    }
    2 < .1
    3 < .1
    4 < .1
}

3) Test: I've started adding a new page with 20 subpages per level each (I.e 20 pages at first sublevel, and each page has 20 subpages and so forth till the 4th level).

Even though I'm far from finished (I started with the intention of adding 20 pages at the first level with a complete 4-level tree of 20pages each), I could already share some results; I've taken screenshots of the admin panel
with various amounts of pages added.

I've had to add about 9000+ pages before the rendering time goes near 20 seconds.

In the latter condition (9000+ pages), with cache enabled and filled, the rendering time is about 300ms

Do you think I could perform some other tests? Or can I consider the issue solved? Thank you

#40 Updated by Julian Hofmann over 1 year ago

Beside the solutions with fluid or use of APC, there's another option to speedup big HMENUs with expAll: use the possibility of caching.
Normally, big parts of these menus does not change on each page, but the whole HMENU is rendered per page. Why not using identical parts of the menu for different pages? All inactive branches of the menu could be cached. This lowers the count of database queries.

For details of such a solution, there's a blogpost: http://stichwort.org/typo3/typoscript/mega-menue-mit-cache-optimieren/ (German only)

#41 Updated by Riccardo De Contardi about 1 year ago

  • Status changed from Needs Feedback to Closed

I think I can close this one for now, as there has been no activity since the last 4 months, and this perfomance issue
should be solved now for versions 7 and 8 (moreover, here a couple of possible strategies have been suggested to reduce the loading times).

If you think that this is the wrong decision or experience the issue again or think that there is still work to be done about this topic, please reopen it or open a new issue with a reference to this one.

#42 Updated by Jonas Ulrich 12 months ago

We've written an alternative, based on the addition to the blog post of the last comment, made on 2018-05-17:
Addition in the blogpost: https://gist.github.com/pgampe/cb29bc0fc1111d1370cc
Our generalized approach, based on that: https://gist.github.com/julrich/d91c1ac1968418e6ae8290f2a2e90afb

Maybe this is helpful for somebody else!
Handles cases like the special pages (index page, search page, pages not inside the root node of the menu, where the replacement RegExp otherwise would be an empty string, replacing everything), logged in users and languages.

Btw. what make you say this: "should be solved now for versions 7 and 8", Riccardo? At least for us, it wasn't (8.7).

#43 Updated by Riccardo De Contardi 12 months ago

@Jonas Ulrich basically there has not been much activity on this issue; I guessed it was solved because of the lack of people complaining about it... It it absolutely possible that I am wrong. I also tried an empiric test 9 months ago and had no feedback...

Please, if you have more information about how to reproduce the problem write it; I'll reopen it if you want.

Regards,

Riccardo

#44 Updated by Jonas Ulrich 12 months ago

Not sure if there is much to do about this, to be honest.

It's just expAll in a HMENU resulting in a lot of SQL calls, and corresponding TYPO3 function calls (render, makeMenu, prepareMenuItems, getSubpagesForPages, etc).

About 120% of the number of pages in SQL calls (e.g. 1000 pages, ~1200 SQL requests).
Approximately 1/8th is spent in actually executing the SQL queries (in a reference profile 841ms (1232 SQL requests) of 6.26s.
The rest (7/8) ist mostly in those mentioned calls.

I've attached a sample profiling, outlining those calls. Sorry about the quality, but had to quickly stitch this together from multiple screenshots:

profiling-hmenu.png

#45 Updated by Riccardo De Contardi 12 months ago

  • Status changed from Closed to New

Reopening.

#46 Updated by Markus Klein 12 months ago

@Jonas

1) Please edit your message and use the thumbnail feature to avoid this huge image inline.
2) Please add the corresponding typoscript which generated this graph.

Thanks.

#47 Updated by Jonas Ulrich 12 months ago

I've changed the image to use the thumbnail, sorry about that.

As our menu is quite complex, I can't really rip it out (multiple .ts-files) easily.
I will see (can't promise anything) if I can add a reduced example. But to be honest, to me it seems that the problem is just unchanged.

I'd hazard a guess, but the minimal example, posted by Jörg Wagner 3 years ago, should also still exhibit those performance penalties for large(r) menus:
https://forge.typo3.org/projects/typo3cms-core/activity?from=2014-12-17

#48 Updated by Jarvis H 7 days ago

  • Related to Bug #87729: MenuProcessor can't cache added

Also available in: Atom PDF