Task #50484
closed
Generating URLs takes very long
0%
Description
Hi all
We found out that generating simple URLs using $cObj->typoLink_URL() or $this->uriBuilder->uriFor() in ExtBase plug-ins take very long compared to other frameworks like Zend.
Time for creating 100 URLs without RealURL:
- standard URLs: 113.239050ms
- absolute URLs: 123.664141ms
- with CHash: 110.864878ms
- without CHash: 87.856054ms
Time for creating 100 URLs with RealURL:
- standard URLs: 194.674969ms
- absolute URLs: 205.719948ms
- with CHash: 193.242073ms
- without CHash: 168.409109ms
Even when disabling RealURL, more than 100ms are necessary to generate 100 URLs while the Zend Framework needs around 10ms to do the same. Usually, product lists in e-commerce sites generate a lot of similar URLs. Is there a possibility to speed up generating the URLs by using some sort of caching?
For testing, I've used a modified version of the efempty plugin and the relevant code is:
public function urlAction()
{
$results = array();
$results[] = 'Time for creating 100 URLs:';
$start = microtime( true );
for( $i = 0; $i < 100; $i++ ) {
$this->uriBuilder->reset()->uriFor( 'url', array(), 'Start' );
}
$results[] = sprintf( '- standard URLs: %1$fms', ( microtime( true ) - $start ) * 1000 );$start = microtime( true );
for( $i = 0; $i < 100; $i++ ) {
$this->uriBuilder->reset()->setCreateAbsoluteUri( true )->uriFor( 'url', array(), 'Start' );
}
$results[] = sprintf( '- absolute URLs: %1$fms', ( microtime( true ) - $start ) * 1000 );$start = microtime( true );
for( $i = 0; $i < 100; $i++ ) {
$this->uriBuilder->reset()->setUseCacheHash( true )->uriFor( 'url', array(), 'Start' );
}
$results[] = sprintf( '- with CHash: %1$fms', ( microtime( true ) - $start ) * 1000 );$start = microtime( true );
for( $i = 0; $i < 100; $i++ ) {
$this->uriBuilder->reset()->setUseCacheHash( false )->uriFor( 'url', array(), 'Start' );
}
$results[] = sprintf( '- without CHash: %1$fms', ( microtime( true ) - $start ) * 1000 );return join( '<br/>', $results );
}Best regards,
Norbert
Files
Updated by Philipp Gampe over 10 years ago
- Status changed from New to Needs Feedback
Please ask just questions in the mailing list/forums (typo3.english is a good place).
This depends a bit on your settings (multiple domains, TS, server, rootline cache, parameters).
Anyway, typolink is known for taking very long. This is due to all the magic that is used.
May I close the bug report?
Updated by Norbert Sendetzky over 10 years ago
Hi Philipp
This depends a bit on your settings (multiple domains, TS, server, rootline cache, parameters).
Sorry, forgot to mention the environment:
- TYPO3 introduction package 6.1
- Single domain
- Plugin placed on one sub-page (/test)
- Caching is enabled in intro package but test action is non-cachable
- No parameters were used (as you can see in the code above)
- Performed on a Lenovo T400
Anyway, typolink is known for taking very long. This is due to all the magic that is used.
May I close the bug report?
If it takes so long even for generating simple use cases, maybe there are possibilities to speed this up by taking some shortcuts. It would be great if we could find out the main bottlenecks and improve typoLink() for 6.2 as part of the performance improvements being on the agenda for 6.2.
Best regards,
Norbert
Updated by Philipp Gampe over 10 years ago
- Estimated time set to 20.00 h
- Translation missing: en.field_remaining_hours set to 20.0
Well, there are not simple shortcuts.
We need to- move the typolink code into an own class
- cover this one with unit tests
- split the code into smaller functions
- cover those with test
- then implement (runtime) caches where it makes sense
The problem is known since very long.
Updated by Ernesto Baschny over 10 years ago
Refactoring typoLink would be a very good thing to do, maybe someone could take a try with it?
I would endorse this as this is a long standing and known issue, but agree with the proposed plan by Philipp, which is covering it with unit tests first and then refactoring it's inner workings. Moving this huge function (>450 lines!) into it's own class (and along with it some of the helper methods it needs) would be a good first refactoring step - after we have unit tests to make sure everything still works after the refactoring.
Just then we can start thinking about adding caches, short circuits to speed up processing, etc.
Updated by Markus Klein over 10 years ago
There's already a first start by Benni: https://review.typo3.org/21785
Updated by Ernesto Baschny over 10 years ago
- Status changed from Needs Feedback to On Hold
- Target version deleted (
6.2.0)
Good finding, Markus, thanks. So let's continue on that track!
Updated by Philipp Wrann about 10 years ago
Also discovered this bottleneck.
In a result-list with 20 results and 3 a tags per result for each link many queries are made (getting different sys_template records, different pages-records).
When using a first-level cache to prevent from multiple uri generation for the same object (so dropping 40 build-link-processes) the parsing time decreased by MAAAAANY milliseconds, i think it went down from 1000 to 800 or something.
THAT is a HUGE bottleneck. And you cant know that without digging very deep, so you wont prevent multiple rendering of the same uri...
So, yep - the typoLink functionallity definitly needs a refactoring.
Updated by Markus Klein about 9 years ago
- Status changed from On Hold to Needs Feedback
- Translation missing: en.field_remaining_hours deleted (
20.0)
Anyone interested in continuing here for CMS 7?
Updated by Alexander Opitz about 9 years ago
As typoLink function is complex and only the first parts are done in splitting typoLink function in better read/testable parts ... I donÄt think we will get something for 7LTS ... so maybe move this to 8 LTS
Updated by Alexander Opitz almost 9 years ago
- Status changed from Needs Feedback to New
Remove needs feedback as task is very clear.
Updated by Benni Mack about 8 years ago
- Status changed from New to Accepted
- Assignee set to Benni Mack
I'll assign it to myself as I'm in charge of cleaning up the typolink part.
Updated by Susanne Moog over 6 years ago
- Category changed from Content Rendering to Link Handling, Site Handling & Routing
Updated by Susanne Moog about 5 years ago
- Status changed from Accepted to Needs Feedback
@Benni Mack: Anything new to add here since the changes in latest versions?
Updated by Benni Mack about 5 years ago
Susanne Moog wrote:
@Benni Mack: Anything new to add here since the changes in latest versions?
By NOT using RealURL anymore and having Site Handling basically re-using the Rootline Utility / Rootline Cache), DB queries get reduced very much. I'm eager to get new information on how the perf stats look like. Esp. TYPO3 v10.0 with even simpler code regarding "cHash" calculation will help even more.
Updated by Benni Mack about 4 years ago
Hey Norbert,
can you re-run the tests with TYPO3 v9 or v10 to create 1000 links to the same page now?
Updated by Benni Mack almost 4 years ago
- Status changed from Needs Feedback to Closed
Closing this issue due to lack of feedback. We've improved link generation again in v9.5.18, so I believe we can close this issue from 7ys ago. Let me know, if I should re-open the ticket once we have more data.