TYPO3 Core

Full VM for VirtualBox 4.2 (1GB): http://philippgampe.info/fileadmin/ubuntu-typo3.ova.zip

Philipp, did you also test the 4 other versions. Im interested to know if the last changes are the source of the bigger performance lost or that is was something we introduced in a earlier version. Compatibility layer etc.....

I ran all tests at least twice and warmed up the caches with a couple of browser requests beforehand. Thus those numbers represent a fully cached condition.
All tests are on the same virtual machine (see link above), on the same DB and a default set of packages/extensions. See the zip file attached for localconf.php, LocalConfiguration.php and PackageStates.php.

Tests are started from host and query the VM. It can be expected that additional I/O load has a bigger impact than on real-life servers.

TYPO3_4-5

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   1.252 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    79.86 [#/sec] (mean)
Time per request:       12.522 [ms] (mean)
Time per request:       12.522 [ms] (mean, across all concurrent requests)
Transfer rate:          96.55 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:    10   12   1.8     12      24
Waiting:       10   12   1.8     12      23
Total:         11   12   1.8     12      24

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     13
  75%     13
  80%     13
  90%     14
  95%     16
  98%     20
  99%     24
 100%     24 (longest request)

TYPO3_4-6

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   1.461 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    68.44 [#/sec] (mean)
Time per request:       14.612 [ms] (mean)
Time per request:       14.612 [ms] (mean, across all concurrent requests)
Transfer rate:          82.74 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:    12   14   1.7     15      26
Waiting:       12   14   1.7     14      26
Total:         13   15   1.7     15      26

Percentage of the requests served within a certain time (ms)
  50%     15
  66%     15
  75%     15
  80%     15
  90%     16
  95%     16
  98%     20
  99%     26
 100%     26 (longest request)

TYPO3_4-7

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   1.388 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    72.02 [#/sec] (mean)
Time per request:       13.884 [ms] (mean)
Time per request:       13.884 [ms] (mean, across all concurrent requests)
Transfer rate:          87.08 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:    12   14   2.1     13      28
Waiting:       12   13   2.1     13      28
Total:         12   14   2.1     13      29

Percentage of the requests served within a certain time (ms)
  50%     13
  66%     14
  75%     14
  80%     14
  90%     15
  95%     18
  98%     22
  99%     29
 100%     29 (longest request)

TYPO3_6-0

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   6.062 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    16.50 [#/sec] (mean)
Time per request:       60.620 [ms] (mean)
Time per request:       60.620 [ms] (mean, across all concurrent requests)
Transfer rate:          19.94 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    57   60   4.2     61      94
Waiting:       57   60   4.2     61      94
Total:         57   61   4.3     61      94

Percentage of the requests served within a certain time (ms)
  50%     61
  66%     61
  75%     62
  80%     62
  90%     63
  95%     65
  98%     70
  99%     94
 100%     94 (longest request)

TYPO3_6-1

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   11.001 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    9.09 [#/sec] (mean)
Time per request:       110.012 [ms] (mean)
Time per request:       110.012 [ms] (mean, across all concurrent requests)
Transfer rate:          10.99 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    90  110   8.1    109     137
Waiting:       90  109   7.7    109     136
Total:         90  110   8.2    109     137

Percentage of the requests served within a certain time (ms)
  50%    109
  66%    112
  75%    112
  80%    113
  90%    121
  95%    124
  98%    135
  99%    137
 100%    137 (longest request)

master as of today

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   14.083 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    7.10 [#/sec] (mean)
Time per request:       140.831 [ms] (mean)
Time per request:       140.831 [ms] (mean, across all concurrent requests)
Transfer rate:          8.58 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   126  141   7.6    139     188
Waiting:      126  140   7.6    139     188
Total:        126  141   7.7    139     190

Percentage of the requests served within a certain time (ms)
  50%    139
  66%    141
  75%    143
  80%    144
  90%    147
  95%    155
  98%    165
  99%    190
 100%    190 (longest request)

Do I see that right that this means:

Longest request 4.5 vs 6.0: factor ~4 slower
Longest request 6.0 vs 6.2: factor ~2 slower
Longest request 4.5 vs 6.2: factor ~8 slower

Oh darling....

Did you use PHP 5.3 as specified in the ticket here?

Yes I am using Ubuntu LTS 12.04 which ships PHP 5.3.10 with Suhosin. I expect that PHP 5.4 is 20-30% faster. PHP 5.5 is even faster.
Keep in mind that this is a VM and that I/O increase count higher here (more PHP cache files).

Also this is a very simple site. Complex sites might do better. Although I did a full cache scenario, therefore this are important numbers nevertheless.

<!DOCTYPE html>
<html lang="en">
<head>

<meta charset="utf-8">
<!-- 
    This website is powered by TYPO3 - inspiring people to share!
    TYPO3 is a free open source Content Management Framework initially created by Kasper Skaarhoj and licensed under GNU/GPL.
    TYPO3 is copyright 1998-2013 of Kasper Skaarhoj. Extensions are copyright of their respective owners.
    Information and contribution at http://typo3.org/
-->

<title>Root-Simple-TEXT</title>
<meta name="generator" content="TYPO3 6.2 CMS">

<link rel="stylesheet" type="text/css" href="typo3temp/stylesheet_8b9c2e8d89.css?1382018318" media="all">

</head>
<body>
<h1>Headline</h1><p>Some longer Text
which might be split across several lines
</p><div>Some footer</div>

</body>
</html>

This are just three TEXT objects.

Yes, especially when the page is extremely simple it shows that the surrounding overhead got slower a lot.

I do not think that we can justify an 400% increase for fully cached pages, because this is the most important scenario.

We really need to find out where this comes from and if we can take certain shortcuts on fully cached pages.

PHP 5.4 is IMHO a must for TYPO3 6.x. This way we already "regain" a bit (although 4.5 would be equally faster as well). Fully cached pages are for sure extremely important and to be optimized first but we should not forget that in many cases USER_INT have to be found. Just think about the felogin which is not AJAX capable.

Philipp,

would you mind installing APC and using it instead of t3lib_cache_backend_FileBackend? (see http://wiki.typo3.org/Caching_framework#t3lib_cache_backend_ApcBackend)


$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_reflection']['backend'] = 't3lib_cache_backend_ApcBackend';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['caching']['cacheConfigurations']['extbase_object']['backend'] = 't3lib_cache_backend_ApcBackend';


We found this has a huge impact on performance.

This should especially be true on an IO-limited VM.

@Xavier: I see this as you do, but in fact about 75% of the (german) hosting companies dont even offer PHP 5.4 (not to think of PHP 5.5), so TYPO3 6.x MUST run
with maximum performance on PHP 5.3...

@Alexander I am aware of APC and the caching framework, however you (usually) do not a memory cache on shared hostings.

@Xavier the impact of PHP 5.4 and 5.5 is much higher on 6.x, because we make better use of OOP then in old CMS installations.

@Frank choosing the right hoster should not be that difficult. You can get good hosting for 10-15€/month (120-180€/a). That should be affortable for every buisiness, especially if their website is not just a place to post the contact information.

• find the bottlenecks introduced with the latest versions
• (if possible) fix the code or
• find shortcuts for fully cached pages (parameters in matching chash)

Intro package, without cache¶

• Open homepage of intro package in 6.2, after clearing all caches
• main() takes 4,235,874 μs to complete
  • run_init::tslib/index_ts.php takes 4,215,982 μs
    • TYPO3\CMS\Core\Core\Bootstrap::loadConfigurationAndInitialize takes 3,108,059 μs
      • TYPO3\CMS\Core\Core\Bootstrap::initializePackageManagement takes 3,100,639 μs
        • TYPO3\CMS\Core\Package\PackageManager::initialize takes 3,092,816 μs
          • TYPO3\CMS\Core\Core\ClassLoader::setPackages takes 3,084,159 μs
            • TYPO3\CMS\Core\Core\ClassAliasMap::setPackages takes 3,083,876 μs
              • TYPO3\CMS\Core\Core\ClassAliasMap::buildMappingFiles takes 3,034,559 μs

Intro package, second run¶

• main() takes 746,521 μs to complete
  • run_init::tslib/index_ts.php takes 730,117 μs
    • 1. TYPO3\CMS\Frontend\Page\PageGenerator::renderContent takes 340,651 μs
    • 2. TYPO3\CMS\Core\Core\Bootstrap::loadExtensionTables takes 102,665 μs

So without cache, I would start by optimizing PackageManager

Xavier Perseguers wrote:

[...]

So without cache, I would start by optimizing PackageManager

+1

I'seen comparable numbers on my dev VM and also typo3-6-2beta1-fully-cached2.png points in that direction (24.33 % in Core\Cache\Frontend\PhpFrontEnd::requireOnce()).

Keep in mind that the cache for the Package Manager and Class Loader are built completely upfront to prevent keeping the alias and class map in memory. These arrays contain about 2500 entries each on average and 3 of them have to be kept in memory.
I'm currently exploring a different approach to this, so please consider optimizing the queries of the session management for frontend and backend users for now which send about 2 queries each for example. Usually noone runs TYPO3 without caches in production and the first single request will take some seconds untill all caches are populated. So I would propose to optimize the cached requests first, and with cached I mean USER and USER_INT as the latter is also cached somehow.

I fully second Tom here. PackageManager Caches are filled on first request and if you do not change packages (which happens rarely on life sites), then the package manager cache will always be "warm".

I also stumbled over session management, which seems to be really ineffective. Having a backend user cookie (not taken into account above as request came from ab) will add at least another three milliseconds even if you are not logged in.

@Thorsten Kahler Kahler

If you do profiling with Xdebug or something else, you will ever see such things like "24.33 % in Core\Cache\Frontend\PhpFrontEnd::requireOnce()" for files that will be included. To get more real numbers you should change your setup.

For Xdebug, disable xdebug.collect_includes ( http://xdebug.org/docs/all_settings ) to get better numbers about the parts that do the trouble.

No this is without xdebug, but with xcache (32MB) enabled. All posted numbers are the second or third run, thus the xcache was always warm.

Are these numbers generated with a bytecode cache or without?

Interesting: Without xcache, the performance seems to be better.

(I will provide the numbers without bytecode cache in the next hour; the next two blocks are generated while running on battery, thus might be lower the needed).

TYPO3_4-5 slower without bytecode cache by a factor 4.5

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        937 bytes

Concurrency Level:      1
Time taken for tests:   5.310 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      123800 bytes
HTML transferred:       93700 bytes
Requests per second:    18.83 [#/sec] (mean)
Time per request:       53.099 [ms] (mean)
Time per request:       53.099 [ms] (mean, across all concurrent requests)
Transfer rate:          22.77 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    48   53   4.4     52      81
Waiting:       47   53   4.4     52      81
Total:         48   53   4.5     52      82

Percentage of the requests served within a certain time (ms)
  50%     52
  66%     53
  75%     56
  80%     57
  90%     58
  95%     59
  98%     61
  99%     82
 100%     82 (longest request)

master faster without bytecode cache ???

$ ab -n 100 "http://ubuntu-typo3.local/dummy/index.php?id=1" 
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking ubuntu-typo3.local (be patient).....done

Server Software:        Apache/2.2.22
Server Hostname:        ubuntu-typo3.local
Server Port:            80

Document Path:          /dummy/index.php?id=1
Document Length:        752 bytes

Concurrency Level:      1
Time taken for tests:   9.905 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      105300 bytes
HTML transferred:       75200 bytes
Requests per second:    10.10 [#/sec] (mean)
Time per request:       99.052 [ms] (mean)
Time per request:       99.052 [ms] (mean, across all concurrent requests)
Transfer rate:          10.38 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:    85   99   5.6     99     134
Waiting:       85   99   5.6     99     134
Total:         86   99   5.7    100     135

Percentage of the requests served within a certain time (ms)
  50%    100
  66%    101
  75%    101
  80%    102
  90%    103
  95%    105
  98%    114
  99%    135
 100%    135 (longest request)

@Philipp:

No this is without xdebug, but with xcache (32MB) enabled. All posted numbers are the second or third run, thus the xcache was always warm.<<<

I spoke to Thorsten Kahler and not to you and your numbers. You also do benchmarking and not profiling.

Let us know if you want to test this in one of our hosting packages. There you can switch instantly between PHP 5.3/5.4 and enable/disable APC, i.e. use the same setup with different PHP/APC configurations.

Hi Jochen,

thats a nice offer. We should define scenarios first what we need to benchmark, the single page from cache isn't the only speed problem.

Alexander Opitz wrote:

@Philipp:

No this is without xdebug, but with xcache (32MB) enabled. All posted numbers are the second or third run, thus the xcache was always warm.<<<

I spoke to Thorsten Kahler and not to you and your numbers. You also do benchmarking and not profiling.

@Alex: I referred to Philipps diagram, so Philipp is the right one to answer.

PhpFrontend::requireOnce() does not do anything except:

return $this->backend->requireOnce($entryIdentifier);

I profiled 6.2 with XDebug, top time eating classes are from the caching framework and class loader.

Notice the last line in the image. File typo3temp/Cache/Code/cache_classes/tx_realurl.php contains the following:

<?php require __DIR__ . '/../../../../typo3conf/ext/realurl/class.tx_realurl.php';

What is that? Including the file from cache , which just includes the real file? If it is the same for every other file, we double amount of require_once classes, which is slow because it works with the file system.

@Thorsten Kahler

Oh I thought it was your diagram, shame on me.

But in real the require_once isn't such cost expensive without profiling, I got such high numbers with Xdebug before I disabled "xdebug.collect_includes".

@Dmitry Dulepov

In refer to your message: "What is that? Including the file from cache , which just includes the real file?"

Please take a look at http://wiki.typo3.org/Blueprints/Packagemanager#Implementation_Details why we have this handling.

@all review testers

After profiling and writing with Thomas Maroschik, I was able to identify a problem, which may also affect your tests.
If you have a symlink inside the path to your "/typo3temp/Cache/Code/" directory, the caching may fail.

To verify this problem, open in /typo3temp/Cache/Code/ your PackageManager_*.php and remove the following Part

__DIR__ !== '{your path}/typo3temp/Cache/Code/cache_core' ? FALSE :

so your cache file should be read:

array (
  'packageStatesConfiguration' =>
  array (
    'packages' =>
    array (
...

And benchmark your installation again.

On my installation the reqests per second changed
with Xdebug from 145 to 229
without Xdebug from 221 to 305

Thanks Alexander.

Anyway, my numbers are from a default install with typo3_src link structure and real folders for the rest (automatically created by the installer).

@Alexander Opitz

Hmmm, this is interesting :)

Do you mean "The backend creates "proxy require" files so that the memory consumption of the autoloader is lowered"? Was memory a real issue for anybody? To me it does not look good to solve memory issues by doubling amount of inclusions.

After I removed that line, I got 6.78 rq/s. With that line it is 3.28 rq/s. Used "ab -n 100" for testing. What hardware do you have with such rq/s numbers?

@Dmitry Dulepov

About memory consumption and the "proxy require" I can't tell much. The Blueprint is written by Thomas Maroschik. But while developing the PackageManager I did benchmarking the autoloader from TYPO3 6.0/6.1 his biggest issues are memory consumption for all the aliases parts. First try to remove this was eliminating the aliasing if not needed. But we came to the point where an alias is needed but isn't autoloaded. The second try from Thomas was using a Symlink structure. It was faster but APC and Xcache didn't performed well on them as they had problems with caching the data. So Thomas came with this solution of the "proxy require", they will be cached by APC, Xcache or OPcache. While using Xdebug, xhprof or something like that it will decrease the performance more then the symlink solution (thats what I get from my numbers).

The mentioned line doesn't have to do with this "proxy require" it is more or less a nitpik which may be removed, it prevents using the PackageManager and PackageObject cache which do not have to do with the "proxy require".

My Hardware/Software:

- 8 core 64Bit CPU
- 12 GB Ram
- Linux Kernel 3.11
- apache 2.4.6
- PHP 5.5.3 with OPcache 7.0.3
- MySQL 5.5.34

Test with "ab -n 1000 -c 100 typo3-master.sphinx"

Is it possible to get some cachegrind dumps?
I am rather busy this week, but reading a cachegrind file might be possible.

Just don't have the time to set up my whole profiling env this week

@Mathias please have a look into the profiles.tar.gz attached to the issue. It includes some "callgrind" files for both 4.5 and 6.2; keep in mind that they are create with a different tool the usual cachegrind files, but work fine in e.g. kcachegrid.

Thanks.
I need to check the includes and called functions to determine the version or do you have it in mind which dump was what?

• is_file vs. file_exists
• isset vs array_key_exists
  Please also keep in mind that I didn't dig into the code yet.
  When you cache file paths you check, do you use absolute paths?
  If not, let's do that as well, since it will remove the entire include_path checks as well.

From what I've seen so far the whole "is the file there" things takes up a huge chunk of time.
This would be a good thing to start digging.

Let's have a chat on that this weekend

@Mathias:
Is is_file really so much faster in the end? Both need to stat the file by the OS syscall.

isset is indeed faster, but it behaves differently when it comes to array values being NULL.
See example#2 here http://us2.php.net/array_key_exists

Therefore such changes might be dangerous.

Markus Klein wrote:

@Mathias:
Is is_file really so much faster in the end? Both need to stat the file by the OS syscall.

• SAN FS (Nexenta) mounted via NFS (Ubuntu 12.04): 12% increase
• SAN FS mounted via iSCSI onto VMWare ESXI (Ubuntu 12.04): 48% increase
• Samsung 840 EVO Native OSX Mountain Lion: 120% increase

So no, it is not "so much faster", but going 1kph faster still makes you win the race :)
I suggest to improve whereever we can, even for the slightest improvement.
I read something interesting about not using require but using include instead, since most of the time you try/catch that anyways (or exception out of it at least).
So it might be possible to get rid of a lot the file_exists (don't think all will be possible, though).

isset is indeed faster, but it behaves differently when it comes to array values being NULL.
See example#2 here http://us2.php.net/array_key_exists

Yeah, that's a typical one.
Personally I consider null values in arrays bad practice but opinions may of course differ here.
At least we should try to get rid of these whereever possible.
Maybe even change them all and then check what breaks.
The speed improvement is tremendous.

Therefore such changes might be dangerous.

No pain, no gain.
If the system was like 10% slower, fine - I wouldn't even bother.
But 800% or heck... even 50% is something we just cannot allow ourselves.

I agree that we must do whatever we can to improve performance. However I would prefer to profile typical calls, find 1-2 main bottlenecks, fix them whichever way makes it drop dramatically in term of cost and start again.

I think we also can't change all file_exists into is_file, as is_file returns false for symlinks.

@Mathias: Which PHP version did you use for your benchmarks?

Alexander Opitz wrote:

I think we also can't change all file_exists into is_file, as is_file returns false for symlinks.

Well, don't get me wrong, I don't know the "new" core and don't have the time to take a look at it since I have a company to run.
I can just take a look at cachegrinds and similar and come up with proposals.
I have to leave it up to you to decide which improvement belongs at which point in the code.

@Mathias: Which PHP version did you use for your benchmarks?

I used 5.3.25 or 27.
Just to suppress all "with 5.x everything will be faster/better/bla" that might come up (ignore if you don't feel addressed :)):
We can't afford to wait for stuff like that.
5.3 is the major player out in the field, waiting for newer versions just states being lazy.
If newer PHP version speed things up - even better.
But as for now we need to focus on what the market demands and the competition delivers in order to stay competitive. And just in case this didn't come through... that's the only thing I care about - TYPO3 being competitive.
If you guys get that done with the "100.000 files to require, although logic dictates it is slower" approach, that's fine with me.
We all just need to be on the same track here.
So sacrifying competitiveness for "by-the-book-code" is not an option :)

Xavier Perseguers wrote:

find 1-2 main bottlenecks, fix them whichever way makes it drop dramatically in term of cost and start again.

agreed.
Apart from that I will try to convince Rupi to revive and supply his old testing setup so we all have a common base to test on.
Luckily this instance covers what you mean by "typical calls", since it features basically everything a 4.3 frontend offered in that time.

So all in all:
I love everything you do for TYPO3, as long as you love everything I do for TYPO3 - joined forces for the greater good.

Probably we can set something to cache at APC instead of file system if APC is available.

Dmitry Dulepov wrote:

Probably we can set something to cache at APC instead of file system if APC is available.

Is APC meanwhile save on shared hosting environments to enable it by default on any installation? That was a blocker some years ago. Is this still true?

APC ist still a problem in shared hosting when you not use FCGI. A solution more general would be better! And every solution which ist fast without any extra software would be even faster with a bytecode cache!

The Install Tool has a "Configuration Preset" which detects if APC is present and has enough free memory and recommends in this case to switch some Extbase caches to it. So it's not "enabled by default" but a recommended setting through the Install Tool if available (and auto-configured this way if you install through the Step Installer and it detects this situation).

The "default" has still to be File or Database based caching - as these are always available.

A speed improvement for fully cached pages: http://forge.typo3.org/issues/53404

http://www.php.net/ChangeLog-5.php#5.5.6

• Improved performance of array_merge() and func_get_args() by eliminating useless copying.

This could bring quite a bit of performance for TYPO3 since array_merge() is used ~400 times in Core.

(Disclaimer: This is just a note and not an excuse for the necessary software improvements.)

A very small performance thing, yet done on every request: http://forge.typo3.org/issues/53702

I think i found another one:

http://forge.typo3.org/issues/53962

And this one is a big slowdown: http://forge.typo3.org/issues/53862

Since 6.2alpha2, many speedups are implemented. The backend really seams faster now. But on my frontend test (empty cached page), nothing changed (compared alpha2 with master of today). The speedup from #53598 is measurable, but we regressed somewhere else so that we don't see any change in the comparison. Till yet, we are slower then 6.1.7 (for this test case).

How does it look for your tests?
@Dmitry Dulepov can you post latest results from your system?

@Dmitry Dulepov You may need the patch from #55022 for your setup (as described above on comment 30).

You consider this solved? What about the huge amount of open sub-tickets?

I would still consider the performance of the new extension manager as horrible, especially on windows. Don't know much about the rest.

Markus, can you please check the performance 7.x vs. 4.5 should be a lot faster, as we can skip the doubled class loading there ...

I'm not checking anything here, just cleaning up the bug tracker.