TYPO3 Core

State: PHP 5.5.12-2ubuntu4.2 (cli) (built: Feb 13 2015 18:56:49)

• Tests: 5548, Assertions: 6905, Skipped: 152
• Time: 14.99 seconds, Memory: 513.75Mb

State: PHP 7.0.0-dev (cli) (built: Feb 25 2015 09:59:30)

• Some sporadic crashes
• Crash in Extbase tests
• Tests: 5548, Assertions: 6916, Failures: 7, Skipped: 137
• Time: 11.17 seconds, Memory: 344.00Mb

(Less skipped tests as PHP 7 runs the PDO tests)

• Tests: 6956, Assertions: 9010, Skipped: 160.
• Time: 17.98 seconds, Memory: 641.00Mb

• No sporadic crashes anymore
• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Tests: 6956, Assertions: 8994, Errors: 23, Failures: 5, Skipped: 145.
• Time: 13.21 seconds, Memory: 284.00Mb

• Tests: 6957, Assertions: 9009, Skipped: 160.
• Time: 17.56 seconds, Memory: 641.25Mb

• (!) DEBUG Build
• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Less skipped tests as it runs the PDO tests
• Tests: 6957, Assertions: 9010, Errors: 22, Failures: 5, Skipped: 145.
• Time: 27.89 seconds, Memory: 314.00Mb

• Tests: 7040, Assertions: 9127, Skipped: 160.
• Time: 18.05 seconds, Memory: 646.00Mb

• (!) DEBUG Build
• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Less skipped tests as it runs the PDO tests
• Tests: 7040, Assertions: 9128, Errors: 22, Failures: 4, Skipped: 145.
• Tests: 7040, Assertions: 9128, Errors: 22, Failures: 3, Skipped: 145. (After commit of #67189)
• Time: 27.34 seconds, Memory: 318.00Mb

• Tests: 7185, Assertions: 9274, Skipped: 160.
• Time: 21.88 seconds, Memory: 654.75Mb

• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Less skipped tests as it runs the PDO tests
• Missing 44 Tests against PHP 5.5?
• Tests: 7141, Assertions: 9234, Errors: 19, Failures: 3, Skipped: 145.
• Time: 12.23 seconds, Memory: 286.00Mb

• Tests: 7228, Assertions: 9306, Skipped: 160.
• Time: 18.66 seconds, Memory: 655.25Mb

• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Less skipped tests as it runs the PDO tests
• Tests: 7228, Assertions: 9315, Errors: 15, Failures: 3, Skipped: 145.
• Time: 13.85 seconds, Memory: 288.00Mb

Nice reading and benchmarks.

http://talks.php.net/fluent15#/

With https://review.typo3.org/#/c/41274/1 I have no errors and no failures in the tests (but still skips the request things) and backend works quite well. Let's join forces :)

• Tests: 7696, Assertions: 10036, Errors: 8, Skipped: 162.
• Time: 20.83 seconds, Memory: 675.25Mb

• crash in extbase/Tests/Unit/Mvc/Web/RequestBuilderTest.php (Test deactivated)
• Less skipped tests as it runs the PDO tests
• Tests: 7696, Assertions: 10028, Errors: 11, Failures: 1, Skipped: 179.
• Time: 14.68 seconds, Memory: 296.00Mb

Fixed my typo3 master for the phpunit runnings, added PDO tests to 5.6.x. Also the PHP7 crash is fixed, so here are new numbers

• Tests: 7694, Assertions: 10071, Skipped: 147.
• Time: 20.84 seconds, Memory: 678.50Mb

• Tests: 7694, Assertions: 10067, Errors: 3, Failures: 1, Skipped: 147.
• Time: 14.48 seconds, Memory: 296.00Mb

After releasing PHP 7.0beta3 all seams fine.

• Tests: 7750, Assertions: 10142, Skipped: 147.
• Time: 20.02 seconds, Memory: 686.75Mb

• Tests: 7750, Assertions: 10142, Skipped: 147.
• Time: 14.89 seconds, Memory: 302.00Mb

Will do regular updates till PHP 7.0 is released which seams to be some days before our TYPO3.CMS 7 LTS.

Status Functional Tests

PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)

Time: 58.46 minutes, Memory: 21.75Mb

OK, but incomplete, skipped, or risky tests!
Tests: 691, Assertions: 10783, Skipped: 4.

PHP 7.0.0-dev (cli) (built: Aug 11 2015 10:27:29)

Time: 51.57 minutes, Memory: 12.00Mb

There were 6 errors:

1) TYPO3\CMS\Core\Tests\Functional\DataHandling\IRRE\ForeignField\Modify\ActionTest::modifyParentContent
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

2) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\IRRE\CSV\Modify\ActionTest::createParentContent
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

3) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\IRRE\CSV\Publish\ActionTest::modifyParentContent
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

4) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\IRRE\ForeignField\PublishAll\ActionTest::deletePage
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

5) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\ManyToMany\Publish\ActionTest::changeCategoryRelationSorting
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

6) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\ManyToMany\Publish\ActionTest::createContentAndCreateRelation
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

FAILURES!
Tests: 691, Assertions: 10681, Errors: 6, Skipped: 4.

As PHP7 do not crash for travis and some other developers, I need to dig deeper before reporting. It only core dumps in the ActionTest's and it won't core dump if you run every test by themself.
Like with phpunit7 -c typo3/sysext/core/Build/FunctionalTests.xml typo3/sysext/core/Tests/Functional/DataHandling/IRRE/ForeignField/Modify/ActionTest.php

Unittests¶

• Time: 20.77 seconds, Memory: 691.00Mb
• Tests: 7810, Assertions: 10218, Skipped: 147.

• DEBUG Build
• Time: 30.48 seconds, Memory: 340.00Mb
• Tests: 7810, Assertions: 10218, Skipped: 147.

Functionals¶

Time: 43.74 minutes, Memory: 14.00Mb

There were 4 errors:

1) TYPO3\CMS\Core\Tests\Functional\DataHandling\FAL\Modify\ActionTest::copyContent
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

2) TYPO3\CMS\Core\Tests\Functional\DataHandling\IRRE\ForeignField\Modify\ActionTest::modifyParentAndDeleteHotelChild
PHPUnit_Framework_Exception: Segmentation fault (core dumped)

3) TYPO3\CMS\Extbase\Tests\Functional\Persistence\TranslationTest::fetchingPostsReturnsGreekPostsWithFallback
PHPUnit_Framework_Exception: Aborted (core dumped)

4) TYPO3\CMS\Workspaces\Tests\Functional\DataHandling\Select\Modify\ActionTest::copyContentOfRelation
PHPUnit_Framework_Exception: Aborted (core dumped)

FAILURES!
Tests: 691, Assertions: 10735, Errors: 4, Skipped: 4.

One third slower but half the memory... :-|

Unittests¶

• PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)
  • Time: 21.21 seconds, Memory: 698.75Mb
  • Tests: 7854, Assertions: 10263, Skipped: 147.

• PHP 7.0.0-dev (cli) (built: Sep 3 2015 12:08:00)
  • Time: 13.01 seconds, Memory: 304.00Mb
  • Tests: 7854, Assertions: 10263, Skipped: 147.

Functionals¶

• PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)
  • Time: 42.63 minutes, Memory: 22.00Mb
  • Tests: 697, Assertions: 10789, Skipped: 4.

• PHP 7.0.0-dev (cli) (built: Sep 3 2015 12:08:00)
  • Time: 39.49 minutes, Memory: 12.00Mb
  • Tests: 697, Assertions: 10789, Skipped: 4.

The times for the functional tests are more dependent on the database and what does happen on the testing system. Don't use them as Benchmark.

Unittests¶

• PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)
  • Time: 24.13 seconds, Memory: 764.00Mb
  • Tests: 8214, Assertions: 10877, Skipped: 148, Incomplete: 1.

• PHP 7.1.0-dev (cli) (built: Sep 17 2015 10:02:25)
  • Time: 18.2 seconds, Memory: 332.00Mb
  • Tests: 8214, Assertions: 10877, Skipped: 148, Incomplete: 1.

• PHP 7.0.0-dev (cli) (built: Sep 17 2015 14:58:54)
  • Time: 15.7 seconds, Memory: 332.00Mb
  • Tests: 8214, Assertions: 10877, Skipped: 148, Incomplete: 1.

Ups, 7.0 is branched off, so this is a 7.1 dev build this time. (Added unit test for PHP 7.0 branch)

Functionals¶

• PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)
  • Time: 40.79 minutes, Memory: 21.75Mb
  • Tests: 697, Assertions: 10789, Skipped: 4.

• PHP 7.1.0-dev (cli) (built: Sep 17 2015 10:02:25)
  • Time: 42.93 minutes, Memory: 12.00Mb
  • Tests: 697, Assertions: 10789, Skipped: 4.

The times for the functional tests are more dependent on the database and what does happen on the testing system. Don't use them as Benchmark.

Unittests¶

• PHP 5.6.4-4ubuntu6.2 (cli) (built: Jul 2 2015 15:29:28)
  • Time: 24,39 seconds, Memory: 778,50Mb
  • Tests: 8240, Assertions: 10918, Failures: 1, Skipped: 158, Incomplete: 1.

• PHP 7.0.0-dev (cli) (built: Sep 28 2015 11:40:46)
  • Time: 17,96 seconds, Memory: 338,00Mb
  • Tests: 8240, Assertions: 10918, Failures: 1, Skipped: 158, Incomplete: 1.

Hey Alexander,

are we PHP7 compliant? I use php7 for my local development now and now problems so far.

We are 7 compliant, this Epic is only open for running phpUnit "Benchmarks" till PHP 7.0.0 is released.

Hint: System updated to Ubuntu 15.10!¶

Unittests¶

• PHP 5.6.11-1ubuntu3 (cli)
  • Time: 27,16 seconds, Memory: 829,75Mb
  • Tests: 8504, Assertions: 11266, Skipped: 143, Incomplete: 1.

• PHP 7.0.0-dev (cli) (built: Oct 27 2015 09:30:09) ( NTS )
  • Time: 18,88 seconds, Memory: 352,00Mb
  • Tests: 8504, Assertions: 11243, Skipped: 158, Incomplete: 1.
  • Need to check what to compile to get less skipped ones (like 5.6.11)

New PHP7 release date: 26th of november

https://wiki.php.net/todo/php70#timetable
http://news.php.net/php.internals/89100

Unittests¶

• PHP 5.6.11-1ubuntu3.1 (cli)
  • Time: 29 seconds, Memory: 826,25Mb
  • Tests: 8473, Assertions: 11153, Skipped: 143, Incomplete: 1.

• PHP 7.0.1-dev (cli) (built: Nov 2 2015 10:42:45) ( NTS )
  • Time: 19,6 seconds, Memory: 354,00Mb
  • Tests: 8473, Assertions: 11130, Skipped: 158, Incomplete: 1.

PHP7 RC8 released: http://php.net/archive/2015.php#id2015-11-26-1

Unittests¶

• PHP 5.6.11-1ubuntu3.1 (cli)
  • Time: 26,07 seconds, Memory: 843,75Mb
  • Tests: 8530, Assertions: 11215, Skipped: 143, Incomplete: 1.

• PHP 7.0.2-dev (cli) (built: Dec 17 2015 11:50:40) ( NTS )
  • Time: 18,6 seconds, Memory: 360,00Mb
  • Tests: 8530, Assertions: 11192, Skipped: 158, Incomplete: 1.

• PHP 7.1.0-dev (cli) (built: Dec 17 2015 13:57:35) ( NTS )
  • Time: 18,5 seconds, Memory: 360,00Mb
  • Tests: 8530, Assertions: 11192, Skipped: 158, Incomplete: 1.

Last Update of the Benchmarks, as TYPO3 8 LTS will be PHP >= 7.0.

• PHP 5.6.11-1ubuntu3.1 (cli)
  • Time: 21,24 seconds, Memory: 804,25Mb
  • Tests: 8144, Assertions: 10684, Skipped: 158, Incomplete: 1.
• PHP 7.1.0-dev (cli) (built: Feb 15 2016 12:56:35) ( NTS )
  • Time: 15,23 seconds, Memory: 340,00Mb
  • Tests: 8144, Assertions: 10684, Skipped: 158, Incomplete: 1.

Upgrading my Ubuntu packages with http://www.lornajane.net/posts/2016/php-7-0-and-5-6-on-ubuntu to have PHP56 and PHP7 installed parallel.

• PHP 5.6.18-3+deb.sury.org~wily+1 (cli)
  • Time: 21,01 seconds, Memory: 805,00Mb
  • Tests: 8144, Assertions: 10684, Skipped: 158, Incomplete: 1.
• PHP 7.0.3-3+deb.sury.org~wily+1 (cli) ( NTS )
  • Time: 15,59 seconds, Memory: 340,00Mb
  • Tests: 8144, Assertions: 10684, Skipped: 158, Incomplete: 1.