Task #56086

Epic #55070: Workpackages

Epic #54542: WP: Importer / Exporter with relations MM/IRRE/FAL

Story #55522: As a developer I want to ensure that further changes have no negative impact on import or export

Impexp functional tests do not fail on wrong assertion

Added by Marc Bastian Heinrichs over 5 years ago. Updated about 2 years ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Tests
Target version:
Start date:
2014-02-18
Due date:
% Done:

100%

TYPO3 Version:
6.2
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Since this commit https://git.typo3.org/Packages/TYPO3.CMS.git/commit/67c4204dcb7131e1ed2b6aa02c2abdec2aa6ce93
from issue #53962

the functional tests in impexp did not fail on wrong assertion.

E.g. if you change a node value in

typo3/sysext/impexp/Tests/Functional/Fixtures/ImportExport/pages-and-ttcontent.xml

und running the simple import test

./typo3conf/ext/phpunit/Composer/vendor/bin/phpunit --colors -v --process-isolation --bootstrap typo3/sysext/core/Build/FunctionalTestsBootstrap.php typo3/sysext/impexp/Tests/Functional/ImportExport/ExportSimpleTest.php

The test is green, but with 0 assertions

OK (1 test, 0 assertions)

If I revert the commit the test fails what is right in this case.

FAILURES!
Tests: 1, Assertions: 2, Failures: 1.

Confirmed by Anja and Olly.


Related issues

Related to TYPO3 Core - Bug #53962: Class loader does not cache non existing classes Closed 2013-11-25
Related to TYPO3 Core - Bug #56187: Add additional configuration for functional tests Closed 2014-02-21

Associated revisions

Revision 86cef9f8 (diff)
Added by Helmut Hummel over 5 years ago

[BUGFIX] Remove class loader in functional test tear down

The functional test teardown does not remove the
class loader but removes the instance directories.
Because of this the TYPO3 class loader is still
triggered (by PHPUnit classes) and tries to write
things to a cache directory that does not exist any more.

Resolves: #56086
Releases: 6.2
Change-Id: Ib6acb588fb8b84c64d04882228822f6398838b6b
Reviewed-on: https://review.typo3.org/27775
Reviewed-by: Helmut Hummel
Tested-by: Helmut Hummel

Revision 9551ac77 (diff)
Added by Helmut Hummel over 5 years ago

[TASK] Add additional configuration for functional tests

Our functional test suit does not fail currently when
tested code emits a PHP warning. This is due to the
fact that the factory configuration uses the
production exception handler which silently discards
warnings.

This change adds additional configuration that
is merged with the factory configuration.

In this configuration the TYPO3 exception handling
is disabled and the core and classes cache is
disabled.

Additionally a new property is added to the base
class to easily add some configuration values that
might be needed for the test case.

Resolves: #56187
Related: #56086
Releases: 6.2
Change-Id: I5c7c2c0d495bae06a56d2285b8d521eaa4ab37d2
Reviewed-on: https://review.typo3.org/27773
Reviewed-by: Helmut Hummel
Tested-by: Helmut Hummel

History

#1 Updated by Marc Bastian Heinrichs over 5 years ago

  • Status changed from New to Accepted

#2 Updated by Marc Bastian Heinrichs over 5 years ago

  • Subject changed from Impexp functional tests did not fail on wrong assertion to Impexp functional tests do not fail on wrong assertion

#3 Updated by Oliver Hader over 5 years ago

That's very strange... I can confirm this behaviour where $this->assertFalse(TRUE) does not lead to failing tests...

#4 Updated by Marc Bastian Heinrichs over 5 years ago

  • Parent task set to #55522

#5 Updated by Marc Bastian Heinrichs over 5 years ago

The core DataHandling tests are not affected.

But if you add

$this->assertTrue(FALSE);

to
TYPO3\CMS\Core\Tests\Functional\Category\Collection\CategoryCollectionTest::checkIfFromArrayMethodSetCorrectProperties()

you get following error:

1) TYPO3\CMS\Core\Tests\Functional\Category\Collection\CategoryCollectionTest::checkIfFromArrayMethodSetCorrectProperties
PHPUnit_Framework_Exception: PHP Fatal error:  Uncaught exception 'PHPUnit_Framework_Error_Warning' with message 'file_put_contents(/home/typo3master/public_html/typo3temp/functional-8715107/typo3temp/Cache/Data/cache_classes/530393bd3daf4.temp): failed to open stream: No such file or directory' in /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php:238
Stack trace:
#0 [internal function]: PHPUnit_Util_ErrorHandler::handleError(2, 'file_put_conten...', '/_REPOS/Git/TYP...', 238, Array)
#1 /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php(238): file_put_contents('/home/typo3mast...', '')
#2 /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Frontend/StringFrontend.php(60): TYPO3\CMS\Core\Cache\Backend\SimpleFileBackend->set('typo3_cms_core_...', '', Array, NULL)
#3 /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Core/ClassLoader.php(168): TYPO3\CMS\Core\Cache\Frontend\StringFrontend->set('typo3_cms_core_...', '')
#4 [internal function]:  in /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php on line 238

Fatal error: Uncaught exception 'PHPUnit_Framework_Error_Warning' with message 'file_put_contents(/home/typo3master/public_html/typo3temp/functional-8715107/typo3temp/Cache/Data/cache_classes/530393bd3daf4.temp): failed to open stream: No such file or directory' in /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php on line 238

PHPUnit_Framework_Error_Warning: file_put_contents(/home/typo3master/public_html/typo3temp/functional-8715107/typo3temp/Cache/Data/cache_classes/530393bd3daf4.temp): failed to open stream: No such file or directory in /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php on line 238

Call Stack:
    5.9686   11310800   1. TYPO3\CMS\Core\Error\AbstractExceptionHandler->handleException(class PHPUnit_Framework_Error_Warning) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Error/AbstractExceptionHandler.php:0
    5.9687   11310888   2. TYPO3\CMS\Core\Error\ProductionExceptionHandler->echoExceptionCLI(class PHPUnit_Framework_Error_Warning) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Error/AbstractExceptionHandler.php:46
    5.9687   11311016   3. TYPO3\CMS\Core\Error\AbstractExceptionHandler->writeLogEntries(class PHPUnit_Framework_Error_Warning, string(3)) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Error/ProductionExceptionHandler.php:78
    5.9689   11313232   4. TYPO3\CMS\Core\Utility\GeneralUtility::sysLog(string(369), string(29), long) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Error/AbstractExceptionHandler.php:71
    5.9689   11314016   5. TYPO3\CMS\Core\Core\ClassLoader->loadClass(string(34)) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Error/AbstractExceptionHandler.php:0
    5.9696   11314264   6. TYPO3\CMS\Core\Cache\Frontend\StringFrontend->set(string(34), string(0), ???, ???) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Core/ClassLoader.php:168
    5.9697   11314448   7. TYPO3\CMS\Core\Cache\Backend\SimpleFileBackend->set(string(34), string(0), array(0), null) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Frontend/StringFrontend.php:60
    5.9701   11314680   8. file_put_contents(string(112), string(0)) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php:238
    5.9701   11316936   9. PHPUnit_Util_ErrorHandler::handleError(long, string(181), string(96), long, array(5)) /_REPOS/Git/TYPO3/TYPO3.CMS_master/typo3/sysext/core/Classes/Cache/Backend/SimpleFileBackend.php:238

with

$this->assertTrue(TRUE);

everything is fine.

#6 Updated by Gerrit Code Review over 5 years ago

  • Status changed from Accepted to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27773

#7 Updated by Helmut Hummel over 5 years ago

Marc Bastian Heinrichs wrote:

you get following error:

These are side effects of a not optimal functional testing bootstrap.

First of all, PHP warnings are discarded by the production exception handler which is active by default.
Because of this many warning that are now triggered are silently ignored

Secondly the functional test teardown does not remove the class loader but removes the instance directories
because of this the TYPO3 class loader is still triggered and tries to write things to a cache directory that does not exist any more.

A temporary workaround is disable caching (at least of core and classes) in functional tests as caching should not happen anyway during testing.

#8 Updated by Gerrit Code Review over 5 years ago

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/27775

#9 Updated by Helmut Hummel over 5 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#10 Updated by Riccardo De Contardi about 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF