TYPO3 Core

Bug/Behaviour verified for: v9,v10,master

=> Findings/notes

Found corresponding functional test: typo3/sysext/frontend/Tests/Functional/SiteHandling/EnhancerSiteRequest/PageTypeDecoratorTest.php

Added testcase(s) [extending DataProvider) for test pageTypeDecoratorIndexCanBePartOfSlug, because for this case there was no corresponding test (only with index and .html suffix)

What was shocking first, these tests passed successfully, even if manual testing in a real install/instance it breaks / displays 404 page not found.

As for now, these new tests MUST fail, but they don't.

I debugged / digged deeper in, and one of the differences is, that the normalizedParamsAttributes are not filled correctly in the functional tests compared to a real frontend request.
As so, scriptName is an empty string "", and as such, no replacement in functional tests. In an apache instance it would be "/index.php" because of the htacces rewriting, ltrimmed to index.php
which is simple str_replaced().

The testframework calls the middlewars, but the NormalizedParams Middleware does not add something (because things in the ServerRequest regarding serverParams are fully empty.

        $urlPath = $previousResult->getTail();
        // Remove the script name (e.g. index.php), if given
        if (!empty($urlPath)) {
            $normalizedParams = $request->getAttribute('normalizedParams');
            if ($normalizedParams instanceof NormalizedParams) {
                $scriptName = ltrim($normalizedParams->getScriptName(), '/');
                if ($scriptName !== '' && strpos($urlPath, $scriptName) !== false) {
                    $urlPath = str_replace($scriptName, '', $urlPath);
                }
            }
        }

So i digged deeper ... and in the testingframework there is the TYPO3\TestingFramework\Core\Functional->retrieveFrontendSubRequestResult method.
Here some values are written to $_SERVER array, but when the new ServerRequest is instanciated, it is not passed, which leads to the empty normalizedParams.
Passing the full $_SERVER array as serverParams argument for the constructur will faile a lot of tests. But using the folling subarray sets, runs all tests normally, and my new tow tests are failing as expected.

$serverRequest = new ServerRequest(
    $uri,
    $request->getMethod(),
    'php://input',
    $request->getHeaders()
    ,[
        'SCRIPT_NAME'   => $_SERVER['SCRIPT_NAME'],
        'HTTP_HOST'     => $_SERVER['HTTP_HOST'],
        'SERVER_NAME'   => $_SERVER['SERVER_NAME'],
    ]
);

Applying my noted workaround/fix for the TYPO3\CMS\Core\Routing\PageRouter->matchRequest() will pass the new tests successfully.

if (!empty($urlPath)) {
    $normalizedParams = $request->getAttribute('normalizedParams');
    if ($normalizedParams instanceof NormalizedParams) {
        $scriptName = ltrim($normalizedParams->getScriptName(), '/');
        if ($scriptName !== '' && strpos($urlPath, $scriptName) !== false) {
            $urlPath = str_replace('/'.$scriptName, '', $urlPath);
        }
    }
}

Note

Created a pull request for the testing framework, see: https://github.com/TYPO3/testing-framework/pull/244

Today Testingframework pullrequest was merged, tagged with new version and raised in master.

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Added a patch as WIP for now: https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Until testingframework has improved, the functional test is useless (before & after), because it does not
initalize the frontend subrequest correctly in such way, that normalizedArguments holds /index.php as
scriptName like it would be in a real webrequest.

As such, the broken replacement which this patch tries to fix can only seen as broken before the patch,
if testingframework bugfix is in place and wired to the core.

See https://github.com/TYPO3/testing-framework/pull/244

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/69854

Patch set 1 for branch 10.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70553

Patch set 2 for branch 10.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/70553