TYPO3 Core

Oliver Hader wrote in #note-1:

Please try this configuration instead:

Thanks for your help pointing out how to optimize it. I've checked it with your modified configuration (dropping defaults). I've tested it on two systems (Dev and Stage), both still only work for the URL '/main-page/10.55776/any-project-slug'. But this worked also before. Did you also check if you still can open the URL '/main-page/'? This still does not work. I've tried to double check everything and even flushed cashes, opened site-module (to force reload YAML config) more than twice.

At least for me it keeps trying to invoke project-controller for URL '/main-page/'. I've expected it to open the physical page as requirements should not be positive resulting to a valid project-page-request.

-- update --
Funny fact. I've tested again and after cache flushing on stage it now works "opposite". The URL '/main-page/', but the project URL '/main-page/10.55776/any-project-slug' dosn't. But neither on the DEV nor Stage I can get a result that both URLs work fine.

URL '/main-page/' should open the regular (physical page uid 4)
URL '/main-page/10.55776/any-project-slug' should invoke the Route for Project::detail Controller

So either there's something else screwed on my two TYPO3 instances or there is anything going completly wrong for routing.

Yes, it worked for me (using TYPO3_CONF_VARS[FE][cacheHash][enforceValidation] = false):

• https://ip12.anyhost.it/en/features → HTTP status 200, no arguments resolved
• https://ip12.anyhost.it/en/features/10.55776 → HTTP status 404, page not found
• https://ip12.anyhost.it/en/features/10.55776/something → HTTP status 200, arguments for tx_searchbar_project resolved

But, when using TYPO3_CONF_VARS[FE][cacheHash][enforceValidation] = true, both 10.55776 and 10.55776/something would be rejected with an HTTP status 404 - since the route values are not statically mapped and no &cHash= query parameter was given. For that scenarios, additional mappers are required (see complete example below again, adding the aspects section and removing the requirements).

routeEnhancers:
  searchIntegration:
    limitToPages: [4]
    type: Extbase
    extension: Searchbar
    plugin: Project
    routes:
      - routePath: '/{doiPrefix}/{projectSlug}'
        _controller: 'Project::detail'
    aspects:
      doiPrefix:
        type: StaticValueMapper
        map:
          '10.55776': '10.55776'
      projectSlug:
        type: StaticValueMapper
        map:
          something: something-internal-value
          project-slug: project-slug-internal-value

The concept of aspects is described at https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Routing/AdvancedRoutingConfiguration.html#aspects - the reason why requirements won't work with aspects is described in the same document at https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Routing/AdvancedRoutingConfiguration.html#aspect-precedence

Oliver Hader wrote in #note-5:

Yes, it worked for me (using TYPO3_CONF_VARS[FE][cacheHash][enforceValidation] = false):

Ah okay, thanks for pointing that out. But at least I think completly disabling cHash-Validation isn't really a good solution. Isn't this possible by using an Route-Aspect/Enchanger to make the cHash pass without completly disabling this?

To completly disabling cHash-validation can (and may be will) have an impact to a TYPO3 Website and also for possibility in DDOS attacks. I think it's not a valid solution to disable such a feature to "fix" a routing-feature that should work in a normal TYPO3 context with cHash enabled.

Just for notice I've already tried to code a simple Pass-Through Aspect that will take any value passing it through as "valid" just like the core-aspects will do, but with mappings.

This is what I've tried:

<?php
declare(strict_types=1);
namespace TYPOworx\MyExtension\Routing\Aspect;

use TYPO3\CMS\Core\Routing\Aspect\StaticMappableAspectInterface;

class SlugPassthroughEnhancer implements StaticMappableAspectInterface
{
    public function generate(string $value) : ?string
    {
        return $value;
    }

    public function resolve(string $value) : ?string
    {
        return $value;
    }
}

The result was so far the same. But when I understand it correct I would have to implement an own requirement feature here as requirements won't work with custom aspects (or any core-aspects as well?!)

I am not advising to disable enforceValidation, I was more referring to possible configuration settings out there and the corresponding behavior.

The SlugPassthroughEnhancer (actually it's an "aspect" not an "enhancer") that was shown above is just tricking TYPO3 and would facilitate DOS scenarios (it does not have to be "distributed") - basically since any value is accepted and cached.

To limit the amount of possible values (to make it "static"), the available aspects (as described at https://docs.typo3.org/m/typo3/reference-coreapi/main/en-us/ApiOverview/Routing/AdvancedRoutingConfiguration.html#aspects) can be used. In https://review.typo3.org/c/Packages/TYPO3.CMS/+/74016 I started a feature a year ago that allows to make parameters "static" in combination with requirements (no need to use aspects or these fake aspects).

Hello Oliver,

thanks for your help and time in this issue. Ok I understand. The SlugPassthroughEnhancer (I don't know yet why I named it Enhancer :-D) was a test trying to fix it. You are right that solution isn't really good at all! But I was trying to figure out how to get it working, then trying to harden it.

Using a debugger in the generate & resolve method by the way only gives result for the project-url (URL '/main-page/10.55776/any-project-slug'). The URL '/main-page/' is then still bypassed to project-controller and there it is not even invoking the custom aspect SlugPassthroughAspect. I would have expected it is invoked if the Router assumes, the route is valid for project::controller. But it doesn't.

Thats a really hard to understand logic as it behaves not as I as developer would assume it works. And I'm also experienced with pure HTTP-Foundation Router (used in Laravel/Symfony) and at least in Laravel I was always able to harden my routes in a way they match only if desired -- even if they're looking very similar. I was even able to separate GET/POST requests.

Never the less TYPO3 routing seems to use HTTP-Foundation Routing under the hood, the Routing with TYPO3 works (at least for my feeling) in a very abstract way that is hard to understand why they are some obviously breaks in logic.

1. I've been trying to harden the route with requirements (also tested without any aspects),
   but the requirements are ignored while the route is matched -- that should not be matched then.
2. I've tried to harden the route using aspect, but the aspect isn't even invoked while the TYPO3 Router matches the route with
   path-segments that are neither given nor set.

Just to clarify... the URL '/main-page/' exists as physical page in TYPO3! And it works with disabling the plugin routing. It does not make any sense that the router will bypass everything to project::detail if the route should not match and while there is a page-slug that matches perfectly.

May be there is something I'm missing here, but for me this doesn't feel like logical routing that can separate routes with explicit rules. It's more like a Hen that is picking until it thinks it get's its first corn and then being satisfied with that. The Hen doesn't seem to be interested the corn is a stone or not.

Gabriel Kaufmann / Typoworx NewMedia wrote in #note-8:

Thats a really hard to understand logic as it behaves not as I as developer would assume it works. And I'm also experienced with pure HTTP-Foundation Router (used in Laravel/Symfony) and at least in Laravel I was always able to harden my routes in a way they match only if desired -- even if they're looking very similar. I was even able to separate GET/POST requests.

TYPO3's routing extends symfony/routing and just focusses on mapping an URL to potential parameters. This happens independent of any application logic (e.g. a controller) in TYPO3 and that's the main difference to Laravel or Symfony, where one can use annotations/attributes inline in the application logic. In TYPO3, the processing might rely on additional conditions defined by TypoScript, that might might change some instructions based on the resolved parameters - thus, having that difference in dispatching the process in the second big difference between TYPO3 and slim frameworks like Laravel or Symfony.

When focussing on the routing part again, Laravel and Symfony would just pass the parameters to controllers, e.g. for /action/{month} invoked as /action/january, the parameter value that hits the controller would be january. However, for TYPO3 (and in reference to previous ext:realurl) the demand was to have a mapping available, that transforms january to 1, and the controller just receives the integer 1. This transformation has to work in both ways of course (URL january <=> 1 controller).

Never the less TYPO3 routing seems to use HTTP-Foundation Routing under the hood, the Routing with TYPO3 works (at least for my feeling) in a very abstract way that is hard to understand why they are some obviously breaks in logic.

Just to mention some of them:

1. I've been trying to harden the route with requirements (also tested without any aspects),
   but the requirements are ignored while the route is matched -- that should not be matched then.
2. I've tried to harden the route using aspect, but the aspect isn't even invoked while the TYPO3 Router matches the route with
   path-segments that are neither given nor set.

I guess(!) that it stems from using defaults and requirements at the same time - which means that for /main-page the defaults for doiPrefix and projectSlug are filled in. AFAIK symfony/routing does not validate requirements again on values from defaults. Thus, with using defaults the route always matches - to avoid that, a prefix would have to be used, like routePath: '/details/{doiPrefix}/{projectSlug}' (details/ being the prefix).

Just to clarify... the URL '/main-page/' exists as physical page in TYPO3! And it works with disabling the plugin routing. It does not make any sense that the router will bypass everything to project::detail if the route should not match and while there is a page-slug that matches perfectly.

I have a similar scenario (based on the introduction package), where it worked with aspects (using the configuration https://forge.typo3.org/issues/101162#note-5). In case other enhancers, aspects, page-type-decorators are configured - maybe trying to isolate the problem by temporarily disabling other routes might help.

In any way, I agree that debugging routing in TYPO3 is difficult & having something like https://symfony.com/doc/4.1/routing/debug.html at hand for developers would probably help and speed up the process.

Hey Oliver,

thanks for your feedback. Good to know - I'm obviously not aline while having such troubles. Well a separate URL segment isn't currently planned (our customer has some specifications on their URLs). But if nothing helps I have to tell them it's required.

I have to keep in mind requirements vs defaults vs aspects combining is no good idea never the less one would assume it should work (while defining empty default-value and a requirement that should not match for non-empty).

As TYPO3 likes to throw exceptions for so many stuff and sometimes "problems", that are not worth an exception (f:image src invalid?!), this would be a case I wish TYPO3 would throw an exception clarifying, that the route-enchange configuration has some "errors" in view of TYPO3 Logic.

Well I have to see what I'll make now... I think if everything is going to break and customer wants their URL specifications, I'll have to write a middleware to handle that. That doesn't feel like "rapid development" at all.

If the customer would not have their TYPO3 must-have for this implementation I've would have done it in laravel.

Gabriel Kaufmann / Typoworx NewMedia wrote in #note-10:

If the customer would not have their TYPO3 must-have for this implementation I've would have done it in laravel.

Well, obviously I cannot help you with that... :)

In case there's more input, configuration, etc. I'd be looking forward to having a look into that again - however, I was not able to reproduce the problem locally with the given configuration from above in TYPO3 v12.4.3-dev.

Oliver Hader wrote in #note-11:

Well, obviously I cannot help you with that... :)

Ok I understand. Thanks for your time. Never the less... for me this still feels like a misbehaviour or at least optimisation as TYPO3 routing lacks more explicit validation for "conflicting" configurations on the one hand and on the other hands a working solution to cope with such routes without any "dirty hacks".

Shouldn't we keep this open for further investigations on how to optimize the route-behaviour?!

Oliver Hader wrote in #note-9:

to avoid that, a prefix would have to be used, like routePath: '/details/{doiPrefix}/{projectSlug}' (details/ being the prefix).

Just for mentioning. Even though I think the customer is hard to convince using another Slug in their URL. I've tested if it works:

routeEnhancers:
  searchIntegration:
    type: Extbase
    extension: SearchBar
    plugin: Project
    #limitToPages is required to set physical page-pid
    limitToPages: [4]
    routes:
      - routePath: '/project/{doiPrefix}/{projectSlug}'
        _controller: 'Project::detail'
        _arguments:
          doiPrefix: doiPrefix
          projectSlug: projectSlug
    requirements:
      doiPrefix: '10.55776'
      projectSlug: '.+'
#    aspects:
#      doiPrefix:
#        type: SlugPassthroughAspect
#      projectSlug:
#        type: SlugPassthroughAspect

Notice the additional virtual path-segment I've prefixed for routePath "/project/". It still does not work for both routes! Project-Page works fine like before. The "/main-page/" doesn't match resulting in error from Project::detail controller.

And also trying to adopt your idea:

    aspects:
      doiPrefix:
        type: StaticValueMapper
        map:
          '10.55776': '10.55776'

Does not work better.

At least I assume there must be something done to make such things working. I don't think this is a should have need that a CMS like TYPO3 should ignore.

Gabriel Kaufmann / Typoworx NewMedia wrote in #note-13:

At least I assume there must be something done to make such things working. I don't think this is a should have need that a CMS like TYPO3 should ignore.

In case it would be a bug, I would totally agree. However, I was not able to reproduce the misbehavior that you experience. It does work for me (without using the prefix, using the configuration show in https://forge.typo3.org/issues/101162#note-5) - it tested that locally, it does work.

I don't know why it does not work on your side. There is nothing more that I or TYPO3 could to do here. It is not about ignoring, in fact it cannot be reproduced.

Oliver Hader wrote in #note-15:

Again, the following is the exact route configuration I was using:

Thanks Oliver,
I'll have to check out. But using the StaticValueMapper for projectSlug as it is will sadly be no option as there are thousands and I don't want to end-up mapping them all manually.

But may be there's a side-way doing a prefetch against the API to check wether it exists or not combined with false/positive caching the result (regardless it exists or not) as compromise.
But as far as I tested in that direction using proof-of-concept it doesn't work as expected for both URL cases with main-site und detail-page url underneath "main-site".

So at least I think there is a bug in the TYPO3-Routing at least for use-cases like my one for "non-physical" URLs that have no record to reflect the page slug and physical pages that overlap with deep-link URLs to a virtual mapped "sub-page". So even if there obviously is no non-workaround solution now, I still think there is a bug to keep in track at least for future optimisations in TYPO3-Routing. So I'm wondering you want to close this issue if there obviously is no solution that really fixes this.

But I will retry with your last end-up configuration to check again if it works with StaticValueMapper and then trying to do it with own customized ValueMapper doing non-static mapping.

I'll let you know here if it works then.