TYPO3 Core

The following works correctly in RTE:

select bodytext from tt_content where pid=913;

<p>mit Umlauten <a href="https://xn--gebudehlle-s5a60a.swiss/">link (umlauts)</a></p>
<p><a href="https://xn--gebudehlle-s5a60a.swiss/sdfsddfs">broken link (umlauts)</a></p>
<p>Encoded: <a href="https://gebäudehülle.swiss">link (encoded)</a></p>
<p><a href="https://gebäudehülle.swiss/sdfsddfs">broken link (encoded)</a></p>

For pages.url, if the already encoded value is used, it is ok:

• https://xn--gebudehlle-s5a60a.swiss/

Not ok, if this is used:

• https://xn--gebudehlle-s5a60a.swiss/

As a workaround you can use the encoded URL (the URL is already converted for me, when I copy it from the browser address bar, for example).

As mentioned above, the problem is in UrlSoftReferenceParser. Possible solutions:

• value is already converted when saved in pages.url (and other fields)
• linkvalidator does not use softref parser to "parse" fields where the content can be used as is (e.g. for pages.url which is of type "input" with "softref" = "url")
• linkvalidator converts the URL before passing it to the sofref parser (not possible for fields where parsing should be performed FIRST)
• the softref parser UrlSoftReferenceParser converts the URL first. We already have code for this in ExternalLinktype::preprocessUrl :

protected function preprocessUrl(string $url): string
    {
        $url = html_entity_decode($url);
        $parts = parse_url($url);
        if ($parts['host'] ?? false) {
            try {
                $newDomain = (string)idn_to_ascii($parts['host']);
                if (strcmp($parts['host'], $newDomain) !== 0) {
                    $parts['host'] = $newDomain;
                    $url = HttpUtility::buildUrl($parts);
                }
            } catch (\Exception | \Throwable $e) {
                // ignore error and proceed with link checking
            }
        }
        return $url;
    }

Also, there is no problem with parsing sys_file_reference.link because it uses the TypolinkSoftReferenceParser.

I am wondering what is the point of having UrlSoftReferenceParser because TypolinkSoftReferenceParser can also parse urls.

Also, pages.url could use type=link and with allowedTypes='url' (though that would not solve the problem with possibly other fields)

pages.url:
- type=input
- softref=url

sys_file_reference.link:
- type=link
- softref=typolink

Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/87609

Patch set 3 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/87609

Patch set 1 for branch 13.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/87618

Patch set 1 for branch 12.4 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/87619