Bug #89488

Epic #85006: Reduce falsely reported broken links

HTML special characters fool linkvalidator

Added by Rainer Plöckl 4 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Should have
Category:
Linkvalidator
Target version:
-
Start date:
2019-10-23
Due date:
% Done:

100%

TYPO3 Version:
8
PHP Version:
7.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

I detect a problem with & within a link when using the linkvalidator.

How to reproduce the problem:
- create link to the following page
https://standards.cen.eu/dyn/www/f?p=204:6:0::::FSP_ORG_ID,FSP_LANG_ID:,22&cs=1A3FFBC44FAB6B2A181C9525249C3A829
(this do not contain any &)
- check the link and it will work
- run linkvalidator, this will report an invalid link for:
https://standards.cen.eu/dyn/www/f?p=204:6:0::::FSP_ORG_ID,FSP_LANG_ID:,22&cs=1A3FFBC44FAB6B2A181C9525249C3A829
(the result is correct, as the webserver report 404 when a & instead of & is used)

I could get this working, when I extend linkvalidator/Classes/Linktype/ExternalLinktype.php with htmlspecialchars_decode()

--- ExternalLinktype.php.orig 2019-10-23 17:03:45.000000000 +0200
+++ ExternalLinktype.php 2019-10-23 17:02:57.000000000 +0200
@ -82,7 +82,7 @
];
$url = $this->preprocessUrl($origUrl);
if (!empty($url)) {
- $isValidUrl = $this->requestUrl($url, 'HEAD', $options);
+ $isValidUrl = $this->requestUrl(htmlspecialchars_decode($url), 'HEAD', $options);
if (!$isValidUrl) {
// HEAD was not allowed or threw an error, now trying GET
$options['headers']['Range'] = 'bytes=0-4048';

Maybe there's a better solution.


Related issues

Related to TYPO3 Core - Bug #89682: Linkvalidator: external URLs containing `& amp ;` or whitespace at the end not working Closed 2019-11-14

Associated revisions

Revision 44df5456 (diff)
Added by Sybille Peters 2 months ago

[BUGFIX] Convert URLs with ampersands in Linkvalidator

The linkref functions return the URLs as they are displayed in the
BE form. This will return '&' as '&' HTML entity.

Linkvalidator should correctly decode the HTML entity.

Resolves: #89488
Resolves: #89682
Releases: master, 9.5
Change-Id: I17b421354d45297423437373b92a93dfbe53d2d9
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/62634
Tested-by: TYPO3com <>
Tested-by: Susanne Moog <>
Tested-by: Daniel Goerz <>
Reviewed-by: Susanne Moog <>
Reviewed-by: Daniel Goerz <>

Revision 1fa19165 (diff)
Added by Sybille Peters 2 months ago

[BUGFIX] Convert URLs with ampersands in Linkvalidator

The linkref functions return the URLs as they are displayed in the
BE form. This will return '&' as '&' HTML entity.

Linkvalidator should correctly decode the HTML entity.

Resolves: #89488
Resolves: #89682
Releases: master, 9.5
Change-Id: I17b421354d45297423437373b92a93dfbe53d2d9
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/62645
Tested-by: TYPO3com <>
Tested-by: Susanne Moog <>
Tested-by: Daniel Goerz <>
Reviewed-by: Susanne Moog <>
Reviewed-by: Daniel Goerz <>

History

#1 Updated by Sybille Peters 4 months ago

  • Assignee set to Sybille Peters
  • Parent task set to #85006

#2 Updated by Sybille Peters 3 months ago

  • Related to Bug #89682: Linkvalidator: external URLs containing `& amp ;` or whitespace at the end not working added

#3 Updated by Gerrit Code Review 2 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/62634

#4 Updated by Gerrit Code Review 2 months ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/62645

#5 Updated by Sybille Peters 2 months ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#6 Updated by Rainer Plöckl 2 months ago

thanks for fixing this bug.

I initially reported the problem against Typo3 8.* But the fix was now "only" done in TYPO3 Version 9. Is it expected, that this change will also be merged to TYPO3 Version 8? I don't see this fix within the new 8.7.30 Release Notes .

#7 Updated by Benni Mack 2 months ago

  • Status changed from Resolved to Closed

#8 Updated by Sybille Peters about 2 months ago

TYPO3 v8 is in priority bugfix mode since Sep 30, 2018, see "Support Times" in Roadmap

I am not sure what qualifies as a "priority bugfix". This might not be it, but I can try to get it backported.

What you can do though is to override the default behaviour. Linkvalidator is extendable by adding a hook for each link type. You could create a class that extends ExternalLinkType and only overrides the preprocessUrl() function. (I see that you already did something similar above).

#9 Updated by Sybille Peters about 2 months ago

BTW, by the line Releases in the commit message of the patch, you can see to which versions it will be backported:

Releases: master, 9.5

https://review.typo3.org/c/Packages/TYPO3.CMS/+/62634

The patch is linked to in this issue in the messages above, such as:

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/62634

#10 Updated by Rainer Plöckl about 2 months ago

Sybille Peters wrote:

TYPO3 v8 is in priority bugfix mode since Sep 30, 2018, see "Support Times" in Roadmap

I am not sure what qualifies as a "priority bugfix". This might not be it, but I can try to get it backported.

I have already added your implementation as backport to fix this in our Typo3 v8 installation.
I was just surprised that the problem was fixed in different versions without comment as reported.

So for me it's okay now, but I just wanted to point out this inconsistency / inaccuracy, not that there was something missed.

Thanks for your effort.

Also available in: Atom PDF