Bug #12225
closed
Feature #13680: Refactor external link, do not follow senseless loop, add user agent
External Links with Redirects are reported as 404
100%
Description
Lets says you have a link to http://www.typo3.org/ and this is a redirection (ir is delivered with code 301) to http://typo3.org then you will receive an 404 as report.
Files
Updated by Chris topher over 13 years ago
Yes, sometimes the result for external links is not correct. But this is not caused by linkvalidator, but by t3lib_div::getURL, which sometimes returns wrong HTTP statuscodes.
See #24483.
Updated by Patrick Gaumond about 13 years ago
- Status changed from New to Accepted
As mentionned, this is a Core bug. We will try to find something to fix this anyway at least for LV itself.
Updated by Daniel Minder about 13 years ago
- File linkvalidator.patch linkvalidator.patch added
Redirections returned by the werbserver are usually well formated, i.e. the Location line contains at least a '/'. Therefore, it's not related to the getURL() bug.
Debugging a little bit more, I found several bugs in the external link checker:
1. The . in the preg_match() call matches also \r (=CR), which are often returned by webservers. (Note that getURL() also sends CRLF in its requests). Therefore, the new URL includes an extra CR, which confuses some webservers. And results in an 404 error.
2. To make it more error proof, "Location" should match only at the beginning of a line.
3. It happens that a webserver returns only a path in the Location line, not a full URL (example: http://www.sea-aeroportimilano.it/). This has to be handled.
4. In case no Location line is returned or it cannot be parsed, it does not make sense to run the loop 5 times. We could exit earlier and deal with it as it was an infinite loop.
A patch for the current sysext version of linkvalidator is attached.
Updated by Philipp Gampe about 13 years ago
- Assignee set to Philipp Gampe
Oops ... have not seen this one yet.
Thanks for your patch. I am currently rewriting the whole redirect stuff to also include cookies, etc.
I will include your changes.
Updated by Mr. Hudson about 13 years ago
Patch set 4 of change I86c7478d46d62f980735e5b80c6766e8bd811938 has been pushed to the review server.
It is available at http://review.typo3.org/1138
Updated by Mr. Hudson about 13 years ago
Patch set 5 of change I86c7478d46d62f980735e5b80c6766e8bd811938 has been pushed to the review server.
It is available at http://review.typo3.org/1138
Updated by Mr. Hudson almost 13 years ago
Patch set 1 of change I49a7bd993e51da0cdd42b15f0bf77150bed4d556 has been pushed to the review server.
It is available at http://review.typo3.org/1705
Updated by Mr. Hudson almost 13 years ago
Patch set 1 of change I0d998123a601d35cc983b13f577161f30bcd302f has been pushed to the review server.
It is available at http://review.typo3.org/2283
Updated by Mr. Hudson almost 13 years ago
Patch set 2 of change I49a7bd993e51da0cdd42b15f0bf77150bed4d556 has been pushed to the review server.
It is available at http://review.typo3.org/1705
Updated by Mr. Hudson over 12 years ago
Patch set 1 of change Ib5d70f4c18fce2617dd9b4d0c675468c5c9558ed has been pushed to the review server.
It is available at http://review.typo3.org/4261
Updated by Mr. Hudson over 12 years ago
Patch set 2 of change Ib5d70f4c18fce2617dd9b4d0c675468c5c9558ed has been pushed to the review server.
It is available at http://review.typo3.org/4261
Updated by Philipp Gampe over 12 years ago
- Status changed from Accepted to Resolved
- % Done changed from 80 to 100
Applied in changeset commit:5096f769049686c74e5c22a42a8a303cbbdc2ff2.
Updated by Chris topher about 12 years ago
- Status changed from Resolved to Closed
Updated by