Project

General

Profile

Actions
Copy link

Feature #13680

closed

Refactor external link, do not follow senseless loop, add user agent

Added by Philipp Gampe about 13 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Could have
Assignee:
Philipp Gampe
Category:
Linkvalidator
Target version:
-
Start date:
2011-01-14
Due date:
% Done:

100%

Estimated time:
(Total: 0.00 h)
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

This takes care of most external link issues.

We still need to wait for the patch for getUrl, but this should be solved soon.
  • add user-agent #13800
  • correct handling of location (Thanks to Daniel Minder)
  • move code up and down
  • remove useless or duplicated code

Files

Download all files
linkval-cleanup.diff (4.29 KB) linkval-cleanup.diff Philipp Gampe, 2011-03-12 17:08
linkval-cleanup_2.diff (4.35 KB) linkval-cleanup_2.diff Philipp Gampe, 2011-03-14 23:41
linkval-user-agent-cleanup-external-url.diff (4.33 KB) linkval-user-agent-cleanup-external-url.diff without cookie support and without head only Philipp Gampe, 2011-05-21 14:24

Subtasks 4 (0 open — 4 closed)

Feature #13794: Redirects in external links should observer CookiesClosedPhilipp Gampe2011-03-11

Actions
Feature #13800: Add user-agent headerClosedPhilipp Gampe2011-03-12

Actions
Bug #13432: 404 for external links with no redirectClosedPhilipp Gampe2011-02-28

Actions
Bug #12225: External Links with Redirects are reported as 404ClosedPhilipp Gampe2011-01-14

Actions

Related issues 1 (0 open — 1 closed)

Related to TYPO3 Core - Bug #13756: External links to big files crash TYPO3ClosedPhilipp Gampe2011-03-10

Actions
Actions
Copy link
 #1

Updated by Philipp Gampe about 13 years ago

  • File deleted (linkval-cleanup.diff)
Actions
Copy link
 #2

Updated by Philipp Gampe about 13 years ago

  • File linkval-cleanup.diff added

I will commit in 48 hours if nobody objects.

Actions
Copy link
 #3

Updated by Chris topher about 13 years ago

Hi Philipp,

do we need the line t3lib_utility_Debug::debug($cookies);?

The rest looks fine.

Actions
Copy link
 #4

Updated by Philipp Gampe about 13 years ago

No, of course not ...

I am currently rewriting the whole loop and I will come up with a patch. It will cover all open issues regarding external link handling.

Actions
Copy link
 #5

Updated by Philipp Gampe about 13 years ago

  • Subject changed from Cleanup external link, do not follow senseless loop to Cleanup external link, do not follow senseless loop, add user agent and cookies
Actions
Copy link
 #6

Updated by Philipp Gampe about 13 years ago

Actions
Copy link
 #7

Updated by Philipp Gampe about 13 years ago

  • File deleted (linkval-cleanup.diff)
Actions
Copy link
 #8

Updated by Philipp Gampe about 13 years ago

I updated the patch ... please test.

Actions
Copy link
 #9

Updated by Philipp Gampe about 13 years ago

  • some code style cleanups
  • add getUrl($url, 3, FALSE, &$report) [needs fix in core]
Actions
Copy link
 #10

Updated by Philipp Gampe about 13 years ago

  • Subject changed from Cleanup external link, do not follow senseless loop, add user agent and cookies to Refactor external link, do not follow senseless loop, add user agent and cookies
Actions
Copy link
 #11

Updated by Daniel Minder about 13 years ago

I tested the patch (with getUrl($url, 2, ...) instead of 3) and found a bug in the Cookie handling.

It's not required that a path is given, and if it's present it does not need to be the first attribute. So, line 117 should be changed to:

if ((preg_match('/Set-Cookie: ([^;]+)/', $line, $cookie))) {

since the actual cookie stops at ; or at the end of the line. I'm not sure if the name or value of the cookie can be a quoted string that includes ;. http://tools.ietf.org/html/draft-ietf-httpstate-cookie-21 defines name and value to be a token as defined in RFC 2616 where token is "1*<any CHAR except CTLs or separators>". Then, the regexp would become more complex...

Actions
Copy link
 #12

Updated by Philipp Gampe about 13 years ago

A Cookie an also have an expires entry.

I think Name= + value + expires + ; + path is pretty much standard.

I will do some tests this afternoon.

Actions
Copy link
 #13

Updated by Philipp Gampe almost 13 years ago

  • Subject changed from Refactor external link, do not follow senseless loop, add user agent and cookies to Refactor external link, do not follow senseless loop, add user agent
Actions
Copy link
 #14

Updated by Philipp Gampe almost 13 years ago

We will deal with cookies later

Actions
Copy link
 #15

Updated by Philipp Gampe almost 13 years ago

  • File linkval-user-agent-cleanup-external-url.diff added

-remove cookie support from patch; it does not always work

Actions
Copy link
 #16

Updated by Philipp Gampe almost 13 years ago

-remove only fetching headers

this was dropped for now in core, so we will have to come up with a better solution for fetching URLs first.

Actions
Copy link
 #17

Updated by Philipp Gampe almost 13 years ago

  • File deleted (linkval-user-agent-cleanup-external-url.diff)
Actions
Copy link
 #19

Updated by Philipp Gampe almost 13 years ago

commited to trunk in r47965
and to 4.5 in r47966

Actions
Copy link
 #20

Updated by Philipp Gampe almost 13 years ago

  • Status changed from Accepted to Resolved
  • % Done changed from 70 to 100

Applied in changeset r47965.

Actions
Copy link
 #21

Updated by Chris topher about 12 years ago

  • Status changed from Resolved to Closed
Actions
Copy link
 #22

Updated by Michael Stucki over 10 years ago

  • Category set to Linkvalidator
Actions
Copy link
 #23

Updated by Michael Stucki over 10 years ago

  • Project changed from 1510 to TYPO3 Core
  • Category changed from Linkvalidator to Linkvalidator
Actions
Copy link

Also available in: Atom PDF