Project

General

Profile

Actions
Copy link

Bug #20035

closed

Crawler does not crawl though relative links in an external page

Added by Dennis van over 15 years ago. Updated over 11 years ago.

Status:
Closed
Priority:
Should have
Assignee:
Jeff Segars
Category:
Indexed Search
Target version:
-
Start date:
2009-02-17
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
4.2
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

The crawler does not crawl through relative links when crawling an external page AND this external page is not just the domain name.

Example:
Relative links work when you crawl http://www.somesite.com/

Relative links DO NOT work when you crawl http://www.somesite.com/somefolder/

This seems to be a small error in class.crawler.php. The original writer seems to have forgotten to add the folder names that were in the url of the page it is crawling.

The diff holds a fix, where the class.crawler.php_back is the original file and class.crawler.php is the fixed file.
(issue imported from #M10463)

Files

Download all files
crawler.diff (609 Bytes) crawler.diff Administrator Admin, 2009-02-17 15:44
0010463_v2.diff (739 Bytes) 0010463_v2.diff Administrator Admin, 2009-02-19 18:09

Related issues 2 (0 open — 2 closed)

Related to TYPO3 Core - Bug #22296: IS cannot not index files if absRefPrefix is set and indexExternalURLs is notClosedDmitry Dulepov2010-03-18

Actions
Related to TYPO3 Core - Bug #22229: External URL only indexes first pageClosedXavier Perseguers2010-03-03

Actions
Actions
Copy link

Also available in: Atom PDF