Project

General

Profile

Bug #86918

Updated by Sybille Peters over 5 years ago

* Some sites require specific HTTP headers, which are normally set in browsers. 
 E.g. an external link validation for the URL "https://www.dpdhl.com/en.html" never ends and finally breaks the scheduler task. 
 The result of some debuggins is the following header default set in \TYPO3\CMS\Linkvalidator\Linktype\ExternalLinktype::checkLink(): 
 <pre> 
 $options = [ 
     'cookies' => GeneralUtility::makeInstance(CookieJar::class), 
     'allow_redirects' => ['strict' => true], 
     'headers' => [ 
         'User-Agent'          => 'TYPO3 linkvalidator', 
         'Accept'              => '*/*', 
         'Accept-Language'     => '*', 
         'Accept-Encoding'     => '*', 
         'Connection'          => 'keep-alive', 
     ], 
 ]; 
 </pre> 


 * 

 Also, some sites don't allow HEAD requests and in that cases the defined fallback GET Request in the mentioned method above won't ever used.   
 So, it would be great if you could decide, for example by a configuration, if you just want to use a simple GET request. 

 * 
 Another point is using the HTTP setting "Range: bytes = 0 – 4048" leads to strange responses in some link cases. A better way would be a possibility to set up this header setting so that it will not always be used.

Back