Feature #70056

Use guzzle

Added by Georg Ringer over 3 years ago. Updated 8 months ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
-
Target version:
-
Start date:
2015-09-23
Due date:
% Done:

100%

PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Add guzzle as depencency and deprecate HttpRequest


Related issues

Related to TYPO3 Core - Bug #67838: Cannot instantiate HttpRequest Closed 2015-06-30
Related to TYPO3 Core - Feature #28626: Refactor t3lib_div::getUrl() Closed 2011-08-01
Related to TYPO3 Core - Bug #75757: FAL: Exception thrown if image for youtube video can't be fetched Closed 2016-04-18
Related to TYPO3 Core - Bug #78196: in installtool there is no curluse under [SYS] Closed 2016-10-08
Related to TYPO3 Core - Bug #84167: Negation Bug in TYPO3 8.7 GuzzleHttp \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() Closed 2018-03-07
Related to TYPO3 Core - Bug #84170: Inconsistent lib value in \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() since GuzzleHttp switch Closed 2018-03-07
Related to TYPO3 Core - Bug #84171: \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() Request-Header format changed from array of strings to associative array in GuzzleHttp conversion Closed 2018-03-07
Related to TYPO3 Core - Bug #84173: TYPO3 8.7 GuzzleHttp \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() returns content_type as array instead of string Closed 2018-03-07
Related to TYPO3 Core - Bug #85491: GeneralUtility:getUrl with includeHeaders=2 uses GET instead of HEAD Closed 2018-07-05

Associated revisions

Revision 23fa188f (diff)
Added by Benni Mack about 3 years ago

[!!!][FEATURE] Introduce Guzzle for making HTTP Requests

The Guzzle library is added as a composer dependency
instead of HttpRequest to request urls from TYPO3.

Guzzle is encapsulated inside a RequestFactory to do
requests and return PSR-7 compliant Response objects
to deal with content. In the future, Guzzle can also be used
to do HTTP requests asynchronously but there is no API
for that yet as there is no current need for that in the
TYPO3 Core.

GeneralUtility::getUrl() now uses Guzzle under the hood,
thus adding headers like the TYPO3 User Agent by default.

A lot of existing TYPO3_CONF_VARS options are now
removed or merged into Guzzle compliant options which
are independant of the implementation (cURL, stream
wrappers, fopen etc).

Resolves: #70056
Releases: master
Change-Id: Ibd14bba944b1590bae1b12c10f26365f20576475
Reviewed-on: https://review.typo3.org/43508
Reviewed-by: Morton Jonuschat <>
Tested-by: Morton Jonuschat <>
Reviewed-by: Markus Klein <>
Tested-by: Markus Klein <>
Reviewed-by: Susanne Moog <>
Tested-by: Susanne Moog <>

Revision 54f807d7 (diff)
Added by Felix Buenemann about 1 year ago

[BUGFIX] Unify value for $report['lib'] to GuzzleHttp

GeneralUtility::getUrl() now always returns 'GuzzleHttp' as lib
in the report, instead of returning
"GuzzleHttp" if an exception was caught and "http" otherwise.

Resolves: #84170
Related: #70056
Releases: master, 8.7
Change-Id: I6a28cb7921a3d784c37912c12f9b250f6b4d493c
Reviewed-on: https://review.typo3.org/56040
Tested-by: TYPO3com <>
Reviewed-by: Markus Klein <>
Tested-by: Markus Klein <>
Reviewed-by: Benni Mack <>
Tested-by: Benni Mack <>

Revision 5869aa20 (diff)
Added by Felix Buenemann about 1 year ago

[BUGFIX] Unify value for $report['lib'] to GuzzleHttp

GeneralUtility::getUrl() now always returns 'GuzzleHttp' as lib
in the report, instead of returning
"GuzzleHttp" if an exception was caught and "http" otherwise.

Resolves: #84170
Related: #70056
Releases: master, 8.7
Change-Id: I6a28cb7921a3d784c37912c12f9b250f6b4d493c
Reviewed-on: https://review.typo3.org/56091
Reviewed-by: Benni Mack <>
Tested-by: Benni Mack <>
Tested-by: TYPO3com <>

Revision 662fb9ae (diff)
Added by Felix Buenemann about 1 year ago

[BUGFIX] Restore getUrl support for list of headers

The change of \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl()
from cURL to GuzzleHttp the format of the $requestHeaders param was
implicitly changed from an array of header strings to an associative
array where the key is the header name and the value is either a single
or an array of values for that header.

This adds back support for the old list of headers format by detecting a
non-associative array and converting it to the Guzzle key/value(s) style.

At the same time the 'old' way is deprecated.

Resolves: #84171
Related: #70056
Releases: master, 8.7
Change-Id: I41b23993957288dfd5294129fa8039aab717461d
Reviewed-on: https://review.typo3.org/56046
Tested-by: TYPO3com <>
Reviewed-by: Frank Naegler <>
Tested-by: Frank Naegler <>
Reviewed-by: Susanne Moog <>
Tested-by: Susanne Moog <>

Revision 284411c1 (diff)
Added by Felix Buenemann about 1 year ago

[BUGFIX] Restore getUrl support for list of headers

The change of \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl()
from cURL to GuzzleHttp the format of the $requestHeaders param was
implicitly changed from an array of header strings to an associative
array where the key is the header name and the value is either a single
or an array of values for that header.

This adds back support for the old list of headers format by detecting a
non-associative array and converting it to the Guzzle key/value(s) style.

At the same time the 'old' way is deprecated.

Resolves: #84171
Related: #70056
Releases: master, 8.7
Change-Id: I41b23993957288dfd5294129fa8039aab717461d
Reviewed-on: https://review.typo3.org/56149
Tested-by: TYPO3com <>
Reviewed-by: Frank Naegler <>
Tested-by: Frank Naegler <>
Reviewed-by: Susanne Moog <>
Tested-by: Susanne Moog <>

History

#1 Updated by Gerrit Code Review over 3 years ago

  • Status changed from New to Under Review

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/43508

#2 Updated by Gerrit Code Review over 3 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/43508

#3 Updated by Gerrit Code Review over 3 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#4 Updated by Gerrit Code Review over 3 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#5 Updated by Christian Kuhn over 3 years ago

First: I want guzzle, too!

But imho, this area needs a full re-work and this patch does not go far enough.

Situation:
At the moment we have 4 (!) ways to fetch remote content:
  • getUrl with curl
  • getUrl with fsock
  • getUrl with file_get_contents
  • http2 with a slim core "fassade" that is extending http2.

Additionally we have a series of configuration options that are half deprecated within typo3_conf_vars. Furthermore getUrl() is also able to retrieve local files.

Technical debts:
  • getUrl: fetching local files and getting remote content are 2 different things, getUrl hides that, but imho a controller should know and steer this. so hiding these 2 concerns is not a good idea. core could help with a helper to find out if a given string is a local page number, a local file or an external ressource, maybe more. So in general i'd like to have getUrl() deprecated!
  • http2: the "fassade" still exposes all http2 details while it should probably hide for instance the configuration setters to consumers.
  • current pending patch with guzzle creates a direct dependency from a controller to guzzle without a fassade. there is not even a way to transfer configuration options from core to initialize guzzle options.
  • there is no sane default configuration / detection done.

If we start fixing this mess with guzzle, I'd require to have some core support that fixes all of the above stuff and does not just add something. I want the technical debt to be solved once and for all, in detail:

  • Have a stable API to fetch remote content that does not change if we substitute the implementation with something else.
  • Have a proper detection of settings within install tool.
  • Do not expose the whole guzzle API to the outside but only things that make sense for consumers, and clearly state what is not exposed. Goal is here to have a package in the core that is properly pre-configured. If extensions need more, they still may use guzzle directly, but then they're on their own.
  • Separate remote fetching from local ressources, help extensions to find out which type of a resource they are handling so they can steer further stuff.
  • Get rid of getUrl()

In general I'd like to finally have a proper modeling for this stuff in the core and a solution path of things that need to be done instead of cowboy hacking guzzle into the system now. The above are just some ideas, I'd love to have some feedback / more detailed discussion and will help with finding a solution path if someone wants to do something like that.

#6 Updated by Helmut Hummel over 3 years ago

  • Status changed from Under Review to New

Christian perfectly outlined how we should proceed. Anyone up to take over this task? Georg?

#7 Updated by Philipp Gampe over 3 years ago

Well, I second this, although I already did most of the work years ago. I hope now since change is more accepted and separation of concerns came into focus that this getUrl mess is finally resolved.

#8 Updated by Gerrit Code Review about 3 years ago

  • Status changed from New to Under Review

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#9 Updated by Gerrit Code Review about 3 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#10 Updated by Gerrit Code Review about 3 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#11 Updated by Gerrit Code Review about 3 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#12 Updated by Gerrit Code Review about 3 years ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#13 Updated by Gerrit Code Review about 3 years ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#14 Updated by Gerrit Code Review about 3 years ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#15 Updated by Gerrit Code Review about 3 years ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#16 Updated by Gerrit Code Review about 3 years ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#17 Updated by Gerrit Code Review about 3 years ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#18 Updated by Gerrit Code Review about 3 years ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#19 Updated by Gerrit Code Review about 3 years ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#20 Updated by Gerrit Code Review about 3 years ago

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#21 Updated by Gerrit Code Review about 3 years ago

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#22 Updated by Gerrit Code Review about 3 years ago

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/43508

#23 Updated by Benni Mack about 3 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#24 Updated by Felix Buenemann about 1 year ago

  • Related to Bug #84167: Negation Bug in TYPO3 8.7 GuzzleHttp \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() added

#25 Updated by Felix Buenemann about 1 year ago

  • Related to Bug #84170: Inconsistent lib value in \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() since GuzzleHttp switch added

#26 Updated by Felix Buenemann about 1 year ago

  • Related to Bug #84171: \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() Request-Header format changed from array of strings to associative array in GuzzleHttp conversion added

#27 Updated by Felix Buenemann about 1 year ago

  • Related to Bug #84173: TYPO3 8.7 GuzzleHttp \TYPO3\CMS\Core\Utility\GeneralUtility::getUrl() returns content_type as array instead of string added

#28 Updated by Markus Klein 11 months ago

  • Related to Bug #85491: GeneralUtility:getUrl with includeHeaders=2 uses GET instead of HEAD added

#29 Updated by Benni Mack 8 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF