Bug #29365

better cHash calculation (avoid cache flooding of USER_INT)

Added by Daniel Poetzinger almost 7 years ago. Updated 9 months ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Caching
Target version:
Start date:
2011-08-30
Due date:
% Done:

100%

TYPO3 Version:
4.6
PHP Version:
Tags:
Complexity:
nightmare
Is Regression:
Sprint Focus:

Description

TYPO3 Caching and the cHash

What is the goal:
1) Content that can be cached should be cached
2) Content that is dynamic should not be cached, but the rest of the page
3) Not more cache entries then required

Problem description short:
---------------------------
Currently TYPO3 takes all parameters to calculate the cHash and have a seperate cache entry for that parameter combination. But for parameters that are evaluated in USER_INT (like uncacheable actions in extbase for example) this don't makes sense and results in two problems a) uncached page because of wrong cHash or b) too much unneccessary page cache variants.

Problem in Detail: (you may want to skip reading this part and check solution below)
---------------------------

Introduction with two Use-Cases:

Use-Case 1:
- You have a single view that displays some data based on a "tx_news_pi1[uid]" parameter.
=> If you have a valid cHash for that parameter TYPO3 caches a separate variant of this page (e.g. id=12&tx_plugin_pi1[uid]&cHash=correct2343243 )

Use-Case 2: using _INT Typoscript Objects for Use-Case:
- You have a search that searches for things based on a query:
=> You are not allowed to have a cHash Parameter then. Your URLs need to look like this:
id=12& tx_search_pi1[query]=test
id=12& tx_search_pi1[query]=test2

=> TYPO3 will then have only one Cache entry for that page. But your plugin will be a USER_INT and therefore can evaluate this parameter each time.

The Problem:
----------------

1) It is not possible to have this behaviour mixed on one page: If you have elements on your page that should be cached based on some parameters but you still want to have elements that needs to be uncached. And there are many use-cases for this!

So to come back to the two examples:
id=12&tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1&cHash= correct2343243
will not work since the cHash is not valid anymore

id=12& tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1
Will not work since with a missing cHash TYPO3 might deliver the wrong cached content for the news extension
id=12& tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1&cHash=thenewcorrectchash213213
Will work but pollute the Cachetable with new entries for every tx_search_pi1 parameter.
(And the additional problem that the cHash may not be possible to calculate)

2) The current extbase uncached actions solution is another example for the same problem. Since extbase replaces USER to USER_INT on the fly if an uncached action is called it needs to have the cHash Parameter set. Which results in a new Cache entry for every Parametercombination of the uncached action. Which makes it pretty useless in most cases.

3) A third aspect of that example is, that parameters like "L" or other Parameters that might be part of some conditions are used for cHash calculation also. That can cause tricky bugs and uncached pages also.

Solution:
-------------

The cHash should (and have to) be only calculated and evaluated for parameters that are used in cachable plugins (or actions).

For the above example that means:
id=12&tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1&cHash= correct2343243
id=12&tx_search_pi1[query]=test2&tx_plugin_pi1[uid]=1&cHash= correct2343243

should both result in a cache hit.

id=12&tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1&cHash=wrongcHash
=> should trigger an cHash error since its a wrong cHash
id=12&tx_search_pi1[query]=test&tx_plugin_pi1[uid]=1
=> should throw an cHash error too, since the cHash is missing but required for tx_plugin_pi1[uid]

The patch set introduced a new class for the cHash calculation because of:
1) it could be xclassed if there are very special requirements to cHash calculation
2) the class could be unit-testes
3) it makes sense to move features from t3lib_div for maintenance reasons


Related issues

Related to TYPO3 Core - Feature #28299: Add hook to tslib_fe->getHash() Resolved 2011-07-18
Related to TYPO3 Core - Bug #20372: Exclude &L parameter from &cHash params Resolved 2009-04-28

Associated revisions

Revision fa52bc83 (diff)
Added by Daniel Poetzinger over 6 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7
Reviewed-on: http://review.typo3.org/4704
Reviewed-by: Fabrizio Branca
Tested-by: Fabrizio Branca
Reviewed-by: Daniel Poetzinger
Tested-by: Daniel Poetzinger
Reviewed-by: Tolleiv Nietsch
Tested-by: Tolleiv Nietsch

Revision 886fa08f (diff)
Added by Daniel Poetzinger over 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

Revision c5b4443e (diff)
Added by Daniel Poetzinger over 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

Revision 354282a9 (diff)
Added by Daniel Poetzinger over 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

Revision d3a91709 (diff)
Added by Daniel Poetzinger over 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

Revision 7c8688f6 (diff)
Added by Daniel Poetzinger over 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

Revision 17ca0d68 (diff)
Added by Daniel Poetzinger almost 5 years ago

[FEATURE] Flexible cHash Calculation

Introduces new configuration options for $TYPO3_CONF_VARS['FE']
  • cHashRequiredParameters
  • cHashExcludedParameters
  • cHashExcludedParametersIfEmpty
  • cHashOnlyForParameters

Moves the adjusted cHash calculation logic to it's own class
t3lib_cacheHash.

Change-Id: I257ddd2042015622872519db18deee1b5071f950
Resolves: #29365
Releases: 4.7

Conflicts:
t3lib/class.t3lib_div.php
t3lib/config_default.php
t3lib/core_autoload.php
typo3/sysext/cms/tslib/class.tslib_content.php
typo3/sysext/cms/tslib/class.tslib_fe.php

History

#1 Updated by Mr. Hudson almost 7 years ago

Patch set 1 of change I257ddd2042015622872519db18deee1b5071f950 has been pushed to the review server.
It is available at http://review.typo3.org/4704

#2 Updated by Mr. Hudson almost 7 years ago

Patch set 2 of change I257ddd2042015622872519db18deee1b5071f950 has been pushed to the review server.
It is available at http://review.typo3.org/4704

#3 Updated by Andreas Wolf almost 7 years ago

  • Category set to Caching
  • Target version changed from 4.6.0 to 4.6.0-beta3
  • Complexity set to medium

IMHO this should be included into 4.6 beta3, to have enough testing before 4.6 release.

#4 Updated by Philipp Gampe almost 7 years ago

Your page includes global Install Tool configuration options.

Now think about a multidomain site with 4 languages and a multitree concept. You can not certainly say which parameter you need on all site or you will choose to much.

I know this can not really go into TS template, because this would turn the cache useless. Maybe you could allow to override those Install Tool settings inside the domain record? I guess those are loaded on each pagehit anyway.
Or one would need to introduce ugly localconf.php conditions which makes troubles, because the install tool does not support those.

Could someone with more knowledge of the TYPO3 bootstrap say what could be a possible solution.

#5 Updated by Oliver Hader almost 7 years ago

  • Target version changed from 4.6.0-beta3 to 4.6.0-RC1

#6 Updated by Xavier Perseguers almost 7 years ago

  • Target version changed from 4.6.0-RC1 to 4.7.0

#7 Updated by Susanne Moog over 6 years ago

  • Complexity changed from medium to nightmare

#8 Updated by Gerrit Code Review over 6 years ago

  • Status changed from New to Under Review

Patch set 3 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#9 Updated by Gerrit Code Review over 6 years ago

Patch set 4 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#10 Updated by Gerrit Code Review over 6 years ago

Patch set 5 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#11 Updated by Gerrit Code Review over 6 years ago

Patch set 6 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#12 Updated by Gerrit Code Review over 6 years ago

Patch set 7 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#13 Updated by Gerrit Code Review over 6 years ago

Patch set 8 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#14 Updated by Gerrit Code Review over 6 years ago

Patch set 9 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#15 Updated by Gerrit Code Review over 6 years ago

Patch set 10 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#16 Updated by Gerrit Code Review over 6 years ago

Patch set 11 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#17 Updated by Gerrit Code Review over 6 years ago

Patch set 12 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#18 Updated by Gerrit Code Review over 6 years ago

Patch set 13 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#19 Updated by Gerrit Code Review over 6 years ago

Patch set 14 for branch master has been pushed to the review server.
It is available at http://review.typo3.org/4704

#20 Updated by Daniel Poetzinger over 6 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#21 Updated by Riccardo De Contardi 9 months ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF