Bug #28514
closed
t3lib_cache_backend_MemcachedBackend's tag-to-identifier and vice-versa is a performance chugger
0%
Description
Using the standard class for Memcached Caching, it does takes 10 calls to memcache to do a single SET operation.
I've added error_log statements for every call made by this class to memcached. See attached class.
Here an excerpt of the resulting log. I've isolated two SETs :
SET set() normal fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc DELETE removeIdentifierFromAllTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag()SET set() normal fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc DELETE removeIdentifierFromAllTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag() %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc //[...] SET set() normal fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 DELETE removeIdentifierFromAllTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc //[...] SET set() normal fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 DELETE removeIdentifierFromAllTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 <pre> There another big issue with this tag/identifier saving process; and that the reason of my custom rewrite of this class. When flushing the cache on our Live servers where almost all of the data was cached, we ran into PHP's 30seconds timeout every time we were hitting the "clear-custom-cache" button in the admin section. I have written two classes in an attempt to fix this; please see attached in .tar.gz file: * Regular: Trying to keep the current system, but reduce the calls and fix the timeout problem. public remove() implementation had to the dropped... * Aggressive: No longer doing anything with tags or saving any identifier... Clearing caching is done using version number appended to cache key... This class leaves all older entries in memcached. The deamon has its own way to ensure free space, so I didn't bothered to empty it...
Files
Updated by Raphaël Riel over 13 years ago
Sorry for above formatting... Since I can't edit of delete the entry and redo... here take 2 :
Using the standard class for Memcached Caching, it does takes 10 calls to memcache to do a single SET operation.
I've added error_log statements for every call made by this class to memcached. See attached class.
Here an excerpt of the resulting log. I've isolated two SETs :
SET set() normal fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc DELETE removeIdentifierFromAllTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findIdentifiersByTag() %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc GET findTagsByIdentifier() fr_SphinxSearchPluginController-indexConfigKeyValueAssoc [...] SET set() normal fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 DELETE removeIdentifierFromAllTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() fr SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062 SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findIdentifiersByTag() %MEMCACHEBE%st_zone_mvc SET addIdentifierToTags() fr_user_zone_cat_menu-getRootlineDataForUid-159062 GET findTagsByIdentifier() fr_user_zone_cat_menu-getRootlineDataForUid-159062
There another big issue with this tag/identifier saving process; and that the reason of my custom rewrite of this class. When flushing the cache on our Live servers where almost all of the data was cached, we ran into PHP's 30seconds timeout every time we were hitting the "clear-custom-cache" button in the admin section.
I have written two classes in an attempt to fix this; please see attached in .tar.gz file:- Regular: Trying to keep the current system, but reduce the calls and fix the timeout problem. public remove() implementation had to the dropped...
- Aggressive: No longer doing anything with tags or saving any identifier... Clearing caching is done using version number appended to cache key... This class leaves all older entries in memcached. The deamon has its own way to ensure free space, so I didn't bothered to empty it...
Updated by Steffen Gebert over 13 years ago
- Category changed from Performance to Caching
Updated by Christian Kuhn over 13 years ago
- Status changed from New to Rejected
- Priority changed from Must have to Won't have this time
- Target version set to 4.6.0-beta1
- TYPO3 Version changed from 4.5 to 4.6
- Complexity set to medium
We are aware of this and the problem is documented in http://wiki.typo3.org/Caching_framework
This is a design problem that can only be solved if the tags-identifier relations are not written to memcache, but to some other backend that can handle it. We have a 'hybrid backend' for this in the works, but it will not be ready for 4.6 anymore (and it must go to FLOW3 first).
I will close this issue now, solutions for this problems must go to FLOW3 and will be backported afterwards. Additionally we have the memcache 'slow' and 'corrupt' problem on the radar but we will not solve it for 4.6 anymore.
If you want to give your backend to others I'd suggest to deliver it as an own implementation and create an extension together with a documentation of what your solution does in advance to the core implementation and what can not be done with your solution currently.
Updated by Ernesto Baschny over 11 years ago
- Target version deleted (
4.6.0-beta1)