Bug #20495: function getByTag() in new caching framework runs out of memory - TYPO3 Core - TYPO3 Forge

Actions

Copy link

Bug #20495

closed

function getByTag() in new caching framework runs out of memory

Added by Rupert Germann over 15 years ago. Updated about 11 years ago.

Status:

Closed

Priority:

Must have

Assignee:

Category:

Target version:

Start date:

2009-05-23

Due date:

% Done:

Estimated time:

TYPO3 Version:

4.3

PHP Version:

5.2

Tags:

Complexity:

Is Regression:

Sprint Focus:

Description

I got a:
Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 27672 bytes) in /srv/SVN/forge.typo3.org/Core/trunk/t3lib/cache/frontend/class.t3lib_cache_frontend_variablefrontend.php on line 107

after I tried to open the cache section of the adminpanel.

Steps to reproduce:
I have a fully cached news singleview page with a huge amount of cached versions. The table cache_pages is ~ 300MB big and it has ~ 5000 entries 99% of them on that very single view page.
If I now try to open the cache section of the admpanel it calls extGetNumberOfCachedPages(). This function calls getByTag() from the new caching framework. getByTag() tries to read the complete cache_pages table to an array which is passed back to extGetNumberOfCachedPages() which counts the array entries to display the number of cached page versions.

in older TYPO3 versions this worked flawlessly with 10000s of pages.

in TYPO3 4.2.6 function extGetNumberOfCachedPages looks like this:

function extGetNumberOfCachedPages($page_id)    {
        $res = $GLOBALS['TYPO3_DB']->exec_SELECTquery('count(*)', 'cache_pages', 'page_id='.intval($page_id));
        list($num) = $GLOBALS['TYPO3_DB']->sql_fetch_row($res);
        return $num;
    }

I suggest to use this function again .

(issue imported from #M11175)

Related issues 1 (0 open — 1 closed)

Actions

Copy link

Updated by Steffen Kamper over 15 years ago

Hi Rupi,

i don't think it's possible to use the old function. If you use eg file-cache there is no DB-table used anymore. There are other possibilities like memcached or APC.

Correct is that reading complete into array is too much and should be avoided.
I'll look to flow3 what happens there.

Actions

Copy link

Updated by Rupert Germann over 15 years ago

hi,

yes you're right, using the simple count query does of yourse only work if there's actually a db to query.
The caching framework needs a method to count cache record/files/entries fast.

It's not only that the complete result is read to an array - first the hashes are fetched (by executing selectgetRows and an additionally foreach loop) and then a single query is done for each entry, reading the complete content of the field content in a array. The field content can contain quite a lot of html code. That means that depending on your server configuration the memory limit will be reached quite fast.

But doing 5001 sql queries to get something like "Cache entries: 5000" is embarassing, ridiciculous, nuts, stupid ....
;-)
you know what I mean.

Actions

Copy link

Updated by Steffen Kamper over 15 years ago

hehe - i absolutely agree :-)

Actions

Copy link

Updated by Ingo Renner over 15 years ago

The DB BE could maybe use a more direct query...

Actions

Copy link

Updated by Karsten Dambekalns over 15 years ago

Fetching everything for counting is indeed nonsense. getByTag() was not intended for this, so if one needs this information, getEntryCountByTag() might make more sense.

Regarding ~28595 - whatever the BE does, we need an API to access it. :)

Actions

Copy link