Bug #35576
Index Queue Worker runs out of memory
| Status: | Closed | Start date: | 2012-04-03 | |
|---|---|---|---|---|
| Priority: | Should have | Due date: | ||
| Assignee: | Ingo Renner | % Done: | 100% |
|
| Category: | Indexing | |||
| Target version: | 2.2 | |||
| TYPO3 Version: | Has patch: | |||
| PHP Version: | Tags: | |||
| Votes: | 1 (View) |
Description
2.6.0EAP : when running the Index Queue Worker scheduler job from the Scheduler backend module with the default 50 documents per run and 128MB memory assigned it often runs out of memory at various points in the processing.
I haven't seen it in previous 2.50-dev installations, but after updating a few development installations to 2.6.0 they all show this symptom.
Related issues
| related to Apache Solr for TYPO3 CMS - Bug #35404: Index queue didn't stop the task | Closed | 2012-03-30 |
History
Updated by Christoph Moeller 12 months ago
We're running into this and Bug #35404, as well. After upgrading from old EAP (released somewhere around mid-november 2011) to new EAP trunk (2012-06-14) last week, we got exactly the same symptoms as Jigal during the nightly full indexing run.
Indexing runs fine until some point where it crashes with this error:
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 262144 bytes) in /var/lib/typo3/typo3_src-4.5.14/t3lib/class.t3lib_tsparser.php on line 125
After that, the scheduler tasks "hangs": green bullet showing "still running" status, but no activity any more. The rest of the site will not be indexed then, until we stop the current run and have it continue, manually.
The scheduler indexqueue worker task is set to run every 5min, scheduler cronjob runs every 60s. No parallel execution whatsoever, nothing that could possibly interfere.
The scheduler task runs fine if set to only index 50 docs per run, it works most of the time, but takes very long to completely index the whole site. If we increase to 80 docs or more, the indexer consumes more RAM than allowed.
We could provide BE/shell access or a remote session to further debug this. Is there any way to speed up this bugfix by sponsoring?
Updated by Christoph Moeller 12 months ago
Additionally, after upgrading to current trunk (as of today) and increasing the memory_limit for CLI and apache to 1GB, the error still occurs - same LOC. So this rather looks like some kind of a heavy memory leak somewhere.
Updated by Ingo Renner 12 months ago
I now had a look at Jari's installation and couldn't run into a memory limit when indexing records. There was a clear increase of memory usage with every page being indexed though.
Jigal and Marco, could you maybe have a look at your installations whether this is the case for you, too?
Also of interest is whether your installations are multi language and/or use access restrictions.
Updated by Ingo Renner 12 months ago
Also, what environment ... OS, PHP version... are you using?
Updated by Christoph Moeller 12 months ago
Ingo Renner wrote:
Also, what environment ... OS, PHP version... are you using?
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1
PHP 5.3.10 (upgraded from SuSE people repository), APC 3.1.9 (via PECL - turned off to debug, of course).
Updated by Ingo Renner 12 months ago
- File 35576.diff added
As it turned out on Jari's installation the increased memory consumption was caused by a missing configuration for config.sys_language_mode. The setting is needed to determine which translations to index. When this setting is not set on the first page being indexed on an index run TSFE is initialized so that the setting is available.
That TSFE object is put into a cache for reuse (usefull for example when indexing records from a common storage page). The cache key includes the page ID and since the page ID changes for each page the setting will never be set and for each page the TSFE will be cached again.
So the easiest solution is to always set config.sys_language_mode.
I also changed the affected code so that we now retrieve the sys_language_mode setting from the root page - it should not change for a site anyway. This way the cache is used better.
Please try the attached patch (without setting config.sys_language_mode) and let me know whether it solves the problem for your installation.
Updated by Ingo Renner 12 months ago
- Status changed from New to Under Review
- Assignee set to Jigal van Hemert
- Target version set to 2.2
- % Done changed from 0 to 100
Updated by Marco Bresch 12 months ago
Hi Ingo,
I tested this patch with v2.6.0 (tags/2.6.0) and it works for me.
Thanks a lot and greetings
Marco :-)
Updated by Jigal van Hemert 12 months ago
Patch Set 1: Verified; Looks good to me, but someone else must approve
As discussed: please use strict mode on in_array().
Updated by Ingo Renner 12 months ago
- Status changed from Under Review to Resolved
- Assignee changed from Jigal van Hemert to Ingo Renner
Thanks for checking the fix Jigal!
Updated by Ingo Renner 11 months ago
- Status changed from Resolved to Closed