Bug #89928: Memory usage issue on CLI, caused by InMemoryLogWriter (adminpanel) - TYPO3 Core - TYPO3 Forge

Actions

Copy link

Bug #89928

closed

Memory usage issue on CLI, caused by InMemoryLogWriter (adminpanel)

Added by Armin Vieweg over 4 years ago. Updated about 4 years ago.

Status:

Closed

Priority:

Should have

Assignee:

Category:

AdminPanel

Target version:

Start date:

2019-12-12

Due date:

% Done:

100%

Estimated time:

TYPO3 Version:

PHP Version:

7.2

Tags:

Complexity:

Is Regression:

Sprint Focus:

Description

I've registered a Symfony command in TYPO3 v9 to import products. After 1000 products I've experienced a memory usage of >500MB.

Thanks to XDebug, I've figured out what causes the memory leak: It's the \TYPO3\CMS\Adminpanel\Log\InMemoryLogWriter registred in adminpanel extension.

When I put this to my extension's ext_localconf.php:

if (PHP_SAPI === 'cli') {
    unset($GLOBALS['TYPO3_CONF_VARS']['LOG']['writerConfiguration'][\TYPO3\CMS\Core\Log\LogLevel::DEBUG]);
}

the memory usage during full import stays below 50M.

The full conversation can be seen here (including screenshots): https://typo3.slack.com/archives/C025BQLFA/p1576143741206100

Actions

Copy link

Updated by Rémy DANIEL about 4 years ago

Hello

I'm experiencing the same symptoms.

A plugin loads many Extbase objects with relationships (which should definitively be @lazy, but this is not the point).
I'm seeing the high memory consumption by the InMemoryLogWriter, only when the Admin panel is activated.

The InMemoryLogWriter collects logs (first problem) from any levels for the admin panel, which stores all that logs in a cache (second problem) and adds them to the page DOM (third problem).

First problem, in my case, the InMemoryLogWriter::$log is full of SignalSlotDispatcher debug entries.

Second problem, I had also the error "The size of BLOB/TEXT data inserted in one transaction is greater than 10% of redo log size. Increase the redo log size using innodb_log_file_size" (the cache entry is reeeeeealy big).

Third problem, the page takes veeeeery long to load with high browser's memory usage, because of the admin panel's data in the DOM.

I fixed my issue by adding the missing @lazy, but maybe the Admin Panel and its InMemoryLogWriter can be optimized is such a way that a huge log activity can't blow up the system.

Cheers.

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63510

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63510

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63555

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63555

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63555

Actions

Copy link

Updated by Gerrit Code Review about 4 years ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/63579

Actions

Copy link