TYPO3 Core

`clearState` did not change since 2015 - please also note that it is annotated as an internal function and not meant to be used by 3rd party code.

If you can, try to find out, what exactly is consuming that memory?

I used your code to perform manual tests and I benchmarked it with blackfire.

Maybe there is a misunderstanding in what clearState() does. It removes all references to entities inside the persistence manager and the persistence session.
This does not necessarily mean that the php garbage collection is triggered. All entities remain in the memory as long as they are used in your code.

What happens here is the following:

$v1 = $this->frontendUserRepository->findAll();

This creates a QueryResultInterface, which at this point in time does not allocate memory for entities.

foreach($v1 as $vv1) {

This triggers $v1 to instantiate all entities and it binds them in it's internal property \TYPO3\CMS\Extbase\Persistence\Generic\QueryResult::$queryResult.

$a1 = $this->frontendUserRepository->findOneByUid($vv1->getUid());

This triggers a new sql query and the instantiation of new objects. I'd consider this a bug because as long as an existing, unchanged entity exists in the session, the database should not be queried again.
As `$a1` is not outside the loop, the memory consumption only increases shortly before the garbage collection kicks in.

$this->objectManager->get(\TYPO3\CMS\Extbase\Persistence\Generic\PersistenceManager::class)->clearState();

This actually works. All references to existing objects are removed.

To have reliable numbers, you should add the following to your test snippet:

unset($v1);
var_dump(memory_get_usage());

This brings your memory consumption down to about the same level it had in the beginning.
If you prove me wrong, I will dig deeper but for now I consider this bug report invalid.

Stumbled on the same issue in an update project in TYPO3 10.

We've been using clearState to manage Extbase' memory appetite for years (since TYPO3 7) and it worked like a charm. A little trait (https://medium.com/@Waldgeist/use-a-generator-to-batch-process-typo3-repository-entries-552589b4d08d) enabled us to process 100k entries no problem, no memory hogging, nothing.

Now the same attempt to batch process feUser entries (21k of them) results in a linear memory growth, no freeing until the memory just runs out. Not PHP version dependent. This is a serious bug and issue for Extbase.

Just wrote this little testcase, which runs out of memory, hasn't before TYPO3 10.

        foreach($this->feUserRepository->findAllBatched(25) as $feUser){
            $this->output->writeln($feUser->getEmail());
            $this->output->writeln(\memory_get_usage());
        }

On the sidenote: Just declaring a function after it has been in use (and there is really no replacement for it) as @interal isn't really the way to go imho. This is a useful and vital tool in Extbase programming.

EDIT: checking the code it all seems quite simple. Too simple to fail. Haven't run that through xDebug but is it somehow possible, the DI causes multiple instance of the persistenceManager to be created instead of just a Singleton? That would maybe explain, why the clearState on the persistenceManager doesn't do anything, because every Repository has its own instance of it. But that's a pretty wild guess...

EDIT: A quick check with PHP 7.4 $ref = \WeakReference::create($feUser); shows the object is still referenced from somewhere within Extbase Framework after it should've been cleared by clearState();

        foreach($this->feUserRepository->findAllBatched(25) as $feUser){
            $ref = \WeakReference::create($feUser);
            unset($feUser);
            $this->persistenceManager->persistAll();
            $this->persistenceManager->clearState();
            $this->output->writeln("Referenced still?: " . $ref->get());
        }