TYPO3 Core

With little effort and a little concurrency it is still possible to trigger the TYPO3 frontend to show a "Page is being generated" message instead of the rendered content of the page.
This happens despite locking that was introduced to get rid of this message.

In general this message appears, when a page that should be cached is hit with empty caches and a second request hits the same page concurrently a few milliseconds later.
The second request will show the message (other than I would have expected to wait until the first page generated the cache).

This happens for the following reasons:

The temporary cache entry is still generated before actual page rendering starts. When page rendering finishes, this temporary cache entry is overwritten with the actual rendered page
A cached page consists of two separate cache entries in two separate caches. A TypoScript cache stored in cache_pagesection and with only the page id as identifier and the page cache stored in cache_pages with a lot of things being part of the identifier like page id, cHash, language...

This means the TypoScript cache is generated only once for a page, while there typically exist multiple page cache entries.

Both caches are protected from being generated multiple times by different requests with locking. The locking itself works fine as far as I investigated. However how the locking of both cache generations and the generation of the temporary cache entry is intertwined, leads to the (to me) unexpected behavior.

The code flow for the first hit with empty caches is roughly as follows:

1. Lock TS cache generation (if empty cache)
2. Lock page generation (if empty cache)
3. Generate TS cache
4. Generate temporary cache entry
5. Release TS lock
6. Generate page and (over-)write page cache
7. Release TS lock
8. Release page generation lock

The culprit is position 5, which was introduced in TYPO3 7.6. Before that version, both locks were only removed after complete page rendering finished.
The problem with that is, that TS lock is released after the temporary cache entry is written

It means a second request that is fired right after the first, will wait on position 1 until the TS lock is released and directly asks for a page cache entry, which is then found; the temporary cache entry.

This means it only need two concurrent requests on empty caches for the second request to show the page is being generated message.
Two requests do not mean a "high load" or "high traffic". It imho is a severe and esay to trigger bug that needs to be fixed, at best down to TYPO3 7.6.

The fix is easy: Remove position 5, or at least move it before generating the temporary cache entry. Both options lead to the same result on empty caches, because it does not matter whether the second request waits for the TS lock or the page lock to be released.

For master, we should entirely get rid of the temporary content, as it is not required any more. In older TYPO3 versions, the code could have proceeded without locking. If locking failed, it was silently ignored. With introduction of the new locking API in TYPO3 7.6 this is not the case any more. If locking fails, TYPO3 fails hard as well. So the poor mans mitigation for concurrent requests is not needed any more.