TYPO3 Core

Markus Klein wrote:

This should never be the case.

Why should this never be the case?
If you let all processes wait you may potentially run into critical load situations. Delivering temporary content to those clients quickly gets them off the TCA stack.

If the load is an issue, this will anyway be a problem, since the "Page is being generated" page will also refresh after some time.
In contrast, if you simply wait for a lock, this does not consume resources.

Either way, i would suggest to have at least a configurable refresh time and an option for disabling this feature, since for pages with long generation times, this can occur quite often.
What do you think?

The accessLock is necessary as we have to prevent resource escalation.
The pageAccess lock depends on the $key, which is a composite of quite some variables. Depending on the locking mode used (eg semaphores), the resources for the lock have to be freed after they are not needed anymore, otherwise a server cloud be potentially flooded with resources for locks.

In \TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController we have

protected function acquireLock($type, $key)
{
            $lockFactory = GeneralUtility::makeInstance(LockFactory::class);
            $this->locks[$type]['accessLock'] = $lockFactory->createLocker($type);
            $this->locks[$type]['pageLock'] = $lockFactory->createLocker(
               $key,
               LockingStrategyInterface::LOCK_CAPABILITY_EXCLUSIVE | LockingStrategyInterface::LOCK_CAPABILITY_NOBLOCK
             );

hence $this->locks['pages']['accessLock'] does not depend on any key, but only on $type='pages', in contrast to $this->locks['pages']['pageLock']

We may destroy a lock resource only if nobody else is currently on that resource. (That is why the pagelock is non-blocking.) Hence we have to ensure that we are the only ones (exclusive) operating on that very pagelock, when doing such actions.
The accessLock in contrast is used only in exclusive mode and it's resource name (and lock name) is not depending on environment or data. The resources of an access lock may NEVER be removed unless the webserver is stopped (application shutdown in terms of a real applications server). In the PHP world we have no notion of application life cycle and hence do not know when a the resource for a lock maybe removed. As a consequence the maximum resource usage of locks must be limited. The access locks are constructed that way, so they only take up very few (four or so, if I remember correctly) resources.

but do I understand it correctly that any request for any page generation requires an exclusive accessLock which is immediately released?

The old implementation in 6.x and earlier was that the lock resources have never been freed. This caused weird server crashes (!) because of exhausted semaphore resources in the kernel.

this seems a good improvement introduced in https://review.typo3.org/#/c/38840/

side remark: I remember having this issue with TYPO3 since 2005, so I think this should really be improved.

In contrast, if you simply wait for a lock, this does not consume resources.

Well, you definitely block a TCP stack, a webserver process/thread and maybe also a PHP-fpm process/thread. Moreover it depends on the locking method used, a lock might need busywaiting.
In the worst case on high load someone flushes all FE caches and a lot of requests would be piled up. The server could easily run into a maximum process limit by the kernel if a non-threaded worker is used. While this might not be the case for most sane setups for high load, we cannot rule that out completely. Better to have a waiting page and the user hits F5 a few seconds after reading it (30s with the autoreload) than a 500 server-error page or no response at all.
The few seconds delay until F5 is hit is usually sufficient to have the caches warmed up again.

An option to disable the behaviour is of course doable. Feel free to push a patch to Gerrit.

hence $this->locks['pages']['accessLock'] does not depend on any key, but only on $type='pages', in contrast to $this->locks['pages']['pageLock']

Exactly. accessLock only depends on a limited set of possible keys so the number of underlying resources has an upper bound. pageLock's key is unique to rendering variant of the current page and includes things like uid, MP, etc etc.

but do I understand it correctly that any request for any page generation requires an exclusive accessLock which is immediately released?

Yep. This is the price we need to pay to avoid race conditions during page generation. Keep in mind that cached pages are of course not affected. We do not lock for cache reads.
(which is not perfect in terms of the reader-writer-problem, but acceptable)

Well, you definitely block a TCP stack, a webserver process/thread and maybe also a PHP-fpm process/thread. Moreover it depends on the locking method used, a lock might need busywaiting.

This is true, but again, if users refresh the page you would anyway have this issue, although possibly distributing the peak load

Better to have a waiting page and the user hits F5 a few seconds after reading it (30s with the autoreload) than a 500 server-error page or no response at all.

This is an awful user experience to simply get an (almost) empty page with barely visible information. Most first-time visitors would simply leave the page, which is a disaster.

An option to disable the behaviour is of course doable. Feel free to push a patch to Gerrit.

would this also be included into 7.6 or only 8?
how should I do it? here's my proposal:
- add a new typo3_conf_vars option for setting the refresh time or disabling this function (incl. documentation)
- add a loading-icon to the "page is being generated" page

Is this allready realised / improved in Typo3 Version 8?

@Michael Christian: unfortunately, this is not yet implemented.
However, there exists an extension https://typo3.org/extensions/repository/view/skip_page_is_being_generated
Maybe it would be good to use that implementation and make it configurable.
Anyone up for that?

Is this allready realised / improved in Typo3 Version 8?

Maybe you got that a bit wrong.
TYPO3 is Open Source and there is no paid dev team. So the "who" is "you"!
If you do not provide a feature or the money for man power by someone who can do it, it will most likely not happen.

@Markus Klein: Aha, maybe you got that a bit wrong. But now it's time to relax. My question has already been answered satisfactorily. See above!

@Markus Mächler: Thank you very much for this informations. That's everything what i wanted to know! I use this Extension with V7.6. ...and then probably with V8. :-) Thanx you!

Seems to be fixed in TYPO3 v9.5.6 and v8.7.25: https://typo3.org/article/typo3-956-and-8725-security-releases-published/

Page is being generated functionality removed
Thanks to a combined effort of Benjamin Franzke, Helmut Hummel and Markus Klein, the infamous "Page is being generated" message functionality has been removed in both TYPO3 v9.5.6 and TYPO3 v8.7.25. Page-locking of concurrent requests creating the page has been heavily improved and simplified, which leads to a better experience for every website visitor and TYPO3 user.