Task #77443

Make "Page is being generated" configurable

Added by Markus Mächler over 3 years ago. Updated 7 months ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Frontend
Target version:
-
Start date:
2016-08-08
Due date:
% Done:

0%

TYPO3 Version:
9
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

The message "Page is being generated" still appears in TYPO3 7.6.10 after the page cache is cleared and concurrent page requests are made.

This should never be the case. Moreover, the hardcoded refresh time of 30s is not suitable either.

I suggest completely removing \TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->tempPageCacheContent
and let the of the second request client wait until the page has been generated by the first request,
which will be the case since there is no "temporary content" in the database cache
and the lock for generating the page in the second request will be acquired as soon as the first request is processed, and subsequently the second request will retrieve the cached paged from the database.
See \TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->getFromCache

public function getFromCache()
{
...
$row = $this->getFromCache_queryRow();
if (!is_array($row)) {
// nothing in the cache, we acquire an exclusive lock now
$this->acquireLock('pages', $lockHash);
//
// from this point on we're the only one working on that page ($lockHash)
//
// query the cache again to see if the data are there meanwhile
$row = $this->getFromCache_queryRow();

Therefore our very simple solution is to just remove the code in \TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController->tempPageCacheContent

public function tempPageCacheContent(){}

We have tested it and it works fine.

Moreover, I am not sure why the "accessLock" introduced in https://review.typo3.org/#/c/38840/ is necessary.
This lock is set regardless of the current page id being handled, and therefore temporarily blocks the concurrent page generation of different pages.


Related issues

Related to TYPO3 Core - Bug #87980: TYPO3 can not handle two concurrent requests to the same (cachable) page without a 503 response Closed 2019-03-22

History

#1 Updated by Markus Klein over 3 years ago

  • Status changed from New to Needs Feedback
  • Priority changed from Must have to -- undefined --

This should never be the case.

Why should this never be the case?
If you let all processes wait you may potentially run into critical load situations. Delivering temporary content to those clients quickly gets them off the TCA stack.

The accessLock is necessary as we have to prevent resource escalation.
The pageAccess lock depends on the $key, which is a composite of quite some variables. Depending on the locking mode used (eg semaphores), the resources for the lock have to be freed after they are not needed anymore, otherwise a server cloud be potentially flooded with resources for locks.
We may destroy a lock resource only if nobody else is currently on that resource. (That is why the pagelock is non-blocking.) Hence we have to ensure that we are the only ones (exclusive) operating on that very pagelock, when doing such actions.
The accessLock in contrast is used only in exclusive mode and it's resource name (and lock name) is not depending on environment or data. The resources of an access lock may NEVER be removed unless the webserver is stopped (application shutdown in terms of a real applications server). In the PHP world we have no notion of application life cycle and hence do not know when a the resource for a lock maybe removed. As a consequence the maximum resource usage of locks must be limited. The access locks are constructed that way, so they only take up very few (four or so, if I remember correctly) resources.
The old implementation in 6.x and earlier was that the lock resources have never been freed. This caused weird server crashes (!) because of exhausted semaphore resources in the kernel.

#2 Updated by Markus Mächler over 3 years ago

Markus Klein wrote:

This should never be the case.

Why should this never be the case?
If you let all processes wait you may potentially run into critical load situations. Delivering temporary content to those clients quickly gets them off the TCA stack.

If the load is an issue, this will anyway be a problem, since the "Page is being generated" page will also refresh after some time.
In contrast, if you simply wait for a lock, this does not consume resources.

Either way, i would suggest to have at least a configurable refresh time and an option for disabling this feature, since for pages with long generation times, this can occur quite often.
What do you think?

The accessLock is necessary as we have to prevent resource escalation.
The pageAccess lock depends on the $key, which is a composite of quite some variables. Depending on the locking mode used (eg semaphores), the resources for the lock have to be freed after they are not needed anymore, otherwise a server cloud be potentially flooded with resources for locks.

In \TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController we have

protected function acquireLock($type, $key)
{
$lockFactory = GeneralUtility::makeInstance(LockFactory::class);
$this->locks[$type]['accessLock'] = $lockFactory->createLocker($type);
$this->locks[$type]['pageLock'] = $lockFactory->createLocker(
$key,
LockingStrategyInterface::LOCK_CAPABILITY_EXCLUSIVE | LockingStrategyInterface::LOCK_CAPABILITY_NOBLOCK
);

hence $this->locks['pages']['accessLock'] does not depend on any key, but only on $type='pages', in contrast to $this->locks['pages']['pageLock']

We may destroy a lock resource only if nobody else is currently on that resource. (That is why the pagelock is non-blocking.) Hence we have to ensure that we are the only ones (exclusive) operating on that very pagelock, when doing such actions.
The accessLock in contrast is used only in exclusive mode and it's resource name (and lock name) is not depending on environment or data. The resources of an access lock may NEVER be removed unless the webserver is stopped (application shutdown in terms of a real applications server). In the PHP world we have no notion of application life cycle and hence do not know when a the resource for a lock maybe removed. As a consequence the maximum resource usage of locks must be limited. The access locks are constructed that way, so they only take up very few (four or so, if I remember correctly) resources.

but do I understand it correctly that any request for any page generation requires an exclusive accessLock which is immediately released?

The old implementation in 6.x and earlier was that the lock resources have never been freed. This caused weird server crashes (!) because of exhausted semaphore resources in the kernel.

this seems a good improvement introduced in https://review.typo3.org/#/c/38840/

#3 Updated by Markus Mächler over 3 years ago

side remark: I remember having this issue with TYPO3 since 2005, so I think this should really be improved.

#4 Updated by Markus Klein over 3 years ago

In contrast, if you simply wait for a lock, this does not consume resources.

Well, you definitely block a TCP stack, a webserver process/thread and maybe also a PHP-fpm process/thread. Moreover it depends on the locking method used, a lock might need busywaiting.
In the worst case on high load someone flushes all FE caches and a lot of requests would be piled up. The server could easily run into a maximum process limit by the kernel if a non-threaded worker is used. While this might not be the case for most sane setups for high load, we cannot rule that out completely. Better to have a waiting page and the user hits F5 a few seconds after reading it (30s with the autoreload) than a 500 server-error page or no response at all.
The few seconds delay until F5 is hit is usually sufficient to have the caches warmed up again.

An option to disable the behaviour is of course doable. Feel free to push a patch to Gerrit.

hence $this->locks['pages']['accessLock'] does not depend on any key, but only on $type='pages', in contrast to $this->locks['pages']['pageLock']

Exactly. accessLock only depends on a limited set of possible keys so the number of underlying resources has an upper bound. pageLock's key is unique to rendering variant of the current page and includes things like uid, MP, etc etc.

but do I understand it correctly that any request for any page generation requires an exclusive accessLock which is immediately released?

Yep. This is the price we need to pay to avoid race conditions during page generation. Keep in mind that cached pages are of course not affected. We do not lock for cache reads.
(which is not perfect in terms of the reader-writer-problem, but acceptable)

#5 Updated by Markus Mächler over 3 years ago

Well, you definitely block a TCP stack, a webserver process/thread and maybe also a PHP-fpm process/thread. Moreover it depends on the locking method used, a lock might need busywaiting.

This is true, but again, if users refresh the page you would anyway have this issue, although possibly distributing the peak load

Better to have a waiting page and the user hits F5 a few seconds after reading it (30s with the autoreload) than a 500 server-error page or no response at all.

This is an awful user experience to simply get an (almost) empty page with barely visible information. Most first-time visitors would simply leave the page, which is a disaster.

An option to disable the behaviour is of course doable. Feel free to push a patch to Gerrit.

would this also be included into 7.6 or only 8?
how should I do it? here's my proposal:
- add a new typo3_conf_vars option for setting the refresh time or disabling this function (incl. documentation)
- add a loading-icon to the "page is being generated" page

#6 Updated by Markus Klein over 3 years ago

  • Subject changed from Page is being generated to Make "Page is being generated" configurable
  • Category set to Frontend
  • Status changed from Needs Feedback to New
  • Assignee deleted (Markus Klein)
  • Priority changed from -- undefined -- to Should have
  • TYPO3 Version changed from 7 to 8
  • Complexity deleted (easy)

This is a topic for CMS 8, as being a feature clearly.

Yes, rst documentation is a requirement for a new feature. CONF_VAR and icon are fine from my side of course.

#7 Updated by Michael Christian over 2 years ago

Is this allready realised / improved in Typo3 Version 8?

#8 Updated by Markus Mächler over 2 years ago

@Michael Christian: unfortunately, this is not yet implemented.
However, there exists an extension https://typo3.org/extensions/repository/view/skip_page_is_being_generated
Maybe it would be good to use that implementation and make it configurable.
Anyone up for that?

#9 Updated by Markus Klein over 2 years ago

Is this allready realised / improved in Typo3 Version 8?

Maybe you got that a bit wrong.
TYPO3 is Open Source and there is no paid dev team. So the "who" is "you"!
If you do not provide a feature or the money for man power by someone who can do it, it will most likely not happen.

#10 Updated by Michael Christian over 2 years ago

@Markus Klein: Aha, maybe you got that a bit wrong. But now it's time to relax. My question has already been answered satisfactorily. See above!

@Markus Mächler: Thank you very much for this informations. That's everything what i wanted to know! I use this Extension with V7.6. ...and then probably with V8. :-) Thanx you!

#11 Updated by Markus Klein over 2 years ago

  • Tracker changed from Bug to Task
  • TYPO3 Version changed from 8 to 9

#12 Updated by Markus Mächler 7 months ago

Seems to be fixed in TYPO3 v9.5.6 and v8.7.25: https://typo3.org/article/typo3-956-and-8725-security-releases-published/

Page is being generated functionality removed
Thanks to a combined effort of Benjamin Franzke, Helmut Hummel and Markus Klein, the infamous "Page is being generated" message functionality has been removed in both TYPO3 v9.5.6 and TYPO3 v8.7.25. Page-locking of concurrent requests creating the page has been heavily improved and simplified, which leads to a better experience for every website visitor and TYPO3 user.

#13 Updated by Markus Klein 7 months ago

  • Related to Bug #87980: TYPO3 can not handle two concurrent requests to the same (cachable) page without a 503 response added

#14 Updated by Markus Klein 7 months ago

  • Status changed from New to Closed

Correct, resolved with #87980.

Also available in: Atom PDF