TYPO3 Core

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/45154

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/45154

This issue still persists in TYPO3 8, Ubuntu 18.04.

The provided patch does work (with small adjustments to accommodate the new LOCK_CAPABILITY_NOBLOCK option):

        $capabilities = self::LOCK_CAPABILITY_SHARED;
        $testLock = GeneralUtility::makeInstance(__CLASS__, str_replace('\\', '_', __CLASS__));
        try {
            if ($testLock->acquire(self::LOCK_CAPABILITY_EXCLUSIVE)) {
                $testLock->release();
                $capabilities |= self::LOCK_CAPABILITY_EXCLUSIVE;
            }
            if ($testLock->acquire(self::LOCK_CAPABILITY_NOBLOCK)) {
                $testLock->release();
                $capabilities |= self::LOCK_CAPABILITY_NOBLOCK;
            }
        } catch (\Exception $exception) {
            throw new LockAcquireException($exception->getMessage(), $exception->getCode(), $exception);
        }

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/57514

Gerrit Code Review wrote:

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/57514

Thanks a lot, this patch save my life on Nuxit hosting !! Will it be in 8.7.20 ?

@Sebastian Kurfuerst Convers: Read my comments in the patch!
The answer is: No, not in this state.

I submitted PR #363 to the documentation. This does not solve this issue, but I think it lessens the severity by adding the information that there may be problems with NFS, gives some information about the locking API and could be extended to address further use cases.

This is one puzzle piece, another could be making locking configurable and adding further locking strategies in the core, I think.

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

Please also see: https://docs.typo3.org/typo3cms/CoreApiReference/ApiOverview/LockingApi/Index.html#caveats

If someone wants to update that page to make it even clearer, please do.

@Markus Klein:

NFS => not suitable for shared locking.
Recommendation: Keep caches local (and therefore the locks for those too).

What I (hope I) understand: The locks are used to lock (some) TYPO3 cache(s), e.g. page cache by default. So you have to find a way to either use a central cache or a distributed cache and configure the cache and the locking accordingly. (Obviously, you can't use one central cache - as is default - and have the locks local for each machine and vice versa).

What I don't understand: If you keep the locks local (e.g. map typo3temp/var/lock or entire typo3temp to local storage), and do the same with the page cache (e.g. with local Redis storage) on a load-balancing distributed system, how do you make sure all caches for a page are cleared when pages are edited?

I don't know a way to do that with out-of-the-box TYPO3 functionality.

Sybille Peters wrote:

@Markus Klein:

NFS => not suitable for shared locking.
Recommendation: Keep caches local (and therefore the locks for those too).

What I (hope I) understand: The locks are used to lock (some) TYPO3 cache(s), e.g. page cache by default. So you have to find a way to either use a central cache or a distributed cache and configure the cache and the locking accordingly. (Obviously, you can't use one central cache - as is default - and have the locks local for each machine and vice versa).

Correct.

What I don't understand: If you keep the locks local (e.g. map typo3temp/var/lock or entire typo3temp to local storage), and do the same with the page cache (e.g. with local Redis storage) on a load-balancing distributed system, how do you make sure all caches for a page are cleared when pages are edited?

I don't know a way to do that with out-of-the-box TYPO3 functionality.

You need to implement this specific to your needs. Mostly a simple shell script that runs a cli command to flush caches is enough.
If you use one of the servers for content editing, you also need to implement a hook that triggers the according cache flush command also on the other servers using the same database.
The thing is that there are so many ways how you can do scaling/loadbalancing that I don't see a generic solution that could be implemented into the core.

One solution could also be to use something like https://github.com/bmack/typo3-redis-lock-strategy for locking to have it independent from the file system.

Of course you can use a distributed locking mechanism too if you share the caches. The redis integration is of course nice when using redis for the caches anyways.

Markus Klein wrote:

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

We are using vagrant for development. It mounts the whole project over NFS. How to do you propose to make a subdirectory of NFS-mounted directory local?

Dmitry Dulepov wrote:

Markus Klein wrote:

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

We are using vagrant for development. It mounts the whole project over NFS. How to do you propose to make a subdirectory of NFS-mounted directory local?

That's indeed a totally different use case. So it is not about load balancing and sharing stuff for multi-head instances, but for local development.
This report got hijacked a little.

Regarding a solution for your use case:
As I posted years ago I'm not a fan of doing this feature detection on every call to createLocker. I consider the fact that a certain property of a locking strategy is there or not there to be of rather static nature (limited to the boundaries of the current instance). So this feature detection of the capabilities should be part of the Install Tool (could be like part of the SilentConfigurationUpgradeService) then.
I would be fine with such a detection logic in the Install Tool, which persists its findings in TYPO3_CONF_VARS.
This could even be done for version 8 then as a bugfix.
What do you think about such an approach?

@Susi

One solution could also be to use something like https://github.com/bmack/typo3-redis-lock-strategy for locking to have it independent from the file system.

There are several extensions floating about which use redis for locking. I believe Benni now recommends the b13 distributed-locks. https://github.com/b13/distributed-locks This one has recent changes. I don't know about the bmack/typo3-redis-lock-strategy.

(I would still prefer a core solution that is maintained long term.)

Really? Require Redis instead of a simple check in php if exclusive locking is allowed?