Bug #72074

FileLockStrategy fails on NFS folders

Added by Dmitry Dulepov over 3 years ago. Updated about 1 month ago.

Status:
New
Priority:
Must have
Assignee:
-
Category:
Locking / Session Handling
Target version:
-
Start date:
2015-12-06
Due date:
% Done:

0%

TYPO3 Version:
8
PHP Version:
Tags:
Complexity:
easy
Is Regression:
No
Sprint Focus:

Description

FileLockStrategy assumes it can always lock exclusively but fails on NFS mounts (example: vagrant environment). There should be a check that exclusive lock is possible before the capability is reported.

The result of the error is the exception:

#1294586098: Could not acquire access lock for "pagesection"" 

RuntimeException thrown in file
/home/***/vendor/typo3/cms/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php in line 4921.

7 TYPO3\CMS\Frontend\Controller\TypoScriptFrontendController::acquireLock("pagesection", "1::")

/home/***/vendor/typo3/cms/typo3/sysext/frontend/Classes/Controller/TypoScriptFrontendController.php:
02357:             // but we protect the access again with a global exclusive lock to avoid race conditions
02358: 
02359:             $this->acquireLock('pagesection', $this->id . '::' . $this->MP);
02360:             //
02361:             // from this point on we're the only one working on that page ($key)

Related issues

Related to TYPO3 Core - Feature #87072: Make locking configurable Under Review 2018-12-04

History

#1 Updated by Gerrit Code Review over 3 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/45154

#2 Updated by Gerrit Code Review over 3 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/45154

#3 Updated by Gerrit Code Review over 3 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/45154

#4 Updated by Wouter Wolters almost 3 years ago

  • Status changed from Under Review to New

Patch was abandoned

#5 Updated by Grigori Prokhorov about 1 year ago

This issue still persists in TYPO3 8, Ubuntu 18.04.

The provided patch does work (with small adjustments to accommodate the new LOCK_CAPABILITY_NOBLOCK option):

        $capabilities = self::LOCK_CAPABILITY_SHARED;
        $testLock = GeneralUtility::makeInstance(__CLASS__, str_replace('\\', '_', __CLASS__));
        try {
            if ($testLock->acquire(self::LOCK_CAPABILITY_EXCLUSIVE)) {
                $testLock->release();
                $capabilities |= self::LOCK_CAPABILITY_EXCLUSIVE;
            }
            if ($testLock->acquire(self::LOCK_CAPABILITY_NOBLOCK)) {
                $testLock->release();
                $capabilities |= self::LOCK_CAPABILITY_NOBLOCK;
            }
        } catch (\Exception $exception) {
            throw new LockAcquireException($exception->getMessage(), $exception->getCode(), $exception);
        }

#6 Updated by Grigori Prokhorov about 1 year ago

  • TYPO3 Version changed from 7 to 8
  • PHP Version set to 7.2
  • Complexity set to easy

#7 Updated by Grigori Prokhorov about 1 year ago

  • TYPO3 Version changed from 8 to 7
  • PHP Version deleted (7.2)

#8 Updated by Gerrit Code Review 12 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/57514

#9 Updated by Gerrit Code Review 12 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/57514

#10 Updated by Sebastien Convers 10 months ago

Gerrit Code Review wrote:

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/57514

Thanks a lot, this patch save my life on Nuxit hosting !! Will it be in 8.7.20 ?

#11 Updated by Markus Klein 9 months ago

@Sebastian Convers: Read my comments in the patch!
The answer is: No, not in this state.

#12 Updated by Sybille Peters 7 months ago

  • Status changed from Under Review to New

Patch was abandoned.

#13 Updated by Sybille Peters 7 months ago

I submitted PR #363 to the documentation. This does not solve this issue, but I think it lessens the severity by adding the information that there may be problems with NFS, gives some information about the locking API and could be extended to address further use cases.

This is one puzzle piece, another could be making locking configurable and adding further locking strategies in the core, I think.

#14 Updated by Sybille Peters 7 months ago

  • Category set to Locking / Session Handling

#15 Updated by Sybille Peters 7 months ago

#16 Updated by Tobias Klepp about 1 month ago

  • TYPO3 Version changed from 7 to 8

My sys_log is full this thousands of this error messages.

Core: Exception handler (WEB): Uncaught TYPO3 Exception: #1294586099: Lock file could not be opened | TYPO3\CMS\Core\Locking\Exception\LockAcquireException thrown in file /var/vhosts/domain.com/vendor/typo3/cms/typo3/sysext/core/Classes/Locking/FileLockStrategy.php in line 95. Requested URL: https://www.domain.com/path/to/page/

and

Core: Error handler (FE): PHP Warning: fopen(/var/vhosts/domain.com/html/typo3temp/var/locks/flock_5809fbcf8380f5ad1454dc2863c41a0a): failed to open stream: No such file or directory in /var/vhosts/domain.com/vendor/typo3/cms/typo3/sysext/core/Classes/Locking/FileLockStrategy.php line 93

I also use NFS to share the file system with two webservers for redundancy.

#17 Updated by Markus Klein about 1 month ago

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

#18 Updated by Sybille Peters about 1 month ago

Please also see: https://docs.typo3.org/typo3cms/CoreApiReference/ApiOverview/LockingApi/Index.html#caveats

If someone wants to update that page to make it even clearer, please do.

#19 Updated by Sybille Peters about 1 month ago

@Markus Klein:

NFS => not suitable for shared locking.
Recommendation: Keep caches local (and therefore the locks for those too).

What I (hope I) understand: The locks are used to lock (some) TYPO3 cache(s), e.g. page cache by default. So you have to find a way to either use a central cache or a distributed cache and configure the cache and the locking accordingly. (Obviously, you can't use one central cache - as is default - and have the locks local for each machine and vice versa).

What I don't understand: If you keep the locks local (e.g. map typo3temp/var/lock or entire typo3temp to local storage), and do the same with the page cache (e.g. with local Redis storage) on a load-balancing distributed system, how do you make sure all caches for a page are cleared when pages are edited?

I don't know a way to do that with out-of-the-box TYPO3 functionality.

#20 Updated by Markus Klein about 1 month ago

Sybille Peters wrote:

@Markus Klein:

NFS => not suitable for shared locking.
Recommendation: Keep caches local (and therefore the locks for those too).

What I (hope I) understand: The locks are used to lock (some) TYPO3 cache(s), e.g. page cache by default. So you have to find a way to either use a central cache or a distributed cache and configure the cache and the locking accordingly. (Obviously, you can't use one central cache - as is default - and have the locks local for each machine and vice versa).

Correct.

What I don't understand: If you keep the locks local (e.g. map typo3temp/var/lock or entire typo3temp to local storage), and do the same with the page cache (e.g. with local Redis storage) on a load-balancing distributed system, how do you make sure all caches for a page are cleared when pages are edited?

I don't know a way to do that with out-of-the-box TYPO3 functionality.

You need to implement this specific to your needs. Mostly a simple shell script that runs a cli command to flush caches is enough.
If you use one of the servers for content editing, you also need to implement a hook that triggers the according cache flush command also on the other servers using the same database.
The thing is that there are so many ways how you can do scaling/loadbalancing that I don't see a generic solution that could be implemented into the core.

#21 Updated by Susanne Moog about 1 month ago

One solution could also be to use something like https://github.com/bmack/typo3-redis-lock-strategy for locking to have it independent from the file system.

#22 Updated by Markus Klein about 1 month ago

Of course you can use a distributed locking mechanism too if you share the caches. The redis integration is of course nice when using redis for the caches anyways.

#23 Updated by Dmitry Dulepov about 1 month ago

Markus Klein wrote:

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

We are using vagrant for development. It mounts the whole project over NFS. How to do you propose to make a subdirectory of NFS-mounted directory local?

#24 Updated by Markus Klein about 1 month ago

Dmitry Dulepov wrote:

Markus Klein wrote:

NFS => not suitable for shared locking.

Recommendation: Keep caches local (and therefore the locks for those too).

We are using vagrant for development. It mounts the whole project over NFS. How to do you propose to make a subdirectory of NFS-mounted directory local?

That's indeed a totally different use case. So it is not about load balancing and sharing stuff for multi-head instances, but for local development.
This report got hijacked a little.

Regarding a solution for your use case:
As I posted years ago I'm not a fan of doing this feature detection on every call to createLocker. I consider the fact that a certain property of a locking strategy is there or not there to be of rather static nature (limited to the boundaries of the current instance). So this feature detection of the capabilities should be part of the Install Tool (could be like part of the SilentConfigurationUpgradeService) then.
I would be fine with such a detection logic in the Install Tool, which persists its findings in TYPO3_CONF_VARS.
This could even be done for version 8 then as a bugfix.
What do you think about such an approach?

Also available in: Atom PDF