Bug #82060
closed
File Abstraction Layer: Extract metadata in storage (scheduler) failing on indexing files that matches denied patterns(['BE']['fileDenyPattern'])
100%
Description
File Abstraction Layer: Extract metadata in storage (scheduler) fails on trying to index the files in denied pattern(['BE']['fileDenyPattern']) by throwing an exception like following::
_TYPO3\CMS\Core\Resource\Exception\InsufficientFileAccessPermissionsExceptionprototypeobject
message => 'You are not allowed to access that file: "browserdetect.inc.php"' (64 chars)
code => 1375955429 (integer)_
And my ['BE']['fileDenyPattern'] is "\.(php[3-7]?|phpsh|phtml)(\..*)?$|^\.htaccess$"
We can avoid this by just adding a condition check in runMetaDataExtraction() function in TYPO3\CMS\Core\Resource\Index\Indexer class as follows
$isAllowed = GeneralUtility::verifyFilenameAgainstDenyPattern($indexRecord['name']);
if($isAllowed){
$fileObject = $this->getResourceFactory()->getFileObject($indexRecord['uid'], $indexRecord);
$this->extractMetaData($fileObject);
}
so can any one please provide a patch with above mentioned solution?
Files
Updated by Ricky Mathew over 6 years ago
The logic is , we need to run the extractMetaData() function only if the file is of allowable type and that is what the condition check will do.
Updated by Ricky Mathew over 6 years ago
- Category set to scheduler
- Target version set to Candidate for patchlevel
Updated by Ricky Mathew over 6 years ago
- File Indexer_patch.patch Indexer_patch.patch added
Attaching the patch file here.
Updated by Ricky Mathew over 6 years ago
--- Indexer_orig.php 2017-08-09 16:02:52.912607837 +0530
+++ Indexer.php 2017-08-09 16:10:08.404625955 +0530
@@ -106,8 +106,12 @@
{
$fileIndexRecords = $this->getFileIndexRepository()->findInStorageWithIndexOutstanding($this->storage, $maximumFileCount);
foreach ($fileIndexRecords as $indexRecord) {
+ $isAllowed = GeneralUtility::verifyFilenameAgainstDenyPattern($indexRecord['name']);
+ if ($isAllowed) {
$fileObject = $this->getResourceFactory()->getFileObject($indexRecord['uid'], $indexRecord);
$this->extractMetaData($fileObject);
+ }
}
}
Updated by Ricky Mathew over 6 years ago
As my membership in core project is still pending, can anyone please take this task and apply this patch?
Updated by Frans Saris over 6 years ago
Hi Ricky,
tnx for the patch. You don't have to be a member of the core to push patches to Gerrit. Like most other OS projects everyone is allowed to push patches.
https://docs.typo3.org/typo3cms/ContributionWorkflowGuide/
Please let us know if you need further assistance with pushing this to Gerrit. On our Slack channel there is always someone around that's willing to assist you. https://forger.typo3.org/slack
Updated by Frans Saris over 6 years ago
btw maybe is't easier to just catch the InsufficientFileAccessPermissionsException. So add a
try {
...
} catch(InsufficientFileAccessPermissionsException $e) {}
arround the call
Updated by Ricky Mathew over 6 years ago
Frans Saris wrote:
btw maybe is't easier to just catch the InsufficientFileAccessPermissionsException. So add a
[...]
arround the call
But that will throw the exception and stops the scheduler execution..But we just need to skip the denied files. that's all
Updated by Frans Saris over 6 years ago
No it does mainly the same as your solution
$fileIndexRecords = $this->getFileIndexRepository()->findInStorageWithIndexOutstanding($this->storage, $maximumFileCount);
foreach ($fileIndexRecords as $indexRecord) {
try {
$fileObject = $this->getResourceFactory()->getFileObject($indexRecord['uid'], $indexRecord);
$this->extractMetaData($fileObject);
} catch(InsufficientFileAccessPermissionsException $e) {}
}
It will still loop thought all items
Updated by Ricky Mathew over 6 years ago
Frans Saris wrote:
No it does mainly the same as your solution
[...]
It will still loop thought all items
Okay.Let me check whether it works for me.
Updated by Gerrit Code Review over 6 years ago
- Status changed from New to Under Review
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53834
Updated by Gerrit Code Review over 6 years ago
Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53834
Updated by Gerrit Code Review over 6 years ago
Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53834
Updated by Gerrit Code Review over 6 years ago
Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/54708
Updated by Ricky Mathew over 6 years ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset 65a053621da2ef4c8ede5556a5e090084d9ea03d.
Updated by Gerrit Code Review about 6 years ago
- Status changed from Resolved to Under Review
Patch set 1 for branch TYPO3_7-6 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/56540
Updated by Ricky Mathew about 6 years ago
- Status changed from Under Review to Resolved
Applied in changeset ee130aa2ff40859db4de6c85c38b355c1a89720e.
Updated by