Bug #87610

Filelist Search - Add constraint to avoid searching of records marked as missing.

Added by Ricky Mathew 7 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Must have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
Start date:
2019-02-01
Due date:
% Done:

100%

TYPO3 Version:
9
PHP Version:
7.2
Tags:
Complexity:
easy
Is Regression:
Sprint Focus:

Description

The filelist search now fetches all records except deleted and hidden on a matching string.It should avoid searching for records marked as missing, since those fetched missing records leads to Folder not found or File not found Exceptions like in the following screenshot.

As a work around i just added the 'missing = 0' constraint at the end of findBySearchWordInMetaData() in TYPO3\CMS\Core\Resource\Index\FileIndexRepository class

$queryBuilder->andWhere(
                $queryBuilder->expr()->eq($sysFileTableName.'.missing', $queryBuilder->createNamedParameter(0, \PDO::PARAM_INT))
        );
        $result = $queryBuilder->execute();
        $fileRecords = [];

If this workaround feels sensible can someone please push it to the gerrit review?

file_not_found.png View (173 KB) Ricky Mathew, 2019-02-01 07:42


Related issues

Related to TYPO3 Core - Bug #87048: File search searches in forbidden folders => InsufficientFolderAccessPermissionsException New 2018-11-30

Associated revisions

Revision ee6b87b9 (diff)
Added by Helmut Hummel 3 months ago

[FEATURE] API to search for files including meta data based on folder

Instead of globally selecting all files with matching
meta data, we now base each search on the selected folder.

To do so, search is now completely based on the persisted
files in sys_file (and their corresponding meta data
in sys_file_metadata).

Additionally we properly evaluate search fields from TCA
so that we now search in all fields defined for sys_file
and sys_file_metadata table.

To achieve that, a new capability "CAPABILITY_HIERARCHICAL_IDENTIFIERS"
is introduced, which drivers can set, that build identifiers
that reflect the directory structure.
For such drivers, the search can be optimized by using
like queries on identifiers, instead of recursively traversing folders,
which can be an expensive operation especially for drivers
handling a remote file system.

Resolves: #87610
Releases: master, 9.5
Change-Id: Ia132465437827b2fdb56004eb73348ce4a05b336
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985
Reviewed-by: Markus Klein <>
Reviewed-by: Georg Ringer <>
Tested-by: TYPO3com <>
Tested-by: Markus Klein <>
Tested-by: Georg Ringer <>

Revision 3e787e77 (diff)
Added by Helmut Hummel 3 months ago

[FEATURE] API to search for files including meta data based on folder

Instead of globally selecting all files with matching
meta data, we now base each search on the selected folder.

To do so, search is now completely based on the persisted
files in sys_file (and their corresponding meta data
in sys_file_metadata).

Additionally we properly evaluate search fields from TCA
so that we now search in all fields defined for sys_file
and sys_file_metadata table.

To achieve that, a new capability "CAPABILITY_HIERARCHICAL_IDENTIFIERS"
is introduced, which drivers can set, that build identifiers
that reflect the directory structure.
For such drivers, the search can be optimized by using
like queries on identifiers, instead of recursively traversing folders,
which can be an expensive operation especially for drivers
handling a remote file system.

Resolves: #87610
Releases: master, 9.5
Change-Id: Ia132465437827b2fdb56004eb73348ce4a05b336
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/60685
Tested-by: TYPO3com <>
Tested-by: Georg Ringer <>
Reviewed-by: Georg Ringer <>

History

#1 Updated by Guido Schmechel 6 months ago

  • Assignee set to Helmut Hummel

https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985 Helmut works on a complete new approach. Maybe he can say something about the current state?

Basically, I'm with you and find that the problem should be solved even for 8.7.

#2 Updated by Gerrit Code Review 6 months ago

  • Status changed from New to Under Review

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#3 Updated by Helmut Hummel 6 months ago

  • Status changed from Under Review to New
  • Assignee deleted (Helmut Hummel)

I created a patch already

https://review.typo3.org/#/c/Packages/TYPO3.CMS/+/58985/

This tackles a whole lot more than just missing files, and already works on a production system in one of my projects like that.

#4 Updated by Gerrit Code Review 6 months ago

  • Status changed from New to Under Review

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#5 Updated by Gerrit Code Review 6 months ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#6 Updated by Gerrit Code Review 6 months ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#7 Updated by Gerrit Code Review 4 months ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#8 Updated by Gerrit Code Review 4 months ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#9 Updated by Gerrit Code Review 4 months ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#10 Updated by Gerrit Code Review 4 months ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#11 Updated by Gerrit Code Review 4 months ago

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#12 Updated by Gerrit Code Review 4 months ago

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#13 Updated by Gerrit Code Review 4 months ago

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#14 Updated by Gerrit Code Review 4 months ago

Patch set 22 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#15 Updated by Benni Mack 4 months ago

  • Target version changed from next-patchlevel to Candidate for patchlevel

#16 Updated by Gerrit Code Review 4 months ago

Patch set 23 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#17 Updated by Gerrit Code Review 4 months ago

Patch set 24 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#18 Updated by Gerrit Code Review 4 months ago

Patch set 25 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#19 Updated by Gerrit Code Review 4 months ago

Patch set 26 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#20 Updated by Gerrit Code Review 4 months ago

Patch set 27 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#21 Updated by Benni Mack 4 months ago

  • Target version changed from Candidate for patchlevel to next-patchlevel

#22 Updated by Gerrit Code Review 4 months ago

Patch set 28 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#23 Updated by Gerrit Code Review 3 months ago

Patch set 29 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#24 Updated by Gerrit Code Review 3 months ago

Patch set 29 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#25 Updated by Gerrit Code Review 3 months ago

Patch set 29 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#26 Updated by Gerrit Code Review 3 months ago

Patch set 30 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#27 Updated by Gerrit Code Review 3 months ago

Patch set 31 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#28 Updated by Gerrit Code Review 3 months ago

Patch set 32 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#29 Updated by Gerrit Code Review 3 months ago

Patch set 33 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#30 Updated by Gerrit Code Review 3 months ago

Patch set 34 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#31 Updated by Gerrit Code Review 3 months ago

Patch set 35 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#32 Updated by Gerrit Code Review 3 months ago

Patch set 35 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#33 Updated by Gerrit Code Review 3 months ago

Patch set 36 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#34 Updated by Gerrit Code Review 3 months ago

Patch set 37 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/58985

#35 Updated by Gerrit Code Review 3 months ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60685

#36 Updated by Helmut Hummel 3 months ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#37 Updated by Benni Mack 3 months ago

  • Status changed from Resolved to Closed

#38 Updated by Markus Klein about 2 months ago

  • Related to Bug #87048: File search searches in forbidden folders => InsufficientFolderAccessPermissionsException added

Also available in: Atom PDF