Bug #80061
closed
FileStorageExtractionTask breaks if file not found
Added by Jonas Renggli about 7 years ago.
Updated over 5 years ago.
Description
Steps to Reproduce¶
- Ensure there's something to index. Add files, reset
last_indexed
field
UPDATE `sys_file`
SET `last_indexed` = 0;
- Delete file in filesystem
- Run scheduler Task "File Abstraction Layer: Extract metadata in storage" (
FileStorageExtractionTask
)
- Run scheduler Task "File Abstraction Layer: Update storage index" (
FileStorageIndexingTask
)
- Run scheduler Task "File Abstraction Layer: Extract metadata in storage" (
FileStorageExtractionTask
)
Actual Behavior¶
Expected Behavior¶
- Skip missing file and continue with next file
- Respect
missing
field in sys_file
(set by FileStorageIndexingTask
)
Files
- Description updated (diff)
I see several ways to fix this issue:
- Respect
missing
field in FileIndexRepository::findInStorageWithIndexOutstanding()
- Catch
Exception\FileDoesNotExistException
and continue in Indexer::runMetaDataExtraction()
- Optionally invoke
FileIndexRepository::markFileAsMissing()
in catch black
I'm currently testing the patches on a project with several thousand files. I'll push it to gerrit as soon as I'm sure it's working.
Unfortunately I don't have a project on TYPO3 v8 so far. Therefore I can't really test it. How is the workflow currently? Do I have to push the fix to master first and then backport it to v7?
Hi Jonas
Tried to reproduce the error. The only case where I could trigger the error was when the record was
also missing in sys_file_metadata - could you maybe check if this is the case?
- Status changed from New to Under Review
Hello,
we experienced some problems in a typo3 8 upgrade with occasionally missing pictures and suspect it might come from the "Update storage index" task.
We experimented some with the info given here and the first thing we noticed is that sys_file entries with "storage=0" will never get indexed, they rest with "last_indexed = 0". Is that the expected behaviour?
Second thing we noticed is that even with parrallel execution set to "no", it looks from the logs as if it does nevertheless (I'll attach a screenshot). Can this break stuff when it runs in parallel with the "Extract metadata in storage" task?
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
- Status changed from Resolved to Closed
Also available in: Atom
PDF