Feature #60221

Read FAL resources in chunks

Added by Bernhard Kraft over 5 years ago. Updated 11 days ago.

Status:
Needs Feedback
Priority:
Could have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
-
Start date:
2014-07-10
Due date:
% Done:

0%

PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Usually file layers like the Linux file system support a basic common set of IO operations.
If you ever have to implement a file system in linux or you are going to implement a java class implementing the file interface you will notice that you will have to implement the following methods:

open()
close()
read()
seek()
tell()
write()

... of course there are a few others like "size()" and for handling access control.

currently FAL only supports open/close as making an instance of the "File" object could get seen as a call to open().

This issue suggests to add additional methods to File() which allows those basic operations.

For the LocalDriver a fopen()/filehandle in the constructor and wrappers for fread/fwrite/fseek/ftell would be sufficient.

Driver for S3 Amazon could for example manage internal position pointers and use the "Range" header to retrieve requested chunks from the storage (http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html)


Related issues

Related to TYPO3 Core - Feature #66466: Tell FAL driver that we will fetch 40 files now Closed 2015-04-17

History

#1 Updated by Markus Klein over 5 years ago

Does this method from the File class fulfill your needs?

/**
 * Returns a path to a local version of this file to process it locally (e.g. with some system tool).
 * If the file is normally located on a remote storages, this creates a local copy.
 * If the file is already on the local system, this only makes a new copy if $writable is set to TRUE.
 *
 * @param boolean $writable Set this to FALSE if you only want to do read operations on the file.
 *
 * @throws \RuntimeException
 * @return string
 */
public function getForLocalProcessing($writable = TRUE) {
    if ($this->deleted) {
        throw new \RuntimeException('File has been deleted.', 1329821486);
    }
    return $this->getStorage()->getFileForLocalProcessing($this, $writable);
}

If you set $writeable = FALSE you will get the absolute path to the file (if the LocalDriver is used).
You can do with this path whatever you like.

#2 Updated by Bernhard Kraft over 5 years ago

This issue is marked as feature and could have. So its rather an idea/suggestion than a pressing requirement!

I don't like to handle FileObjects using different drivers specifically. Usually I try to write code which works on any driver.

The reason for this issue is just a proposal. For example when working with large files (CSV, etc.) this could be useful.

#3 Updated by Mathias Schreiber over 5 years ago

  • Target version changed from 7.0 to 7.1 (Cleanup)

#4 Updated by Benni Mack almost 5 years ago

  • Target version changed from 7.1 (Cleanup) to 7.4 (Backend)

#5 Updated by Susanne Moog over 4 years ago

  • Target version changed from 7.4 (Backend) to 7.5

#6 Updated by Benni Mack over 4 years ago

  • Target version changed from 7.5 to 8 LTS

#7 Updated by Riccardo De Contardi about 3 years ago

  • Target version changed from 8 LTS to 9.0

#8 Updated by Tymoteusz Motylewski over 2 years ago

  • Related to Feature #66466: Tell FAL driver that we will fetch 40 files now added

#9 Updated by Susanne Moog about 2 years ago

  • Target version deleted (9.0)

#10 Updated by Benni Mack 11 days ago

  • Status changed from New to Needs Feedback

We have ResourceStorage->streamFile() since TYPO3 v9. I think the only valid solution would be to integrate something like FlySystem as a lowlevel abstraction layer to solve these kind of issues. What do you think? Did some new ideas come up in the past years?

Also available in: Atom PDF