Project

General

Profile

Actions

Feature #60221

closed

Read FAL resources in chunks

Added by Bernhard Kraft almost 10 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Could have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
-
Start date:
2014-07-10
Due date:
% Done:

0%

Estimated time:
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

Usually file layers like the Linux file system support a basic common set of IO operations.
If you ever have to implement a file system in linux or you are going to implement a java class implementing the file interface you will notice that you will have to implement the following methods:

open()
close()
read()
seek()
tell()
write()

... of course there are a few others like "size()" and for handling access control.

currently FAL only supports open/close as making an instance of the "File" object could get seen as a call to open().

This issue suggests to add additional methods to File() which allows those basic operations.

For the LocalDriver a fopen()/filehandle in the constructor and wrappers for fread/fwrite/fseek/ftell would be sufficient.

Driver for S3 Amazon could for example manage internal position pointers and use the "Range" header to retrieve requested chunks from the storage (http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectGET.html)


Related issues 1 (0 open1 closed)

Related to TYPO3 Core - Feature #66466: Tell FAL driver that we will fetch 40 files nowClosed2015-04-17

Actions
Actions #1

Updated by Markus Klein almost 10 years ago

Does this method from the File class fulfill your needs?

/**
 * Returns a path to a local version of this file to process it locally (e.g. with some system tool).
 * If the file is normally located on a remote storages, this creates a local copy.
 * If the file is already on the local system, this only makes a new copy if $writable is set to TRUE.
 *
 * @param boolean $writable Set this to FALSE if you only want to do read operations on the file.
 *
 * @throws \RuntimeException
 * @return string
 */
public function getForLocalProcessing($writable = TRUE) {
    if ($this->deleted) {
        throw new \RuntimeException('File has been deleted.', 1329821486);
    }
    return $this->getStorage()->getFileForLocalProcessing($this, $writable);
}

If you set $writeable = FALSE you will get the absolute path to the file (if the LocalDriver is used).
You can do with this path whatever you like.

Actions #2

Updated by Bernhard Kraft almost 10 years ago

This issue is marked as feature and could have. So its rather an idea/suggestion than a pressing requirement!

I don't like to handle FileObjects using different drivers specifically. Usually I try to write code which works on any driver.

The reason for this issue is just a proposal. For example when working with large files (CSV, etc.) this could be useful.

Actions #3

Updated by Mathias Schreiber over 9 years ago

  • Target version changed from 7.0 to 7.1 (Cleanup)
Actions #4

Updated by Benni Mack almost 9 years ago

  • Target version changed from 7.1 (Cleanup) to 7.4 (Backend)
Actions #5

Updated by Susanne Moog over 8 years ago

  • Target version changed from 7.4 (Backend) to 7.5
Actions #6

Updated by Benni Mack over 8 years ago

  • Target version changed from 7.5 to 8 LTS
Actions #7

Updated by Riccardo De Contardi about 7 years ago

  • Target version changed from 8 LTS to 9.0
Actions #8

Updated by Tymoteusz Motylewski almost 7 years ago

  • Related to Feature #66466: Tell FAL driver that we will fetch 40 files now added
Actions #9

Updated by Susanne Moog over 6 years ago

  • Target version deleted (9.0)
Actions #10

Updated by Benni Mack about 4 years ago

  • Status changed from New to Needs Feedback

We have ResourceStorage->streamFile() since TYPO3 v9. I think the only valid solution would be to integrate something like FlySystem as a lowlevel abstraction layer to solve these kind of issues. What do you think? Did some new ideas come up in the past years?

Actions #11

Updated by Christian Kuhn over 2 years ago

  • Status changed from Needs Feedback to Closed

Closing. This has seems to be solved with fal layers meanwhile. Please re-open or create active action points using new issues if this is still a topic.

Actions

Also available in: Atom PDF