Project

General

Profile

Actions

Bug #95122

open

TYPO3 falesly indexes file CHANGE time (ctime) into sys_file.creation_date.

Added by Andrea Herzog-Kienast over 3 years ago. Updated 2 months ago.

Status:
Needs Feedback
Priority:
Should have
Assignee:
-
Category:
File Abstraction Layer (FAL)
Target version:
-
Start date:
2021-09-06
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
12
PHP Version:
7.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Updated description

TYPO3 indexes a file's mtime into sys_file.modification_date, which is correct. The mtime = modification time of file content.
TYPO3 indexes a file's ctime into sys_file.creation_date, which is wrong . The ctime = change time of the file node (including name, access modifiers, and content).

ctime >= mtime, basically always.

The "c" stands for "change", not "creation".

TYPO3 also labels this sys_file.creation_date field in the backend as "Created At", which is wrong, as it contains semantically the filesystem change time of the file, as noted above.

Original ticket Description

Hey guys,

It seems that create date and Last modified date in FAL is reversed (info and ref view.) Checked it in TYPO3 7/8/10.

Cheers,

Andrea


Files

last-mod-1.PNG (7.14 KB) last-mod-1.PNG Andrea Herzog-Kienast, 2021-09-06 14:34
Bildschirmfoto 2021-09-06 um 16.32.53.png (43.2 KB) Bildschirmfoto 2021-09-06 um 16.32.53.png In List View its correct Christoph Schwob, 2021-09-06 14:41
Actions #1

Updated by Christoph Schwob over 3 years ago

Also occurs in current master.

But there it's only on ElementInformation and not on FileList's default view.

Actions #2

Updated by Andreas Kienast over 3 years ago

This is a Linux specialty. Linux (at least the commonly used ext4 file system) doesn't store the actual creation date in a file's metadata in the file system, but the last inode change time, becoming the actual result of filectime(). With renaming a file, the inode changes as well, thus the "creation date" gets updated in FAL storage as well.

I don't think there's much we can do here but not relying on the dates the file system reports to TYPO3, which requires major rewrites of FAL, AFAICS.

Actions #3

Updated by Andrea Herzog-Kienast over 3 years ago

Jepp, but strange noboby noticed. As I can see: Uploaded file in FAL gets a timestamp, however upload date. If you change the filename LINUX "thinks" - a new file. Changing Metadata doesn't have any influences to date times.

Actions #4

Updated by Urs Braem over 2 years ago

  • TYPO3 Version changed from 10 to 11

I can confirm with TYPO3 11 on centOS.
In the databases I looked at,


SELECT name, FROM_UNIXTIME(creation_date) as created, FROM_UNIXTIME(modification_date) as modified
    FROM sys_file
        WHERE creation_date < modification_date

returns zero or next to it. That means that creation_date is always newer or equal to modification_date (which is the wrong way round, creation_date should stay fixed, while modification_date should be raised sometimes.

Actions #5

Updated by S P 12 months ago

  • TYPO3 Version changed from 11 to 12

The core file Indexer has this:

  $mappingInfo = [
            // 'driverKey' => 'fileProperty' Key is from the driver, value is for the property in the file
            'size' => 'size',
            'atime' => null,
            'mtime' => 'modification_date',
            'ctime' => 'creation_date',
            'mimetype' => 'mime_type',
        ];

The `ctime` field of a file is the CHANGE time, it is NOT the creation time. The TYPO3 core got this totally wrong, since ever. In most file systems an actual creation time of a file is not known.

ctime (filesystem node change time) gets updates whenever anything regarding the file changes (like name, access modifiers, content, ...).
But mtime (content modification time) gets only updates when the content changes.

So, usually ctime >= mtime.

Actions #6

Updated by S P 12 months ago

  • Subject changed from FAL Last modified / create date to TYPO3 falesly indexes file CHANGE time (ctime) into sys_file.creation_date.
Actions #7

Updated by S P 12 months ago

  • Description updated (diff)
Actions #8

Updated by S P 12 months ago

  • Description updated (diff)
Actions #9

Updated by S P 12 months ago

  • Description updated (diff)
Actions #10

Updated by Benni Mack 2 months ago

  • Status changed from New to Needs Feedback

This is what we got: https://stackoverflow.com/questions/4401320/php-how-can-i-get-file-creation-date#:~:text=Use%20filectime.,creation%20time%20of%20the%20file.

So since we cannot guarantee this information, I suggest to drop it instead of keeping wrong information around?

Actions #11

Updated by S P 2 months ago

The vast majority of TYPO3 installations runs on POSIX-based environments. So it is safe to assume that ctime is basically "wrong" (considering the field name) on all TYPO3 setups out there.

I suggest dropping all of these fields ({a,c,m}time) from sys_file. There's no reason to have them in the DB, IMHO (they can easily be confused with crdate and tstamp from sys_file_metadata or sys_file, which hold this kind of information for the time info of the SQL row instead of the physical file they represent).

A getter in the models for these file properties could fetch them directly from the physical file. For a ctime getter add a doc block describing that on Linux it is the change time, on Windows it is the creation time.

Actions #12

Updated by Garvin Hicking 2 months ago

We also need to think about remote file storage synching (ni direct filesystem getter possible) plus using this info to get sorted output or date grouping without needing to touch every possible physical file. And intermediate DB cache access for this is very useful.

Actions #13

Updated by S P 2 months ago

Garvin Hicking wrote in #note-12:

using this info to get sorted output or date grouping without needing to touch every possible physical file. And intermediate DB cache access for this is very useful.

True, but only if the stored value is actually reliable (which it is not and can never be, because on POSIX-based system the physcial file creation time is never available).

The only to-a-degree-reliable value is the mtime, because this is available on all file systems in some kind of way (but can include slightly different information, like only timestamps of file content changes or also timestamps of filesystem metadata changes). The FAL API should not provide technical unreliable information that neither developers nor editors can influence or verify in any way.

Actions

Also available in: Atom PDF