Project

General

Profile

Actions

Bug #106364

open

Epic #77562: Misbehaviors with datetime values and timezones

DatetimeElement stores format=time in seconds, while extbase interprets it as timezone-offset-mapped UTC timestamp, when php timezone is not UTC

Added by Mogens Fiebrandt 12 days ago. Updated 6 days ago.

Status:
Under Review
Priority:
Should have
Assignee:
-
Category:
Extbase
Target version:
Start date:
2025-03-12
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
13
PHP Version:
8.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

When configuring a field TCA as:

'config' => [
    'type' => 'datetime',
    'format' => 'time',
],

And the field is stores as int timestamp in the Database.
And the Timezone in PHP is not set to UTC.

Then the time is stored in the timezone configured in PHP and not in UTC.

This behavior differs from when using format datetime (default) in the TCA configuration, and leads to errors i.e. when outputting the time in fluid with f:format.date.

/sysext/backend/Classes/Form/Element/DatetimeElement.php
For a date in "datetime" format there is applied a "Fake UTC-0" normalization to the items value.
This normalization is not applied, when the field is in format "time". But should.

Edit: Extbase interprets format=time differently, see https://forge.typo3.org/issues/106364#note-5

When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.


Files

clipboard-202503140019-kp0sk.png (2.52 KB) clipboard-202503140019-kp0sk.png Mogens Fiebrandt, 2025-03-13 23:19
clipboard-202503140024-27b4z.png (9.47 KB) clipboard-202503140024-27b4z.png Mogens Fiebrandt, 2025-03-13 23:24
clipboard-202503140034-bxqhd.png (9.58 KB) clipboard-202503140034-bxqhd.png Mogens Fiebrandt, 2025-03-13 23:34
clipboard-202503140046-p3otj.png (51 KB) clipboard-202503140046-p3otj.png Mogens Fiebrandt, 2025-03-13 23:46

Related issues 3 (1 open2 closed)

Related to TYPO3 Core - Feature #105549: Replace fake UTC0 ISO8601 date strings by unqualified (local) ISO8601Under Review2024-11-06

Actions
Related to TYPO3 Core - Task #104309: Use ISO dates in FormEngine date handlingClosed2024-07-04

Actions
Related to TYPO3 Core - Task #103496: Use ISO date format by defaultClosedMathias Brodala2024-03-28

Actions
Actions #2

Updated by Garvin Hicking 12 days ago

  • Related to Feature #105549: Replace fake UTC0 ISO8601 date strings by unqualified (local) ISO8601 added
  • Related to Task #104309: Use ISO dates in FormEngine date handling added
Actions #3

Updated by Garvin Hicking 12 days ago

  • Related to Task #103496: Use ISO date format by default added
Actions #4

Updated by Benjamin Franzke 11 days ago · Edited

  • Status changed from New to Needs Feedback

Hi,

thanks for you report.

I can not reproduce this with the following timezone settings, set via TYPO3 config or php.ini:

$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'UTC';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'America/New_York';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'Europe/Berlin';

# also tested in php.ini
date.timezone = Europe/Berlin
date.timezone = America/New_York

Test times:

  • 21:24 = 21 * 60 * 60 + 24 * 60 = 77040 seconds
  • 3:59 = 3 * 60 * 60 + 59 * 60 = 14340 seconds

Tested with inputdatetime_5 from https://github.com/TYPO3/typo3/blob/v13.4.7/typo3/sysext/styleguide/Configuration/TCA/tx_styleguide_elements_basic.php#L306-L313

In any timezone the value 21:24 is stored as 77040 in the database, and 3:59 as 14340.

select inputdatetime_5 from tx_styleguide_elements_basic where uid in (16,17);
+-----------------+
| inputdatetime_5 |
+-----------------+
|           77040 |
|           14340 |
+-----------------+

Can you iterate on concrete values that you used and the timezone and value that you expected to see in database?

Maybe it's not the mapping to the database but the mapping from the database values that is broken (as you refer to errors when outputting the time)?

For a date in "datetime" format there is applied a "Fake UTC-0" normalization to the items value.
This normalization is not applied, when the field is in format "time". But should.

As Garvin already mentioned, we plan to streamline and simplify this confusing normalization, but currently it is actually intended to not normalize the format=time dates to align with the DataHandler (which processes the data for persistence once submitted) does not substract the normalization for format=time (as mentioned, planed to be streamlined: https://review.typo3.org/86888).

Updated by Mogens Fiebrandt 10 days ago

Hi. Thank you for your replies.

I tested again in following system:
- TYPO3 13.4.7
- PHP 8.3.4 (Not 8.2 as mentioned in the ticket description. Sorry for that)
- Timezone set via php.ini to "Europe/Berlin". Checked in phpinfo in the environment module in the Backend.

For testing purposes i used the fields starttime and endtime, configured like that:

'starttime' => [
    'exclude' => true,
    'label' => 'LLL:EXT:frontend/Resources/Private/Language/locallang_ttc.xlf:starttime_formlabel',
    'config' => [
        'type' => 'datetime',
    ],
],
'endtime' => [
    'exclude' => true,
    'label' => 'LLL:EXT:frontend/Resources/Private/Language/locallang_ttc.xlf:endtime_formlabel',
    'config' => [
        'type' => 'datetime',
        'format' => 'time',
    ],
],

I cleared all caches and opened an entry in the list module.
Checked the fields in the Database. Both 0 (zero).
Now i entered the following to the fields (09:30):

And i would now expect, that both fields would have the same value in the database. And both values should be saved as the timestamp of 1970-01-01 08:30:00 UTC.

09:30 in Berlin = 08:30 in UTC = 8 * 60 * 60 + 30 * 60 = 30600

But the values in the fields differs as shown here:

When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.

Actions #6

Updated by Mogens Fiebrandt 10 days ago

I patched my Version like this and everything works fine:

Actions #7

Updated by Benjamin Franzke 10 days ago

  • Subject changed from DatetimeElement stores time in wrong timezone, when php timezone is not UTC to DatetimeElement stores format=time in seconds, while extbase interprets it as timezone-offset-mapped UTC timestamp, when php timezone is not UTC
  • Category changed from DataHandler aka TCEmain to Extbase

Thanks for you input. The values are actually as expected.
But I fully understand the frustration regarding exbtase's interpretation of these values.

And i would now expect, that both fields would have the same value in the database. And both values should be saved as the timestamp of 1970-01-01 08:30:00 UTC.

No, format=time does not map time offsets, it just stores seconds (hour * 3600 + minutes * 60, which happens to look like a unix timestamp on 1970) – but these seconds do not relate to any date (also not 1970) by definition, therefore they are not timezone shifted (like you'd expect given the comparison with format=datetime).

This has been designed (by intent or by accident, i don't know) many years ago, see old documentation
https://docs.typo3.org/m/typo3/reference-tca/10.4/en-us/ColumnsConfig/Type/inputDateTime.html#eval
https://docs.typo3.org/m/typo3/reference-tca/6.2/en-us/Reference/Columns/Input/Index.html#eval
(sections about time/timesec alway refer to a fixed database value, while date/datetime incorporate timezone-specific values)
and I even checked out older instances (like v10) do behave like this as well.

When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.

We need to fix extbase to properly map integer-based format=time(sec) fields to and from the database.
This is of course "breaking", since application that worked around the broken state would be influenced by any fix that changes interpretation of database values.
Maybe we can add a feature toggle…

I'm going to update the issue subject to reflect that we've got an extbase issue here.

Actions #8

Updated by Benjamin Franzke 10 days ago

  • Description updated (diff)
Actions #9

Updated by Benjamin Franzke 10 days ago

  • Parent task set to #77562
Actions #10

Updated by Gerrit Code Review 10 days ago

  • Status changed from Needs Feedback to Under Review

Patch set 1 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/88591

Actions #11

Updated by Mogens Fiebrandt 9 days ago

Ok. Thank you for clarification.

Actions #12

Updated by Gerrit Code Review 6 days ago

Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/88591

Actions

Also available in: Atom PDF