Bug #106364
openEpic #77562: Misbehaviors with datetime values and timezones
DatetimeElement stores format=time in seconds, while extbase interprets it as timezone-offset-mapped UTC timestamp, when php timezone is not UTC
0%
Description
When configuring a field TCA as:
'config' => [ 'type' => 'datetime', 'format' => 'time', ],
And the field is stores as int timestamp in the Database.
And the Timezone in PHP is not set to UTC.
Then the time is stored in the timezone configured in PHP and not in UTC.
This behavior differs from when using format datetime (default) in the TCA configuration, and leads to errors i.e. when outputting the time in fluid with f:format.date.
/sysext/backend/Classes/Form/Element/DatetimeElement.php
For a date in "datetime" format there is applied a "Fake UTC-0" normalization to the items value.
This normalization is not applied, when the field is in format "time". But should.
Edit: Extbase interprets format=time differently, see https://forge.typo3.org/issues/106364#note-5
When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.
Files
Updated by Garvin Hicking 3 days ago
Please check out:
https://review.typo3.org/c/Packages/TYPO3.CMS/+/86888
https://review.typo3.org/c/Packages/TYPO3.CMS/+/85262
Maybe that helps in moving things forward.
Updated by Garvin Hicking 3 days ago
- Related to Feature #105549: Replace fake UTC0 ISO8601 date strings by unqualified (local) ISO8601 added
- Related to Task #104309: Use ISO dates in FormEngine date handling added
Updated by Garvin Hicking 3 days ago
- Related to Task #103496: Use ISO date format by default added
Updated by Benjamin Franzke 3 days ago
· Edited
- Status changed from New to Needs Feedback
Hi,
thanks for you report.
I can not reproduce this with the following timezone settings, set via TYPO3 config or php.ini:
$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'UTC';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'America/New_York';
$GLOBALS['TYPO3_CONF_VARS']['SYS']['phpTimeZone'] = 'Europe/Berlin';
# also tested in php.ini
date.timezone = Europe/Berlin
date.timezone = America/New_York
Test times:
21:24
=21 * 60 * 60 + 24 * 60
=77040
seconds3:59
=3 * 60 * 60 + 59 * 60
=14340
seconds
Tested with inputdatetime_5
from https://github.com/TYPO3/typo3/blob/v13.4.7/typo3/sysext/styleguide/Configuration/TCA/tx_styleguide_elements_basic.php#L306-L313
In any timezone the value 21:24
is stored as 77040
in the database, and 3:59
as 14340
.
select inputdatetime_5 from tx_styleguide_elements_basic where uid in (16,17); +-----------------+ | inputdatetime_5 | +-----------------+ | 77040 | | 14340 | +-----------------+
Can you iterate on concrete values that you used and the timezone and value that you expected to see in database?
Maybe it's not the mapping to the database but the mapping from the database values that is broken (as you refer to errors when outputting the time)?
For a date in "datetime" format there is applied a "Fake UTC-0" normalization to the items value.
This normalization is not applied, when the field is in format "time". But should.
As Garvin already mentioned, we plan to streamline and simplify this confusing normalization, but currently it is actually intended to not normalize the format=time
dates to align with the DataHandler (which processes the data for persistence once submitted) does not substract the normalization for format=time
(as mentioned, planed to be streamlined: https://review.typo3.org/86888).
Updated by Mogens Fiebrandt 2 days ago
- File clipboard-202503140019-kp0sk.png clipboard-202503140019-kp0sk.png added
- File clipboard-202503140024-27b4z.png clipboard-202503140024-27b4z.png added
- File clipboard-202503140034-bxqhd.png clipboard-202503140034-bxqhd.png added
Hi. Thank you for your replies.
I tested again in following system:
- TYPO3 13.4.7
- PHP 8.3.4 (Not 8.2 as mentioned in the ticket description. Sorry for that)
- Timezone set via php.ini to "Europe/Berlin". Checked in phpinfo in the environment module in the Backend.
For testing purposes i used the fields starttime and endtime, configured like that:
'starttime' => [ 'exclude' => true, 'label' => 'LLL:EXT:frontend/Resources/Private/Language/locallang_ttc.xlf:starttime_formlabel', 'config' => [ 'type' => 'datetime', ], ], 'endtime' => [ 'exclude' => true, 'label' => 'LLL:EXT:frontend/Resources/Private/Language/locallang_ttc.xlf:endtime_formlabel', 'config' => [ 'type' => 'datetime', 'format' => 'time', ], ],
I cleared all caches and opened an entry in the list module.
Checked the fields in the Database. Both 0 (zero).
Now i entered the following to the fields (09:30):
And i would now expect, that both fields would have the same value in the database. And both values should be saved as the timestamp of 1970-01-01 08:30:00 UTC.
09:30 in Berlin = 08:30 in UTC = 8 * 60 * 60 + 30 * 60 = 30600
But the values in the fields differs as shown here:
When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.
Updated by Mogens Fiebrandt 2 days ago
I patched my Version like this and everything works fine:
Updated by Benjamin Franzke 1 day ago
- Subject changed from DatetimeElement stores time in wrong timezone, when php timezone is not UTC to DatetimeElement stores format=time in seconds, while extbase interprets it as timezone-offset-mapped UTC timestamp, when php timezone is not UTC
- Category changed from DataHandler aka TCEmain to Extbase
Thanks for you input. The values are actually as expected.
But I fully understand the frustration regarding exbtase's interpretation of these values.
And i would now expect, that both fields would have the same value in the database. And both values should be saved as the timestamp of 1970-01-01 08:30:00 UTC.
No, format=time
does not map time offsets, it just stores seconds (hour * 3600 + minutes * 60
, which happens to look like a unix timestamp on 1970) – but these seconds do not relate to any date (also not 1970) by definition, therefore they are not timezone shifted (like you'd expect given the comparison with format=datetime).
This has been designed (by intent or by accident, i don't know) many years ago, see old documentation
https://docs.typo3.org/m/typo3/reference-tca/10.4/en-us/ColumnsConfig/Type/inputDateTime.html#eval
https://docs.typo3.org/m/typo3/reference-tca/6.2/en-us/Reference/Columns/Input/Index.html#eval
(sections about time/timesec alway refer to a fixed database value, while date/datetime incorporate timezone-specific values)
and I even checked out older instances (like v10) do behave like this as well.
When i now reopen the entity in the list module both fields still show 9:30. But when loading the fields as DateTime object via an Extbase model, then the starttime is 8:30 UTC and the endtime is 9:30 UTC.
We need to fix extbase to properly map integer-based format=time(sec) fields to and from the database.
This is of course "breaking", since application that worked around the broken state would be influenced by any fix that changes interpretation of database values.
Maybe we can add a feature toggle…
I'm going to update the issue subject to reflect that we've got an extbase issue here.
Updated by Gerrit Code Review 1 day ago
- Status changed from Needs Feedback to Under Review
Patch set 1 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/88591
Updated by Mogens Fiebrandt about 18 hours ago
Ok. Thank you for clarification.