Bug #79249

Epic #77562: Misbehaviors with datetime values and timezones

TCA fields with eval types "time" or "timesec" are incorrectly stored in the database

Added by Dominique Kreemers almost 3 years ago. Updated about 1 year ago.

Status:
Closed
Priority:
Must have
Assignee:
Category:
DataHandler aka TCEmain
Target version:
Start date:
2017-01-10
Due date:
% Done:

100%

TYPO3 Version:
8
PHP Version:
7.0
Tags:
Complexity:
Is Regression:
Yes
Sprint Focus:
Stabilization Sprint

Description

As mentioned in the title, TCA fields with eval value "time" or "timesec" are stored incorrectly to the database in TYPO3 8.5.1.
In my setup i will get the value "1484084700" in the database if i select 23:45 for the give field.

The documentation suggests that the value will just hold the seconds from 1970-01-01 00:00:00 to the selected time:

https://docs.typo3.org/typo3cms/TCAReference/Reference/Columns/Input/Index.html#eval

The display will be like "23:45" while the database will be "85500".

In TYPO3 8.4.1 this works as expected. It seems like the changes from #77702 have changed this behaviour.

Following code block is responsible for this issue:
(/typo3/sysext/core/Classes/DataHandling/DataHandler.php @Line #2693)

case 'time':
case 'timesec':
case 'date':
case 'datetime':
    // a hyphen as first character indicates a negative timestamp
    if ((strpos($value, '-') === false && strpos($value, ':') === false) || strpos($value, '-') === 0) {
        $value = (int)$value;
    } else {
        // ISO 8601 dates
        $dateTime = new \DateTime($value);
        // The returned timestamp is always UTC
        $value = $dateTime->getTimestamp();
    }
    // $value is a UTC timestamp here.
    // The value will be stored in the server’s local timezone, but treated as UTC, so we brute force
    // subtract the offset here. The offset is subtracted instead of added because the value is stored
    // in the timezone, but interpreted as UTC, so if we switched the server to UTC, the correct
    // value would be returned.
    if ($value !== 0 && !$this->dontProcessTransformations) {
        $value -= date('Z', $value);
    }
    break;


Related issues

Related to TYPO3 Core - Task #77702: Use moment.js and ISO-8601 dates in the backend Closed 2016-08-30
Related to TYPO3 Core - Bug #79557: Date fields DataMapper handling problem New 2017-01-30
Duplicated by TYPO3 Core - Bug #79975: TCA -> input -> eval: 'time' not relative timestamp Closed 2017-02-23
Duplicated by TYPO3 Core - Bug #79957: TCA fields with eval types "time" or "timesec" decrease on every save Closed 2017-02-22
Duplicated by TYPO3 Core - Bug #80679: Input field eval datetime in FlexForm converts to wrong timestamp Closed 2017-04-04

Associated revisions

Revision edd26bf2 (diff)
Added by Markus Klein over 2 years ago

[BUGFIX] Make date/time fields working again

This patch solves an issue with time-fields and a couple of
inconsistencies for time(sec), date and datetime fields in BE:

- BackendUtility::time() is superfluous as gmdate() can be used
- Ensure stored data is identical to TYPO3 7
- Only use ISO dates in UTC timezone to communicate with JS
(time fields use 1970-01-01 as date)
- Only write ISO dates in UTC back to hidden FormEngine fields
- Do that for all date/time fields to simplify JS code
- Take care of necessary timezone conversion for date(time) fields
- Correctly handle manual input
- Fix record titles if a date field is the label field
- Fix wrong Acceptance tests
- Fix wrong Unit tests
- Remove superfluous data-date-offset attribute for DateTimePicker
- Streamline FormEngineValidation date handling code
- Refactor DateTimePicker: fix code duplication, work in UTC only

Resolves: #79249
Releases: master, 8.7
Change-Id: I12ef6b6f59f0843182e093f8f274989161820894
Reviewed-on: https://review.typo3.org/51242
Tested-by: TYPO3com <>
Reviewed-by: Johannes Kasberger <>
Tested-by: Johannes Kasberger <>
Reviewed-by: Christian Kuhn <>
Tested-by: Christian Kuhn <>

Revision 1022875b (diff)
Added by Markus Klein over 2 years ago

[BUGFIX] Make date/time fields working again

This patch solves an issue with time-fields and a couple of
inconsistencies for time(sec), date and datetime fields in BE:

- BackendUtility::time() is superfluous as gmdate() can be used
- Ensure stored data is identical to TYPO3 7
- Only use ISO dates in UTC timezone to communicate with JS
(time fields use 1970-01-01 as date)
- Only write ISO dates in UTC back to hidden FormEngine fields
- Do that for all date/time fields to simplify JS code
- Take care of necessary timezone conversion for date(time) fields
- Correctly handle manual input
- Fix record titles if a date field is the label field
- Fix wrong Acceptance tests
- Fix wrong Unit tests
- Remove superfluous data-date-offset attribute for DateTimePicker
- Streamline FormEngineValidation date handling code
- Refactor DateTimePicker: fix code duplication, work in UTC only

Resolves: #79249
Releases: master, 8.7
Change-Id: I12ef6b6f59f0843182e093f8f274989161820894
Reviewed-on: https://review.typo3.org/53166
Tested-by: TYPO3com <>
Reviewed-by: Christian Kuhn <>
Tested-by: Christian Kuhn <>

History

#1 Updated by Gerrit Code Review almost 3 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#2 Updated by Gerrit Code Review almost 3 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#3 Updated by Dominique Kreemers almost 3 years ago

  • Is Regression changed from No to Yes
  • Sprint Focus set to Stabilization Sprint

#4 Updated by Gerrit Code Review almost 3 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#5 Updated by Mona Muzaffar over 2 years ago

  • Related to Epic #80852: Datetime handling in backend added

#6 Updated by Markus Klein over 2 years ago

  • Parent task set to #81489

#7 Updated by Markus Klein over 2 years ago

  • Parent task changed from #81489 to #77562

#8 Updated by Markus Klein over 2 years ago

  • Duplicated by Bug #79975: TCA -> input -> eval: 'time' not relative timestamp added

#9 Updated by Riccardo De Contardi over 2 years ago

  • Related to deleted (Epic #80852: Datetime handling in backend)

#10 Updated by Gerrit Code Review over 2 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#11 Updated by Markus Klein over 2 years ago

  • Duplicated by Bug #79957: TCA fields with eval types "time" or "timesec" decrease on every save added

#12 Updated by Markus Klein over 2 years ago

  • Duplicated by Bug #80679: Input field eval datetime in FlexForm converts to wrong timestamp added

#13 Updated by Markus Klein over 2 years ago

  • Category changed from Backend User Interface to DataHandler aka TCEmain
  • Assignee set to Markus Klein
  • Target version set to next-patchlevel

#14 Updated by Markus Klein over 2 years ago

This can be tested easily with the extension provided in the parent ticket.

#15 Updated by Gerrit Code Review over 2 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#16 Updated by Gerrit Code Review over 2 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#17 Updated by Gerrit Code Review over 2 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#18 Updated by Gerrit Code Review over 2 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#19 Updated by Gerrit Code Review over 2 years ago

Patch set 1 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53166

#20 Updated by Gerrit Code Review over 2 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#21 Updated by Gerrit Code Review over 2 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#22 Updated by Gerrit Code Review over 2 years ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#23 Updated by Gerrit Code Review over 2 years ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#24 Updated by Gerrit Code Review over 2 years ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#25 Updated by Gerrit Code Review over 2 years ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#26 Updated by Gerrit Code Review over 2 years ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/51242

#27 Updated by Gerrit Code Review over 2 years ago

Patch set 2 for branch TYPO3_8-7 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/53166

#28 Updated by Markus Klein over 2 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#29 Updated by Tizian Schmidlin over 2 years ago

Will this be backported to 7.6? Maybe for the next patch level?

#30 Updated by Markus Klein over 2 years ago

7.6 actually works, no need to backport anything.

We adjusted v8 to work like v7 again.

#31 Updated by Markus Klein about 2 years ago

  • Related to Bug #79557: Date fields DataMapper handling problem added

#32 Updated by Benni Mack about 1 year ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF