Actions
Bug #96057
closedTypo3 does not handle UTF-characters from U+10000 and higher correctly
Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Backend User Interface
Target version:
Start date:
2021-11-23
Due date:
% Done:
0%
Estimated time:
TYPO3 Version:
10
PHP Version:
7.4
Tags:
Backend, UTF-8, Database
Complexity:
medium
Is Regression:
No
Sprint Focus:
Description
While checking out what's the problem with embedding Twitter Posts with emoticons in text, i found our that typo3 cannot handle UTF-characters correctly when saving them to the database:
- Saving to the database throws error:
1: These fields of record <nnn> in table "tt_content" have not been saved correctly: bodytext! The values might have changed due to type casting of the database.
- The database, table and column-charset are all UTF-8, General CI.
- The Typo3-UTF-8-Settings for database connection and Locale are UTF-8 as well.
- Typo3 cannot write to the database; when i insert the characters directly with SQL into the database, the frontend shows up correctly, while opening the record in the backend only shows up an "?" instead of the proper character; so Typo3 frontend can handle it, Typo3 backend not.
- Typo3 has no problem with UTF-8-Character up to U+FFFF. These can be saved and worked with in the backend as well. The trouble starts with code pages from U+1000 and above. There is the split where Typo3 backend can handle UTF-8 (or is this UFT-16 then?!? Don't know that part).
- Database analyzer is fine in this case - the columns have proper type (text oder mediumtext). Referring here to completely different cases with the same error, where sometimes this was an solution.
Please consider investigation into it and solve it.
Thank you,
Thomas
Updated by Georg Ringer almost 3 years ago
- Status changed from New to Needs Feedback
thanks for reporting the issue! Do you use a proper utf8mb4 connection? See https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.5/Feature-80398-Utf8mb4OnMysqlByDefaultForNewInstances.html for details
Updated by Thomas Sam Wittich almost 3 years ago
Hi Georg,
thank you for that hint. It solved the problem. Had no utf8-mb4 charset, but just utf8.
Updated by Wouter Wolters almost 3 years ago
- Status changed from Needs Feedback to Closed
Suggestion from Georg resolved the issue for the reporter. Closing as solved.
Actions