Feature #61542

Add two-digit ISO 639-1 keys to sys_language

Added by Benni Mack about 7 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Should have
Assignee:
Category:
Localization
Target version:
Start date:
2014-09-11
Due date:
% Done:

100%

Estimated time:
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

The language handling of records in TYPO3
is solely based on UIDs of the sys_language DB table,
but no reference to the real language used.

The ISO 639-1 defines the language identifiers
(ISO language code) completely (182 entries).

If the real language key was needed before in
TYPO3, one could install static_info_tables
which takes a field ("static_lang_isocode") in the
various places of the TYPO3 Core and fetches
the isocode via separate SQL-Queries.

The change introduces the ISO language
two-letter-keys natively in the core in order to
1. use less SQL queries in FE and BE if
static_info_tables was installed
2. remove hard coded dependencies of
EXT:static_info_tables
3. and always ensure that the core
includes the ISO code

Additionally one can now use the
$TSFE->sys_language_isocode all the time for
working with the ISO code instead of the
UID parameter, meaning that isocode can now be
set with a TypoScript parameter as well.

A Migration Wizard moves values put in the old
DB field (which is a UID reference to
static_languages) to the new DB field "iso2code".

The old field is deprecated to use without the
new field filled "iso2code".

In TYPO3 CMS 8 the static_lang_isocode field
will be removed within the CMS core.


Related issues

Related to TYPO3 Core - Bug #22119: Include ISO_3166-1 alpha2 in sys_languagesClosed2010-02-14

Actions
Related to TYPO3 Core - Task #68616: Upgrade Wizard without installed ext:static_info_tables failesClosedAnja Leichsenring2015-07-29

Actions
Related to TYPO3 Core - Task #69929: Unsorted list of ISO codesClosed2015-09-18

Actions
Related to TYPO3 Core - Bug #81922: Upgrade Wizards: Field 'language_isocode' do not existsClosed2017-07-19

Actions
#1

Updated by Stefan Neufeind about 7 years ago

Sounds great. Didn't we also even discuss going one step further to allow "arbitrary" languages to be defined, also with country-specific additions (like de_CH) or completely "invented" languages for other use-cases? Can this be integrated with this change maybe?

#2

Updated by Sigfried Arnold about 7 years ago

i suggested this some time ago, but i don't think this is a good idea

if you go for i19n, then ISO 639-1 is the right way to do - in most cases ...

but lots of "international" sites are in fact localized (l10n) - so ISO 639-1 would not be enough, this need to be combined with ISO 3166 Alpha 2 to be compilant to RFC 3066

For example "english" could be represented in multiple forms

en (english, plain)
en-US (english, targeted to audience in the USA)

this is important in countries with multiple languages for example switzerland - de-CH significantly differs from de-AT or de-DE

even more "problematic" is, that TYPO3 mixes things up and uses both:
db fields are sometimes named i18n_ - for example "pages.I18n_cfg" or "pages_language_overlay.I18n_diffsource" where other tables are namend l10n_ - for example "sys_category.l10n_parent"

if this is added, it should be done like static_info_tables.

the issue #1 Benjamin Mack states (takes more SQL queries) is not true, since this can be resolved with a simple join - it still would be 1 query (bit a bit more complex, but this should not be an impact to the overall performance)

i guess it is important to let people choose if the want to i18n or l10n their website

#3

Updated by Daniel Maier about 7 years ago

We should also consider that in ISO 639-1 there are no separate languages codes for the different versions of the Chinese language.

ISO 639-1 language code "zh" is both used for traditional Chinese (mainly used for example in Taiwan) and simplified Chinese (mainly used in mainland China). As far as I know they are more than just different dialects; the differences between them seem bigger than differences between e.g German dialects; they for example also differ in the used characters.

What do you think about that? How could we handle this?

#4

Updated by Benni Mack about 7 years ago

Sigfried Arnold wrote:

i suggested this some time ago, but i don't think this is a good idea

For example "english" could be represented in multiple forms

en (english, plain)
en-US (english, targeted to audience in the USA)

this is important in countries with multiple languages for example switzerland - de-CH significantly differs from de-AT or de-DE

I agree here. However, I would consider combining this info with a separate field. One is stating the language (ISO 639-1) and the other the more specific locale.

even more "problematic" is, that TYPO3 mixes things up and uses both:
db fields are sometimes named i18n_ - for example "pages.I18n_cfg" or "pages_language_overlay.I18n_diffsource" where other tables are namend l10n_ - for example "sys_category.l10n_parent"

Yes. Another thing to resolve.

if this is added, it should be done like static_info_tables.

What do you mean? Adding it in a separate table? I feel like having a string in the DB (finally) for the isocode is finally going in the right direction. I don't see a benefit having this info in the DB for now.

the issue #1 Benjamin Mack states (takes more SQL queries) is not true, since this can be resolved with a simple join - it still would be 1 query (bit a bit more complex, but this should not be an impact to the overall performance)

Yes. I agree, but this is not how it is currently in the core (as static_info_tables is not part of the core).

#5

Updated by Gerrit Code Review about 7 years ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#6

Updated by Gerrit Code Review almost 7 years ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#7

Updated by Gerrit Code Review almost 7 years ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#8

Updated by Gerrit Code Review almost 7 years ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#9

Updated by Gerrit Code Review almost 7 years ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#10

Updated by Gerrit Code Review almost 7 years ago

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#11

Updated by Gerrit Code Review almost 7 years ago

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#12

Updated by Martin Kutschker almost 7 years ago

As real world sites use IETF language tags this issue (patch) should have follow-ups to add a second field for ISO 3166-1 alpha-2 country codes. An even more perfect solution (a third patch) would add also a field for ISO 15924 script codes.

So IMHO this is definitely a step in the right direction, it must not be the last one.

http://en.wikipedia.org/wiki/IETF_language_tag
http://en.wikipedia.org/wiki/ISO_3166-1
http://en.wikipedia.org/wiki/ISO_15924

#13

Updated by Gerrit Code Review almost 7 years ago

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#14

Updated by Gerrit Code Review almost 7 years ago

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#15

Updated by Gerrit Code Review almost 7 years ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#16

Updated by Gerrit Code Review almost 7 years ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#17

Updated by Gerrit Code Review almost 7 years ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#18

Updated by Gerrit Code Review almost 7 years ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#19

Updated by Gerrit Code Review almost 7 years ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#20

Updated by Gerrit Code Review almost 7 years ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/32884

#21

Updated by Benni Mack almost 7 years ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100
#22

Updated by Gerrit Code Review almost 7 years ago

  • Status changed from Resolved to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at http://review.typo3.org/35022

#23

Updated by Markus Klein almost 7 years ago

  • Status changed from Under Review to Resolved
#24

Updated by Riccardo De Contardi almost 4 years ago

  • Status changed from Resolved to Closed
#25

Updated by Benni Mack over 1 year ago

  • Related to Bug #81922: Upgrade Wizards: Field 'language_isocode' do not exists added

Also available in: Atom PDF