Project

General

Profile

Actions

Feature #93334

open

New syntax for labels

Added by Benni Mack over 3 years ago. Updated 3 months ago.

Status:
New
Priority:
Should have
Assignee:
-
Category:
Localization
Target version:
-
Start date:
2021-01-21
Due date:
% Done:

0%

Estimated time:
PHP Version:
Tags:
Complexity:
Sprint Focus:

Description

The syntax for "LLL:" has several drawbacks

  • It references a path (also uses EXT:) which makes it hard to override
  • Changing label files is more error prone
  • The paths are unnecessary longer (EXT:my_ext/Resources/Private/Language/locallang_.xlf) and it is confusing for newcomers to understand both the folder structure and the naming (what does "locallang" mean? or "LLL"?).

For this reason, I propose a more "speakable" approach, since we talk about "labels" in this case (not to be confused with "translations" in content).

What I want to address
  • Making the syntax more consistent - find better solutions for "LLL:"
  • Decouple the label identifier from the file name when addressing the label (making transitions from label names easier)
  • Decouple the label identifier from an extension name + path
  • Shorten the identifier massively for its usage
  • Make it easier for people who CREATE such files to address.

A new syntax proposal (totally up for discussion)

"label://vendor.package.namespace/resource-name/identifier-in-resource"

Examples

Previously: LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:general
New: label://typo3.core/form.tabs/general

Previously: LLL:EXT:core/Resources/Private/Language/locallang_tca.xlf:pages.tabs.metadata
New: label://typo3.core/tca/pages.tabs.metadata

For a TER extension: LLL:EXT:news/Resources/Private/Language/locallang.xlf:news.title
New: label://georgringer.news/default/news.title

For other extensions: LLL:EXT:my_ext/Resources/Private/Language/locallang_modules.xlf:my-module
New: label://a-random-namespace/modules/my-module

Usages

Fluid:

<f:translate identifier="label://a-random-namespace/modules/my-module">Fallback string</f:translate>

TypoScript:

page.10 = TEXT
page.10.lang = label://my.site/resource/site-name

PHP:

$langService->label('label://my.site/resource/site-name')

So, under the hood this is what happens:
  • A "LabelProviderService" contains all available files registered within the system
  • The existing "languageService" gets extended to translate the new "label://" syntax into the actual resource + identifier
  • The existing "languageService->label()" is added to make the syntax more understandable, and falls back if no "LLL:" or "label://" syntax is used
  • For existing extensions, existing labels can be provided automatically by checking for Resources/Private/Language/locallang.xlf and registering with the composer-name syntax automatically, making any registration for language file very easy and consistent via a ServiceProvider during build-time.

Manual registration would be possible as well

LabelProvider->registerResource($pathToFile = '...', $namespace = 'my.site', $resourceName = 'default');

Backwards-compatibility

  • We still allow "LLL:" for a LONG time (just like we keep GeneralUtility::makeInstance() until a long time)

Still up for discussion

  • Overriding labels from extensions locally
  • Dealing with translation server files
Actions #1

Updated by Benni Mack over 3 years ago

  • Description updated (diff)
Actions #2

Updated by Benni Mack over 3 years ago

  • Description updated (diff)
Actions #3

Updated by Xavier Perseguers over 3 years ago

Beware that locallang_ prefix is purely a convention (except for filename locallang.xlf in the context of Fluid).

I've sticked myself with this convention but others may not, and this is possibly breaking if you then choose to drop support for the LLL:EXT: syntax altogether and this could be really really breaking if people are handling their translation on a custom server where it may be quite hard to rename translations of a file they could have named Resources/Private/Language/whatever.xlf.

Actions #4

Updated by Xavier Perseguers over 3 years ago

I've a hard time figuring out how this example would work:

Previously: LLL:EXT:core/Resources/Private/Language/Form/locallang_tabs.xlf:general
New: label://typo3.core/form.tabs/general

You take the part /Form/locallang_tabs to create form.tabs but this is all lowercase, how is it supposed to work on file-sensitive file systems? And what it for some reason I used /FoRM/ as directory? Do you plan to check all possible combinations? having the identifier Form.tabs sounds more logical.

Actions #5

Updated by Claus Due over 3 years ago

I think this might be a wrong direction to go in. A couple of problems come to my mind:

  1. Label files don't really have an identifier as such - currently, you are free to use any filename you wish. This makes it impossible to consistently reference a certain XLF file by identifier alone.
  2. Vendor seems redundant (unless part of the goal is to be able to reference files from non-extensions which I'm also not sure is desirable).
  3. "label" is far too generic - one of the advantages of "LLL" is that it is readily identifiable as a language resource located within a file.
  4. It would make it much harder to track any references to a label in a file when you only have the XLF file as starting point. Right now you can scan for the filename or the path of the XLF file to find any label references, if this was a translated path this wouldn't be possible (and/or would cause noise when searching if identifiers are common, which they very frequently are).

On the other hand what might be a better direction is to allow a new shorthand notation for such references, e.g.:

lll://extkey/labelname

Which would exclusively work to reference an XLF file named exactly "locallang.xlf" that exists in the conventional location in extension with key "extkey". The protocol name is less ambiguous, it's even shorter than including the vendor, and it encourages following conventions in terms of how to name XLF files.

It might also be possible to compromise slightly and allow something like:

lll://extkey/customfile.xlf/labelname

Where the fact that this is a custom file can be identified by the number of slash-separated segments in the reference.

However, I'd still be somewhat against this simply for the fact that it makes it impossible to scan for occurrences of a certain label unless you mentally convert the file path and labelname to the expected protocol syntax.

That being said I think it's a reasonable idea to collect all XLF files from extensions and store them in a format that does not require XML parsing, e.g. a hash map in which such labels could be looked up quite quickly. IMHO this is very preferable compared to the current "parse as needed" approach - and can be done in warmup along with things like DI rebuild. (and it's much easier to override things in a hash-map than registering an override file for an XLF). Throw a PSR-14 event into the mix which for example allows loading files from an external source or overriding specific labels and you've also solved the two "Still up for discussion" bullet points ;)

Actions #6

Updated by Jonas Eberle over 3 years ago

I would rather not have the infamous `locallang` become a convention.
Also a vendor is absolutely not necessary IMHO and would feel like bloat.

I do like a PHP API as a replacement for https://docs.typo3.org/m/typo3/reference-coreapi/master/en-us/ApiOverview/Internationalization/ManagingTranslations.html#custom-translations, though!

Actions #7

Updated by Benni Mack over 3 years ago

Thanks for your feedback all.

I personally don't want this to be "all magic" in terms of "locallang_tabs.xlf" becomes "tabs" as resource identifier - not at all. I'd rather have (in the long term) have extensions register their files with their prefixes as they fit.

In addition, we also have files outside of extensions (not using EXT:) which also need to somehow be registered via a system-wide configuration option.

I did not take a ext_localconf.php example (because it's not available during compile-time) but maybe the registration helps to understand:

$labelProvider = GeneralUtility::makeInstance(LabelProvider::class);
$labelProvider->register('news', 'tabs', DIR . '/Resources/Private/Language/labels.xlf');

?>

This code can also be put in AdditionalConfiguration to register custom files. I still see roughly 50% from agencies in TYPO3 v9/v10 projects using XLF files inside fileadmin/ for instance, so we need to consider that as well (that's why "magic" would only be available for extensions putting their files in the right folder).

It's similar to the Icon Registry, but just for labels.

A few additional notes:
  • I strongly recommend going away from the "let's use extension names as namespace", because then label files can be placed anywhere (local folder, or other composer packages (or whatever packages) could ship their label files, and you don't need to adapt the usages anymore, when you move files.

I don't want "magic" (I tried to find ways to make the registration simple though), bottom line is: I think this is a registry for labels which isn't solely based on file paths.

I think however we disagree on one point (which is fine, since I requested feedback): I proposed this idea, because I believe this resource concept helps to separate concerns a file location vs. referencing a "localized string", which was also addressed similarly by e.g. "puli" project a few years back.

Also just to mention this again, because it might have not been clear enough: "dropping LLL: syntax", I don't see this in ANY foreseeable future for TYPO3 at all.

Actions #8

Updated by Benni Mack over 3 years ago

Since this came up in slack, I'll add this to the ticket as well.

I'm also not sold on "label" as prefix, but LLL "local language label" seems very odd to me, and definitively uncommon outside the TYPO3 world. "xlf://" would be possible as well, ofc. But as usual: "Naming is hard", "label" is too broad, I agree.

Btw... Drupal has a magic function called `t()` which is used to translate a label :)

Actions #9

Updated by Helmut Hummel over 2 years ago

I like the idea of having way to register translation resources (aka xlf files), and map them to an (arbitrary) but namespaced and installation unique, or rather "globally" unique, identifier.

These identifiers must be conflict free. No two packages/ extensions must use the same. Since this only is achievable with a registration, and I'd rather not introduce another registry for that, I suggest to use Composer package names. By doing so, we have a registry, that enforces vendor name ownership (Packagist), and have a defined syntax {vendor}/{name}

I would suggest though to not only look at label, but think about a generic solution to handle all and reference all resources, like configuration files, template files, TCA files, etc.

All these resources benefit from a clean way to reference and override them.

Actions #10

Updated by Benni Mack 3 months ago

So, some more thoughts from my scratchpad.

What we should / could do before / in advance:
  • Get rid of "default" and use "en" everywhere.

I've been digging into symfony/translate package, which has a translator interface, and separates a "identifier" in three parts:

- catalogue (default "app") - that would be our "namespace"
- domain - by default "messages" in symfony - that's the identifier for the file within a package for example
- messageId - that would be the part of the string within the XLF file

We could then define a new syntax that consists of this structure:

Reference syntax could be "label://{catalogue}/{domain}/{message.id}"

In contrast - our current syntax: LLL:EXT:core/Resources/Private/Language/locallang_mod.xlf:labels.depth_0

Examples of how a new reference syntax could then look like:

  • label://app/messages/labels.depth_0
  • label://EXT:core/mod/labels.depth_0
  • label://typo3.cms-core/mod/labels.depth_0
  • label://core.main/labels.depth_0
  • label://typo3.cms-core/core/labels.depth_0
Some more examples:
  • LLL:EXT:backend/Resources/Private/Language/locallang.xlf:login.link
  • label://extension.backend/locallang/login.link
  • label://{catalogue}/{domain}/{id}

Some ideas:
- We could map "Resources/Private/Language/locallang.xlf" to the default domain "messages". We could also map the extension key (or vendor namespace) to a catalogue
- We could ship a default "app" namespace for a TYPO3 project to allow XLF directly

Registration:
- Either we allow PackageManager to deal with this and register default namespaces and domains by scanning the XLF
- We could also define this in the XLF?
- Every package could define their "domain"

We could create a Mapper from the old syntax to the new one.

Regarding Helmut's comments, I'm not sure if we find a good way to achieve this for the different use-cases, but I'm open for solutions here.

Actions

Also available in: Atom PDF