Bug #62921

l10n_mode with relations does not work in Extbase

Added by Dmitry Dulepov over 4 years ago. Updated over 1 year ago.

Must have
Extbase + l10n
Target version:
Start date:
Due date:
% Done:


TYPO3 Version:
PHP Version:
Is Regression:
Sprint Focus:


Original issue is here: https://forge.typo3.org/issues/29293 I believe it is incorrectly closed by Marc Bastian Heinrichs.

Suppose in $TCA:

        'documents' => array(
            'label' => '...',
            'l10n_mode' => 'exclude',
            'config' => array(
                'allowed' => 'tx_documents_document',
                'autoSizeMax' => 30,
                'internal_type' => 'db',
                'maxitems' => 20,
                'max_size' => 128000,
                'minitems' => 0,
                'foreign_table' => 'tx_documents_document',
                'MM' => 'tx_documents_ref',
                'MM_foreign_select' => 1,
                'MM_match_fields' => array(
                    'ident' => 'documents'
                'MM_opposite_field' => 'file_usage',
                'prepend_tname' => 1,
                'type' => 'group'

When fetching objects for 'documents' field, there should be no language overlay done at all because there are no translated records (l10n_mode=exclude). However doLanguageAndWorkspaceOverlay in typo3/sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php does not care and tries to fetch translated records and unsets valid rows in the default language because there are no translated rows (how there can be?). So you always get no docs in languages other than default.

From TCA reference about l10n_mode:

exclude – Field will not be shown in TCEforms if this record is a localization of the default language. (Works basically like a display condition.) Excluded fields will not be copied when a language-copy is made. May have frontend implications similar to "mergeIfNotBlank".
mergeIfNotBlank – Field will be editable but if the field value is blank the value from the default translation is used (this can be very useful for images shared from the default record). Requires frontend support. In the backend the effect is that the field content is not copied when a new "localization copy" is made.

defLangBining.png View (139 KB) Dmitry Dulepov, 2015-03-18 10:37

Related issues

Related to TYPO3 Core - Bug #65859: Support l10n_mode in extbase Needs Feedback 2015-03-20
Related to TYPO3 Core - Bug #57272: Extbase doesn't handle FAL translations correctly Closed 2014-03-25
Related to TYPO3 Core - Bug #77754: Typo3DbQueryParser does not respect enablefields for translated objects Closed 2016-09-01
Related to TYPO3 Core - Bug #75851: Extbase 110n_mode exclude + repository orderings Needs Feedback 2016-04-21
Duplicated by TYPO3 Core - Bug #76519: Extbase not respecting l10n_mode = mergeIfNotBlank for mm relations Closed 2016-06-08
Duplicated by TYPO3 Core - Bug #70948: repository doesn't handle l10n_mode "exclude" in some case Closed 2015-10-22
Duplicated by TYPO3 Core - Bug #75850: Extbase sys_language_mode + l10n_mode exclude Closed 2016-04-21


#1 Updated by Peter Niederlag about 4 years ago

TCA says "Requires frontend support". To me it seems extbase is just lacking this frontend support.

Dmitry, not sure wether I understood your use case properly. This is my use case:

As Developer I want to be able to assign a relation on a record in the default translation that must not be changed in the translation of the record.
When I use the \TYPO3\CMS\Extbase\Persistence\QueryInterface to lookup records by a category
Then I expect to find records in the language of the Frontend

'faculty' => array(
            'exclude' => 0,
            'l10n_mode' => 'exclude',
            'l10n_display' => 'hideDiff,defaultAsReadonly',
            'label' => 'LLL:EXT:luhjobs/Resources/Private/Language/locallang_db.xlf:tx_luhjobs_domain_model_vacancy.faculty',
            'config' => array(
                'type' => 'select',
                'foreign_table' => 'tx_luhjobs_domain_model_faculty',
                'foreign_table_where' => 'AND tx_luhjobs_domain_model_faculty.sys_language_uid IN (-1,0)',
                'items' => array(
                'minitems' => 1,
                'maxitems' => 1,
// QueryInterface
$constraints[] = $query->equals('faculty', $uid);
// resulting SQL(relevant parts only) in case of Frontend with ?L=1
tx_luhjobs_domain_model_vacancy.type = '19' AND tx_luhjobs_domain_model_vacancy.intern = 0
    tx_luhjobs_domain_model_vacancy.sys_language_uid IN (1,-1)
        tx_luhjobs_domain_model_vacancy.sys_language_uid=0 AND tx_luhjobs_domain_model_vacancy.uid NOT IN
                SELECT tx_luhjobs_domain_model_vacancy.l10n_parent FROM tx_luhjobs_domain_model_vacancy WHERE tx_luhjobs_domain_model_vacancy.l10n_parent>0 AND tx_luhjobs_domain_model_vacancy.sys_language_uid=1 AND tx_luhjobs_domain_model_vacancy.deleted=0

#2 Updated by Peter Niederlag about 4 years ago

  • Status changed from New to Needs Feedback

Simple Workaround, that works for me:

'l10n_mode' => '',
'l10n_display' => 'hideDiff,defaultAsReadonly',

This way the relation from the original record is copied when translating the record in the Backend (l10n_mode=''). l10n_display=defaultAsReadonly takes care of making the field not editable.

Could that work out for you as well? Btw. https://forge.typo3.org/issues/29293 yields 403 to me.

#3 Updated by Dmitry Dulepov about 4 years ago

Peter, I think you understood me correctly if I understood you correctly :)

There can be fields that should be changed in translated records (title is a typical example) and there can be fields that are common for the record in the default language and translated record. Those common fields are marked with 'l10n_mode' => 'exclude'. However it is not ight that Extbase lacks FE support. There was support but it was broken due to misunderstanding of TYPO3 localisation. It is quite easy to track where it was made.

For our own purposes I made it working correctly with an Xclass:

namespace Snowflake\Xyz\Xclass;

class Typo3DbQueryParser extends \TYPO3\CMS\Extbase\Persistence\Generic\Storage\Typo3DbQueryParser {

     * Fixes incorrect localisation handling in Extbase.
     * @param string $tableName
     * @param array $sql
     * @param \TYPO3\CMS\Extbase\Persistence\Generic\QuerySettingsInterface $querySettings
    protected function addSysLanguageStatement($tableName, array &$sql, $querySettings) {
        if (is_array($GLOBALS['TCA'][$tableName]['ctrl'])) {
            if (!empty($GLOBALS['TCA'][$tableName]['ctrl']['languageField'])) {
                $sql['additionalWhereClause'][] = $tableName . '.' . $GLOBALS['TCA'][$tableName]['ctrl']['languageField'] . '=0';


#4 Updated by Dmitry Dulepov about 4 years ago

Peter, https://forge.typo3.org/issues/62921#note-2 is not a solution but a workaround. You are trying to make incorrect localisation code in Extbase work properly. However it is not $TCA that should be changed but Extbase.

Check FE localisation guide and see that Extbase does it incorrectly.

I am against removing symptoms, I am for fixing the issue.

#5 Updated by Dmitry Dulepov about 4 years ago

https://forge.typo3.org/issues/29293 can only be found through Google cache because it was in the MVC project and the project is deleted, so all its issues.

#6 Updated by Dmitry Dulepov about 4 years ago

  • Complexity changed from nightmare to easy

#7 Updated by Peter Niederlag about 4 years ago

I think extbase does not respect/support "l10_mode" in TCA, at least I can't find any reference to "l10_mode" in extbase code or it's history. TYPO3-Core/TCA provides l10n_mode which results in a hugh difference in data handling, which extbase should be aware of and take into account.

Are you sure the code in your xclass covers all use cases of translations in a proper way?

But, you are probably right, the workaround with l10_mode='' probably has a huge problem when the relation of the original record is updated. The translation will still point to the old relation and due to l10n_display="defaultAsReadonly" there is no way to fix it. At least that's what I expect to happen without testing it.

#8 Updated by Dmitry Dulepov about 4 years ago

There is no need for Extbase to do anything about l10n_mode. This happens in PageRepository::getRecordOverlay(). Extbase calls that function already.

And, yes, my code is correct. Again, see the FE localisation guide document by Kasper. I was a co-author and the first reviewer.

#9 Updated by Peter Niederlag about 4 years ago

Are we talking about the table "pages" only, or are we talking about the generic approach in extbase to handle translations? The pages table is very special in that regard as you probably know.

Yes, for sure extbase must know about l10n_mode. It just results in a completly different data-handling of the core. As the data in the database is managed very different based on l10n_mode anyone trying to make sql-queries for the data must absolutly take the l10n_mode into account.

l10n_mode="exclude" ist even rather simple, but how would you expect the logic for l10n_mode="mergeIfNotBlank" to be implemented without teaching extbase about l10n_mode?

#10 Updated by Dmitry Dulepov about 4 years ago

Peter, it does not matter what table it is. Extbase does NOT need to know about l10n_mode. Everything is handled by the core in PageRepository::getRecordOverlay() and related functions.

Look at https://git.typo3.org/Packages/TYPO3.CMS.git/blob/HEAD:/typo3/sysext/frontend/Classes/Page/PageRepository.php#l408 and https://git.typo3.org/Packages/TYPO3.CMS.git/blob/HEAD:/typo3/sysext/frontend/Classes/Page/PageRepository.php#l1427.

Core already does all that.

Extbase calls getRecordOverlay() here: https://git.typo3.org/Packages/TYPO3.CMS.git/blob/HEAD:/typo3/sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php#l766

There is no any need for Extbase to care about l10n_mode because it is done by the core api. Look at the code!

Any more questions?

#11 Updated by Peter Niederlag about 4 years ago

Best thing really would be to have proper test-scenarios to cover all different cases. Then we'd know for sure when it works.

Thx for drawing my attention again to the fact that extbase does use PageRepository::getRecordOverlay() to apply overlays to a record. This definitly does work fine.

However I am still not 100% convinced that it's really sufficient for lookups on the whole property chain, as the problem is not in applying the overlay. The problem start's in the beginning when trying to find the proper record to apply the overlay on.

Can we check that please once more really good?

Let's say we have a simple n:1 (select) relation like:

jobs.type => jobtypes

Now, I(extbase) needs to lookup all jobs where the type=3 and that should be displayed when the requested Frontend language is 1(non-default!).

Just looking for recordes with sys_language_uid=0 (that's what imho your xclass boils down to) will miss all records that are not translations but "standalone foreign language" records (having l10n_parent=0 and sys_language_uid=1).

#12 Updated by Dmitry Dulepov about 4 years ago

I can only ask you to read again the FE localisation guide. There is no such thing as "standalone foreign language records" in TYPO3. TYPO3 always requires a record in the main language for having localized records. All other interpretations are wrong by design. Just read the doc: http://docs.typo3.org/typo3cms/FrontendLocalizationGuide/CoreSupportForLocalization/TcaTcemainTceforms/Index.html

What you write about, can only happen if somebody uses localisation in TYPO3 incorrectly.

#13 Updated by Peter Niederlag about 4 years ago

Now we get somewhere else. The localisation guide imho does not explicitly state that TYPO3-Core does not support a "standalone foreign language record". The TYPO3-Core has always allowed to use the system in such a way without any exception or error message. So I'd consider it to be a valid use case and have always thought so.

If we can agree on this being a non-valid use case we should try to prevent it and can remove support for it. Do you think that's doable?

#14 Updated by Dmitry Dulepov about 4 years ago

The TYPO3-Core has always allowed to use the system in such a way without any exception or error message. So I'd consider it to be a valid use case and have always thought so.

Well, you thought so but TYPO3 core does not really allow this :) You can create such records but they will only be visible in he List module, which lists everything with the goal to help detect all kind of issues. But all other TYPO3 core functions require records in the main language to work with localisations. There is no single function that supports "standalone" localised records.

Many people are mistaken and think that it is possible to create a record in any language and just use it. It does not work like that in TYPO3. There is always the main record and the overlay. Main record can be hidden in FE if it is unused but it has to exist. It is a strange but no more strange than other things in TYPO3 (see versions and workspaces, for example of other strange decisions).

If an extension uses records in the translated language directly, it means only one thing: its author is unaware of getRecordOverlay() and how core localisation works.

#15 Updated by Peter Niederlag about 4 years ago


So what is the reason for admins having to explicitly enable strict forcing the overlay thing?

// tsref
// http://docs.typo3.org/typo3cms/TyposcriptReference/Setup/Config/Index.html#sys-language-overlay
// 0 being the default
config.sys_language_overlay = 1
// Page-TSconfig
// http://docs.typo3.org/typo3cms/TSconfigReference/PageTsconfig/Mod/Index.html#web-page-mod-web-layout
mod.web_layout {
  defLangBinding = 1

Actually these settings are non-default and must be enabled. So to me it seems like whoever designed it had the most flexibilty in mind allowing different approaches including "standalone foreign language records" as well as "enforce translation/hide elements without a parent in default language".

#16 Updated by Dmitry Dulepov about 4 years ago

"strict" has its uses in some situations when it is necessary to enforce translations.

"defLangBinding" is a rudiment. In 2007 there were three people working on EXT:templavoila: Kasper, Robert and me. Kasper was silent for a couple of months, Robert and me were working on various functions. Robert did everything related to localisations while I fixed various bugs from the bug tracker. At one moment Kasper appeared and said that localisation is all wrong. Robert, in fact, implemented what you describe as "standalone translated records" but in TemplaVoila. Kasper wanted to change it completely and rewrite this code to fully follow TYPO3 core way (main language record + language overlays). However there were lots of installs already that used templavoila with multiple languages. We argued for a couple of weeks. In the end we decided to implement it properly but also add options to ensure compatibility with existing sites. Everybody, who used the "wrong" approach, would have to use TSConfig to alter how page module and TemplaVoila work with languages. All new installations should use "proper" approach with main record and overlays.

The problem also was that many people used standard page module with TemplaVoila because editors knew how to use it. So Kasper implemented support for the "wrong" templavoila localisations also in the standard page module. defLangBinding is one of those options. If you check TYPO3 core, you will see that it is used only in the page module (see defLangBinding.png attached here). However this is not supported anywhere else in the core. Even more, if you look how ontent fetching works, you will see that it is only main record language + overlays. Code is here: https://git.typo3.org/Packages/TYPO3.CMS.git/blob/HEAD:/typo3/sysext/frontend/Classes/ContentObject/ContentContentObject.php#l72 (see lines 72 to 91). Since TemplaVoila is not using core for rendering records, the change was needed only in the Page module.

So there is no support in the core for anything except main language record + language overlay. Record in the main language is mandatory and "standalone" translated records are not supported by the core.

#17 Updated by Peter Niederlag about 4 years ago

Hi Dmitry,

first of all it's a pleasure for me to share knowledge with you. I am still very convinced it's just working, as I have been using TYPO3 this way with customers for almost 15 years now. Actually I ran a bunch of tests in BE and FE while writing all these comments. From practical point:

- standalone translated records in tt_content work
- standalone translated records are supported in page module in BE

There are options to disable this functionality (see my last comment), but to be honest I've always kept the flexibility that was available and requested by our customers. BTW the overlay api was imho not even existant in the very beginning(?) and got implemented/adjusted(?) with the WS/version feature, which was added some time later (by Kasper).

It would be way easier if TYPO3 was not as flexibel as it is, but hey, that's TYPO3 :) We can't even agree on wether this is a feature (my POW) or a bug (your POW). And, TYPO3 CMS's still powerfull and works. That's awesome, isn't it?

Should we go for a poll or an RFC on AC-ML?

#18 Updated by Bodo Eichstädt about 4 years ago

Peter, tt_content is somehow the worst test case. What was mentioned before, the records in another language without relation to their "parent" record (= in the default language) is different with tt_content. It might look like it works, just because you actually do not browse the CE directly but their parent, the page with an URL. The desired effect might be totally different with other tables. So for tt_content is actually does not matter much. Correct me if I am wrong. Yes agree, this strict, overlay etc is pain in the ass for a beginning developer with his/her first EXT.

#19 Updated by Peter Niederlag about 4 years ago

No, it does not look like it works, it definitly works, for almost fifteen years and as of today(tested on 6.2-dev). You can create content in any translation as you want, and it works in FE as you would expect it ("Out of the Box"). And I have adapted this working model for extensions/other records as well. List module afaik still allows to disable language view, hiding any parent/child translation relations. You won't convience me here. If there is a majority that shares your point of view, I am fine to adapt myself to it.

#20 Updated by Dmitry Dulepov about 4 years ago

No, Peter, it cannot work for content out of the box :) Here the code that creates the query to fetch content: https://git.typo3.org/Packages/TYPO3.CMS.git/blob/HEAD:/typo3/sysext/frontend/Classes/ContentObject/ContentObjectRenderer.php#l8027

        if ($conf['languageField']) {
            // The sys_language record UID of the content of the page
            $sys_language_content = (int)$GLOBALS['TSFE']->sys_language_content;

            if ($GLOBALS['TSFE']->sys_language_contentOL
                && isset($GLOBALS['TCA'][$table])
                && !empty($GLOBALS['TCA'][$table]['ctrl']['languageField'])
                && !empty($GLOBALS['TCA'][$table]['ctrl']['transOrigPointerField'])
            ) {
                // Sys language content is set to zero/-1 - and it is expected that whatever routine processes the output will
                // OVERLAY the records with localized versions!
                $languageQuery = $conf['languageField'] . ' IN (0,-1)';
                // Use this option to include records that don't have a default translation
                // (originalpointerfield is 0 and the language field contains the requested language)
                if (!empty($conf['includeRecordsWithoutDefaultTranslation'])) {
                    $languageQuery .= ' OR (' . $GLOBALS['TCA'][$table]['ctrl']['transOrigPointerField'] . ' = 0 AND ' .
                        $conf['languageField'] . ' = ' . $sys_language_content . ')';
            } else {
                $languageQuery = $conf['languageField'] . ' = ' . $sys_language_content;
            $query .= ' AND (' . $languageQuery . ')';
The only way that it could work is when you make custom adjustments to the core:
  • you set select.includeRecordsWithoutDefaultTranslation = 1
  • you unset $TCA['tt_content']['ctrl']['languageField'] or 'transOrigPointerField'
  • you set select. languageField in your configuration

However the case above is completely different from this bug report because it is about a custom configuration of the content object.

Do you use a modified core like most companies do? Or do you use fluidcontent, which depends on Extbase and uses the same wrong code?

The problem with Extbase is that it fetches the translated record record directly. So the overlay code, which is called after, is wrong. Here is the commit that broke it: https://review.typo3.org/#/c/10188/

#21 Updated by Dmitry Dulepov about 4 years ago

Yes agree, this strict, overlay etc is pain in the ass for a beginning developer with his/her first EXT.

I fully agree with this. I hate this scheme and I would prefer standalone records myself. But if they are implemented, support for l10n_mode is required!

#22 Updated by Peter Niederlag about 4 years ago


You are aware of the fact that $GLOBALS['TSFE']->sys_language_contentOL is unset by default?

// have a look at
$this->sys_language_contentOL = $this->config['config']['sys_language_overlay'];
// $this->sys_language_contentOL is intialized with 0 in Controller
// $this->config['config']['sys_language_overlay'] is NOT set by default
// so we end up with $this->sys_language_contentOL = NULL

So for the language part of the query we get:

$languageQuery = $conf['languageField'] . ' = ' . $sys_language_content;

Testing on the most plain TS implementation I could think of with a pristine 6.2-dev branch:

page.50 < styles.content.get

[globalVar = GP:L = 1]
config.sys_language_uid = 1
config.language = en
config.locale_all = en_GB.UTF-8

Now, of course the whole story does get different when I set

config.sys_language_overlay = 1

This whole thing probably needs some cleanup/rethinking but I am quite sure it does work like I describe.

Just one additional note to your proposed patch on extbase query-parser. Regardless of our current focus of the discussion of the language overlays, as soon as the relation can be edited on a translation of a record the patch will fail as well. When you search on a property.path of a domain model (a relation), you must take into account that the translated record could have a different relation as the original one. Unless we really simplify this as well and make relations editable only on original records (Which definitly won't be sufficient for workspace/version OL)

#23 Updated by Peter Niederlag about 4 years ago

Or, to put it simple: The whole idea/concept of querying in the default language and applying overlays afterwards is terribly broken as soon as you run queries across relations. Give me all news records in english translation that have category 18 assigned. There is no way to get this working by querying only the default language and applying overlays afterwards. Unless you make the restriction that the relation can't be set on the translated record.

#24 Updated by Dmitry Dulepov about 4 years ago

I agree that it is broken. The idea is bad from the beginning. However this is official way. And when you specify l10n_mode=exclude to manage those categories only on the main language record, you expect it to work. It doesn't.

We are not talking here about concepts. This bug is only about making Extbase work properly with l10n_mode=exclude. If you want to change localisation concept, open another bug report, please.

#25 Updated by Peter Niederlag about 4 years ago

It's broken, we agree. We don't agree on the official way, which has an high impact on the approach to fix the problem. I'd like to keep this Bug-Item for reference to get this thing fixed as it has a great valuable discussion and a lot of background knowledge. My fix/suggestion will however be very different from the solution you suggest. I can setup another issue if you want to keep this one for yourself, just let me know.

#26 Updated by Dmitry Dulepov about 4 years ago

I prefer that you make another issue. This issue is very specific. If you want to make more generic change, it is not for this issue.

#27 Updated by Alexander Opitz over 3 years ago

  • Assignee set to Dmitry Dulepov

What is the state of this issue? Is it fixed in TYPO3 7 LTS?

#28 Updated by Dmitry Dulepov over 3 years ago

Not fixed I think. Otherwise it would be green.

You can still use the patch locally though.

#29 Updated by Dmitry Dulepov almost 3 years ago

Similar issue: #57272

#30 Updated by Ruben Schmidmeister almost 3 years ago

My original issue: https://forge.typo3.org/issues/76519

Always a good example is the news extension:

'categories' => [
    'exclude' => 1,
    'l10n_mode' => 'mergeIfNotBlank',
    'label' => $ll . 'tx_news_domain_model_news.categories',
    'config' => [
        'type' => 'select',
        'renderType' => 'selectTree',
        'treeConfig' => [
            'dataProvider' => \GeorgRinger\News\TreeProvider\DatabaseTreeDataProvider::class,
            'parentField' => 'parent',
            'appearance' => [
                'showHeader' => true,
                'allowRecursiveMode' => true,
                'expandAll' => true,
                'maxLevels' => 99,
        'MM' => 'sys_category_record_mm',
        'MM_match_fields' => [
            'fieldname' => 'categories',
            'tablenames' => 'tx_news_domain_model_news',
        'MM_opposite_field' => 'items',
        'foreign_table' => 'sys_category',
        'foreign_table_where' => ' AND (sys_category.sys_language_uid = 0 OR sys_category.l10n_parent = 0) ORDER BY sys_category.sorting',
        'size' => 10,
        'autoSizeMax' => 20,
        'minitems' => 0,
        'maxitems' => 99,

Let's say I have a news entry in the default langauge with some categories selected.
Now I translate that news to another language.

The expected behaviour for me would be that the categories are inherited from the original article, but they're empty in the frontend.
Naturally, when I add categories to the translated article, those are shown in the frontend.

#31 Updated by Daniel Goerz almost 3 years ago

  • Status changed from Needs Feedback to New
  • TYPO3 Version changed from 6.2 to 7

#32 Updated by Dmitry Dulepov almost 3 years ago

  • Subject changed from l10n_mode=exclude with relations does not work in Extbase to l10n_mode with relations does not work in Extbase

#33 Updated by Markus Klein over 2 years ago

  • Status changed from New to Accepted
  • Complexity deleted (easy)

#34 Updated by Jan Kornblum about 2 years ago

Are you sure this only affects relations? I've a table with several l10n_mode=exclude fields (integers). With 7.6, a query which compares against this fields ($query->equals()) results in no records are beeing found at all when sys_language_uid > 0. In 4.5 it used to work...

#35 Updated by Markus Klein about 2 years ago

@Kornblum: Yes broken. You can try my core patch https://review.typo3.org/49332 if it helps you.

#36 Updated by Dmitry Dulepov over 1 year ago

  • Assignee deleted (Dmitry Dulepov)

#37 Updated by Tymoteusz Motylewski over 1 year ago

  • Category changed from Extbase to Extbase + l10n

#38 Updated by Riccardo De Contardi over 1 year ago

  • Related to Bug #75851: Extbase 110n_mode exclude + repository orderings added

Also available in: Atom PDF