Bug #86762

Site-Configuration defined language fallbacks not working

Added by Carsten Hülsmann 7 months ago. Updated 20 days ago.

Status:
Closed
Priority:
Must have
Category:
Link Handling, Site Handling & Routing
Target version:
-
Start date:
2018-10-26
Due date:
% Done:

100%

TYPO3 Version:
9
PHP Version:
7.2
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

Hi,
maybe i'm just expecting the wrong but it seems there is an issue with the language fallback functionality.

I have made a site configuration with 2 languages (/en/, /de/) while en is default and de has a fallback to en.

My expectation is now that if i have not translated (en) content elements on the page and i visit the /de/-Version of it, that TYPO3 renders the english content element. But this doesn't happen. I tried debugging but there were no specific errors or something regarding this behaviour.

Here the current site configuration:

rootPageId: 1
base: 'https://mybaseurl.com'
baseVariants: {  }
languages:
  -
    title: English
    enabled: true
    languageId: '0'
    base: /en/
    typo3Language: default
    locale: en_US.UTF-8
    iso-639-1: en
    navigationTitle: English
    hreflang: en-US
    direction: ltr
    flag: us
  -
    title: German
    enabled: true
    languageId: '1'
    base: /de/
    typo3Language: de
    locale: de_DE.UTF-8
    iso-639-1: de
    navigationTitle: Deutsch
    hreflang: de-DE
    direction: ltr
    fallbackType: fallback
    fallbacks: '0'
    flag: de
errorHandling: {  }
routes: {  }

Typo3DbBackend.diff View (974 Bytes) David Bruchmann, 2019-02-26 07:49

PageRepository.diff View (5.74 KB) David Bruchmann, 2019-02-26 07:49


Related issues

Related to TYPO3 Core - Bug #86595: No Language Fallback Closed 2018-10-08
Related to TYPO3 Core - Bug #19114: sys_language_mode content_fallback with a defined fallback chain does not output expected fallback content Closed 2008-07-16
Related to TYPO3 Core - Feature #88137: Create multi-step fallback for content and arbitrary records New 2019-04-11
Duplicated by TYPO3 Core - Bug #87121: Content fallback does not work, when page translation does not exist yet Closed 2018-12-11

Associated revisions

Revision 064f9043 (diff)
Added by Benni Mack about 1 month ago

[FEATURE] Re-introduce mixed overlay mode for content fallback

This "feature" solves a lot of issues, but I really wanted to make sure
to cover the generic issues around this topic.

In order to understand this, we need to understand the different
logic when talking about "language fallbacks", but I don't want
to warm up how it was before.

This is how it works:
- fallbackChain: defines which page translation should be checked
when linking or resolving a page in a specific language
- fallbackType: Now that we covered all relevant cases, the naming
could be better, but here we go, it's actually defining the logic for fetching
content.
- fallbackType="strict" > only show the content that is viable in the target language
but this is based on "overlays". Fetch all "language=0" records, do overlays
and remove the ones that have no overlays. However, take the ones that have no
language parent and render them as well ("includeWithoutDefaultTranlsation")
This is what we call "do overlays with floating".
This is recommended to do in most classic translation cases, with different languages
fallbackType="fallback" > Do overlays: Fetch all "language=0" records, do overlays
but KEEP the ones that have no overlays. However, take the ones that have no
language parent and render them as well ("includeWithoutDefaultTranlsation")
This is what we call "do overlays in mixed mode".
Useful if your translation is Swiss-German but your default language is "German"
fallbackType="free" - Do not do overlays, just fetch all records of the target language
Could be seen as "free mode" as we do it in TYPO3 Page module.

The new free option is therefore new, also the "fallback" functionality
is actually showing more content than before (thus, different, but maybe we could fix that!)
as we have the "mixed" mode back.
Also the "language fallback" is now possible for any fallbackType.

Now, what's still missing - but out of scope - is actually a way to fetch content
with multiple possibilities for overlaying. I call this "forward language overlays"
however, this is a feature that is theoretically possible but not in v9 anymore.

This patch restores the max. types of use cases back for TYPO3 Core.

The following things that are really gone for good now and won't come back:
- config.sys_language_mode = ignore
- config.sys_language_mode =
- Option includeRecordsWithoutDefaultTranslation (= always enabled) is not needed anymore

Also, there are no ways anymore to use inconsistent multiple TypoScript settings which
do not make sense depending if the translated page does not exist (l18n_cfg)
but still using TypoScript conditions for that.

Resolves: #86762
Resolves: #86712
Releases: master, 9.5
Change-Id: I8b3144410f7d7ed1d705d42f16a46f190275387a
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367
Tested-by: TYPO3com <>
Tested-by: Nikolaj Wojtkowiak-Pfänder <>
Tested-by: Steven Hilgendorff
Tested-by: Ralf Merz <>
Tested-by: Anja Leichsenring <>
Reviewed-by: Nikolaj Wojtkowiak-Pfänder <>
Reviewed-by: Ralf Merz <>
Reviewed-by: Anja Leichsenring <>

Revision 8f8c632e (diff)
Added by Benni Mack about 1 month ago

[FEATURE] Re-introduce mixed overlay mode for content fallback

This "feature" solves a lot of issues, but I really wanted to make sure
to cover the generic issues around this topic.

In order to understand this, we need to understand the different
logic when talking about "language fallbacks", but I don't want
to warm up how it was before.

This is how it works:
- fallbackChain: defines which page translation should be checked
when linking or resolving a page in a specific language
- fallbackType: Now that we covered all relevant cases, the naming
could be better, but here we go, it's actually defining the logic for fetching
content.
- fallbackType="strict" > only show the content that is viable in the target language
but this is based on "overlays". Fetch all "language=0" records, do overlays
and remove the ones that have no overlays. However, take the ones that have no
language parent and render them as well ("includeWithoutDefaultTranlsation")
This is what we call "do overlays with floating".
This is recommended to do in most classic translation cases, with different languages
fallbackType="fallback" > Do overlays: Fetch all "language=0" records, do overlays
but KEEP the ones that have no overlays. However, take the ones that have no
language parent and render them as well ("includeWithoutDefaultTranlsation")
This is what we call "do overlays in mixed mode".
Useful if your translation is Swiss-German but your default language is "German"
fallbackType="free" - Do not do overlays, just fetch all records of the target language
Could be seen as "free mode" as we do it in TYPO3 Page module.

The new free option is therefore new, also the "fallback" functionality
is actually showing more content than before (thus, different, but maybe we could fix that!)
as we have the "mixed" mode back.
Also the "language fallback" is now possible for any fallbackType.

Now, what's still missing - but out of scope - is actually a way to fetch content
with multiple possibilities for overlaying. I call this "forward language overlays"
however, this is a feature that is theoretically possible but not in v9 anymore.

This patch restores the max. types of use cases back for TYPO3 Core.

The following things that are really gone for good now and won't come back:
- config.sys_language_mode = ignore
- config.sys_language_mode =
- Option includeRecordsWithoutDefaultTranslation (= always enabled) is not needed anymore

Also, there are no ways anymore to use inconsistent multiple TypoScript settings which
do not make sense depending if the translated page does not exist (l18n_cfg)
but still using TypoScript conditions for that.

Resolves: #86762
Resolves: #86712
Releases: master, 9.5
Change-Id: I8b3144410f7d7ed1d705d42f16a46f190275387a
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/60456
Tested-by: TYPO3com <>
Tested-by: Anja Leichsenring <>
Reviewed-by: Anja Leichsenring <>

History

#1 Updated by Riccardo De Contardi 7 months ago

Maybe has this been already solved with https://review.typo3.org/58698?

#2 Updated by Carsten Hülsmann 7 months ago

I already tried this and also this one: https://review.typo3.org/#/c/58612/

But my issue stays the same. Over that there seems to be the issue that CE's with lang = -1 (all Languages) don't get rendered in default language.

#3 Updated by Carsten Hülsmann 7 months ago

Are really only 2 of us having these issues? I would appreciate to get some kind of feedback.

#4 Updated by Stefan Neufeind 7 months ago

  • Related to Bug #86595: No Language Fallback added

#5 Updated by Carsten Hülsmann 7 months ago

Small update from here: Just updated to latest 9.5.1 with the same behaviour.

#6 Updated by Frank Liebich 7 months ago

Solution for language fallback with an existing translated page using site-configuration and config.sys_language_overlay

config.sys_language_overlay = 1

change in \typo3\sysext\frontend\Classes\Controller\TypoScriptFrontendController.php
line 2272

$languageAspect = LanguageAspectFactory::createFromSiteLanguage($siteLanguage);

to
$languageAspect = LanguageAspectFactory::createFromSiteLanguage($siteLanguage, $this->config['config'] ?? []);

and in \typo3\v9\typo3\sysext\core\Classes\Context\LanguageAspectFactory.php
replace the function starting on line 100

public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect

with
public static function createFromSiteLanguage(SiteLanguage $language, array $config): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
        } else {
            $fallbackOrder = [0];
        }

        switch ((string)($config['sys_language_overlay'] ?? '')) {
            case '1':
                $overlayType = LanguageAspect::OVERLAYS_MIXED;
                break;
            case '0':
                $overlayType = LanguageAspect::OVERLAYS_OFF;
                break;
            case 'hideNonTranslated':
            default:
                $overlayType = LanguageAspect::OVERLAYS_ON_WITH_FLOATING;
        }
        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, $overlayType, $fallbackOrder);
    }

that way I got the fallback back.
It would be nice to set the overlay type in the site-configuration module, so that config.sys_language_overlay is not needed anymore

#7 Updated by Carsten Hülsmann 7 months ago

Thanks for this. Works as expected but breaks routingEnhancers at the same time:

Too few arguments to function TYPO3\CMS\Core\Context\LanguageAspectFactory::createFromSiteLanguage(), 1 passed in .../typo3/sysext/core/Classes/Routing/Aspect/PersistedAliasMapper.php on line 261 and exactly 2 expected
in .../typo3/sysext/core/Classes/Context/LanguageAspectFactory.php line 100


     *
     * @param SiteLanguage $language
     * @return LanguageAspect
     */
    public static function createFromSiteLanguage(SiteLanguage $language, array $config): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {

Maybe i will look deeper into this later (if there's time).

#8 Updated by Carsten Hülsmann 6 months ago

I now had the chance to looked deeper into this.

However it seems the content fallback was not respected for the routing-initiative.
In fact it's not possible to setup the setting sys_language_overlay is not respected in the LanguageAspectFactory if you use site-configuration, cause it's not part of the array $language.

I solved this temporarily now for me by adding this into the existing if statements of createFromSiteLanguage. BUt it would be great if this would take place in a more usable way for everyone.

Original function:

 public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
        } else {
            $fallbackOrder = [0];
        }

        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, LanguageAspect::OVERLAYS_ON_WITH_FLOATING, $fallbackOrder);
    }

Current function:
public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
            $overlayType = LanguageAspect::OVERLAYS_MIXED;
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
            $overlayType = LanguageAspect::OVERLAYS_ON_WITH_FLOATING;
        } else {
            $fallbackOrder = [0];
        }

        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, $overlayType, $fallbackOrder);
    }

#9 Updated by Helmut Hummel 6 months ago

  • Duplicated by Bug #87121: Content fallback does not work, when page translation does not exist yet added

#10 Updated by Anja Leichsenring 5 months ago

  • Status changed from New to In Progress
  • Assignee set to Anja Leichsenring

#11 Updated by Gerrit Code Review 5 months ago

  • Status changed from In Progress to Under Review

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#12 Updated by Gerrit Code Review 5 months ago

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#13 Updated by Gerrit Code Review 5 months ago

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#14 Updated by Gerrit Code Review 5 months ago

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#15 Updated by Gerrit Code Review 5 months ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#16 Updated by Gerrit Code Review 5 months ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#17 Updated by Gerrit Code Review 5 months ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#18 Updated by Gerrit Code Review 5 months ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#19 Updated by Steffen Dietrich 5 months ago

Just found another bug with a colleague of mine while testing it. Here's how to reproduce it:

- create a 2nd Site configuration with fallback to default language
- have a page with both languages
- add a new content element on default language and a new (not translated) content element for the 2nd language

expected results on the translated page(multiple possibilities):
a) only the new, free mode content element is shown
b) the new, free mode content element + the fallback elements are shown

actual result on the translated page:
only the fallback content element is shown, the free mode element doesn't appear at all

#20 Updated by Gerrit Code Review 5 months ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#21 Updated by Gerrit Code Review 5 months ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#22 Updated by Gerrit Code Review 5 months ago

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#23 Updated by Gerrit Code Review 5 months ago

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

#24 Updated by Jarvis H 4 months ago

I just commented this in the patch review here: https://review.typo3.org/59344 but just to sum it up in the issue comments here:

In the patch review I tested the following:

fallbackType: ‘strict’

connected mode: works (only translated records are shown)
free mode: works (only copied records or records which only exists in current language are shown)

fallbackType: ‘fallback’

connected mode: works (non-translated records show in default language + all translated records are shown)
free mode: works (default language records, copied records and records which only exists in current language are all shown)

This is all good, however config.sys_language_overlay seems to not work, at least changing between "0,1,hideNonTranslated" makes no difference.

#25 Updated by Ralf Merz 4 months ago

  • Related to Bug #19114: sys_language_mode content_fallback with a defined fallback chain does not output expected fallback content added

#26 Updated by Wolfgang Kleinbach 4 months ago

The patch only works for content elements on a localized page.
When the page has not been localized (yet), you will get an 404 error.
Example: First language is German, second language is English (with fallback to German). In frontend you're on a page with an English translation. There's a (sub-)menu linked to pages which do not have any English translation at all. Clicking one of the links, you will get 404 instead of the German fallback page.

This issue is related to https://review.typo3.org/#/c/59488, where the issue is well explained. Unfortunately the bugfix provided there, doesn't work.

#27 Updated by Kevin Ditscheid 4 months ago

@Wolfgang Kleinbach I submitted a Patch for that 404 issue https://review.typo3.org/#/c/59675/ as part of this ticket: https://forge.typo3.org/issues/86595

#28 Updated by Wolfgang Kleinbach 4 months ago

@Kevin Ditscheid, your patch works on my installation (9.5.4). Thanks!

#29 Updated by Gerrit Code Review 3 months ago

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#30 Updated by Gerrit Code Review 3 months ago

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#31 Updated by Gerrit Code Review 3 months ago

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#32 Updated by Gerrit Code Review 3 months ago

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#33 Updated by Gerrit Code Review 3 months ago

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#34 Updated by Gerrit Code Review 3 months ago

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#35 Updated by Gerrit Code Review 3 months ago

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#36 Updated by Gerrit Code Review 3 months ago

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#37 Updated by Gerrit Code Review 3 months ago

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

#38 Updated by Gerrit Code Review 3 months ago

Patch set 22 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

#39 Updated by Gerrit Code Review 3 months ago

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59344

#40 Updated by David Bruchmann 3 months ago

Having the following configuration my issue regrettable is not fixed by the latest patch (21):


languages:
  -
    title: Deutsch
    enabled: true
    languageId: '0'
    base: /
    typo3Language: de
    locale: de_CH.UTF-8
    iso-639-1: de
    navigationTitle: Deutsch
    hreflang: de-CH
    direction: ltr
    flag: de
  -
    title: Français
    enabled: true
    languageId: '1'
    base: 'https://french.domain.com/'
    typo3Language: fr
    locale: fr_FR.UTF-8
    iso-639-1: fr
    navigationTitle: Français
    hreflang: fr-FR
    direction: ltr
    fallbackType: strict
    flag: fr
  -
    title: Italiano
    enabled: true
    languageId: '2'
    base: 'https://italian.domain.com/'
    typo3Language: it
    locale: it_IT.UTF-8
    iso-639-1: it
    navigationTitle: Italiano
    hreflang: it-IT
    direction: ltr
    fallbackType: fallback
    fallbacks: '1'
    flag: it

The Parameters in the last text-line of the patch seems being set correctly IMO, but never have the expected impact on the output:
array(4 items)
   0 => 2 (integer)
   1 => 2 (integer)
   2 => 'mixed' (5 chars)
   3 => array(2 items)
      0 => '1' (1 chars)
      1 => 'pageNotFound' (12 chars)

In FE the fallback for Italian [2] is realized to German [0] instead of French [1].

#41 Updated by Martin Weymayer 3 months ago

Hello,

there is another bug concerning default language and setting content elements to "all languages". Is this bugfix also fixing this problem https://forge.typo3.org/issues/87093?next_issue_id=87092&prev_issue_id=87094?

Martin

#42 Updated by David Bruchmann 3 months ago

Having the following configuration my issue regrettable is not fixed by the latest patch (21):


languages:
  -
    title: Deutsch
    enabled: true
    languageId: '0'
    base: /
    typo3Language: de
    locale: de_CH.UTF-8
    iso-639-1: de
    navigationTitle: Deutsch
    hreflang: de-CH
    direction: ltr
    flag: de
  -
    title: Français
    enabled: true
    languageId: '1'
    base: 'https://french.domain.com/'
    typo3Language: fr
    locale: fr_FR.UTF-8
    iso-639-1: fr
    navigationTitle: Français
    hreflang: fr-FR
    direction: ltr
    fallbackType: strict
    flag: fr
  -
    title: Italiano
    enabled: true
    languageId: '2'
    base: 'https://italian.domain.com/'
    typo3Language: it
    locale: it_IT.UTF-8
    iso-639-1: it
    navigationTitle: Italiano
    hreflang: it-IT
    direction: ltr
    fallbackType: fallback
    fallbacks: '1'
    flag: it

The Parameters in the last text-line of the patch seems being set correctly IMO, but never have the expected impact on the output:
array(4 items)
   0 => 2 (integer)
   1 => 2 (integer)
   2 => 'mixed' (5 chars)
   3 => array(2 items)
      0 => '1' (1 chars)
      1 => 'pageNotFound' (12 chars)

So the function LanguageAspectFactory::createFromSiteLanguage returns in my case
return GeneralUtility::makeInstance(LanguageAspect::class, 2, 2, 'mixed', [1,'pageNotFound']);

In FE the fallback for Italian [2] is realized to German [0] instead of French [1].

Beside that I think the comment-line "* Site Languages always run in floating mode" is not 100% accurate.
Another related comment-line in class LanguageAspect is quite confusing:

public const OVERLAYS_ON_WITH_FLOATING = 'includeFloating';    // "hideNonTranslated" + records that are only available in polish

Languages should be written upper CamelCase but I never ever had a site with Polish, so what does that comment mean?

#43 Updated by David Bruchmann 3 months ago

I've fixed method PageRepository::getRecordOverlay and the languageOverlay for most records is working fine with it.
Part of the fix is the line

$languageChain = $sys_language_content . (count($fallbackChain) ? ',' . $fallbackChain[0] : '');

For tx_news for some reason this is not returning the expected value '2,1' of my case but only '2'. it might be that it's called from another method than PageRepository::getLanguageOverlay where I never added the additional parameter. It's possible to calculate $fallbackChain again in the method and drop the additional parameter, probably then everything will work like expected, else all method-calls have to be adjusted.

Other content-elements, including an individual extension are 'translated' correct.

One issue I discovered is that by splitting the whole query in "$query" and "$olrow", the DB-result is returned correct, if I chain all together its wrong.

Here is the fix:

class PageRepository implements LoggerAwareInterface
{

[...]

    /**
     * Master helper method to overlay a record to a language.
     *
     * Be aware that for pages the languageId is taken, and for all other records the contentId.
     * This might change through a feature switch in the future.
     *
     * @param string $table the name of the table, should be a TCA table with localization enabled
     * @param array $row the current (full-fletched) record.
     * @return array|null
     */
    public function getLanguageOverlay(string $table, array $row)
    {
        // table is not localizable, so return directly
        if (!isset($GLOBALS['TCA'][$table]['ctrl']['languageField'])) {
            return $row;
        }
        try {
            /** @var LanguageAspect $languageAspect */
            $languageAspect = $this->context->getAspect('language');
            if ($languageAspect->doOverlays()) {
                if ($table === 'pages') {
                    return $this->getPageOverlay($row, $languageAspect->getId());
                }
                return $this->getRecordOverlay(
                    $table,
                    $row,
                    $languageAspect->getContentId(),
                    $languageAspect->getOverlayType() === $languageAspect::OVERLAYS_MIXED ? '1' : 'hideNonTranslated',
                    $languageAspect->getFallbackChain()
                );
            }
        } catch (AspectNotFoundException $e) {
            // no overlays
        }
        return $row;
    }

[...]

    /**
     * Creates language-overlay for records in general (where translation is found
     * in records from the same table)
     *
     * @param string $table Table name
     * @param array $row Record to overlay. Must contain uid, pid and $table]['ctrl']['languageField']
     * @param int $sys_language_content Pointer to the sys_language uid for content on the site.
     * @param string $OLmode Overlay mode. If "hideNonTranslated" then records without translation will not be returned  un-translated but unset (and return value is FALSE)
     * @param array fallbackChain for translations
     * @throws \UnexpectedValueException
     * @return mixed Returns the input record, possibly overlaid with a translation.  But if $OLmode is "hideNonTranslated" then it will return FALSE if no translation is found.
     */
    public function getRecordOverlay($table, $row, $sys_language_content, $OLmode = '', $fallbackChain=[])
    {
        foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_page.php']['getRecordOverlay'] ?? [] as $className) {
            $hookObject = GeneralUtility::makeInstance($className);
            if (!$hookObject instanceof PageRepositoryGetRecordOverlayHookInterface) {
                throw new \UnexpectedValueException($className . ' must implement interface ' . PageRepositoryGetRecordOverlayHookInterface::class, 1269881658);
            }
            $hookObject->getRecordOverlay_preProcess($table, $row, $sys_language_content, $OLmode, $this);
        }

        $tableControl = $GLOBALS['TCA'][$table]['ctrl'] ?? [];

        if (!empty($tableControl['languageField'])
            // Return record for ALL languages untouched
            // TODO: Fix call stack to prevent this situation in the first place
            && (int)$row[$tableControl['languageField']] !== -1
            && !empty($tableControl['transOrigPointerField'])
            && $row['uid'] > 0
            && ($row['pid'] > 0 || in_array($tableControl['rootLevel'] ?? false, [true, 1, -1], true))) {
            // Will try to overlay a record only if the sys_language_content value is larger than zero.
            if ($sys_language_content > 0) {
                // Must be default language, otherwise no overlaying
                if ((int)$row[$tableControl['languageField']] === 0) {
                    // Select overlay record:
                    $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
                        ->getQueryBuilderForTable($table);
                    $queryBuilder->setRestrictions(
                        GeneralUtility::makeInstance(FrontendRestrictionContainer::class)
                    );
                    $languageChain = $sys_language_content . (count($fallbackChain) ? ',' . $fallbackChain[0] : '');
                    $query = $queryBuilder->select('*')
                        ->from($table)
                        ->where(
                            $queryBuilder->expr()->eq(
                                'pid',
                                $queryBuilder->createNamedParameter($row['pid'], \PDO::PARAM_INT)
                            ),
                            $queryBuilder->expr()->in(
                                $tableControl['languageField'],
                                $queryBuilder->createNamedParameter(GeneralUtility::intExplode(',', $languageChain), Connection::PARAM_INT_ARRAY)
                            ),
                            $queryBuilder->expr()->eq(
                                $tableControl['transOrigPointerField'],
                                $queryBuilder->createNamedParameter($row['uid'], \PDO::PARAM_INT)
                            )
                        )
                        ->add('orderBy', 'FIELD(' . $tableControl['languageField'] . ', ' . $languageChain . ')', true)
                        ->setMaxResults(1);
                    $olrow = $query->execute()
                        ->fetch();

                    $this->versionOL($table, $olrow);
                    // Merge record content by traversing all fields:
                    if (is_array($olrow)) {
                        if (isset($olrow['_ORIG_uid'])) {
                            $row['_ORIG_uid'] = $olrow['_ORIG_uid'];
                        }
                        if (isset($olrow['_ORIG_pid'])) {
                            $row['_ORIG_pid'] = $olrow['_ORIG_pid'];
                        }
                        foreach ($row as $fN => $fV) {
                            if ($fN !== 'uid' && $fN !== 'pid' && isset($olrow[$fN])) {
                                $row[$fN] = $olrow[$fN];
                            } elseif ($fN === 'uid') {
                                $row['_LOCALIZED_UID'] = $olrow['uid'];
                            }
                        }
                    } elseif ($OLmode === 'hideNonTranslated' && (int)$row[$tableControl['languageField']] === 0) {
                        // Unset, if non-translated records should be hidden. ONLY done if the source
                        // record really is default language and not [All] in which case it is allowed.
                        unset($row);
                    }
                } elseif ($sys_language_content != $row[$tableControl['languageField']]) {
                    unset($row);
                }
            } else {
                // When default language is displayed, we never want to return a record carrying
                // another language!
                if ($row[$tableControl['languageField']] > 0) {
                    unset($row);
                }
            }
        }

        foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_page.php']['getRecordOverlay'] ?? [] as $className) {
            $hookObject = GeneralUtility::makeInstance($className);
            if (!$hookObject instanceof PageRepositoryGetRecordOverlayHookInterface) {
                throw new \UnexpectedValueException($className . ' must implement interface ' . PageRepositoryGetRecordOverlayHookInterface::class, 1269881659);
            }
            $hookObject->getRecordOverlay_postProcess($table, $row, $sys_language_content, $OLmode, $this);
        }

        return $row;
    }

[...]
}

EDIT:
Not yet mentioned clearly enough is the behavior that it seems that fallback to the main-language is always done if a fallback is defined (it doesn't matter which one).
That also means that any posts are shown in the original language even they had to be hidden based on the fallback-settings.
Additionally that means that the simple case of Carsten Hülsmann never covers many issues, especially if all records are translated.
Based on my example I posted before that means that a post which is neither translated to French, nor to Italian, shouldn't be shown at all on the Italian page but is shown in German.
The essence of my settings is that German is main-language, French has no fallback and is therefore strict, Italian has fallback to French only.

I'm quite sure that there still exist cases which might not be covered by my verification. I still will post here a change for extbase which is not covered by my posted code.

#44 Updated by David Bruchmann 3 months ago

Attached the diff-files PageRepository.diff and for Extbase Typo3DbBackend.diff.

#45 Updated by Gerrit Code Review 3 months ago

Patch set 23 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

#46 Updated by Gerrit Code Review 3 months ago

Patch set 24 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

#47 Updated by Gerrit Code Review 3 months ago

Patch set 25 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

#48 Updated by Gerrit Code Review 3 months ago

Patch set 26 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

#49 Updated by Andreas Fernandez 3 months ago

  • Status changed from Under Review to New

#50 Updated by Wolfgang Kleinbach 3 months ago

The problem still exists with TYPO3 9.5.5.
To fix it, you just can apply the changes, which Carsten Hülsmann describes in his note above: https://forge.typo3.org/issues/86762#note-8.

#51 Updated by Nikolaj Wojtkowiak-Pfänder 3 months ago

Can confirm: Problem still exists in 9.5.5 :(

#52 Updated by David Bruchmann 2 months ago

I remarked not all cases are covered by my patches.
Some TypoScript Objects might missing in output and some extbase extensions too.

#53 Updated by Gerrit Code Review about 2 months ago

  • Status changed from New to Under Review

Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

#54 Updated by Gerrit Code Review about 2 months ago

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

#55 Updated by Markus Opahle about 2 months ago

The new patch only works if you call the page via id query param (?id=...). If you use the page path, the pagerouter will ignore the fallbackchain for all types but "fallback". See https://github.com/TYPO3/TYPO3.CMS/blob/9.5/typo3/sysext/core/Classes/Routing/PageRouter.php#L122

#56 Updated by Gerrit Code Review about 2 months ago

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

#57 Updated by Gerrit Code Review about 2 months ago

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

#58 Updated by Benni Mack about 2 months ago

  • Related to Feature #88137: Create multi-step fallback for content and arbitrary records added

#59 Updated by Gerrit Code Review about 2 months ago

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

#60 Updated by Gerrit Code Review about 1 month ago

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60456

#61 Updated by Benni Mack about 1 month ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

#62 Updated by Benni Mack 20 days ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF