TYPO3 Core

Maybe has this been already solved with https://review.typo3.org/58698?

I already tried this and also this one: https://review.typo3.org/#/c/58612/

But my issue stays the same. Over that there seems to be the issue that CE's with lang = -1 (all Languages) don't get rendered in default language.

Are really only 2 of us having these issues? I would appreciate to get some kind of feedback.

Small update from here: Just updated to latest 9.5.1 with the same behaviour.

Solution for language fallback with an existing translated page using site-configuration and config.sys_language_overlay

config.sys_language_overlay = 1

change in \typo3\sysext\frontend\Classes\Controller\TypoScriptFrontendController.php
line 2272

$languageAspect = LanguageAspectFactory::createFromSiteLanguage($siteLanguage);

$languageAspect = LanguageAspectFactory::createFromSiteLanguage($siteLanguage, $this->config['config'] ?? []);

and in \typo3\v9\typo3\sysext\core\Classes\Context\LanguageAspectFactory.php
replace the function starting on line 100

public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect

public static function createFromSiteLanguage(SiteLanguage $language, array $config): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
        } else {
            $fallbackOrder = [0];
        }

        switch ((string)($config['sys_language_overlay'] ?? '')) {
            case '1':
                $overlayType = LanguageAspect::OVERLAYS_MIXED;
                break;
            case '0':
                $overlayType = LanguageAspect::OVERLAYS_OFF;
                break;
            case 'hideNonTranslated':
            default:
                $overlayType = LanguageAspect::OVERLAYS_ON_WITH_FLOATING;
        }
        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, $overlayType, $fallbackOrder);
    }

that way I got the fallback back.
It would be nice to set the overlay type in the site-configuration module, so that config.sys_language_overlay is not needed anymore

Thanks for this. Works as expected but breaks routingEnhancers at the same time:

Too few arguments to function TYPO3\CMS\Core\Context\LanguageAspectFactory::createFromSiteLanguage(), 1 passed in .../typo3/sysext/core/Classes/Routing/Aspect/PersistedAliasMapper.php on line 261 and exactly 2 expected
in .../typo3/sysext/core/Classes/Context/LanguageAspectFactory.php line 100

     *
     * @param SiteLanguage $language
     * @return LanguageAspect
     */
    public static function createFromSiteLanguage(SiteLanguage $language, array $config): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {

Maybe i will look deeper into this later (if there's time).

I now had the chance to looked deeper into this.

However it seems the content fallback was not respected for the routing-initiative.
In fact it's not possible to setup the setting sys_language_overlay is not respected in the LanguageAspectFactory if you use site-configuration, cause it's not part of the array $language.

I solved this temporarily now for me by adding this into the existing if statements of createFromSiteLanguage. BUt it would be great if this would take place in a more usable way for everyone.

Original function:

 public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
        } else {
            $fallbackOrder = [0];
        }

        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, LanguageAspect::OVERLAYS_ON_WITH_FLOATING, $fallbackOrder);
    }

public static function createFromSiteLanguage(SiteLanguage $language): LanguageAspect
    {
        $languageId = $language->getLanguageId();
        $fallbackType = $language->getFallbackType();
        if ($fallbackType === 'fallback') {
            $fallbackOrder = $language->getFallbackLanguageIds();
            $fallbackOrder[] = 'pageNotFound';
            $overlayType = LanguageAspect::OVERLAYS_MIXED;
        } elseif ($fallbackType === 'strict') {
            $fallbackOrder = [];
            $overlayType = LanguageAspect::OVERLAYS_ON_WITH_FLOATING;
        } else {
            $fallbackOrder = [0];
        }

        return GeneralUtility::makeInstance(LanguageAspect::class, $languageId, $languageId, $overlayType, $fallbackOrder);
    }

Patch set 10 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 11 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 12 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Just found another bug with a colleague of mine while testing it. Here's how to reproduce it:

- create a 2nd Site configuration with fallback to default language
- have a page with both languages
- add a new content element on default language and a new (not translated) content element for the 2nd language

expected results on the translated page(multiple possibilities):
a) only the new, free mode content element is shown
b) the new, free mode content element + the fallback elements are shown

actual result on the translated page:
only the fallback content element is shown, the free mode element doesn't appear at all

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59344

I just commented this in the patch review here: https://review.typo3.org/59344 but just to sum it up in the issue comments here:

In the patch review I tested the following:

fallbackType: ‘strict’

connected mode: works (only translated records are shown)
free mode: works (only copied records or records which only exists in current language are shown)

fallbackType: ‘fallback’

connected mode: works (non-translated records show in default language + all translated records are shown)
free mode: works (default language records, copied records and records which only exists in current language are all shown)

This is all good, however config.sys_language_overlay seems to not work, at least changing between "0,1,hideNonTranslated" makes no difference.

The patch only works for content elements on a localized page.
When the page has not been localized (yet), you will get an 404 error.
Example: First language is German, second language is English (with fallback to German). In frontend you're on a page with an English translation. There's a (sub-)menu linked to pages which do not have any English translation at all. Clicking one of the links, you will get 404 instead of the German fallback page.

This issue is related to https://review.typo3.org/#/c/59488, where the issue is well explained. Unfortunately the bugfix provided there, doesn't work.

@Wolfgang Podbregar Kleinbach I submitted a Patch for that 404 issue https://review.typo3.org/#/c/59675/ as part of this ticket: https://forge.typo3.org/issues/86595

@Kevin no-lastname-given Ditscheid, your patch works on my installation (9.5.4). Thanks!

Patch set 13 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 14 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 15 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 16 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 17 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 18 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 19 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 20 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/59676

Patch set 22 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

Patch set 21 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59344

Having the following configuration my issue regrettable is not fixed by the latest patch (21):

languages:
  -
    title: Deutsch
    enabled: true
    languageId: '0'
    base: /
    typo3Language: de
    locale: de_CH.UTF-8
    iso-639-1: de
    navigationTitle: Deutsch
    hreflang: de-CH
    direction: ltr
    flag: de
  -
    title: Français
    enabled: true
    languageId: '1'
    base: 'https://french.domain.com/'
    typo3Language: fr
    locale: fr_FR.UTF-8
    iso-639-1: fr
    navigationTitle: Français
    hreflang: fr-FR
    direction: ltr
    fallbackType: strict
    flag: fr
  -
    title: Italiano
    enabled: true
    languageId: '2'
    base: 'https://italian.domain.com/'
    typo3Language: it
    locale: it_IT.UTF-8
    iso-639-1: it
    navigationTitle: Italiano
    hreflang: it-IT
    direction: ltr
    fallbackType: fallback
    fallbacks: '1'
    flag: it

array(4 items)
   0 => 2 (integer)
   1 => 2 (integer)
   2 => 'mixed' (5 chars)
   3 => array(2 items)
      0 => '1' (1 chars)
      1 => 'pageNotFound' (12 chars)

Hello,

there is another bug concerning default language and setting content elements to "all languages". Is this bugfix also fixing this problem https://forge.typo3.org/issues/87093?next_issue_id=87092&prev_issue_id=87094?

Martin

Having the following configuration my issue regrettable is not fixed by the latest patch (21):

languages:
  -
    title: Deutsch
    enabled: true
    languageId: '0'
    base: /
    typo3Language: de
    locale: de_CH.UTF-8
    iso-639-1: de
    navigationTitle: Deutsch
    hreflang: de-CH
    direction: ltr
    flag: de
  -
    title: Français
    enabled: true
    languageId: '1'
    base: 'https://french.domain.com/'
    typo3Language: fr
    locale: fr_FR.UTF-8
    iso-639-1: fr
    navigationTitle: Français
    hreflang: fr-FR
    direction: ltr
    fallbackType: strict
    flag: fr
  -
    title: Italiano
    enabled: true
    languageId: '2'
    base: 'https://italian.domain.com/'
    typo3Language: it
    locale: it_IT.UTF-8
    iso-639-1: it
    navigationTitle: Italiano
    hreflang: it-IT
    direction: ltr
    fallbackType: fallback
    fallbacks: '1'
    flag: it

array(4 items)
   0 => 2 (integer)
   1 => 2 (integer)
   2 => 'mixed' (5 chars)
   3 => array(2 items)
      0 => '1' (1 chars)
      1 => 'pageNotFound' (12 chars)

return GeneralUtility::makeInstance(LanguageAspect::class, 2, 2, 'mixed', [1,'pageNotFound']);

In FE the fallback for Italian [2] is realized to German [0] instead of French [1].

Beside that I think the comment-line "* Site Languages always run in floating mode" is not 100% accurate.
Another related comment-line in class LanguageAspect is quite confusing:

public const OVERLAYS_ON_WITH_FLOATING = 'includeFloating';    // "hideNonTranslated" + records that are only available in polish

I've fixed method PageRepository::getRecordOverlay and the languageOverlay for most records is working fine with it.
Part of the fix is the line

$languageChain = $sys_language_content . (count($fallbackChain) ? ',' . $fallbackChain[0] : '');

Other content-elements, including an individual extension are 'translated' correct.

One issue I discovered is that by splitting the whole query in "$query" and "$olrow", the DB-result is returned correct, if I chain all together its wrong.

Here is the fix:

class PageRepository implements LoggerAwareInterface
{

[...]

    /**
     * Master helper method to overlay a record to a language.
     *
     * Be aware that for pages the languageId is taken, and for all other records the contentId.
     * This might change through a feature switch in the future.
     *
     * @param string $table the name of the table, should be a TCA table with localization enabled
     * @param array $row the current (full-fletched) record.
     * @return array|null
     */
    public function getLanguageOverlay(string $table, array $row)
    {
        // table is not localizable, so return directly
        if (!isset($GLOBALS['TCA'][$table]['ctrl']['languageField'])) {
            return $row;
        }
        try {
            /** @var LanguageAspect $languageAspect */
            $languageAspect = $this->context->getAspect('language');
            if ($languageAspect->doOverlays()) {
                if ($table === 'pages') {
                    return $this->getPageOverlay($row, $languageAspect->getId());
                }
                return $this->getRecordOverlay(
                    $table,
                    $row,
                    $languageAspect->getContentId(),
                    $languageAspect->getOverlayType() === $languageAspect::OVERLAYS_MIXED ? '1' : 'hideNonTranslated',
                    $languageAspect->getFallbackChain()
                );
            }
        } catch (AspectNotFoundException $e) {
            // no overlays
        }
        return $row;
    }

[...]

    /**
     * Creates language-overlay for records in general (where translation is found
     * in records from the same table)
     *
     * @param string $table Table name
     * @param array $row Record to overlay. Must contain uid, pid and $table]['ctrl']['languageField']
     * @param int $sys_language_content Pointer to the sys_language uid for content on the site.
     * @param string $OLmode Overlay mode. If "hideNonTranslated" then records without translation will not be returned  un-translated but unset (and return value is FALSE)
     * @param array fallbackChain for translations
     * @throws \UnexpectedValueException
     * @return mixed Returns the input record, possibly overlaid with a translation.  But if $OLmode is "hideNonTranslated" then it will return FALSE if no translation is found.
     */
    public function getRecordOverlay($table, $row, $sys_language_content, $OLmode = '', $fallbackChain=[])
    {
        foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_page.php']['getRecordOverlay'] ?? [] as $className) {
            $hookObject = GeneralUtility::makeInstance($className);
            if (!$hookObject instanceof PageRepositoryGetRecordOverlayHookInterface) {
                throw new \UnexpectedValueException($className . ' must implement interface ' . PageRepositoryGetRecordOverlayHookInterface::class, 1269881658);
            }
            $hookObject->getRecordOverlay_preProcess($table, $row, $sys_language_content, $OLmode, $this);
        }

        $tableControl = $GLOBALS['TCA'][$table]['ctrl'] ?? [];

        if (!empty($tableControl['languageField'])
            // Return record for ALL languages untouched
            // TODO: Fix call stack to prevent this situation in the first place
            && (int)$row[$tableControl['languageField']] !== -1
            && !empty($tableControl['transOrigPointerField'])
            && $row['uid'] > 0
            && ($row['pid'] > 0 || in_array($tableControl['rootLevel'] ?? false, [true, 1, -1], true))) {
            // Will try to overlay a record only if the sys_language_content value is larger than zero.
            if ($sys_language_content > 0) {
                // Must be default language, otherwise no overlaying
                if ((int)$row[$tableControl['languageField']] === 0) {
                    // Select overlay record:
                    $queryBuilder = GeneralUtility::makeInstance(ConnectionPool::class)
                        ->getQueryBuilderForTable($table);
                    $queryBuilder->setRestrictions(
                        GeneralUtility::makeInstance(FrontendRestrictionContainer::class)
                    );
                    $languageChain = $sys_language_content . (count($fallbackChain) ? ',' . $fallbackChain[0] : '');
                    $query = $queryBuilder->select('*')
                        ->from($table)
                        ->where(
                            $queryBuilder->expr()->eq(
                                'pid',
                                $queryBuilder->createNamedParameter($row['pid'], \PDO::PARAM_INT)
                            ),
                            $queryBuilder->expr()->in(
                                $tableControl['languageField'],
                                $queryBuilder->createNamedParameter(GeneralUtility::intExplode(',', $languageChain), Connection::PARAM_INT_ARRAY)
                            ),
                            $queryBuilder->expr()->eq(
                                $tableControl['transOrigPointerField'],
                                $queryBuilder->createNamedParameter($row['uid'], \PDO::PARAM_INT)
                            )
                        )
                        ->add('orderBy', 'FIELD(' . $tableControl['languageField'] . ', ' . $languageChain . ')', true)
                        ->setMaxResults(1);
                    $olrow = $query->execute()
                        ->fetch();

                    $this->versionOL($table, $olrow);
                    // Merge record content by traversing all fields:
                    if (is_array($olrow)) {
                        if (isset($olrow['_ORIG_uid'])) {
                            $row['_ORIG_uid'] = $olrow['_ORIG_uid'];
                        }
                        if (isset($olrow['_ORIG_pid'])) {
                            $row['_ORIG_pid'] = $olrow['_ORIG_pid'];
                        }
                        foreach ($row as $fN => $fV) {
                            if ($fN !== 'uid' && $fN !== 'pid' && isset($olrow[$fN])) {
                                $row[$fN] = $olrow[$fN];
                            } elseif ($fN === 'uid') {
                                $row['_LOCALIZED_UID'] = $olrow['uid'];
                            }
                        }
                    } elseif ($OLmode === 'hideNonTranslated' && (int)$row[$tableControl['languageField']] === 0) {
                        // Unset, if non-translated records should be hidden. ONLY done if the source
                        // record really is default language and not [All] in which case it is allowed.
                        unset($row);
                    }
                } elseif ($sys_language_content != $row[$tableControl['languageField']]) {
                    unset($row);
                }
            } else {
                // When default language is displayed, we never want to return a record carrying
                // another language!
                if ($row[$tableControl['languageField']] > 0) {
                    unset($row);
                }
            }
        }

        foreach ($GLOBALS['TYPO3_CONF_VARS']['SC_OPTIONS']['t3lib/class.t3lib_page.php']['getRecordOverlay'] ?? [] as $className) {
            $hookObject = GeneralUtility::makeInstance($className);
            if (!$hookObject instanceof PageRepositoryGetRecordOverlayHookInterface) {
                throw new \UnexpectedValueException($className . ' must implement interface ' . PageRepositoryGetRecordOverlayHookInterface::class, 1269881659);
            }
            $hookObject->getRecordOverlay_postProcess($table, $row, $sys_language_content, $OLmode, $this);
        }

        return $row;
    }

[...]
}

EDIT:
Not yet mentioned clearly enough is the behavior that it seems that fallback to the main-language is always done if a fallback is defined (it doesn't matter which one).
That also means that any posts are shown in the original language even they had to be hidden based on the fallback-settings.
Additionally that means that the simple case of Carsten Hülsmann never covers many issues, especially if all records are translated.
Based on my example I posted before that means that a post which is neither translated to French, nor to Italian, shouldn't be shown at all on the Italian page but is shown in German.
The essence of my settings is that German is main-language, French has no fallback and is therefore strict, Italian has fallback to French only.

I'm quite sure that there still exist cases which might not be covered by my verification. I still will post here a change for extbase which is not covered by my posted code.

Patch set 23 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

Patch set 24 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

Patch set 25 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

Patch set 26 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/59676

The problem still exists with TYPO3 9.5.5.
To fix it, you just can apply the changes, which Carsten Hülsmann describes in his note above: https://forge.typo3.org/issues/86762#note-8.

Can confirm: Problem still exists in 9.5.5 :(

I remarked not all cases are covered by my patches.
Some TypoScript Objects might missing in output and some extbase extensions too.

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

The new patch only works if you call the page via id query param (?id=...). If you use the page path, the pagerouter will ignore the fallbackchain for all types but "fallback". See https://github.com/TYPO3/TYPO3.CMS/blob/9.5/typo3/sysext/core/Classes/Routing/PageRouter.php#L122

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60367

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/60456