TYPO3 Core

Here some more detailed information:

$logicAnd[] = $query->equals('city.country', $country);
$logicAnd[] = $query->equals('travelType', $travelType);

$travelType and $country is an object of the correct type form their model. The travelType equals statement works in default language and all other languages.

But the joined city.country is not working for any other language than the default language.

This worked in 4.7 and does not work anymore in 6.0.4

I also run into this problem.

Our site has german and english objects. On other language-pages I need the english objects, but the doLanguageAndWorkspaceOverlay only gives me the german. I tried to force the query giving me english objects, but it doesn't work:

if($GLOBALS['TSFE']->sys_language_uid != 0){
  $query->getQuerySettings()->setRespectSysLanguage(FALSE);
  $query->getQuerySettings()->setSysLanguageUid(1);
  $object = $query->matching( $query->equals('l18n_parent',$hotel) )->execute()->getFirst();
  return $object;
}

Johannes

@Felix: Can you drop a line on this one?

$query = $this->createQuery();
$query->getQuerySettings()->setRespectSysLanguage(false);
$query->getQuerySettings()->setRespectStoragePage(false);
$query->getQuerySettings()->setSysLanguageUid(1);
$result = $query->matching(
    $query->logicalAnd(
        $query->equals('sys_language_uid', 1),
        $query->equals('l10n_parent', 1)
    )
)->execute();

works fine for me. Please note that in contrary to Extbase 4.7 the following line is needed since Extbase 6.0 if the requested page language is not set to the desired translation to be fetched (which is uncommon but there might be usecases for that):

$query->getQuerySettings()->setSysLanguageUid(1);

There is still a difference in the handling as Extbase before 6.0 did only perform an overlay on non translated records. Now if a translated record is fetched, Extbase fetches the original language record and performs an overlay with the language id set in the query settings.

This is the related change: https://review.typo3.org/10188

Hello,

here my language setup:

config {
  linkVars = L
  sys_language_uid = 0
  language = de
  locale_all = de_DE.UTF-8
  htmlTag_langKey = de-DE
  sys_language_mode = content_fallback
  sys_language_overlay = hideNonTranslated

}
[globalVar = GP:L = 4]
config.sys_language_uid = 4
config.language = en
config.locale_all = en-GB.UTF-8
config.htmlTag_langKey = en-GB
[global]

This is my usecase:
Model A
-->relation n:1 Model B

I have an admin tool where the fe user should be able to translate records of A and also select their object B. Because of the overlay, even if I set setSysLanguage(4) its for me currently impossible to add B, its only getting related to the sys_language(0).

Regards,
Dominic

Hello

my language-setup:

config {
  htmlTag_langKey = de
  metaCharset = utf-8
  linkVars = l,m
  sys_language_uid = 0
  language = de
  locale_all = de_DE
  [...]
}
# English
[globalVar = GP:l = 1] && [PIDupinRootline = {$pid_tourismus}]
    config.sys_language_uid = 1
    config.language = en
    config.locale_all = english
[global]

# Francais
[globalVar = GP:l = 2] && [PIDupinRootline = {$pid_tourismus}]
    config.sys_language_uid = 2
    config.language = fr
    config.locale_all = fr_FR
[global]

# Dutch
[globalVar = GP:l = 3] && [PIDupinRootline = {$pid_tourismus}]
    config.sys_language_uid = 3
    config.language = nl
    config.locale_all = nl_NL
[global]

Because 75% are only in german, there is no more configuration inside the Main-TS. Inside the subtree with other languages we use TS-Configuration:

TCAdefaults.pages.l18n_cfg = 2

--

@Helmuth: I tried the things in the way you described - for me this not really worked.

In waiting, the bug is resolved, I fix with this :

//for each repository
  public function initializeObject() {
    $this->defaultQuerySettings->setRespectSysLanguage(TRUE);

    $language = $_GET['L'];
    if($language) {
      $this->defaultQuerySettings->setSysLanguageUid($language);
    }
  }

But sys_language_overlay hideNonTranslated not work anymore. It's induced by the bug or it's an other bug ?

In Typo3DbBackend, it not would be judicious to add a condition on the overlay ?

protected function addSysLanguageStatement($tableName, array &$sql, $querySettings) {
...

//this
  if (isset($GLOBALS['TCA'][$tableName]['ctrl']['transOrigPointerField'])
    && $querySettings->getSysLanguageUid() > 0
    && $GLOBALS['TSFE']->sys_language_contentOL != "hideNonTranslated" 
  ) {

//instead
  if (isset($GLOBALS['TCA'][$tableName]['ctrl']['transOrigPointerField'])
    && $querySettings->getSysLanguageUid() > 0
  ) {

//  add this sql clause  :
//  OR (tx_EXT_domain_model_TABLE.sys_language_uid=0
//    AND tx_EXT_domain_model_TABLE.uid NOT IN (
//      SELECT tx_atolactacontenu_domain_model_fiche.l10n_parent
//      FROM tx_EXT_domain_model_TABLE
//      WHERE tx_atolactacontenu_domain_model_fiche.l10n_parent>0
//      AND tx_EXT_domain_model_TABLE.sys_language_uid>0
//      AND tx_EXT_domain_model_TABLE.deleted=0))
  }
}

Hello,

We developed an extension to generate a language menu (http://typo3.org/extensions/repository/view/grds_language_menu).
For this we need to know, which languages the current page is translated to, so that we can add a css class "nonTranslated" to the link if the current page isn't available in that language.
We did this by mapping the table "pages_language_overlay" to a class named "Translation". Then we could get all translations for a page by pid from our "TranslationRepository", setting "respectSysLanguage" to false.

This isn't working anymore because of the bug mentioned above.

Although in our (arguably very uncommon) use case the problem goes further. Extbase tries to load the original "pages_language_overlay" record with a sys_language_id of 0.
But obviously we get an empty result, because the original resides in the table "pages" and not in the table "pages_language_overlay".

To make things worse,

count($translations)

Maybe this is a problem of typo3/extbase or maybe we need to change our approach to get translations of a page. Your help would be appreciated.

Regards,
Marc

@Felix, Anja & Alexander:
What's up with this issue?! Should be fixed on #ecs13ws but wasn't! Will it be fixed sometime or not?

best regards
Johannes

Marc Steiner wrote:

Hello,

We developed an extension to generate a language menu (http://typo3.org/extensions/repository/view/grds_language_menu).
For this we need to know, which languages the current page is translated to, so that we can add a css class "nonTranslated" to the link if the current page isn't available in that language.
We did this by mapping the table "pages_language_overlay" to a class named "Translation". Then we could get all translations for a page by pid from our "TranslationRepository", setting "respectSysLanguage" to false.

This isn't working anymore because of the bug mentioned above.

Although in our (arguably very uncommon) use case the problem goes further. Extbase tries to load the original "pages_language_overlay" record with a sys_language_id of 0.
But obviously we get an empty result, because the original resides in the table "pages" and not in the table "pages_language_overlay".

Please test this change: https://review.typo3.org/#/c/28333/

Regards,
Stanislas

Did anyone found a solution of hint?

I tried now with 6.2.4, but it did not work.
Trying to display all translations of one item on a page. Using "statement" sux, but works:

$query->statement(' SELECT * FROM tx_news_domain_model_news WHERE (uid="'.$uid.'" OR l10n_parent="'.$uid.'") AND uid > 0 AND hidden=0 AND deleted=0 AND (starttime=0 OR starttime<='.time().' ) AND ( endtime=0 OR endtime>='.time().' ) ORDER BY sys_language_uid ASC');

If I try

$query->getQuerySettings()->setRespectSysLanguage(FALSE);
$query->getQuerySettings()->setRespectStoragePage(FALSE);
$query->matching(
  $query->logicalOr(
    $query->equals('uid',$uid),
    $query->equals('l10n_parent',$uid)
  )
);

It's sad, to be forced for using "statement"-queries :-(

Johannes

argh

So this won't be fixed in 6.2?

Is there at least a workaround for the following:

        $result = $query->statement('
                    SELECT
                            tx_myext_domain_model_category.*
                    FROM
                            tx_myext_domain_model_category
                    WHERE
                            tx_myext_domain_model_category.l10n_parent = ' . (int) $uid . ' AND
                            tx_myext_domain_model_category.sys_language_uid = ' . (int) $sys_language_uid .
                            \TYPO3\CMS\Backend\Utility\BackendUtility::BEenableFields('tx_myext_domain_model_category') .
                            \TYPO3\CMS\Backend\Utility\BackendUtility::deleteClause('tx_myext_domain_model_category')
        )->execute();

None of the following options seem to change the output:
$query->getQuerySettings()->setRespectSysLanguage(FALSE);
$query->getQuerySettings()->setLanguageOverlayMode(FALSE);
$query->getQuerySettings()->setLanguageMode('ignore');
$query->getQuerySettings()->setLanguageUid(1);

I always get the default language record instead.
If i change the query to $query->getQuerySettings()->setReturnRawQueryResult(TRUE); i will get the expected result.

Background:
We are using two sysFolders (EN,DE) both only have records with sys_language_uid=0. We are doing this due to overlay limitations. However related records like this tx_myext_domain_model_category are still translated and always as sys_language_uid=0 attached. Now we would like to fetch this translations in Backend context (CommandController aka Task) but this fails always.

It seems like the record gets correctly translated in

    /**
     * Returns the object data matching the $query.
     *
     * @param QueryInterface $query
     * @return array
     */
    public function getObjectDataByQuery(QueryInterface $query) {
        $statement = $query->getStatement();
        if ($statement instanceof Statement) {
            $rows = $this->getObjectDataByRawQuery($statement);
        } else {
            $rows = $this->getRowsByStatementParts($query);
        }

        $rows = $this->doLanguageAndWorkspaceOverlay($query->getSource(), $rows, $query->getQuerySettings());
        return $rows;
    }

    /**
     * Maps a single row on an object of the given class
     *
     * @param string $className The name of the target class
     * @param array $row A single array with field_name => value pairs
     * @return object An object of the given class
     */
    protected function mapSingleRow($className, array $row) {
        if ($this->identityMap->hasIdentifier($row['uid'], $className)) {
            $object = $this->identityMap->getObjectByIdentifier($row['uid'], $className);
        } else {
            $object = $this->createEmptyObject($className);
            $this->identityMap->registerObject($object, $row['uid']);
            $this->thawProperties($object, $row);
            $object->_memorizeCleanState();
            $this->persistenceSession->registerReconstitutedEntity($object);
        }
        return $object;
    }

Vitoandre is right: The function "mapSingleRow" uses $row['uid'] to look up an object in the identityMap. So the original record is found, not the one with overlayed data, which has the same uid, but overlayed data. The difference is, that the overlay record row has a key "_LOCALIZED_UID" which should be used at this point if set.

We implemented a custom DataMapper class which overwrites the "mapSingleRow" method.

1. HACK extbase in your extension (in your ext_localconf.php) to register a "CustomQueryResult" class:

// The code below is NO PUBLIC API!
/** @var $extbaseObjectContainer \TYPO3\CMS\Extbase\Object\Container\Container */
$extbaseObjectContainer = \TYPO3\CMS\Core\Utility\GeneralUtility::makeInstance('TYPO3\\CMS\\Extbase\\Object\\Container\\Container');

$extbaseObjectContainer->registerImplementation('TYPO3\CMS\Extbase\Persistence\QueryResultInterface', 'YOURVENDOR\YOUREXT\Persistence\Storage\CustomQueryResult');
unset($extbaseObjectContainer);

2. Implement a simple CustomQueryResult class:

class CustomQueryResult extends \TYPO3\CMS\Extbase\Persistence\Generic\QueryResult {

    /**
     * @var \YOURVENDOR\YOUREXT\Persistence\Storage\CustomDataMapper
     * @inject
     */
    protected $dataMapper;

}

3. Implement the CustomDataMapper class and overwrite the method "mapSingleRow":

class CustomDataMapper extends \TYPO3\CMS\Extbase\Persistence\Generic\Mapper\DataMapper {

    /**
     * Maps a single row on an object of the given class
     *
     * @param string $className The name of the target class
     * @param array $row A single array with field_name => value pairs
     * @return object An object of the given class
     */
    protected function mapSingleRow($className, array $row) {
        $uid = isset($row['_LOCALIZED_UID']) ? $row['_LOCALIZED_UID'] : $row['uid'];
        if ($this->identityMap->hasIdentifier($uid, $className)) {
            $object = $this->identityMap->getObjectByIdentifier($uid, $className);
        } else {
            $object = $this->createEmptyObject($className);
            $this->identityMap->registerObject($object, $uid);
            $this->thawProperties($object, $row);
            $object->_memorizeCleanState();
            $this->persistenceSession->registerReconstitutedEntity($object);
        }
        return $object;
    }
}

That did the trick for us! But I would like to know if there is another way to handle this problem, besides patching the 6.2.x source.

Dear reporters. Thanks for the input so far. However I fear we must become more specific in this topic to achieve progress.

Please give feedback in the following form:

===================================================

Use case¶

Short description what your use case is for disabling the sysLanguageHandling.
Especially explicitly mention if your use case needs to cover only reading objects or additionally also writing objects

Minimal (but complete) example code¶

Please provide all necessary code (Models, sql, TCA …) which is needed to reproduce the issue. The less code it is, the better. Please only include code or configuration which is mandatory to reproduce the issue

Describe the actual result you see on your system¶

Describe the expected result you want to see¶

====================================================

B. St. wrote:

Vitoandre is right: The function "mapSingleRow" uses $row['uid'] to look up an object in the identityMap. So the original record is found, not the one with overlayed data, which has the same uid, but overlayed data.

The question is: How did the original record end up in the identity map?

can you provide a use case description and example code like specified above?

3. Implement the CustomDataMapper class and overwrite the method "mapSingleRow":

That did the trick for us! But I would like to know if there is another way to handle this problem, besides patching the 6.2.x source.

Question is: What other use cases does such a change break? At first sight it may look reasonable for your case, but changing that, will break other cases.

The only chance to make progress here is to collect use cases, write them down and lastly nail them down in functional tests

Hello Helmut

Like you wish, I will make again a sketch of my use-case.

what we need¶

our site has 4 languages: 0=german, 1=english, 2=french, 3=dutch
we use an extension for showing hotels, all items are available in german and english
on french and dutch pages we like to show the english items, but the page around our hotel-item (menu, etc) should be in french or dutch

this works for LIST with following code

...
if($GLOBALS['TSFE']->sys_language_uid != 0){
            $query->getQuerySettings()->setRespectSysLanguage(FALSE);
            $query->getQuerySettings()->setSysLanguageUid(1);
            $langsearch = $query->logicalAnd($query->in('sys_language_uid',array(1)));
            $constraint[] = $langsearch;
}
...

but this works not for SINGLE: everytime we get the german items

$query = $this->createQuery();
        $query->getQuerySettings()->setRespectStoragePage(FALSE);
        if($GLOBALS['TSFE']->sys_language_uid != 0){
            $query->getQuerySettings()->setRespectSysLanguage(FALSE);
            $query->getQuerySettings()->setSysLanguageUid(1);
            $query = $query->matching(
                $query->logicalAnd(
                    $query->equals('t3_origuid',$hotel),
                    $query->in('sys_language_uid',array(1))
                )
            );
        }else{
            $query = $query->matching(
                $query->equals('uid',$hotel)
            );
        }
        return $query->execute()->getFirst(); 

what we need¶

the possibility to force another language inside the query.

what happens¶

if I debug the query it runs fine till doLanguageOverlay overwrites my query by his own and switches back to fallback-language (german).

Hoping my worst english lets you understand what we're looking for. If not, please ask me for more details.

Johannes

hi there

i have currently the same issue.

use-case¶

we build a import-extension which loads products from a external sqlite database and stores them as extbase "product"-objects.
there is a automatic import each day. instead of deleting all the objects with every new import, i would like to update them.
while working with the standard language, there is no problem. i use a method like "findOneByProductNr", get the entry, set the desired values and finally update the object.
if i want to get and update the translated objects i run into the problem which is described on this page.

example code¶

ProductRepository.php

public function findOneByProductNrAndLanguage($productNr, $lang) {
        $query = $this->createQuery();

        $query->getQuerySettings()->setRespectSysLanguage(FALSE);
        $query->getQuerySettings()->setSysLanguageUid($lang);

        $constraints = array();
        $constraints[] = $query->equals('productNr', $productNr);
        $constraints[] = $query->equals('sysLanguageUid', $lang);

        if (!empty($constraints)) {
            $query->matching($query->logicalAnd($constraints));
        }

        return $query->execute()->getFirst();

    }

result¶

the method above only returns the object in the standard language.

expected result¶

if i call the method for example with findOneByProductNrAndLanguage(1,2) i want to get the object with productNr 1 in the language 2 (not 0)

edit to be clear: i work in a backend module and i dont have problems to write the objects. i create the translated objects in my Service Class like this:

if ($importLang === $standardLang) {
    $product->setL10nParent(0);
} else {
    $product->setL10nParent($config['parent']->getUid());
    $product->setSysLanguageUid($importLang);
}

i can't say if im able to update the object, because i never got a translated and persistent one to work with.

Sorry, I cannot report in more detail right now.

But with sys_language_mode = strict the behaviour is like expected. This workaround is ok for us and content_fallback is not necessary, I think.

I found an easy hack workaround for my Extbase repository.

Set

public $languageUid = 0;

in

typo3/sysext/extbase/Classes/Persistence/Generic/Typo3QuerySettings.php

In

typo3_src/typo3/sysext/extbase/Classes/Persistence/Repository.php

I set a static property:

public static $workaround_languageUid = false;

Then I check in createQuery(); if self::$workaround_languageUid isn't false. If so, I set the languageUid in $this->defaultQuerySettings.

/**
     * Returns a query for objects of this repository
     *
     * @return \TYPO3\CMS\Extbase\Persistence\QueryInterface
     * @api
     */
    public function createQuery($language_uid = false) {
        $query = $this->persistenceManager->createQueryForType($this->objectType);
        if ($this->defaultOrderings !== array()) {
            $query->setOrderings($this->defaultOrderings);
        }
        if ($this->defaultQuerySettings !== NULL) {
            /**
             * This static property is a workaround for switching the language uid
             * via the query builder from Extbase.
             *
             * @see https://forge.typo3.org/issues/45873
             */
            if(self::$workaround_languageUid !== false){
                $this->defaultQuerySettings->languageUid = self::$workaround_languageUid;
            }
            $query->setQuerySettings(clone $this->defaultQuerySettings);
        }

        return $query;
    }

In my custom code method I switch \TYPO3\CMS\Extbase\Persistence\Repository::$workaround_languageUid = 2; to whatever it should be. Since I don't need instances from \TYPO3\CMS\Extbase\Persistence\Repository it does the job.

What do you think?

It seems still relevant for 6.2 LTS.

In a backend module a customer wants to list records in default language and in separate lines other languages.
Playing with querySettings seems not to help

An (very ugly) workarround is to overwrite the TCA languageField. That disables the overlay in extbase.
Maybe it helps if someone needs an urgent solution.

public function getLocalizedProjectsFromDefaultProject($project) {
    // Disable extbase language overlay if languageField is empty
    $languageField = $GLOBALS['TCA']['tx_projectlist_projects']['ctrl']['languageField'];
    unset($GLOBALS['TCA']['tx_projectlist_projects']['ctrl']['languageField']);

    $query = $this->createQuery();
    $query->getQuerySettings()->setRespectSysLanguage(FALSE);

    $result = $query->execute();

    // Enforce query and enable extbase language overlay from here
    $result->toArray();
    $GLOBALS['TCA']['tx_projectlist_projects']['ctrl']['languageField'] = $languageField;

    return $result;
}

As Alex Kellner said this issue is still relevant.
I use the following workaround:

        $query->matching( $query->equals( 'uid', $uid ) );

        //https://forge.typo3.org/issues/45873
        //Set transOrigPointerTable skip the record overlay (which always returns the default language entry)
        $GLOBALS['TCA']['tx_news_domain_model_news']['ctrl']['transOrigPointerTable'] = '';

        $result = $query->execute()->getFirst();

Maybe this helps someone.

Thanks all! and especially Christoph Bessei,

setting

$GLOBALS['TCA']['tx_news_domain_model_news']['ctrl']['transOrigPointerTable'] = '';

I used a migrateCommandController to manipulate tt_content records (just once while upgrading to 7.6) and could not manipulate translated tt_content elements because the uid shown was always the uid of default language. In my case i do not care which language the content is in, i just want to set some fields to different values and leave all the rest as is.

Markus Klein wrote:

Not reading the whole story here, is this still an issue on 7.6?

Hello Markus.
Yes, it's still a problem. Like Mathias Schreiber said, translation-handling will be processed and rewritten in TYPO3 9 or 10 :-( .

I found a related problem:

When e.g. creating a query in a repository with

$query = $this->createQuery();
        $query->getQuerySettings()
            ->setRespectStoragePage(true)
            ->setLanguageUid($languageId)
            ->setLanguageMode('strict')
            ->setIgnoreEnableFields(true);

        return $query->execute();

The "->setIgnoreEnableFields(true)" is ignored for fetching the localized record, since in
https://github.com/TYPO3/TYPO3.CMS/blob/master/typo3/sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php#L615

the query settings are not passed to the PageRepository, which in turn will still include the enableFields restriction although it should not, as specified in the querySettings

https://github.com/TYPO3/TYPO3.CMS/blob/master/typo3/sysext/frontend/Classes/Page/PageRepository.php#L542

Which results in no translated records being fetched when they are hidden.

This is an issue in TYPO3 7.6 and TYPO3 8

A workaround for this is temporarily disabling the TCA enableFields and reenabling it.

$tmpEnableFields = $GLOBALS['TCA'][$table]['ctrl']['enablecolumns'];
        unset($GLOBALS['TCA'][$table]['ctrl']['enablecolumns']);

Christoph Bessei wrote:

As Alex Kellner said this issue is still relevant.
I use the following workaround:

[...]

Maybe this helps someone.

As for TYPO3 8 LTS you need to unset transOrigPointerField instead of transOrigPointerTable.

$transOrigPointerField = $GLOBALS['TCA']['tx_news_domain_model_news']['ctrl']['transOrigPointerField'];
$GLOBALS['TCA']['tx_news_domain_model_news']['ctrl']['transOrigPointerField'] = '';

//execute your query

$GLOBALS['TCA']['tx_news_domain_model_news']['ctrl']['transOrigPointerField'] = $transOrigPointerField;

Tymoteusz Motylewski wrote:

Should be resolved with new consistent language handling available since v9

Considering this issue was created 2013 - there should be at least a FIX for 8.7. This is a serious bug.

Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/75103

Patch set 3 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/75103

Patch set 4 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/75103

Patch set 5 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/75103