TYPO3 Core

Hi
Please describe what you're doing. Is the import using DataHandler to import and sync data (it should)? Or are you using extbase for it.
Can you paste a code fragment you use to import/sync data?
Without this info it's not possible to give any hints.

Hey,

im using exclusively Extbase for the whole import.

    /**
     * @param ObjectStorage|AbstractEntity[] $objectList
     * @param int $language
     * @return ObjectStorage|AbstractEntity[]
     */
    protected function getTranslatedEntityList(ObjectStorage $objectList, int $language): ObjectStorage
    {
        $translatedList = new ObjectStorage();
        foreach ($objectList as $object) {
            $translatedObject = $this->translationService->translate($object, $language);
            $translatedList->attach($translatedObject);
        }
        return $translatedList;
    }

    public function syncProductTranslations(int $languageUid)
    {
        /** @var Product $product */
        foreach ($this->productRepository->findByVehicle(false) as $product) {
            $translatedProduct = $this->translationService->translate($product, $languageUid);
            $translatedProduct->setCategoryList($this->getTranslatedEntityList($product->getCategoryList(), $languageUid));
            $this->productRepository->update($translatedProduct);
            $this->productRepository->persistAll();
        }
    }

My translate-method is giving the already translated records, which could be translated with the standard typo3 backend.
The method can also create translations, but because they already exist in my case, there are no error causes in this part.

    /**
     * @param AbstractEntity $entity
     * @param int $targetLanguageUid
     * @return AbstractEntity
     */
    public function translate(AbstractEntity $entity, int $targetLanguageUid): AbstractEntity
    {
        /** @var AbstractRepository $repository */
        $repository = $this->objectManager->get(ClassNamingUtility::translateModelNameToRepositoryName(get_class($entity)));
        /** @var AbstractEntity $translatedEntity */
        $translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
        if (empty($translatedEntity)) {
            $table = $this->dataMapper->convertClassNameToTableName(get_class($entity));
            $this->dataHandler->start([], [
                $table => [
                    $entity->getUid() => ['localize' => $targetLanguageUid]
                ]
            ], null);
            $this->dataHandler->process_cmdmap();
            $translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
        }
        return $translatedEntity;
    }

    /**
     * @param int $uid UID des Datensatzes in der Standardsprache
     * @param int $languageUid Sprache (sys_language_uid)
     * @return object
     */
    public function findTranslationByUid($uid, $languageUid)
    {
        $query = $this->createQuery();
        $query->getQuerySettings()->setRespectSysLanguage(false);
        $query->matching(
            $query->logicalAnd(
                $query->equals('l10n_parent', $uid),
                $query->equals('sys_language_uid', $languageUid)
            )
        );
        return $query->setLimit(1)->execute()->getFirst();
    }

My data model is clicked together with the extension builder.

If u have further questions or need more code examples, feel free to comment here.

I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.

Tymoteusz Motylewski wrote:

I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.

Yes im don't using datahandler and would not rewrite my import to use datahandler only because an bug in extbase.
You don't need to know the whole process. I already gave all the information about the relevant part, where the bug occurs. I attached there also an error log of the exception.

1. Create a entity in the default language in TYPO3 backend.
2. Translate the entity in a language in TYPO3 backend.
3. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Works.
4. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Doesn't work because there are already relations.

Error points: sysext/extbase/Classes/Persistence/Generic/Backend.php line 537
$this->insertRelationInRelationtable($object, $parentObject, $parentPropertyName, $sortingPosition);
--> Its just an insert without checking if there is already a relation in the relation table.

sysext/extbase/Classes/Persistence/Generic/Backend.php line 780
$res = $this->storageBackend->addRow($relationTableName, $row, true);

sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php line 151
throw new SqlErrorException($e->getPrevious()->getMessage(), 1470230766);

If u want, i can build a minimalistic example extension for you.

Well, you're already mixing extbase and datahandler :)

Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.

Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.

Please create an minimal extension to reproduce the issue.
Thanks

This is still an unresolved issue.
A workaround is to remove all relations with a query before persisting.

Gion Koch wrote:

This is still an unresolved issue.
A workaround is to remove all relations with a query before persisting.

Ye thats how we solved our problem for a big product import every night. But managing relations with translated domain objects doesn't work with extbase for domain models that have 'l10nmode' => 'exclude' in the TCA configuration. It creates a relation in the database, but not from the original language record in the uid_foreign, but rather from the translated record uid, so its not valid for TYPO3.

Our final solution now was to create all relations in default language and then copy over doctrine all relations to all languages (also way more performant and optimizable).

It seems that I have the same problem. I debugged it a very long time as there are some very basic questions:
- Which UID exists in _LOCALIZED_UID?
- Which UID will be used for relational columns of translated records? The UID of the related record in default language or the UID of the translated record.

For now I have folowing answers:
- _LOCALIZED_UID = UID of the record in foreign language
- The UID of the record in default language

Please correct me, if I'm wrong.

With this adaption I'm wondered about following lines in Extbase:

if ($value instanceof AbstractDomainObject
    && $value->_hasProperty('_localizedUid')
    && $value->_getProperty('_localizedUid') > 0
) {
    $plainValue = (int)$value->_getProperty('_localizedUid');
} else {
    $plainValue = $this->dataMapper->getPlainValue($value);
}

I don't think that this is correct. Please have a look into a translated tt_content record. These records are connected with page records in default language. IMO this behavior we should use for extbase, too.

Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Hi Stefan,Anton
Thanks for digging into the topic.
The language handling is much better since 9.5, but not perfect.
Please make sure to read this changelog, so you know the concept what is documented and expected right now.
https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.5/Important-82363-MakeExtBaseTranslationHandlingConsistentWithTyposcript.html

Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 10 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 11 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 12 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 13 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 14 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112

Patch set 15 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112