Bug #88853

Updating relations of translated domain objects does not work if they already exist

Added by Anton Fries about 2 months ago. Updated about 2 months ago.

Status:
Needs Feedback
Priority:
Should have
Category:
Extbase + l10n
Target version:
Start date:
2019-07-29
Due date:
% Done:

0%

TYPO3 Version:
9
PHP Version:
7.2
Tags:
extbase, translation, relation, sql-error
Complexity:
medium
Is Regression:
Sprint Focus:

Description

Hey,

for my product import i must to sync relations of translated domain object models to every translated record. TYPO3 does this already right if i click on the "Save"-Button in the backend, because in my TCA configuration its set to "l10n_exclude". But manually saving does not work for > 2500 products on a regular basis.

It works for the first time to insert m:n-relations, but for every next try it tries to insert the value, which of course already exists instead of checking for an already existing relation. Is it possible to quickly fix this or just ignore the sql error and continue with my import? Ignoring would be fine, if i only add relations instead of updating them. Exception: TYPO3\CMS\Extbase\Persistence\Generic\Storage\Exception\SqlErrorException ]
Duplicate entry '2636-13674' for key 'PRIMARY'.

My current solution is to just delete all relations via sql before my import starts:
DELETE FROM tx_gwproductcatalog_product_category_mm WHERE uid_local IN (SELECT uid FROM tx_gwproductcatalog_domain_model_product WHERE sys_language_uid != 0);

I attached the error log calling this from a backend module.

gwcatalog.zip (26.4 KB) Anton Fries, 2019-08-06 10:07

History

#1 Updated by Tymoteusz Motylewski about 2 months ago

Hi
Please describe what you're doing. Is the import using DataHandler to import and sync data (it should)? Or are you using extbase for it.
Can you paste a code fragment you use to import/sync data?
Without this info it's not possible to give any hints.

#2 Updated by Tymoteusz Motylewski about 2 months ago

  • Status changed from New to Needs Feedback

#3 Updated by Anton Fries about 2 months ago

Hey,

im using exclusively Extbase for the whole import.

    /**
     * @param ObjectStorage|AbstractEntity[] $objectList
     * @param int $language
     * @return ObjectStorage|AbstractEntity[]
     */
    protected function getTranslatedEntityList(ObjectStorage $objectList, int $language): ObjectStorage
    {
        $translatedList = new ObjectStorage();
        foreach ($objectList as $object) {
            $translatedObject = $this->translationService->translate($object, $language);
            $translatedList->attach($translatedObject);
        }
        return $translatedList;
    }

    public function syncProductTranslations(int $languageUid)
    {
        /** @var Product $product */
        foreach ($this->productRepository->findByVehicle(false) as $product) {
            $translatedProduct = $this->translationService->translate($product, $languageUid);
            $translatedProduct->setCategoryList($this->getTranslatedEntityList($product->getCategoryList(), $languageUid));
            $this->productRepository->update($translatedProduct);
            $this->productRepository->persistAll();
        }
    }

My translate-method is giving the already translated records, which could be translated with the standard typo3 backend.
The method can also create translations, but because they already exist in my case, there are no error causes in this part.

    /**
     * @param AbstractEntity $entity
     * @param int $targetLanguageUid
     * @return AbstractEntity
     */
    public function translate(AbstractEntity $entity, int $targetLanguageUid): AbstractEntity
    {
        /** @var AbstractRepository $repository */
        $repository = $this->objectManager->get(ClassNamingUtility::translateModelNameToRepositoryName(get_class($entity)));
        /** @var AbstractEntity $translatedEntity */
        $translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
        if (empty($translatedEntity)) {
            $table = $this->dataMapper->convertClassNameToTableName(get_class($entity));
            $this->dataHandler->start([], [
                $table => [
                    $entity->getUid() => ['localize' => $targetLanguageUid]
                ]
            ], null);
            $this->dataHandler->process_cmdmap();
            $translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
        }
        return $translatedEntity;
    }

Related find-method:
    /**
     * @param int $uid UID des Datensatzes in der Standardsprache
     * @param int $languageUid Sprache (sys_language_uid)
     * @return object
     */
    public function findTranslationByUid($uid, $languageUid)
    {
        $query = $this->createQuery();
        $query->getQuerySettings()->setRespectSysLanguage(false);
        $query->matching(
            $query->logicalAnd(
                $query->equals('l10n_parent', $uid),
                $query->equals('sys_language_uid', $languageUid)
            )
        );
        return $query->setLimit(1)->execute()->getFirst();
    }

My data model is clicked together with the extension builder.

If u have further questions or need more code examples, feel free to comment here.

#4 Updated by Tymoteusz Motylewski about 2 months ago

I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.

#5 Updated by Anton Fries about 2 months ago

Tymoteusz Motylewski wrote:

I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.

Yes im don't using datahandler and would not rewrite my import to use datahandler only because an bug in extbase.
You don't need to know the whole process. I already gave all the information about the relevant part, where the bug occurs. I attached there also an error log of the exception.

1. Create a entity in the default language in TYPO3 backend.
2. Translate the entity in a language in TYPO3 backend.
3. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Works.
4. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Doesn't work because there are already relations.

Error points: sysext/extbase/Classes/Persistence/Generic/Backend.php line 537
$this->insertRelationInRelationtable($object, $parentObject, $parentPropertyName, $sortingPosition);
--> Its just an insert without checking if there is already a relation in the relation table.

sysext/extbase/Classes/Persistence/Generic/Backend.php line 780
$res = $this->storageBackend->addRow($relationTableName, $row, true);

sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php line 151
throw new SqlErrorException($e->getPrevious()->getMessage(), 1470230766);

If u want, i can build a minimalistic example extension for you.

#6 Updated by Tymoteusz Motylewski about 2 months ago

Well, you're already mixing extbase and datahandler :)

Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.

Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.

Please create an minimal extension to reproduce the issue.
Thanks

#7 Updated by Anton Fries about 2 months ago

  • File gwcatalog.zip added
  • File DEV_ Galileo Webagentur [TYPO3 CMS 9.5.8].html added

Tymoteusz Motylewski wrote:

Well, you're already mixing extbase and datahandler :)

Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.

Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.

Please create an minimal extension to reproduce the issue.
Thanks

Okay got a minimalistic extension running.
You need to activate the extension and applying the database wizard.
Theres a backend module called "Katalogimport" with one action.
1. First time clicking the action it ensures that there is test data and translated test data. It adds an relation between a translated product and a translated category like TYPO3 backend would do it.
2. Second time clicking the action it tries again to create/overwrite the relation between the translated product and the translated category (could also be another category) where it throws an error.

#8 Updated by Anton Fries about 2 months ago

Anton Fries wrote:

Tymoteusz Motylewski wrote:

Well, you're already mixing extbase and datahandler :)

Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.

Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.

Please create an minimal extension to reproduce the issue.
Thanks

Okay got a minimalistic extension running.
You need to activate the extension and applying the database wizard.
Theres a backend module called "Katalogimport" with one action.
1. First time clicking the action it ensures that there is test data and translated test data. It adds an relation between a translated product and a translated category like TYPO3 backend would do it.
2. Second time clicking the action it tries again to create/overwrite the relation between the translated product and the translated category (could also be another category) where it throws an error.

Forgot that u dont have our gwbase extension. Updated so the findTranslationByUid is directly in the AbstractRepository and no dependency injection from other extensions

#9 Updated by Anton Fries about 1 month ago

  • File deleted (gwcatalog.zip)

#10 Updated by Anton Fries about 1 month ago

  • File deleted (DEV_ Kunde [TYPO3 CMS 9.5.8].html)

#11 Updated by Anton Fries about 1 month ago

  • File deleted (DEV_ Galileo Webagentur [TYPO3 CMS 9.5.8].html)

Also available in: Atom PDF