Bug #88853
openUpdating relations of translated domain objects does not work if they already exist
Added by Anton Fries about 5 years ago. Updated about 1 month ago.
0%
Description
Hey,
for my product import i must to sync relations of translated domain object models to every translated record. TYPO3 does this already right if i click on the "Save"-Button in the backend, because in my TCA configuration its set to "l10n_exclude". But manually saving does not work for > 2500 products on a regular basis.
It works for the first time to insert m:n-relations, but for every next try it tries to insert the value, which of course already exists instead of checking for an already existing relation. Is it possible to quickly fix this or just ignore the sql error and continue with my import? Ignoring would be fine, if i only add relations instead of updating them. Exception: TYPO3\CMS\Extbase\Persistence\Generic\Storage\Exception\SqlErrorException ]
Duplicate entry '2636-13674' for key 'PRIMARY'.
My current solution is to just delete all relations via sql before my import starts:
DELETE FROM tx_gwproductcatalog_product_category_mm WHERE uid_local IN (SELECT uid FROM tx_gwproductcatalog_domain_model_product WHERE sys_language_uid != 0);
I attached the error log calling this from a backend module.
Files
gwcatalog.zip (26.4 KB) gwcatalog.zip | Anton Fries, 2019-08-06 10:07 |
Updated by Tymoteusz Motylewski about 5 years ago
Hi
Please describe what you're doing. Is the import using DataHandler to import and sync data (it should)? Or are you using extbase for it.
Can you paste a code fragment you use to import/sync data?
Without this info it's not possible to give any hints.
Updated by Tymoteusz Motylewski about 5 years ago
- Status changed from New to Needs Feedback
Updated by Anton Fries about 5 years ago
Hey,
im using exclusively Extbase for the whole import.
/**
* @param ObjectStorage|AbstractEntity[] $objectList
* @param int $language
* @return ObjectStorage|AbstractEntity[]
*/
protected function getTranslatedEntityList(ObjectStorage $objectList, int $language): ObjectStorage
{
$translatedList = new ObjectStorage();
foreach ($objectList as $object) {
$translatedObject = $this->translationService->translate($object, $language);
$translatedList->attach($translatedObject);
}
return $translatedList;
}
public function syncProductTranslations(int $languageUid)
{
/** @var Product $product */
foreach ($this->productRepository->findByVehicle(false) as $product) {
$translatedProduct = $this->translationService->translate($product, $languageUid);
$translatedProduct->setCategoryList($this->getTranslatedEntityList($product->getCategoryList(), $languageUid));
$this->productRepository->update($translatedProduct);
$this->productRepository->persistAll();
}
}
My translate-method is giving the already translated records, which could be translated with the standard typo3 backend.
The method can also create translations, but because they already exist in my case, there are no error causes in this part.
/**
* @param AbstractEntity $entity
* @param int $targetLanguageUid
* @return AbstractEntity
*/
public function translate(AbstractEntity $entity, int $targetLanguageUid): AbstractEntity
{
/** @var AbstractRepository $repository */
$repository = $this->objectManager->get(ClassNamingUtility::translateModelNameToRepositoryName(get_class($entity)));
/** @var AbstractEntity $translatedEntity */
$translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
if (empty($translatedEntity)) {
$table = $this->dataMapper->convertClassNameToTableName(get_class($entity));
$this->dataHandler->start([], [
$table => [
$entity->getUid() => ['localize' => $targetLanguageUid]
]
], null);
$this->dataHandler->process_cmdmap();
$translatedEntity = $repository->findTranslationByUid($entity->getUid(), $targetLanguageUid);
}
return $translatedEntity;
}
Related find-method:
/**
* @param int $uid UID des Datensatzes in der Standardsprache
* @param int $languageUid Sprache (sys_language_uid)
* @return object
*/
public function findTranslationByUid($uid, $languageUid)
{
$query = $this->createQuery();
$query->getQuerySettings()->setRespectSysLanguage(false);
$query->matching(
$query->logicalAnd(
$query->equals('l10n_parent', $uid),
$query->equals('sys_language_uid', $languageUid)
)
);
return $query->setLimit(1)->execute()->getFirst();
}
My data model is clicked together with the extension builder.
If u have further questions or need more code examples, feel free to comment here.
Updated by Tymoteusz Motylewski about 5 years ago
I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.
Updated by Anton Fries about 5 years ago
Tymoteusz Motylewski wrote:
I think that mixing extbase persistence with datahandler is an overkill.
I dont know the whole process (what data is comming in and how its process then), but For any batch processing of data I would use datahandler only, without the need to first map incomming data to objects just to take uid, pass it to datahandler and fetch some object again.
Yes im don't using datahandler and would not rewrite my import to use datahandler only because an bug in extbase.
You don't need to know the whole process. I already gave all the information about the relevant part, where the bug occurs. I attached there also an error log of the exception.
1. Create a entity in the default language in TYPO3 backend.
2. Translate the entity in a language in TYPO3 backend.
3. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Works.
4. Use $translatedEntity->setEntityList() (e.g. categoryList) on a m:n relation on the translated entity in extbase. Doesn't work because there are already relations.
Error points: sysext/extbase/Classes/Persistence/Generic/Backend.php line 537
$this->insertRelationInRelationtable($object, $parentObject, $parentPropertyName, $sortingPosition);
--> Its just an insert without checking if there is already a relation in the relation table.
sysext/extbase/Classes/Persistence/Generic/Backend.php line 780
$res = $this->storageBackend->addRow($relationTableName, $row, true);
sysext/extbase/Classes/Persistence/Generic/Storage/Typo3DbBackend.php line 151
throw new SqlErrorException($e->getPrevious()->getMessage(), 1470230766);
If u want, i can build a minimalistic example extension for you.
Updated by Tymoteusz Motylewski about 5 years ago
Well, you're already mixing extbase and datahandler :)
Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.
Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.
Please create an minimal extension to reproduce the issue.
Thanks
Updated by Anton Fries about 5 years ago
- File gwcatalog.zip added
- File DEV_ Galileo Webagentur [TYPO3 CMS 9.5.8].html added
Tymoteusz Motylewski wrote:
Well, you're already mixing extbase and datahandler :)
Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.
Please create an minimal extension to reproduce the issue.
Thanks
Okay got a minimalistic extension running.
You need to activate the extension and applying the database wizard.
Theres a backend module called "Katalogimport" with one action.
1. First time clicking the action it ensures that there is test data and translated test data. It adds an relation between a translated product and a translated category like TYPO3 backend would do it.
2. Second time clicking the action it tries again to create/overwrite the relation between the translated product and the translated category (could also be another category) where it throws an error.
Updated by Anton Fries about 5 years ago
- File gwcatalog.zip gwcatalog.zip added
Anton Fries wrote:
Tymoteusz Motylewski wrote:
Well, you're already mixing extbase and datahandler :)
Extbase main purpose (but not only of course) is to allow to build DDD style FE extensions.
Datahanlder strong side is data processing (also in batches), so it's main usage is in BE - where data is being edited.Of course you can build importer in extbase, if you want. You just need to know that different tools have different strong sides.
Please create an minimal extension to reproduce the issue.
ThanksOkay got a minimalistic extension running.
You need to activate the extension and applying the database wizard.
Theres a backend module called "Katalogimport" with one action.
1. First time clicking the action it ensures that there is test data and translated test data. It adds an relation between a translated product and a translated category like TYPO3 backend would do it.
2. Second time clicking the action it tries again to create/overwrite the relation between the translated product and the translated category (could also be another category) where it throws an error.
Forgot that u dont have our gwbase extension. Updated so the findTranslationByUid is directly in the AbstractRepository and no dependency injection from other extensions
Updated by Anton Fries about 5 years ago
- File deleted (
DEV_ Kunde [TYPO3 CMS 9.5.8].html)
Updated by Anton Fries about 5 years ago
- File deleted (
DEV_ Galileo Webagentur [TYPO3 CMS 9.5.8].html)
Updated by Gion Koch over 4 years ago
This is still an unresolved issue.
A workaround is to remove all relations with a query before persisting.
Updated by Anton Fries over 4 years ago
Gion Koch wrote:
This is still an unresolved issue.
A workaround is to remove all relations with a query before persisting.
Ye thats how we solved our problem for a big product import every night. But managing relations with translated domain objects doesn't work with extbase for domain models that have 'l10nmode' => 'exclude' in the TCA configuration. It creates a relation in the database, but not from the original language record in the uid_foreign, but rather from the translated record uid, so its not valid for TYPO3.
Our final solution now was to create all relations in default language and then copy over doctrine all relations to all languages (also way more performant and optimizable).
Updated by Susanne Moog over 4 years ago
- Status changed from Needs Feedback to Accepted
Updated by Stefan Froemken over 4 years ago
It seems that I have the same problem. I debugged it a very long time as there are some very basic questions:
- Which UID exists in _LOCALIZED_UID?
- Which UID will be used for relational columns of translated records? The UID of the related record in default language or the UID of the translated record.
For now I have folowing answers:
- _LOCALIZED_UID = UID of the record in foreign language
- The UID of the record in default language
Please correct me, if I'm wrong.
With this adaption I'm wondered about following lines in Extbase:
if ($value instanceof AbstractDomainObject && $value->_hasProperty('_localizedUid') && $value->_getProperty('_localizedUid') > 0 ) { $plainValue = (int)$value->_getProperty('_localizedUid'); } else { $plainValue = $this->dataMapper->getPlainValue($value); }
I don't think that this is correct. Please have a look into a translated tt_content record. These records are connected with page records in default language. IMO this behavior we should use for extbase, too.
Updated by Gerrit Code Review over 4 years ago
- Status changed from Accepted to Under Review
Patch set 1 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 4 years ago
Patch set 2 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 4 years ago
Patch set 3 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 4 years ago
Patch set 4 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Tymoteusz Motylewski over 4 years ago
Hi Stefan,Anton
Thanks for digging into the topic.
The language handling is much better since 9.5, but not perfect.
Please make sure to read this changelog, so you know the concept what is documented and expected right now.
https://docs.typo3.org/c/typo3/cms-core/master/en-us/Changelog/9.5/Important-82363-MakeExtBaseTranslationHandlingConsistentWithTyposcript.html
Updated by Gerrit Code Review over 4 years ago
Patch set 5 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 4 years ago
Patch set 6 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 3 years ago
Patch set 7 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 3 years ago
Patch set 8 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 9 for branch master of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 10 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 11 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 12 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 13 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 2 years ago
Patch set 14 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Gerrit Code Review over 1 year ago
Patch set 15 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/64112
Updated by Tymoteusz Motylewski about 1 month ago
- Assignee deleted (
Tymoteusz Motylewski)