Bug #59349
closedUpdate Reference Index failed by SQL errors (INSERT INTO sys_refindex )
0%
Description
I'll update a website from 4.7.17 to 6.2.3. By updating the reference Index I get this errors:
76054 records from 57 tables were checked/updated.
Record backend_layout:1 had 1 added indexes and 0 deleted indexes
Record backend_layout:3 had 1 added indexes and 0 deleted indexes
Record tx_news_domain_model_media:1 had 1 added indexes and 0 deleted indexes ...
Record tx_news_domain_model_file:1 had 1 added indexes and 0 deleted indexes ...
Record tx_gridelements_backend_layout:1 had 1 added indexes and 0 deleted indexes ...
As a result of this I have set debugSQL to true and get these SQL errors:
INSERT INTO sys_refindex (tablename,recuid,field,flexpointer,softref_key,softref_id,sorting,deleted,workspace,ref_table,ref_uid,ref_string,hash) VALUES ('tx_news_domain_model_media','119','image','','','','0','0','0',NULL,NULL,'','238eecf9c5c65e14bb7bbbb39f1ba839')
debug_backtrace require(.2.3/typo3/sysext/lowlevel/dbint/index.php),.2.3/typo3/mod.php#45 // TYPO3\CMS\Lowlevel\View\DatabaseIntegrityView->main#39 // TYPO3\CMS\Lowlevel\View\DatabaseIntegrityView->func_refindex#221 // TYPO3\CMS\Core\Database\ReferenceIndex->updateIndex#318 // TYPO3\CMS\Core\Database\ReferenceIndex->updateRefIndexTable#958 // TYPO3\CMS\Core\Database\DatabaseConnection->exec_INSERTquery#143 // TYPO3\CMS\Core\Database\DatabaseConnection->debug#215
OR
INSERT INTO sys_refindex (tablename,recuid,field,flexpointer,softref_key,softref_id,sorting,deleted,workspace,ref_table,ref_uid,ref_string,hash) VALUES ('tx_gridelements_backend_layout','15','icon','','','','0','0','0',NULL,NULL,'','903729ceee40d48661c4b5ce1f3e8e82')
These records are in affect to the 'image' or 'icon' field with 'group' configuration:
'image' => array( 'exclude' => 0, 'l10n_mode' => 'copy', 'label' => $ll . 'tx_news_domain_model_media.media', 'config' => array( 'type' => 'group', 'internal_type' => 'file', 'allowed' => $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext'], 'max_size' => $GLOBALS['TYPO3_CONF_VARS']['BE']['maxFileSize'], 'uploadfolder' => 'uploads/tx_news', 'show_thumbs' => 1, 'size' => 1, 'minitems' => 1, 'maxitems' => 1, ) ), 'icon' => array( 'exclude' => 1, 'label' => 'LLL:EXT:gridelements/Resources/Private/Language/locallang_db.xml:tx_gridelements_backend_layout.icon', 'config' => array( 'type' => 'group', 'internal_type' => 'file', 'allowed' => 'jpg,gif,png', 'uploadfolder' => 'uploads/tx_gridelements', 'show_thumbs' => 1, 'size' => 5, 'minitems' => 0, 'maxitems' => 2, ) ),
Files
Updated by Johannes C. Laxander over 10 years ago
exec_INSERTquery
caller: TYPO3\CMS\Core\Database\DatabaseConnection::exec_INSERTquery
ERROR: Column 'ref_uid' cannot be null
Updated by Markus Klein over 10 years ago
Thanks.
You reported this for 6.2, but I guess you ran the ref index update on 4.7. So the bug is on 4.7 then, right?
Updated by Jo Hasenau over 10 years ago
That's what I assumed too, but according to the discussion in the German list, the update has been done on 6.2.3 after the migration tool gave the advice to run a reference index update before.
Updated by Markus Klein over 10 years ago
Well I guess we have to rephrase this text.
You should update the reference index BEFORE upgrading in the old version.
Updated by Jo Hasenau over 10 years ago
It doesn't make a difference though, since this is still a bug in the reference index updater, which should not insert NULL values but the correct defaults.
Updated by Jo Hasenau over 10 years ago
BTW: Here is the discussion http://forum.typo3.org/index.php/t/204087/
It seems that the reference index had been updated with 4.7.x and everything was reported to be OK.
Then the upgrade was done using "Migrate all file links of RTE-enabled fields to FAL" which produced the report "There are currently 52 links to update. Update failed!"
There was a hint to do a reference index upgrade, which is why the update has been run on 6.2.3 again, which produced the SQL errors mentioned above.
IMHO the problem is, that there is neither a reference table nor a reference uid, when the reference points to a non FAL based group field of the internal type "file".
Updated by Markus Klein over 10 years ago
I guess there are three issues actually.
- First one is that internal_type 'file' seems to be not handled correctly.
- Second one is that there's a missing check somewhere that the ref_table is not valid
- Third one is that the upgrade wizard obviously does not work in all cases, since it should be able to convert all links in the first run.
So we need a DB dump or somebody debugging the upgrade wizard on this instance to figure out why it says the update fails. "What's wrong with these links".
Updated by Johannes C. Laxander over 10 years ago
Both is possible. It is a test instance, and so you can got an admin login by email.
Unless you'd prefer a DB Dump?
Updated by Markus Klein over 10 years ago
Hi!
Well admin sounds good already, but how many content elements are in this instance?
If it is over 100, I'd prefer if you could do a real debugging session.
Updated by Johannes C. Laxander over 10 years ago
tt_content has ca. 1800 records.
Sorry I do not understand what you mean:
"if you could do a real debugging session" ?
Updated by Markus Klein over 10 years ago
debuggin with your IDE via xdebug to find out which records are affected by this problem.
Updated by Johannes C. Laxander over 10 years ago
I'm not a developer, and so I have no IDE installed.
Updated by Markus Klein over 10 years ago
Ok, do you know if your server has xdebug enabled for remote debugging?
Updated by Johannes C. Laxander over 10 years ago
No, xdebug.so is not pre-installed, but it is possible to do this. Provider ist jweiland.net and from support I#ve got this answer:
Ja, die Nutzung von xdebug ist möglich. In den php Einstellung in der Domainkonfiguration muss die domainspezifische Einstellung aktiviert werden. Dann muss unter Zend-Extensions der Pfad zu einer lokalen xdebug.so eintragen werden.
Unter den sonstigen Einstellungen müsste dann noch ein Eintrag
xdebug.remote_enable = on
vorgenommen werden. Alternativ können die Einstellungen auch in lokale php.ini
Dateien gepackt werden, die dann in das jeweilige PHP Quellcode Verzeichnis kopiert werden.
Eine Anleitung zur generellen Aktivierung von weiteren PHP-Erweiterungen finden Sie unter dem nachfolgenden Link:
http://jweiland.net/typo3-hosting/anleitung/php/weitere-php-module.html
Updated by Johannes C. Laxander over 10 years ago
What is the status? Did you wait for something?
Updated by Markus Klein over 10 years ago
Sorry, I it's currently not possible for me to invest more time into getting debugging running on your hosting provider.
If you want, you can send me a complete DB dump and filesystem dump of the 4.7.17 instance, so I can try an upgrade locally on my dev-server, where I've all necessary tools at hand.
Updated by Johannes C. Laxander over 10 years ago
- File Dokument-0.pdf Dokument-0.pdf added
- File Dokument-1.pdf Dokument-1.pdf added
- File Dokument-2.pdf Dokument-2.pdf added
- File Dokument-3.pdf Dokument-3.pdf added
- File Dokument-4.pdf Dokument-4.pdf added
- File Dokument-5.pdf Dokument-5.pdf added
- File Dokument-6.pdf Dokument-6.pdf added
Ich verfasse diesen Beitrag in deutsch und bitte um Verständnis und Nachsicht, da meine Sprachkenntnisse nicht ausreichen, diesen Beitrag in englischer Sprache zu schreiben.
Ich habe noch einmal einen Versuch unternommen das Upgrade von TYPO3 4.7.19 auf Version 6.2.4 zu machen. Leider mit demselben Ergebnis wie schon oben berichtet. Mein Vorgehen und die Erkenntnisse habe ich Dokument-0.pdf" zusammengefasst.
Kurz-Fazit:
Der Referenz-Index ist unter TYPO3 4.7.19 noch OK. Und direkt nach der Umstellung auf den TYPO3-Source 6.2.4 tritt das Problem auf.
Ich sollte die Website eigentlich schon umgestellt haben, komme hier aber einfach nicht weiter. Habe ich die falsche Vorgehensweise? Ist es möglicherweise ein Bug im TYPO3 Code 6.2.x? Wobei kann ich evtl. helfen, um hier möglichst schnell zu einem Ergebnis zu kommen, das mich in die Lage versetzt, die Umstellung zu Ende zu bringen?
Danke für jede Unterstützung.
Updated by Markus Klein over 10 years ago
- Status changed from New to Needs Feedback
Why do you update the reference index again during the update steps??
Please continue on the mailing lists since this is the bugtracker. (e.g. german mailing list - I'm not reading this list though, just the english ones)
Once you sorted out the issue, you can update this ticket here with the precise reproduction instructions.
I'm changing the status of the ticket meanwhile.
Thank you.
Updated by Johannes C. Laxander over 10 years ago
- File screenshot-01.pdf screenshot-01.pdf added
- File screenshot-02.pdf screenshot-02.pdf added
Update the reference index again during the update steps: please have a look to screenshot-01.pdf
When I do this not the update on wizard step "Migrate all file links of RTE-enabled fields to FAL" failed. Please have a look to screenshot-02.pdf (All 20 records are shown in file "Update failed - Migrate all file links of RTE-enabled fields to FAL.txt". Go back to the updade wizard bring me back to the step "Migrate all file links of RTE-enabled fields to FAL" as shown in file "screenshot-01.pdf.
Updated by Markus Klein over 10 years ago
- Are the files reported missing really missing or are they present?
- Are those files really used in the the 4.7 installation? (Please check the DB with the uids specified in the error messages)
We need to find out, if the data is inconsistent on 4.7 or if the update wizards do something wrong.
Updated by Johannes C. Laxander over 10 years ago
- File screenshot-03.pdf screenshot-03.pdf added
- File DB-Check-uids-in-error messages.pdf DB-Check-uids-in-error messages.pdf added
Are the files reported missing really missing or are they present?
Yes, the files are really missing, except the jpg-files in folder fileadmin/user_upload/bilder/kairos/: kabine-A.jpg and kabine-B.jpg (last 4 messages)
Are those files really used in the the 4.7 installation? (Please check the DB with the uids specified in the error messages)
Yes, the files are really used in the 4.7 installation. Some of the records are hidden. - See screenshot-03.pdf and file DB-Check-uids-in-error messages.pdf
Updated by Jo Hasenau about 10 years ago
I just pointed out, what the actual problem is, but maybe this has been overlooked somehow ;-)
IMHO the problem is, that there is neither a reference table nor a reference uid, when the reference points to a non FAL based group field of the internal type "file".
This results in insert queries with NULL values, which are throwing errors with fields set to NOT NULL.
Updated by Alexander Opitz almost 10 years ago
- Category set to File Abstraction Layer (FAL)
- Status changed from Needs Feedback to New
Updated by Jo Hasenau over 9 years ago
Seems this has been fixed together with #64253. In line 198 of typo3/sysext/core/Classes/Database/ReferenceIndex.php there is now a check that should prevent the NULL values from being inserted.
foreach ($relations as $k => $datRec) { if (!is_array($relations[$k])) { continue; } ...