Project

General

Profile

Actions
Copy link

Bug #92968

closed

URL with result including unicode char > 3 bye leads to exception on DB insert in linkvalidator check if charset utf8 and kills scheduler task.

Added by Sybille Peters almost 4 years ago. Updated 6 months ago.

Status:
Closed
Priority:
Should have
Assignee:
-
Category:
Linkvalidator
Target version:
-
Start date:
2020-12-01
Due date:
% Done:

0%

Estimated time:
TYPO3 Version:
9
PHP Version:
Tags:
Complexity:
Is Regression:
Sprint Focus:

Description

System

  • TYPO3 9.5.23
  • database: charset / collation - not with 4-byte Unicode
+----------------------------+------------------------+----------+
| DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME | SQL_PATH |
+----------------------------+------------------------+----------+
| utf8                       | utf8_general_ci        | NULL     |
+----------------------------+------------------------+----------+
  • can be resolved by changing charset / collation to handle 4-byte unicode characters (utf8mb4 / utf8mb4_unicode_ci) but in any case exception should probably be caught and not lead to scheduler task aborting

Reproduce

Result

Exception occurs during link checking: "Incorrect string value" on insert of DB record

An exception occurred while executing 'INSERT INTO `tx_linkvalidator_link` (`headline`, `record_pid`, `record_uid`, `table_name`, `link_title`, `field`, `last_check`, `link_type`, `url`, `url_response`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params ["Konzert im Sommersemester 2005:", 79485, 352326, "tt_content", "Fl\u00f6te", "bodytext", 1606826589, "external", "http:\/\/de.wikipedia.org\/wiki\/Querfl%F6te", "\x61\x3a\x32\x3a\x7b\x73\x3a\x35\x3a\x22\x76\x61\x6c\x69\x64\x22\x3b\x62\x3a\x30\x3b\x73\x3a\x31\x31\x3a\x22\x65\x72\x72\x6f\x72\x50\x61\x72\x61\x6d\x73\x22\x3b\x61\x3a\x33\x3a\x7b\x73\x3a\x39\x3a\x22\x65\x72\x72\x6f\x72\x54\x79\x70\x65\x22\x3b\x69\x3a\x34\x30\x30\x3b\x73\x3a\x39\x3a\x22\x65\x78\x63\x65\x70\x74\x69\x6f\x6e\x22\x3b\x73\x3a\x32\x33\x39\x3a\x22\x43\x6c\x69\x65\x6e\x74\x20\x65\x72\x72\x6f\x72\x3a\x20\x60\x47\x45\x54\x20\x68\x74\x74\x70\x3a\x2f\x2f\x64\x65\x2e\x77\x69\x6b\x69\x70\x65\x64\x69\x61\x2e\x6f\x72\x67\x2f\x77\x69\x6b\x69\x2f\x51\x75\x65\x72\x66\x6c\x25\x46\x36\x74\x65\x60\x20\x72\x65\x73\x75\x6c\x74\x65\x64\x20\x69\x6e\x20\x61\x20\x60\x34\x30\x30\x20\x42\x61\x64\x20\x52\x65\x71\x75\x65\x73\x74\x60\x20\x72\x65\x73\x70\x6f\x6e\x73\x65\x3a\x0a\x3c\x21\x44\x4f\x43\x54\x59\x50\x45\x20\x68\x74\x6d\x6c\x3e\x0a\x3c\x68\x74\x6d\x6c\x20\x63\x6c\x61\x73\x73\x3d\x22\x63\x6c\x69\x65\x6e\x74\x2d\x6e\x6f\x6a\x73\x22\x20\x6c\x61\x6e\x67\x3d\x22\x64\x65\x22\x20\x64\x69\x72\x3d\x22\x6c\x74\x72\x22\x3e\x0a\x3c\x68\x65\x61\x64\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x63\x68\x61\x72\x73\x65\x74\x3d\x22\x55\x54\x46\x2d\x38\x22\x2f\x3e\x0a\x3c\x74\x69\x74\x6c\x65\x3e\x55\x6e\x67\xc3\xbc\x6c\x74\x69\x67\x65\x72\x20\x54\x69\x74\x65\x6c\x20\xe2\x20\x28\x74\x72\x75\x6e\x63\x61\x74\x65\x64\x2e\x2e\x2e\x29\x0a\x22\x3b\x73\x3a\x37\x3a\x22\x6d\x65\x73\x73\x61\x67\x65\x22\x3b\x73\x3a\x32\x36\x37\x3a\x22\x41\x6e\x20\x65\x72\x72\x6f\x72\x20\x6f\x63\x63\x75\x72\x72\x65\x64\x20\x28\x34\x30\x30\x29\x3a\x20\x22\x43\x6c\x69\x65\x6e\x74\x20\x65\x72\x72\x6f\x72\x3a\x20\x60\x47\x45\x54\x20\x68\x74\x74\x70\x3a\x2f\x2f\x64\x65\x2e\x77\x69\x6b\x69\x70\x65\x64\x69\x61\x2e\x6f\x72\x67\x2f\x77\x69\x6b\x69\x2f\x51\x75\x65\x72\x66\x6c\x25\x46\x36\x74\x65\x60\x20\x72\x65\x73\x75\x6c\x74\x65\x64\x20\x69\x6e\x20\x61\x20\x60\x34\x30\x30\x20\x42\x61\x64\x20\x52\x65\x71\x75\x65\x73\x74\x60\x20\x72\x65\x73\x70\x6f\x6e\x73\x65\x3a\x0a\x3c\x21\x44\x4f\x43\x54\x59\x50\x45\x20\x68\x74\x6d\x6c\x3e\x0a\x3c\x68\x74\x6d\x6c\x20\x63\x6c\x61\x73\x73\x3d\x22\x63\x6c\x69\x65\x6e\x74\x2d\x6e\x6f\x6a\x73\x22\x20\x6c\x61\x6e\x67\x3d\x22\x64\x65\x22\x20\x64\x69\x72\x3d\x22\x6c\x74\x72\x22\x3e\x0a\x3c\x68\x65\x61\x64\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x63\x68\x61\x72\x73\x65\x74\x3d\x22\x55\x54\x46\x2d\x38\x22\x2f\x3e\x0a\x3c\x74\x69\x74\x6c\x65\x3e\x55\x6e\x67\xc3\xbc\x6c\x74\x69\x67\x65\x72\x20\x54\x69\x74\x65\x6c\x20\xe2\x20\x28\x74\x72\x75\x6e\x63\x61\x74\x65\x64\x2e\x2e\x2e\x29\x0a\x22\x2e\x22\x3b\x7d\x7d"]: Incorrect string value: '\xE2 (tru...' for column `unit376dev1`.`tx_linkvalidator_link`.`url_response` at row 1


Tue, 01 Dec 2020 13:27:27 +0100 [ERROR] request="c0169dd94b6d9" component="TYPO3.CMS.Scheduler.Task.AbstractTask": Task failed to execute successfully. Class: TYPO3\CMS\Linkvalidator\Task\ValidatorTask, UID: 38, Code: 0, An exception occurred while executing 'INSERT INTO `tx_linkvalidator_link` (`headline`, `record_pid`, `record_uid`, `table_name`, `link_title`, `field`, `last_check`, `link_type`, `url`, `url_response`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params ["Konzert im Sommersemester 2005:", 38638, 140117, "tt_content", "Fl\u00f6te", "bodytext", 1606825646, "external", "http:\/\/de.wikipedia.org\/wiki\/Querfl%F6te", "\x61\x3a\x32\x3a\x7b\x73\x3a\x35\x3a\x22\x76\x61\x6c\x69\x64\x22\x3b\x62\x3a\x30\x3b\x73\x3a\x31\x31\x3a\x22\x65\x72\x72\x6f\x72\x50\x61\x72\x61\x6d\x73\x22\x3b\x61\x3a\x33\x3a\x7b\x73\x3a\x39\x3a\x22\x65\x72\x72\x6f\x72\x54\x79\x70\x65\x22\x3b\x69\x3a\x34\x30\x30\x3b\x73\x3a\x39\x3a\x22\x65\x78\x63\x65\x70\x74\x69\x6f\x6e\x22\x3b\x73\x3a\x32\x33\x39\x3a\x22\x43\x6c\x69\x65\x6e\x74\x20\x65\x72\x72\x6f\x72\x3a\x20\x60\x47\x45\x54\x20\x68\x74\x74\x70\x3a\x2f\x2f\x64\x65\x2e\x77\x69\x6b\x69\x70\x65\x64\x69\x61\x2e\x6f\x72\x67\x2f\x77\x69\x6b\x69\x2f\x51\x75\x65\x72\x66\x6c\x25\x46\x36\x74\x65\x60\x20\x72\x65\x73\x75\x6c\x74\x65\x64\x20\x69\x6e\x20\x61\x20\x60\x34\x30\x30\x20\x42\x61\x64\x20\x52\x65\x71\x75\x65\x73\x74\x60\x20\x72\x65\x73\x70\x6f\x6e\x73\x65\x3a\x0a\x3c\x21\x44\x4f\x43\x54\x59\x50\x45\x20\x68\x74\x6d\x6c\x3e\x0a\x3c\x68\x74\x6d\x6c\x20\x63\x6c\x61\x73\x73\x3d\x22\x63\x6c\x69\x65\x6e\x74\x2d\x6e\x6f\x6a\x73\x22\x20\x6c\x61\x6e\x67\x3d\x22\x64\x65\x22\x20\x64\x69\x72\x3d\x22\x6c\x74\x72\x22\x3e\x0a\x3c\x68\x65\x61\x64\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x63\x68\x61\x72\x73\x65\x74\x3d\x22\x55\x54\x46\x2d\x38\x22\x2f\x3e\x0a\x3c\x74\x69\x74\x6c\x65\x3e\x55\x6e\x67\xc3\xbc\x6c\x74\x69\x67\x65\x72\x20\x54\x69\x74\x65\x6c\x20\xe2\x20\x28\x74\x72\x75\x6e\x63\x61\x74\x65\x64\x2e\x2e\x2e\x29\x0a\x22\x3b\x73\x3a\x37\x3a\x22\x6d\x65\x73\x73\x61\x67\x65\x22\x3b\x73\x3a\x32\x36\x37\x3a\x22\x41\x6e\x20\x65\x72\x72\x6f\x72\x20\x6f\x63\x63\x75\x72\x72\x65\x64\x20\x28\x34\x30\x30\x29\x3a\x20\x22\x43\x6c\x69\x65\x6e\x74\x20\x65\x72\x72\x6f\x72\x3a\x20\x60\x47\x45\x54\x20\x68\x74\x74\x70\x3a\x2f\x2f\x64\x65\x2e\x77\x69\x6b\x69\x70\x65\x64\x69\x61\x2e\x6f\x72\x67\x2f\x77\x69\x6b\x69\x2f\x51\x75\x65\x72\x66\x6c\x25\x46\x36\x74\x65\x60\x20\x72\x65\x73\x75\x6c\x74\x65\x64\x20\x69\x6e\x20\x61\x20\x60\x34\x30\x30\x20\x42\x61\x64\x20\x52\x65\x71\x75\x65\x73\x74\x60\x20\x72\x65\x73\x70\x6f\x6e\x73\x65\x3a\x0a\x3c\x21\x44\x4f\x43\x54\x59\x50\x45\x20\x68\x74\x6d\x6c\x3e\x0a\x3c\x68\x74\x6d\x6c\x20\x63\x6c\x61\x73\x73\x3d\x22\x63\x6c\x69\x65\x6e\x74\x2d\x6e\x6f\x6a\x73\x22\x20\x6c\x61\x6e\x67\x3d\x22\x64\x65\x22\x20\x64\x69\x72\x3d\x22\x6c\x74\x72\x22\x3e\x0a\x3c\x68\x65\x61\x64\x3e\x0a\x3c\x6d\x65\x74\x61\x20\x63\x68\x61\x72\x73\x65\x74\x3d\x22\x55\x54\x46\x2d\x38\x22\x2f\x3e\x0a\x3c\x74\x69\x74\x6c\x65\x3e\x55\x6e\x67\xc3\xbc\x6c\x74\x69\x67\x65\x72\x20\x54\x69\x74\x65\x6c\x20\xe2\x20\x28\x74\x72\x75\x6e\x63\x61\x74\x65\x64\x2e\x2e\x2e\x29\x0a\x22\x2e\x22\x3b\x7d\x7d"]:

Incorrect string value: '\xE2 (tru...' for column `dbname`.`tx_linkvalidator_link`.`url_response` at row 1 - {"exception":"Doctrine\\DBAL\\Driver\\Mysqli\\MysqliException: Incorrect string value: '\\xE2 (tru...' for column `dbname`.`tx_linkvalidator_link`.`url_response` at row 1 in /var/www/mysite/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/Mysqli/MysqliStatement.php:151\nStack trace:
#0 /var/www/mysite/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1062): Doctrine\\DBAL\\Driver\\Mysqli\\MysqliStatement->execute(Array)
#1 /var/www/mysite/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(779): Doctrine\\DBAL\\Connection->executeUpdate('INSERT INTO `tx...', Array, Array)
#2 /var/www/mysite/htdocs/typo3/sysext/core/Classes/Database/Connection.php(199): Doctrine\\DBAL\\Connection->insert('`tx_linkvalidat...', Array, Array)
#3 /var/www/mysite/htdocs/typo3/sysext/linkvalidator/Classes/LinkAnalyzer.php(241): TYPO3\\CMS\\Core\\Database\\Connection->insert('tx_linkvalidato...', Array)
Actions
Copy link
 #1

Updated by Sybille Peters almost 4 years ago

  • Subject changed from URL with result "bad request" leads to exception on DB insert in linkvalidator check and kills scheduler task. to URL with result including unicode char > 3 bye leads to exception on DB insert in linkvalidator check if charset utf8 and kills scheduler task.
Actions
Copy link
 #2

Updated by Sybille Peters almost 4 years ago

In later version, the json_encode will already effectively truncate the string (json_encode returns false), so the exception on INSERT does not happen, but the error message will be empty.

Actions
Copy link
 #3

Updated by Sybille Peters 6 months ago

  • Description updated (diff)
Actions
Copy link
 #4

Updated by Sybille Peters 6 months ago

  • Status changed from New to Closed

This is rather a database administration problem:

  • either do not have string which cannot be encoded with current encoding / collation
  • use appropriate collation in DB

This is not a linkvalidator specific problem and this will also result in problems when saving.

About the exception throwing - there are some pros and cons with throwing exceptions - currently I think it is better if the exception is thrown, rather than suppressed and generic error message displayed.

Actions
Copy link

Also available in: Atom PDF