Bug #92287

After upgrading to ver. 9.5.21 uploading of pdf files causes an exception

Added by Robert von Hackwitz 15 days ago. Updated 9 days ago.

Status:
Resolved
Priority:
Must have
Assignee:
-
Category:
Backend User Interface
Start date:
2020-09-11
Due date:
% Done:

100%

TYPO3 Version:
9
PHP Version:
7.2
Tags:
Complexity:
Is Regression:
Yes
Sprint Focus:

Description

Until version 9.5.20 in sysext/core/Classes/Resource/Index/Indexer.php, method extractRequiredMetaData there was a check to verify if uploaded file was an Image:

if ($fileObject->getType() == File::FILETYPE_IMAGE && $this->storage->getDriverType() === 'Local') {

The new check introduced in ver. 9.5.21

if ($this->storage->getDriverType() === 'Local' && GeneralUtility::inList(strtolower($GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext'] ?? ''), $fileObject->getExtension())) {

assumes pdf as image files (due to imagefile_ext ...).
This causes wrong value for metadata and throws the exception.

Then: or we remove ".pdf" from Default Configuration $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext'] or we have to revert to old "if ($fileObject->getType() == File::FILETYPE_IMAGE"

Schermata da 2020-09-11 11-09-23.png View (195 KB) Robert von Hackwitz, 2020-09-11 11:09

exception.log View (17.8 KB) Sybille Peters, 2020-09-14 12:56

Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf (205 KB) Sybille Peters, 2020-09-14 13:42


Related issues

Related to TYPO3 Core - Bug #91967: Properties width and height are not extracted to meta data for PDF files Resolved 2020-08-10
Duplicated by TYPO3 Core - Bug #92328: TYPO3 v9.5.21 - Renaming PDF files in filelist causes error: Data truncated for column 'height' at row 1 Closed 2020-09-17

Associated revisions

Revision 1451e560 (diff)
Added by Benni Mack 9 days ago

[BUGFIX] Deliver detected file type for IM identify

Although not used in TYPO3 Core directly, the identify
command by ImageMagick now delivers the file extension
from IM plus the file type identified by IM as well.

This saves some regexp magic, and a foreach() loop,
as the "-format" output delivers proper IM results.

While previously this change was only targeted
for master, it is now backported to 9.5 because
otherwise https://review.typo3.org/65277 causes
a regression when indexing PDF files.

Related: #91967
Resolves: #92287
Resolves: #23348
Releases: 9.5
Change-Id: Ia380ee34dbbef7f9f763ecc088ee0c9b83167ce6
Reviewed-on: https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711
Tested-by: Sybille Peters <>
Tested-by: Daniel Sattler <>
Tested-by: Georg Ringer <>
Tested-by: Helmut Hummel <>
Tested-by: TYPO3com <>
Reviewed-by: Georg Ringer <>
Reviewed-by: Benni Mack <>

History

#1 Updated by Georg Ringer 14 days ago

  • Related to Bug #91967: Properties width and height are not extracted to meta data for PDF files added

#2 Updated by Helmut Hummel 13 days ago

  • Status changed from New to Needs Feedback

Thanks for the report.
The error here isn't that pdf files are treated as images,
but that a wrong height is extracted in your case.
The question is: what code caused a height of "842+0+0"

Since there are multiple mechanisms to detect image dimensions
it would be good to know which one fails for you.

Thanks.

#3 Updated by Helmut Hummel 13 days ago

Helmut Hummel wrote:

The question is: what code caused a height of "842+0+0"

This seems GraphicsMagick/ImageMagick related. Can you post your full TYPO3 `GFX` configuration
and the exact version of IM/GM you are using? Also, if possible, can you upload a PDF file here, that causes the error for you?

I can't reproduce this with my configuration and IM/GM versions

#4 Updated by Helmut Hummel 13 days ago

  • Status changed from Needs Feedback to Accepted

Never mind. Could now reproduce it in 9.5

#5 Updated by Gerrit Code Review 13 days ago

  • Status changed from Accepted to Under Review

Patch set 1 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#6 Updated by Gerrit Code Review 13 days ago

Patch set 2 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#7 Updated by Gerrit Code Review 13 days ago

Patch set 3 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#8 Updated by Gerrit Code Review 13 days ago

Patch set 4 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#9 Updated by Gerrit Code Review 13 days ago

Patch set 4 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#10 Updated by Gerrit Code Review 13 days ago

Patch set 4 for branch 9.5 of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/65711

#11 Updated by Robert von Hackwitz 12 days ago

Hi, thanks for your help

#12 Updated by Sybille Peters 12 days ago

We have the same problem. This is reproducible with 9.5.21 and not with 9.5.20.

It happens - as already mentioned - on uploading a PDF in the filelist.

TYPO3 tries to write "842+0+0" into height which is an INT field.

Core: Exception handler (WEB): Uncaught TYPO3 Exception: An exception occurred while executing 'UPDATE `sys_file_metadata` SET `width` = ?, `height` = ?, `tstamp` = ? WHERE `uid` = ?' with params ["595", "842+0+0", 1600070132, 259702]: Data truncated for column 'height' at row 1 | Doctrine\DBAL\Exception\DriverException thrown in file /var/www/mysite/htdocs/typo3_src-9.5.21/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php in line 106. Requested URL: https://mysite.de/typo3/index.php?route=%%2Fajax%%2Ffile%%2Fprocess&token=--AnonymizedToken--

I attached the exception dump from the logs.

This is the GFX setting:

GFX
gdlib = 1
gdlib_png =
gif_compress = 1
imagefile_ext = gif,jpg,jpeg,tif,tiff,bmp,pcx,tga,png,pdf,ai,svg
jpg_quality = 80
processor = GraphicsMagick
processor_allowFrameSelection = 1
processor_allowTemporaryMasksAsPng =
processor_allowUpscaling = 1
processor_colorspace = RGB
processor_effects =
processor_enabled = 1
processor_interlace = None
processor_path = /usr/bin/
processor_path_lzw = /usr/bin/
processor_stripColorProfileByDefault = 1
processor_stripColorProfileCommand = +profile '*'
thumbnails = 1
thumbnails_png = 1

#13 Updated by Sybille Peters 12 days ago

I updloaded a file with which I can reproduce the problem

gm identify

This is the output of GraphicsMagick (which is used) gm:

$ gm identify Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf
Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf[0] PDF 595x842+0+0 DirectClass 8-bit 2.9Mi 0.000u 0m:0.000011s
Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf[1] PDF 595x842+0+0 DirectClass 8-bit 2.9Mi 0.010u 0m:0.005813s

As you can see, it outputs "595x842+0+0" for width x height

identify

This is the output of ImageMagick:

$ identify Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf
Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf[0] PDF 595x842 595x842+0+0 16-bit sRGB 57543B 0.000u 0:00.000
Antragsformular_PROMOS_Fernweh-Zuschuss_2021-1.pdf[1] PDF 595x842 595x842+0+0 16-bit sRGB 47945B 0.000u 0:00.000

As you can see, it outputs "595x842" for width x height

gm version

$gm version
GraphicsMagick 1.3.34 2019-12-24 Q16 http://www.GraphicsMagick.org/
Copyright (C) 2002-2019 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.

Feature Support:
  Native Thread Safe       yes
  Large Files (> 32 bit)   yes
  Large Memory (> 32 bit)  yes
  BZIP                     yes
  DPS                      no
  FlashPix                 no
  FreeType                 yes
  Ghostscript (Library)    no
  JBIG                     yes
  JPEG-2000                yes
  JPEG                     yes
  Little CMS               yes
  Loadable Modules         yes
  Solaris mtmalloc         no
  OpenMP                   yes (201107 "3.1")
  PNG                      yes
  TIFF                     yes
  TRIO                     no
  Solaris umem             no
  WebP                     yes
  WMF                      yes
  X11                      yes
  XML                      yes
  ZLIB                     yes

Host type: x86_64-redhat-linux-gnu

Configured using the command:
  ./configure  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--disable-dependency-tracking' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--localstatedir=/var' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--enable-shared' '--disable-static' '--docdir=/usr/share/doc/GraphicsMagick-1.3.34' '--with-lcms2' '--with-magick_plus_plus' '--with-modules' '--with-perl' '--with-perl-options=INSTALLDIRS=vendor ' '--with-quantum-depth=16' '--enable-quantum-library-names' '--with-threads' '--with-wmf' '--with-x' '--with-xml' '--without-dps' '--without-gslib' '--with-gs-font-dir=/usr/share/fonts/default/Type1' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4

Final Build Parameters:
  CC       = gcc -std=gnu99
  CFLAGS   = -fopenmp -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -DFT_ENCODING_PRC=FT_ENCODING_GB2312 -Wall -pthread
  CPPFLAGS = -I/usr/include/freetype2 -I/usr/include/libxml2
  CXX      = g++
  CXXFLAGS = -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -pthread
  LDFLAGS  = -Wl,-z,relro
  LIBS     = -llcms2 -lfreetype -lXext -lSM -lICE -lX11 -llzma -lbz2 -lz -lltdl -lm -lpthread

#14 Updated by Sybille Peters 12 days ago

Possible workaround: switch to ImageMagick for now.

#15 Updated by Simon Gilli 12 days ago

  • Priority changed from Should have to Must have

#16 Updated by Georg Ringer 9 days ago

  • Duplicated by Bug #92328: TYPO3 v9.5.21 - Renaming PDF files in filelist causes error: Data truncated for column 'height' at row 1 added

#17 Updated by Benni Mack 9 days ago

  • Status changed from Under Review to Resolved
  • % Done changed from 0 to 100

Also available in: Atom PDF