TYPO3 Core

Can you give a bit more details about versions of IM, PHP, webserver, etc.; if safe_mode is on/off, graphics settings, etc.?

With current dev version on Windows Vista/XAMPP/PHP5.3/GraphicsMagick the commands with quotes work for me (there are other problems which I hope to fix this month)

Example for reading JPG images in the install tool:
"D:/webserver/xampp/GraphicsMagick/gm.exe" convert +profile '*' -geometry 170x136! -colorspace RGB -quality 70 "D:/webserver/xampp/htdocs/typo3/sysext/install/imgs/jesus.jpg"[0] "D:/webserver/xampp/htdocs/typo3temp/pics/install_read_jpg.jpg"

Hi Jigal,

I'm running Windows 2008 with IIS7, php 5.2.14 (safe mode off), IM ImageMagick 6.6.5-8 2010-11-10 Q16.

I tried to isolate the issue, and apparently it lies within PHP. I created a simple php script and ran it:

$exec = 'E:/gm/gm.exe convert +profile \'*\' -geometry 170x136! -colorspace RGB -quality 70

"E:/websites/misc/im/jesus.jpg"[0] "E:/websites/misc/im/install_read_jpg.jpg"';

exec($exec);

This script works flawlessly (takes the jesus.jpg and creates a new install_read_jpg.jpg file). However, as soon as I quote the executable path (it therefore reads "E:/gm/gm.exe" instead of E:/gm/gm.exe), there is no result.

The same with IM instead of GM.

However, on PHP 5.3.3, even quoted path works (but some other issues are introduced, for instance this one - http://bugs.php.net/bug.php?id=52523).

Since I have IM and GM in environment PATH variable, calling plain convert.exe and gm.exe (unquoted!) also works.

Although I agree that putting an absolute & quoted path is a more "sane" (and safe) decision, I am afraid that strictly using quotes in new versions will break compatibility with earlier PHP versions (pre-5.3) (at least in combination with IIS7 on Windows). If removing quotes is unacceptable, I would suggest an optional global setting to disable quoting.

If there is any other information I can provide, don't hesitate to ask.

Yours, Ziga

I've searched for this issue and it seems that PHP 5.2 had or has a bug with the actual execution of the exec() function with quoted commands.
Using short path names (yea olde 8.3 format or DOS style path names) should work, because these contain no spaces.

I haven't found a reliable way (yet) to get a "short" path name for a "long" path.

Unfortunatelly - you can't rely on 8.3 names, because the underlying OS doesn't necessarily support it. On my installation of Win2k8, 8.3 naming was by default disabled (http://www.windowsreference.com/windows-7/control-8dot3-naming-in-windows-2008-and-windows-7/).

I can confirm though, that after enabling 8.3 naming (and creating a new directory with IM), calling e:\testte~1\convert.exe (unquoted) works. (Unquoted. Again, quoted "e:\testtesttest\convert.exe" doesn't work.)

Further searching suggests that the problem is only present if more than two double quotes are present. Can you test if this is indeed true?

$exec = '"E:/gm/gm.exe" convert +profile \'*\' -geometry 170x136! -colorspace RGB -quality 70 E:/websites/misc/im/jesus.jpg⁰ E:/websites/misc/im/install_read_jpg.jpg';

This should work fine (it's not a solution, but just a confirmation of the actual problem).

Can you test if either of these workarounds work on your installation?

1. Put a dummy command in front of the double quoted command. e.g.
$exec = 'E: & "E:/gm/gm.exe" convert +profile \'*\' -geometry 170x136! -colorspace RGB -quality 70 "E:/websites/misc/im/jesus.jpg"[0] "E:/websites/misc/im/install_read_jpg.jpg"';

2. Put the actual command with double quotes in a .bat file and execute that. Please test if the path/name of the .bat file works if surrounded with double quotes.

Thanks for the testing so far!

(Apparently, bugtracker ate part of my previous post at the beginning.)

I first tried the following code:

$exec = '"E:/gm/gm.exe" convert +profile \'*\' -geometry 170x136! -colorspace RGB -quality 70

E:/websites/misc/im/jesus.jpg⁰ E:/websites/misc/im/install_read_jpg.jpg';

This works.

After putting double quotes around the output-file parameter, it doesn't work any more.

Then I tried your code under 1st bullet (with E: & dummy command).

The following code:

$exec = 'E: & "E:/gm/gm.exe" convert +profile \'*\' -geometry 170x136! -colorspace RGB -quality 70

"E:/websites/misc/im/jesus.jpg"[0] "E:/websites/misc/im/install_read_jpg.jpg"';

Works. :-)

I then created test.bat file with the following content:

"E:/gm/gm.exe" convert +profile '*' -geometry 170x136! -colorspace RGB -quality 70 "E:/websites/misc/im/jesus.jpg"[0] "E:/websites/misc/im/install_read_jpg.jpg"

(I un-escaped the single-quotes after +profile parameter.)

I ran test.bat through cmd.exe. It works, the install_read_jpg.jpg file IS created.

Then I tried $exec = 'test.bat'. Works.

$exec = '"test.bat"'; Also works.

$exec = '"e:/websites/misc/im/test.bat"'; Also works.

Thus, I can confirm that bug is only present if more than two double quotes are present in the command. However, then there is a dummy E: & command before, the command works even with more than 2 double quotes.

Patch submitted to core list

I tested the attached patch. (Unfortunatelly I was unable to "patch" it using patch.exe, as it either crashed or only patched 1 line - this is probably an issue exclusively on my side, therefore I "manually" patched the file.)

After application, the following command is called in "Read JPG test" (and other tests) in install tool:

e: & "e:/ImageMagick/convert.exe" +profile '*' geometry 170x136! -colorspace RGB -quality 70 "E:/websites/--.test/typo3/sysext/install/imgs/jesus.jpg"[0] "E:/websites/---.test/typo3temp/pics/install_read_jpg.jpg"

Resulting files are successfully created, as are those generated on the frontend.

Thank you very much for spending time on this issue!

@Žiga Dolhar : your vote in the typo3.teams.core newsgroup / mailinglist is useful to get this patch in 4.5!

This probably also needs to be done in 4.4, since we are also quoting the filename in 4.4.5 (to be able to have spaces in the path to the IM executable, fixed in #17489). This is indeed a very ugly and stupid PHP bug, thanks for finding a workaround for it. Let's see if more people can test it.

The patch works for me ONLY partially. It fixed the problem to generate images by the TYPO3 install tool, but not to create images dynamically by TypoScript and gifbuilder. After some debugging I found differences in the file locations (absolute/relative) in the command string, which ends in successfully created images or a failure without any returned value. See the following command strings:

Install tool (with patch 16656):
$cmd = 'C: & "C:/ImageMagick-6.5.9-Q16/convert.exe" -geometry 170x136! -colorspace RGB -quality 70 "C:/T3demo/typo3/sysext/install/imgs/jesus.jpg"[0] "C:/T3demo/typo3temp/pics/install_read_jpg.jpg"';

By content rendering (TypoScript, Gifbuilder, with patch 16656):
$cmd = 'C: & "C:/ImageMagick-6.5.9-Q16/convert.exe" -geometry 170x136! -colorspace RGB -quality 70 "typo3/sysext/install/imgs/jesus.jpg"[0] "typo3temp/pics/install_read_jpg.jpg"';

After further testing on different mashines
- Windows XP and Windows 2003 Server
- PHP 5.2.8 and 5.3.3
- ImageMagick 6.5.9-Q16
- Apache 2.2
I recieved the following results, which I will simplify by using following constants for more clarity:

COMMAND = C:/ImageMagick-6.5.9-Q16/convert.exe
OPTIONS = -geometry 170x136! -colorspace RGB -quality 70
SOURCEFILE_ABSOLUTE = C:/T3demo/typo3/sysext/install/imgs/jesus.jpg
SOURCEFILE_RELATIVE = typo3/sysext/install/imgs/jesus.jpg
TARGETFILE_ABSOLUTE = C:/T3demo/typo3temp/pics/install_read_jpg.jpg
TARGETFILE_RELATIVE = typo3temp/pics/install_read_jpg.jpg

Please take special attention to the diffent sets of the double quotes.

Samples WITHOUT patch 16656:

1. Command string similar to the install tool
=> does not work in PHP 5.2.8, works in PHP 5.3.3
-> $cmd = '"COMMAND" OPTIONS "SOURCEFILE_ABSOLUTE"[0] "TARGETFILE_ABSOLUTE"';

2. Command string by TS and gifbuilder
=> does not work in PHP 5.2.8, works in PHP 5.3.3
-> $cmd = '"COMMAND" OPTIONS "SOURCEFILE_RELATIVE"[0] "TARGETFILE_RELATIVE"';

3. Command string by TS and gifbuilder WITHOUT double quotes to the command!
=> works in PHP 5.2.8 and PHP 5.3.3
-> $cmd = 'COMMAND OPTIONS "SOURCEFILE_RELATIVE"[0] "TARGETFILE_RELATIVE"';

Samples WITH patch 16656:

4. Command string similar to the install tool
=> works in PHP 5.2.8 and PHP 5.3.3
-> $cmd = 'C: & "COMMAND" OPTIONS "SOURCEFILE_ABSOLUTE"[0] "TARGETFILE_ABSOLUTE"';

5. Command string by TS and gifbuilder
=> does not work in PHP 5.2.8 and PHP 5.3.3
-> $cmd = 'C: & "COMMAND" OPTIONS "SOURCEFILE_RELATIVE"[0] "TARGETFILE_RELATIVE"';

6. Command string by TS and gifbuilder WITHOUT double quotes to the command!
=> does not work in PHP 5.2.8 and PHP 5.3.3
-> $cmd = 'C: & COMMAND OPTIONS "SOURCEFILE_RELATIVE"[0] "TARGETFILE_RELATIVE"';

7. Complete command string in additional double quotes
=> does not work in PHP 5.2.8 and PHP 5.3.3
-> $cmd = 'C: & ""COMMAND" OPTIONS "SOURCEFILE_RELATIVE"[0] "TARGETFILE_RELATIVE""';

SOLUTIONS:
Based on these results, I see two possibilities to fix this problem:

1. First, quick and dirty:
I do not use the patch and have to remove the double quotes to the command string, see sample #3. This only works, until there are NO space character in the command string!!!

see file: t3lib/class.t3lib_div.php, line 5937. 
   I have replaced the escapeshellarg() function with the escapeshellcmd() function.

- $path = escapeshellarg($path . (($command 'composite') ? $combineScript : $command) . $isExt);
+ $path = escapeshellcmd($path . (($command 'composite') ? $combineScript : $command) . $isExt);

2. Certainly correct and better, I am not sure ...
Inclusion of the patch 16656 and additional expanding the relative file location to absolute file locations. But this seems to be done in some methods in the "t3lib_div class", any time the "imageMagickCommand" will be invoked. For example:

> line 465: method gif_compress()
> line 494: method png_to_gif_by_imagemagick()
-> line 511: method read_png_gif()

I'm not sure if this will be the right way?

Yours, Christian

Hi Christian,

Thanks for testing!

The patch will only affect PHP < 5.3 on Windows, so no command prefix will be executed on PHP 5.3.3.
From the test cases you mention I conclude that relative paths are also a problem, but that is not in the scope of this issue. Some test cases might also be influenced by issue #22243 (unquote filenames doesn't work correctly and frame (the [0] part) should be placed inside the quotes). I'm trying hard to get that into 4.5 too.

The patch given here is rather lengthy and the prepended dummy command looks like a real hack. I propose to use my solution from #0017447 which is <b>a simple one-liner</b>. It uses as second pair of double quotes around the IM/GM command string if PHP<5.3 and OS==WIN. This is a solution recommended by a poster on php.net (posting of "luko post cz", 30-Jan-2008, on http://php.net/manual/en/ref.exec.php). Looks quite clean to me.
I have tested the patches given in #0017447 on several machines with PHP5.3.x and 5.2.x with TYPO3 4.4.5 and 4.5.0.

Attached v3 patches sent to the core list. After checking the PHP source code the approach with extra double quotes is the solution which comes closest to the way PHP itself works.

Committed to trunk rev: 10556
4_5 rev: 10557 (will appear in 4.5.1)
4_4 rev: 10558 (will appear in 4.4.7)