Feature #88537
closedEpic #101608: File Abstraction Layer Changes for v13
WebP image format support
100%
Description
Hey,
i think that support for WebP image format would be a great feature in TYPO3. It provides great lossless and lossy compression for image on the web.
More about WebP format: https://developers.google.com/speed/webp/
WebP image format will have a great impact for website optimization and performance.
PS: i think this is a "small" feature, so i didn't start topic on TYPO3 Decisions.
Files
Updated by Benni Mack over 5 years ago
- Status changed from New to Needs Feedback
Hi René,
I agree with that feature. We already use it in our projects in v9 without any adaptions to TYPO3 Core.
A few things we need to do for this to make this happen for Core.
1. Add this to $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext'] (probably along with JPEG XR, JPEG 2000)
2. This should however depend if the systems' IM/GM supports webp - question is how can we identify this?
3. We'd need to change templates to have a webp with a picture tag
What do you think?
Updated by Rene Tobias over 5 years ago
- File phpinfo.JPG phpinfo.JPG added
Hey Benni,
For 1. i think this is the easiest of all 3 point am i right? For 2. PHP Info gives you that information (see screenshot) - could this be helpful for you? So you can implement some phpcheck in the core? 3. - sory i dont understand witch templates do you mean?
Updated by Andreas Kießling over 5 years ago
Hi René,
in the meantime, https://extensions.typo3.org/extension/webp/ might work for you.
It hooks into the generation of processed files, all you then need to configure is a webserver rewrite depending on the accept header. Integration was thus quite easy, but i did not enable / test it for user uploaded files, only for processed ones.
The author was also very quick to respond to bug reports.
HTH
Andreas
Updated by Hannes Strangmeier over 5 years ago
There are several ways to identify the IM/GM support, although some have some drawbacks.
One is:
convert --version | grep Delegates
This will ONLY give you the built-in delegates, which might not include webp (in case IM was not built with webp), even though it is shipped with an external delegate through delegate.xml
You can get those using:
convert -list delegate
Note that this is a bit more tricky to tackle down, since there can be multiple entries for webp.
With all the tests i have done so far, I came to the conclusion that using IM to convert images to webp is not as good as using cwebp. For most pictures I converted cwebp was about 3 times faster than imagemagick while the filesize was approximately the same (less than 1% difference in filesize).
Also note that ImageMagick in some cases falls back to a default-quality (I think 83% or something like that), when you do not define it yourself. cwebp on the other hand defaults to 75, but using cwebp directly does not rely on some layer (e.g. IM) in between when configuring it with parameters.
So the core should make sure to always set a quality to get similar results across different setups (e.g. use [GFX][jpg_quality] or introduce something similar for webp).
Note to phpinfo: This only tells you if the gdlib has webp-support. I am not sure where TYPO3 uses gd, but I guess in most cases IM/GM handles the images. So this does not tell you anything about IM/GM.
Having a full picturetag+webp support in the core would also be great in my point of view :)
Updated by Hannes Strangmeier over 5 years ago
Some more insights with a fresh test:
I installed a fresh VM with Ubuntu 18.04, which is shipped with ImageMagick 6.9.7-4 WITHOUT built-in webp-support:
convert --version Version: ImageMagick 6.9.7-4 Q16 x86_64 20170114 http://www.imagemagick.org Copyright: © 1999-2017 ImageMagick Studio LLC License: http://www.imagemagick.org/script/license.php Features: Cipher DPC Modules OpenMP Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jpeg lcms lqr ltdl lzma openexr pangocairo png tiff wmf x xml zlib
Anyway, you can use IM to convert files to webp after installing webp:
sudo apt-get install webp
ImageMagick has the corresponding delegates:
hstr@hstr-VirtualBox:~/testpics$ convert -list delegate | grep webp png<= webp "cwebp' -quiet %Q '%i' -o '%o" webp => "dwebp' -pam '%i' -o '%o"
I was not able to figure out how to define the quality within ImageMagick, since -quality and define webp:.. did not work and it always uses it's own default. Here's a quick example:
hstr@hstr-VirtualBox:~/testpics$ convert -quality 100 test.jpg test_im_100.webp hstr@hstr-VirtualBox:~/testpics$ convert -quality 75 test.jpg test_im_75.webp hstr@hstr-VirtualBox:~/testpics$ ls -al total 1344 drwxr-xr-x 2 hstr hstr 4096 Jun 17 13:51 . drwxr-xr-x 18 hstr hstr 4096 Jun 17 13:46 .. -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:51 test_im_100.webp -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:51 test_im_75.webp -rw-r--r-- 1 hstr hstr 768608 Jun 17 12:35 test.jpg
So let's just compare the sizes and time needed to create webp-files from now on:
ImageMagick took around 1.8 seconds:
hstr@hstr-VirtualBox:~/testpics$ time convert -quality 75 test.jpg test_im_75.webp real 0m1,894s user 0m1,820s sys 0m0,024s
While cwebp took only 0.6 seconds:
hstr@hstr-VirtualBox:~/testpics$ time cwebp test.jpg -q 75 -o test_cwebp_75.webp Saving file 'test_cwebp_75.webp' File: test.jpg Dimension: 2048 x 1536 Output: 296064 bytes Y-U-V-All-PSNR 38.59 43.30 45.03 39.78 dB block count: intra4: 9000 intra16: 3288 (-> 26.76%) skipped block: 1092 (8.89%) bytes used: header: 258 (0.1%) mode-partition: 40946 (13.8%) Residuals bytes |segment 1|segment 2|segment 3|segment 4| total macroblocks: | 5%| 24%| 22%| 47%| 12288 quantizer: | 36 | 34 | 28 | 18 | filter level: | 11 | 8 | 24 | 17 | real 0m0,655s user 0m0,539s sys 0m0,012s
Filesize was basically the same:
hstr@hstr-VirtualBox:~/testpics$ ls -la total 1636 drwxr-xr-x 2 hstr hstr 4096 Jun 17 13:57 . drwxr-xr-x 18 hstr hstr 4096 Jun 17 13:46 .. -rw-r--r-- 1 hstr hstr 296064 Jun 17 13:57 test_cwebp_75.webp -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:51 test_im_100.webp -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:56 test_im_75.webp -rw-r--r-- 1 hstr hstr 768608 Jun 17 12:35 test.jpg
No errors were detected by webpinfo, so I assume both files are equally valid webp-files:
hstr@hstr-VirtualBox:~/testpics$ webpinfo test_im_75.webp test_cwebp_75.webp File: test_im_75.webp RIFF HEADER: File size: 296112 Chunk VP8 at offset 12, length 296100 Width: 2048 Height: 1536 Alpha: 0 Animation: 0 Format: Lossy (1) No error detected. File: test_cwebp_75.webp RIFF HEADER: File size: 296064 Chunk VP8 at offset 12, length 296052 Width: 2048 Height: 1536 Alpha: 0 Animation: 0 Format: Lossy (1) No error detected.
If anyone is interested in further investigation for the qualtiy-settings, this should be a good start:
https://imagemagick.org/script/command-line-options.php#quality
Note that 75 is the basic PNG-Quality and the delegates show that the input-format for webp is png, if I understand it correctly.
Nonetheless, speed is much faster using cwebp... and cwebp is needed for IM at least with a basic Ubuntu-installation, unless you want to compile IM by yourself. So my advise would be to use cwebp if accessible and IM, if TYPO3 can not access cwebp AND the delegates indicate that it is available (note: all delegates, not only built-in delegates).
I might be able to test built-in support within the next weeks.
Updated by Hannes Strangmeier over 5 years ago
after compiling imagemagick with built-in webp-support (and switchting to version 7), the time to generate webp-files is basically equal to cwebp:
hstr@hstr-VirtualBox:~/testpics$ time convert -quality 75 test.jpg test_im7.webp real 0m0,653s user 0m0,566s sys 0m0,036s hstr@hstr-VirtualBox:~/testpics$ ls -la total 1932 drwxr-xr-x 2 hstr hstr 4096 Jun 17 15:31 . drwxr-xr-x 18 hstr hstr 4096 Jun 17 15:33 .. -rw-r--r-- 1 hstr hstr 296064 Jun 17 13:57 test_cwebp_75.webp -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:51 test_im_100.webp -rw-r--r-- 1 hstr hstr 296112 Jun 17 13:56 test_im_75.webp -rw-r--r-- 1 hstr hstr 299402 Jun 17 15:32 test_im7.webp -rw-r--r-- 1 hstr hstr 768608 Jun 17 12:35 test.jpg
If one wants to reproduce this, I compiled libwebp and imagemagick from scratch as described in step 1 and 2 here:
https://github.com/rosell-dk/webp-convert/wiki/Installing-Imagick-extension
Sidenote: With built-in webp-support it looks like you can set the quality in imagemagick using -quality
Updated by Hannes Strangmeier over 5 years ago
My personal conclusion so far:
Just define cwebp as a requirement, unless you want to dig through all the dependencies coming along with webp in imagemagick...
Updated by Benni Mack over 4 years ago
Hannes Strangmeier wrote:
My personal conclusion so far:
Just define cwebp as a requirement, unless you want to dig through all the dependencies coming along with webp in imagemagick...
I see your point.
I'm fine with opening up TYPO3 Core API to also support webp, but we are not able to support webp as a requirement, as roughly 70% of current TYPO3 installations run on a server-stack that does not even have 64 bit systems available, nor any possibility to add more third-party packages.
So: Adapt API to support it - Yes. Let's discuss what needs to be done
Adding this as a requirement for the sake of building webp natively - not gonna happen at this moment, otherwise we'd loose a large customer base.
Updated by Hannes Strangmeier over 4 years ago
I see your point, too and i'm totally fine with it :)
I can try to do some more testing with cwebp/IM/GM in different constellations regarding delegates and so on "soonish". Thereby also providing more information on how to detect the support for webp in that regard (although i assume that the ways described above are sufficient).
Question is:
Which capabilities should be tested?
I would say something like:
- converting jpg/png/gif to webp and vice versa
- scaling webp images
- combinding webp images
- crop webp images
When PDF files are being processed by IM/GM + GS, they are converted into a bitmap at first, before being saved as a jpeg/png/whatever (at least if i remember it correctly). This should not have an impact on the creation of webp files in general, but there might be the need for two steps when using native cwebp (as IM/GM should do that by themselves).
I'm also wondering if something like "scale this png and convert it to webp" would be multiple steps for TYPO3? What is currently happening, if you for example scale a png and you output-format is jpg?
Updated by Oliver Hader over 4 years ago
Issue #88537 contained some additional links and info in markup (which is duplicated here)
Hi!
It would be great to think about usage of modern images formats like JPEG 2000, JPEG XR, WebP, Avif (https://reachlightspeed.com/blog/using-the-new-high-performance-avif-image-format-on-the-web-today/) in TYPO3.
Main problem could be, that not all browses/devices support these formats (see https://caniuse.com/avif, https://caniuse.com/webp, https://caniuse.com/jpeg2000, https://caniuse.com/jpegxr). So a fallback solution would be great, maybe something like:
<picture> <source srcset="image.webp" type="image/webp"> <source srcset="image.jpg" type="image/jpeg"> <img src="image.jpg"> </picture>
Possible solution: A editor should be capable of adding two or more version of an image. But makes only sense, if the server could handle these formats.
There´s already a discussion about webp in #88537.
Best,
Christoph
Updated by Oliver Hader over 4 years ago
- Has duplicate Feature #92088: Usage of modern image formats added
Updated by Christoph Werner almost 4 years ago
Hi!
Just a brief info: Safari 14 (released in september 2020) supports webp (see https://developer.apple.com/documentation/safari-release-notes/safari-14-release-notes) too, so just older safari, IE and KaiOS Browser can´t handle webp (see https://caniuse.com/webp). Maybe a fallback solution isn´t that important any more.
Best
Christoph
Updated by Jonas Eberle almost 4 years ago
In my oppinion the HTTP `Accept` header should be respected in any case and the "well-established" image formats will be needed for some years to come. MacOS browser support is coupled to the OS version btw (https://caniuse.com/?search=webp).
I think the question here is more towards a general multi-format image delivery framework.
Updated by Riccardo De Contardi over 3 years ago
- Related to Bug #93961: Images not converting to webp in image viewhelpers added
Updated by Riccardo De Contardi over 3 years ago
- Related to Feature #93981: Google webp and Apple heic/heif support for TYPO3 - Make default targetfileextensions for processed images configurable added
Updated by Benni Mack about 1 year ago
- Status changed from Needs Feedback to Accepted
- Parent task set to #101608
Updated by Sybille Peters 12 months ago
Some time has passed here. In my opinion, I don't think we have to make it so complicated. Also, I noticed the current default behaviour is inconsistent:
- we allow upload of .webp images by default, so people can upload them (which they probably will)
- the support is not good: the image thumbnail is not created in the file list. You can't add .webp images as images, for example in content elements textmedia, textpic
So, I think we should support it by default in v13 at least and make it consistent. And that requires to add it to $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext'] etc. by default.
There are then these options:
1. If the default is used, it is possible to use .webp.
2. If people want to use a webp with fallback approach (e.g. with picture srcset and supply webp with fallback, as Oliver posted above), they can use one of the existing extensions available, or follow one of the descriptions in blog posts, but then upload of .webp imaages should be prevented, so they should change their configuration. But, I think this is less relevant nowadays and you can start to expect browser support in newer TYPO3 version
3. If they don't want to support .webp at all, they should configure it away, so it is not possible to upload it at all, same as in 2.
Do we need extra checks to see if ImageMagick / GraphicsMagick supports it? On my systems it all works out-of-the-box, I don't know if this is something which needs so much extra scrutiny, but would be preferable to have a check somewhere.
So, a solution could look like this:
1. add webp to list $GLOBALS['TYPO3_CONF_VARS']['GFX']['imagefile_ext']
2. add webp to list $GLOBALS['TYPO3_CONF_VARS']['SYS']['mediafile_ext']
3. ideally, check if webp images can be scaled (install tool?)
4. related, off scope here: add additional options specific for Webp, see #96925
5. related, off scope here: make it easier to restrict more file extensions from being uploaded, see #102644
extra Info:
GraphicsMagick claims it need libwebp.
http://www.graphicsmagick.org/formats.html
Requires libwebp from "https://developers.google.com/speed/webp/". WebP is good for small photos for the web and is supported by Google's Chrome and Firefox.
In my systems this is installed out of the box via package dependency (e.g. RHEL 9).
Updated by Sybille Peters 12 months ago
- Related to Bug #96925: Make webp quality settings adjustable in TYPO3 added
Updated by Felix Nagel 12 months ago
@Sybille Peters I've tested webp support about a year ago and the server (Uberspace v7) did not support creation of webp. So a check if the system is able to create webp image format would be a good thing I guess.
Updated by Gerrit Code Review 12 months ago
- Status changed from Accepted to Under Review
Patch set 1 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/82196
Updated by Gerrit Code Review 12 months ago
Patch set 2 for branch main of project Packages/TYPO3.CMS has been pushed to the review server.
It is available at https://review.typo3.org/c/Packages/TYPO3.CMS/+/82196
Updated by Benni Mack 12 months ago
- Status changed from Under Review to Resolved
- % Done changed from 0 to 100
Applied in changeset 38485bb005c174a88715234ba5caa48f36cf17ee.