WebP 2

WebP 2 is an experimental image codec based on WebP. WebP 2 will not be released as an image format but is used as a playground for image compression experiments.

This package contains the library that can be used in other programs to encode or decode Webp 2 images, as well as command line tools.

See http://developers.google.com/speed/webp for the first version of WebP.

What to expect?

The WebP 2 experimental codec is mostly pushing the features of WebP further in terms of compression efficiency. The new features (like 10b HDR support) are kept minimal. The axis of experimentation are:

• more efficient lossy compression (~30% better than WebP)
• better visual degradation at very low bitrate
• improved lossless compression
• improved transparency compression
• animation support
• ultra-light previews
• lightweight incremental decoding
• small container overhead, tailored specifically for image compression
• full 10bit architecture (HDR10)
• strong focus on software implementation, fully multi-threaded

The use cases remain mostly the same as WebP: transfer over the wire, faster web, smaller apps, better user experience...
WebP 2 is primarily tuned for the typical content available on the Web and Mobile apps: medium-range dimensions, transparency, short animations, thumbnails.

As of Nov. 2020, WebP 2 is only partially optimized and, roughly speaking 5x slower than WebP for lossy compression. It still compresses 2x faster than AVIF, but takes 3x more time to decompress. The goal is to reach decompression speed parity.

Side-by-side codec comparisons can be found at:

• 2022-10-04
• 2021-12-15
• 2021-08-10

Building

Prerequisites

A compiler (e.g., gcc 6+, clang 7+ or Microsoft Visual Studio 2017+ are recommended) and CMake.

On a Debian-like system the following should install everything you need for a minimal build:

$ sudo apt install build-essential cmake

Compiling

$ mkdir build && cd build
$ cmake ..
$ make -j

Configuration options:

• WP2_ENABLE_SIMD: enable any SIMD optimization.
• WP2_ENABLE_BITTRACE: enable tracing.

For additional options see:

$ cmake .. -LH

Compiling with Android NDK

The latest NDK for Android can be retrieved from the Android download page.

Assuming the variable NDK_ROOT is positioned correctly to point to the NDK's directory, the Android binaries can be built with:

$ mkdir build && cd build
$ cmake .. -DCMAKE_TOOLCHAIN_FILE=../cmake/android.cmake \
           -DWP2_ANDROID_NDK_PATH=${NDK_ROOT}
$ make -j

Extra configuration option:

• ANDROID_ABI: one of armeabi-v7a,armeabi-v7a with NEON,arm64-v8a,x86,x86_64... (default is arm64-v8a)

Binaries

cwp2

cwp2 is a tool to encode images in webp2.

Usage:

$ cwp2 in_file [options] [-o out_file]

Example for a single image:

$ cwp2 -q 70 input.png -o output.wp2

Example for an animation, with list of frames and durations in ms:

$ cwp2 -q 70 -f frame1.png 10 frame2.png 20 frame3.png 5 -o output.wp2

Important options:

* The quality factor range corresponds to:

Use cwp2 -h to see a full list of available options.

dwp2

dwp2 is a tool to decode webp2 images.

Usage:

$ dwp2 in_file [options] [-o out_file]

Use dwp2 -h to see a full list of available options.

vwp2

vwp2 is a visual inspection and debugging tool. You need OpenGL and GLUT to build it.

To open any image (jpeg, png, etc.) then compress it in WebP 2 and view the result:

$ vwp2 in_file...

vwp2 takes most of the same flags as cwp2, e.g. -q for quality. Encoding parameters can also be changed dynamically in the tool using key bindings.

Press h to a list of key bindings.

Use the top left menu or press v and shift+v to cycle between views.

Press i to show or hide info (note this hides the menu).

To view an already compressed file, use:

$ vwp2 -d path/to/image.wp2

rd_curve

rd_curve is a command-line tool for compressing images at multiple quality levels using different codecs (webp2, webp, jpeg, av1) to create rate-distortion curves (rd curves). An rd curve is a plot of distortion (difference between source and encoded image) vs bits per pixel, for different quality settings.

$ rd_curve [options] input_file

rd_curve takes most of the same flags as cwp2, e.g. -q for quality, -effort, and so on.

By default, only the webp2 codec is used. Use -webp, -jpeg or -av1 flags to add other codecs.

By default, results are printed as plain text on standard output.

With the -html flag, rd_curve outputs an html file. It also saves compressed images (turns on the -save option). Use the -save_folder option to set the directory where images are saved.

$ rd_curve input.png -webp -jpeg -av1 -html -save_folder $(pwd) > myfile.html

get_disto

get_disto computes the difference between two images (typically the compressed file and the original file)

$ get_disto [options] compressed_file orig_file

get_disto outputs in order:

• compressed file size
• overall PSNR (or other metric, if specified through flags)
• PSNR (or other metric) for A, R, G and B channels, in order
• compressed file bits per pixel

API

Encoding API

Encoding functions are available in the header src/wp2/encode.h.

Encoding an image

#include "imageio/image_dec.h"
#include "src/wp2/base.h"
#include "src/wp2/encode.h"

WP2::ArgbBuffer input_buffer;
WP2Status status = WP2::ReadImage("path/to/image.png", &input_buffer);
if (status != WP2_STATUS_OK) { /* handle error */ }
WP2::EncoderConfig config;
config.quality = 70;
WP2::MemoryWriter writer;
status = WP2::Encode(input_buffer, &writer, config);
if (status != WP2_STATUS_OK) { /* handle error */ }
// do something with writer.mem_

Encoding an animation

#include "imageio/image_dec.h"
#include "src/wp2/base.h"
#include "src/wp2/encode.h"

WP2::ArgbBuffer frame1, frame2;
WP2Status status = WP2::ReadImage("path/to/frame1.png", &frame1);
if (status != WP2_STATUS_OK) { /* handle error */ }
status = WP2::ReadImage("path/to/frame2.png", &frame2);
if (status != WP2_STATUS_OK) { /* handle error */ }

WP2::AnimationEncoder encoder;
status = encoder.AddFrame(frame1, /*duration_ms=*/100);
if (status != WP2_STATUS_OK) { /* handle error */ }
status = encoder.AddFrame(frame2, /*duration_ms=*/50);
if (status != WP2_STATUS_OK) { /* handle error */ }

WP2::EncoderConfig config;
config.quality = 70;
WP2::MemoryWriter writer;
status = encoder.Encode(&writer, config, /*loop_count=*/1);
if (status != WP2_STATUS_OK) { /* handle error */ }
// do something with writer.mem_

Decoding API

Decoding functions are available in the header src/wp2/decode.h.

Simple decoding

This is mainly just one function to call:

#include "src/wp2/base.h"
#include "src/wp2/decode.h"

const std::string data = ...
WP2::ArgbBuffer output_buffer;
WP2Status status = WP2::Decode(data, &output_buffer);

If the file is a WebP 2 animation, output_buffer will contain the first frame.

Please have a look at the file src/wp2/decode.h for further details.

Animation decoding

To decode all the frames of an animation, the more advanced Decoder API can be used. See tests/test_decoder_api.cc for common use cases.

WP2::ArrayDecoder decoder(data, data_size);
uint32_t duration_ms;
while (decoder.ReadFrame(&duration_ms))  {
  // A frame is ready. Use or copy its 'duration_ms' and 'decoder.GetPixels()'.
}
if (decoder.GetStatus() != WP2_STATUS_OK) { /* error */ }

Incremental decoding

If you want to start decoding before all the data is available, you can use the Decoder API. Use an WP2::ArrayDecoder if the data is stored in an array that progressively gets larger, with old bytes still available as new bytes come in. Use a WP2::StreamDecoder if data is streamed, with old bytes no longer available as new bytes come in. You can also subclass WP2::CustomDecoder to fit your needs. See tests/test_decoder_api.cc for common use cases.

Below is an example with WP2::StreamDecoder.

WP2::StreamDecoder decoder;
while (/*additional data is available in some 'new_data[]' buffer*/) {
  decoder.AppendInput(new_data, new_data_size);
  while (decoder.ReadFrame()) {
    // ReadFrame() returns true when an entire frame is available
    // (a still image is considered as a single-frame animation).
    // The canvas is stored in GetPixels() till the next call to ReadFrame().

    // Use the whole GetPixels().
  }
  if (decoder.Failed()) break;
  if (!decoder.GetDecodedArea().IsEmpty()) {
    // Use the partially GetDecodedArea() of GetPixels().
  }
}
if (decoder.GetStatus() != WP2_STATUS_OK) { /* error */ }

Development

Bugs

Please report all bugs to the issue tracker: https://bugs.chromium.org/p/webp2

Contributing

See CONTRIBUTING.md for details on how to submit patches.

One of the easiest ways to contribute is to report cases with compression artifacts or surprising output size. These ‘bad cases’ are very useful to help improve the compression library!

Please use the tracker to report such issues, making sure to include:

• the version or revision used (using “git rev-parse HEAD” for instance)

• the version of the compiler used (if you compiled your own version)

• the problematic source image (will only be used for debugging)

• the exact command line to use to reproduce the issue

• the output file, if applicable.

Coding conventions and style

• Code must follow the Google C++ style guide.

• const is used for variables everywhere possible. Input parameters use const references.

  void DoSomething(const Type1& const input, Type2* const out) { }
  

• Do not use C++ exceptions.

• Do not use std containers in the main library, e.g. no std::vector/set/map (but they can be used in tests). For vectors, use WP2::Vector instead.

• Use uint32_t for sizes, width, height, loop indices, etc.

• Most functions should return a WP2Status.

File layout

• CMakeLists.txt and cmake/* are used to build the project with CMake. Makefile is unmaintained.

• doc/* contains the container and format specifications.

• examples/* and extras/* contain executables and tools.

  • cwp2.cc and dwp2.cc are the main entry points to the library.

• imageio/* contains image-reading and image-writing functions. Several formats are handled such as PNG, WebP etc.

• presubmit/* contains the continuous integration testing scripts (run by Jenkins when a patch of libwebp2 is sent to Gerrit).

• swig/* contains the SWIG wrapper for Python and wp2_js/* contains the Javascript interface.

• src/* contains the core library.

  • src/wp2/* contains the public API headers. Consult encode.h and decode.h to interact with libwebp2.

  • src/enc/* contains the encoder-related sources.

    • src/enc/main_enc.cc contains the encoder entry point.

  • src/dec/* contains the decoder-related sources.

    • src/dec/main_dec.cc contains the still image decoding entry functions.

    • src/dec/incr/decoder_stages.cc contains the incremental and animation Decoder class implementation.

  • src/common/* and src/utils/* contain the files that are used by both encode and decode functions.

    • src/common/header_enc_dec.h is used to code image-wise BitstreamFeatures.

    • src/common/global_params.h contains the frame-wise GlobalParams.

    • src/enc/tile_enc.h and src/dec/tile_dec.h contain the tile-wise coding functions.

  • src/dsp/* contains the low-level, platform-optimized algorithms.

• tests/* contains the tests:

  • test_simple_enc_dec.cc and test_decoder_api.cc are simple examples to start with.

  • tests/include/* and tests/tools/* contain the helper headers and tools for testing.

  • tests/testdata/* contains some sample images used by the tests.

  • tests/bench/* contains the performance-measuring tools. Requires Google Benchmark.

  • tests/fuzz/* contains the fuzzing tools. See OSS-Fuzz for more information.

Testing

Dependencies

The following dependencies are needed for some tests to pass:

$ sudo apt install libpng-dev libjpeg-dev libtiff-dev libgif-dev libwebp-dev

Google Test Integration

Most tests have a dependency on the GoogleTest library. To run them all, the WP2_ENABLE_TESTS cmake variable must be turned on at cmake generation time (ON by default), and the GTEST_SOURCE_DIR cmake variable must be set to the path of the Google Test source directory (../googletest by default):

$ git clone https://github.com/google/googletest.git path/to/googletest
$ cmake path/to/wp2 -DWP2_ENABLE_TESTS=ON -DGTEST_SOURCE_DIR=path/to/googletest

Alternatively you may install the library instead. CMake should then be able to find it automatically afterwards:

$ sudo apt install libgtest-dev

Running the tests

To run the tests you can use ctest after the build:

$ make
$ ctest

Or run them one by one from the tests source folder, so that the testdata folder is accessible:

$ cd ../tests
$ ../build/tests/test_simple_encode

Discussion

Email: [email protected]

Web: https://groups.google.com/a/webmproject.org/g/webp-discuss