go-libwebp

Status: Experimental.

This package provides a means to encode webp images. It does not wrap the entire libwebp API surface. It does provide a way to avoid CGO.

This package provides several options for interaction with libwebp:

1. dynamic libraries (purego)
2. WebAssembly (wazero)
3. transpiled C to Go (ccgo)

Each option is called a “backend”.

Package dynamic provides a dynamic binding to libwebp. This requires the consumer to acquire the shared objects. Package wasm runs libwebp as a sandboxed WebAssembly module under wazero, from a single embedded artifact that serves every platform. Package transpiled provides a Go translation of the libwebp source.

If the shared objects are available at runtime, dynamic bindings will be preferred.

Backends

Which backends are compiled in is a build-time choice, so a binary carries only what it uses.

Sizes are for a trivial program built with -ldflags='-s -w'; a bare one is 1.4 MiB. Most of the wasm backend’s footprint is wazero’s compiler, not the 0.6 MiB module.

The transpiled tag displaces the wasm backend rather than adding to it, so no build carries both pure-Go fallbacks.

webp.Backend() reports which backend is live. The dynamic backend binds whatever libwebp the host provides, which may be a different version than this package was built against, and may encode to different bytes.

Tags select what is linked, not what is downloaded: the module requires wazero, purego and modernc.org/libc regardless.

Performance

Geomean over the 14-image corpus in ./bench, relative to dynamic, on a Ryzen 9 7900X3D. Lower is better.

Megapixels per second, 1280x720 photographic:

Same source, six builds. SIMD accounts for the whole wasm/transpiled difference, and lossless barely uses it:

All backends encode byte-identically; the harness checks that before measuring.

Reproduce:

tools/build-native-bench-libs.sh     # native shared objects to compare against
tools/build-wasm-emscripten.sh       # SIMD and scalar modules
tools/build-wasm.sh                  # zig scalar module

go run ./bench -so bench/lib -wasm \
  "wasm-zig=bench/lib/libwebp-zig-scalar.wasm,\
wasm-em=bench/lib/libwebp-em-scalar.wasm,\
wasm-em-simd=bench/lib/libwebp-em-simd.wasm"

Package transpiled is a ccgo-based translation from libwebp in the hope to bring webp encoder into Go space.

Example

go run ./cmd encode ./cmd/megopher.png

Can consume JPEG and PNG images.

Rationale

There exists a webp decoder for Go golang.org/x/image/webp that does not include an encoder. Using it is obtuse: if you need an encoder you need to hack together another dependency. This package extends golang.org/x/image/webp with an encoder translated from the libwebp project, providing a single package webp codec.

WebAssembly

lib/wasm/webp/libwebp.wasm is checked in, built from the vendored c-lib source. Two build scripts are kept, both reproducing a module wazero can run:

• tools/build-wasm-emscripten.sh — produces the checked-in module. Emscripten is required for SIMD specifically: libwebp has no simd128 DSP backend, so its SSE2/SSE4.1 kernels are reached through Emscripten’s SSE-to-simd128 compat headers, which wasi-sdk and zig do not ship. Also emits a scalar module for comparison.
• tools/build-wasm.sh — a zig-only scalar build. No emsdk needed; useful as a lighter-weight toolchain and as a control when comparing.

SIMD roughly halves lossy-encode and decode time but does nothing for lossless encode, which is entropy-coding bound.

Run tools/test-matrix.sh to build and test every supported tag combination.

Toolchain

tools/transpile.sh regenerates lib/transpiled with ccgo v4, one file per GOOS/GOARCH, on an amd64 Linux box. It needs:

ccgo v4 supplies target predefines through --goos/--goarch but not target libc headers, so each cross target needs a header source. zig covers darwin from its bundled any-macos-any headers, which is why no macOS SDK or osxcross install is required. It does not cover linux: ccgo cannot parse zig’s generic-glibc bits/types.h, so linux/arm64 uses a real cross compiler and linux/amd64 uses the host.

Two patches are applied during transpilation. libwebp declares GetCoeffs as a volatile function pointer, which ccgo cannot emit a call through; threading is disabled here so dropping volatile is safe. And tools/sharpyuv_stub.c replaces the sharpyuv library, whose sharpyuv_gamma.c is libwebp’s only caller of expf/logf — modernc.org/libc provides those for linux only. Nothing reaches sharpyuv unless WebPConfig.use_sharp_yuv is set, which this package never does.

Dynamic

To make use of the dynamic bindings, ensure you supply the appropriate shared objects at runtime. It is the consumer’s responsibility to acquire the shared object. This projects contains a script that will build amd64 shared-objects and place them in lib/dynamic/webp/blobs (tools/compile-dynamic-libraries.sh).

Testing

Go side testing involves a lossless compression test; byte-wise comparison against a golden test image. In addition, there is a fuzz harness that executes against the transpiled code - encoding and subsequently decoding images of random size and color.

Features

• support for amd64 Windows, Linux, Darwin
• support for dynamic linking and transpiled backends
  • if you want speed and must avoid CGO, use dynamic linking
  • if you want to avoid CGO without further hassle, use transpilation

Contributors

This repo represents a joint effort between Chris Waldon (~whereswaldon) and myself. Thanks Chris for figuring out the build process for compiling libwebp to Go!

We owe a great debt to @cznic for both creating ccgo and fixing numerous small problems we encounted while compiling libwebp.