# go-libwebp

Status: Experimental.

This package provides a means to encode webp images.
It does _not_ wrap the entire libwebp API surface.
It _does_ provide a way to avoid CGO.

This package provides several options for interaction with libwebp:
1. dynamic libraries (purego)
2. WebAssembly (wazero)
3. transpiled C to Go (ccgo)

Each option is called a "backend".

Package dynamic provides a dynamic binding to libwebp. This requires the consumer to acquire the shared objects.
Package wasm runs libwebp as a sandboxed WebAssembly module under wazero, from a single
embedded artifact that serves every platform.
Package transpiled provides a Go translation of the libwebp source.

If the shared objects are available at runtime, dynamic bindings will be preferred.

## Backends

Which backends are compiled in is a build-time choice, so a binary carries only what it uses.

| build                        | backends                 | binary  |
| ---------------------------- | ------------------------ | ------- |
| *(default)*                  | dynamic, then wasm       | 5.7 MiB |
| `-tags transpiled`           | dynamic, then transpiled | 3.4 MiB |
| `-tags nowasm`               | dynamic only             | 1.8 MiB |
| `-tags nodynamic`            | wasm only                | 5.7 MiB |
| `-tags nodynamic,transpiled` | transpiled only          | 3.3 MiB |
| `-tags nodynamic,nowasm`     | none — build error       | —       |

Sizes are for a trivial program built with `-ldflags='-s -w'`; a bare one is 1.4 MiB.
Most of the wasm backend's footprint is wazero's compiler, not the 0.6 MiB module.

The `transpiled` tag displaces the wasm backend rather than adding to it, so no build
carries both pure-Go fallbacks.

`webp.Backend()` reports which backend is live. The dynamic backend binds whatever
libwebp the host provides, which may be a different version than this package was built
against, and may encode to different bytes.

Tags select what is linked, not what is downloaded: the module requires wazero, purego
and `modernc.org/libc` regardless.

## Performance

Geomean over the 14-image corpus in `./bench`, relative to dynamic, on a Ryzen 9 7900X3D.
Lower is better.

| operation             | dynamic | wasm | transpiled |
| --------------------- | ------- | ---- | ---------- |
| encode lossless       | 1.0x    | 2.6x | 2.8x       |
| encode lossy (q=0.75) | 1.0x    | 2.6x | 4.3x       |
| decode                | 1.0x    | 2.2x | 4.5x       |

Megapixels per second, 1280x720 photographic:

| operation             | dynamic | wasm | transpiled |
| --------------------- | ------- | ---- | ---------- |
| encode lossless       | 4.0     | 1.7  | 1.7        |
| encode lossy (q=0.75) | 25.7    | 10.0 | 5.1        |
| decode                | 309     | 133  | 56         |

Same source, six builds. SIMD accounts for the whole wasm/transpiled difference, and
lossless barely uses it:

| operation       | native SIMD | native scalar | wasm zig | wasm em | wasm em+SIMD | transpiled |
| --------------- | ----------- | ------------- | -------- | ------- | ------------ | ---------- |
| encode lossless | 1.0x        | 1.2x          | 3.2x     | 3.2x    | 2.6x         | 2.8x       |
| encode lossy    | 1.0x        | 2.3x          | 4.8x     | 4.7x    | 2.6x         | 4.3x       |
| decode          | 1.0x        | 2.1x          | 4.8x     | 4.7x    | 2.2x         | 4.5x       |

All backends encode byte-identically; the harness checks that before measuring.

Reproduce:

```
tools/build-native-bench-libs.sh     # native shared objects to compare against
tools/build-wasm-emscripten.sh       # SIMD and scalar modules
tools/build-wasm.sh                  # zig scalar module

go run ./bench -so bench/lib -wasm \
  "wasm-zig=bench/lib/libwebp-zig-scalar.wasm,\
wasm-em=bench/lib/libwebp-em-scalar.wasm,\
wasm-em-simd=bench/lib/libwebp-em-simd.wasm"
```

Package transpiled is a [`ccgo`-based](https://pkg.go.dev/modernc.org/ccgo/v3) translation from [libwebp](https://github.com/webmproject/libwebp/)
in the hope to bring webp encoder into Go space.

## Example

`go run ./cmd encode ./cmd/megopher.png` 

Can consume JPEG and PNG images.

## Rationale

There exists a webp decoder for Go `golang.org/x/image/webp` that does not include an encoder. Using it is obtuse: if you need an encoder you need to hack together another dependency.
This package extends `golang.org/x/image/webp` with an encoder translated from the libwebp project, providing a single package webp codec.

## WebAssembly

`lib/wasm/webp/libwebp.wasm` is checked in, built from the vendored `c-lib` source. Two
build scripts are kept, both reproducing a module wazero can run:

- `tools/build-wasm-emscripten.sh` — produces the checked-in module. Emscripten is
  required for SIMD specifically: libwebp has no simd128 DSP backend, so its SSE2/SSE4.1
  kernels are reached through Emscripten's SSE-to-simd128 compat headers, which wasi-sdk
  and zig do not ship. Also emits a scalar module for comparison.
- `tools/build-wasm.sh` — a zig-only scalar build. No emsdk needed; useful as a
  lighter-weight toolchain and as a control when comparing.

SIMD roughly halves lossy-encode and decode time but does nothing for lossless encode,
which is entropy-coding bound.

Run `tools/test-matrix.sh` to build and test every supported tag combination.

## Toolchain

`tools/transpile.sh` regenerates `lib/transpiled` with
[`ccgo`](https://pkg.go.dev/modernc.org/ccgo/v4) v4, one file per GOOS/GOARCH, on an
amd64 Linux box. It needs:

| tool                     | for                                         |
| ------------------------ | ------------------------------------------- |
| `ccgo` v4                | the transpiler                              |
| `zig`                    | darwin headers                              |
| `x86_64-w64-mingw32-gcc` | windows headers                             |
| `aarch64-linux-gnu-gcc`  | linux/arm64 headers                         |

ccgo v4 supplies target predefines through `--goos`/`--goarch` but not target libc
headers, so each cross target needs a header source. zig covers darwin from its bundled
`any-macos-any` headers, which is why no macOS SDK or osxcross install is required. It
does not cover linux: ccgo cannot parse zig's `generic-glibc` `bits/types.h`, so
linux/arm64 uses a real cross compiler and linux/amd64 uses the host.

Two patches are applied during transpilation. libwebp declares `GetCoeffs` as a
`volatile` function pointer, which ccgo cannot emit a call through; threading is
disabled here so dropping `volatile` is safe. And `tools/sharpyuv_stub.c` replaces the
sharpyuv library, whose `sharpyuv_gamma.c` is libwebp's only caller of `expf`/`logf` —
`modernc.org/libc` provides those for linux only. Nothing reaches sharpyuv unless
`WebPConfig.use_sharp_yuv` is set, which this package never does.

## Dynamic

To make use of the dynamic bindings, ensure you supply the appropriate shared objects at runtime. 
It is the consumer's responsibility to acquire the shared object. This projects contains a script
that will build amd64 shared-objects and place them in `lib/dynamic/webp/blobs` (`tools/compile-dynamic-libraries.sh`).

## Testing

Go side testing involves a lossless compression test; byte-wise comparison against a golden test image.
In addition, there is a fuzz harness that executes against the transpiled code - encoding and subsequently
decoding images of random size and color.

## Features

- support for amd64 Windows, Linux, Darwin
- support for dynamic linking and transpiled backends
  - if you want speed and must avoid CGO, use dynamic linking
  - if you want to avoid CGO without further hassle, use transpilation

## Contributors

This repo represents a joint effort between Chris Waldon (~whereswaldon) and myself.
Thanks Chris for figuring out the build process for compiling libwebp to Go!

We owe a great debt to [@cznic](https://gitlab.com/cznic) for both creating `ccgo` and fixing numerous small
problems we encounted while compiling `libwebp`.