nativeaudio

audio playback for Go
Log | Files | Refs | README | LICENSE

README.md (6387B)

      1 # nativeaudio
      2 
      3 `nativeaudio` is a convenience package that provides a Go interface over
      4 native audio APIs.
      5 
      6 Output is PCM s16le, playable directly. Stacks that take an `io.Reader`
      7 of s16le, such as oto and Ebitengine, need no adapter. For the ones that
      8 want float64 sample pairs, such as beep, the `pcm` subpackage converts.
      9 
     10 This package was inspired by the utter lack of audio decoders for AAC, 
     11 which it turns out is a closed codec that Windows and macOS have bought
     12 licenses for. 
     13 
     14 Windows is implemented via the Media Foundation and macOS is implemented
     15 on AudioToolbox/AVFoundation.
     16 
     17 Linux links FFmpeg's libraries directly. Other platforms, and any build
     18 without cgo, run the FFmpeg binary as a sub-process instead.
     19 
     20 The linking is dynamic, which is the case LGPL exists to permit. The
     21 original concern was Go's *static* linking, which LGPL genuinely does not
     22 sit well with; dynamic linking does not raise it. Nothing is vendored or
     23 redistributed either way, so the patent question still sits with whoever
     24 installed FFmpeg.
     25 
     26 One caveat worth knowing. Many distributions build FFmpeg with
     27 `--enable-gpl`, which makes the libraries GPL rather than LGPL; Arch's is
     28 GPL-3.0. The FSF's position is that linking, statically or dynamically,
     29 makes a combined work, so anyone shipping a binary linked against such a
     30 build inherits that obligation. Building with `CGO_ENABLED=0` selects the
     31 sub-process path, which is arm's length and raises no such question.
     32 
     33 
     34 `go get git.sr.ht/~jackmordaunt/nativeaudio`
     35 
     36 ```go
     37 package main
     38 
     39 import (
     40 	"log"
     41 
     42 	"git.sr.ht/~jackmordaunt/nativeaudio"
     43 )
     44 
     45 func main() {
     46 	d, err := nativeaudio.New()
     47 	if err != nil {
     48 		log.Fatal(err)
     49 	}
     50 	defer d.Close()
     51 
     52 	pcm, format, err := d.DecodeFile("audio.m4a")
     53 	if err != nil {
     54 		log.Fatal(err)
     55 	}
     56 	log.Printf("%d bytes of PCM, %+v", len(pcm), format)
     57 }
     58 ```
     59 
     60 Or, to just hear it:
     61 
     62 ```go
     63 import "git.sr.ht/~jackmordaunt/nativeaudio/play"
     64 
     65 play.File("audio.m4a")
     66 ```
     67 
     68 ## API
     69 
     70 - `New()` returns a `Decoder` holding the platform state. Close it when you
     71   are done. Independent parts of a program can each hold their own, and
     72   closing one does not disturb another.
     73 - `Decoder.DecodeFile(path)` and `Decoder.Decode(data)` return s16le PCM
     74   and a `Format`. The core package has no audio-output dependency.
     75 - `Decoder.StreamFile(path)` and `Decoder.Stream(data)` return a `Stream`,
     76   an `io.Reader` over the same PCM, so a long track never has to sit in
     77   memory whole. `Format` is known before the first read. Close it when
     78   done. Every native backend decodes incrementally, and the sub-process
     79   backend pipes when given a file; only decoding from memory without a
     80   native backend buffers up front.
     81 - `Format` reports `SampleRate`, `Channels` and `BytesPerSample`, which is
     82   always 2.
     83 - A `Decoder` is safe for concurrent use, and `Close` waits for decodes
     84   already in flight.
     85 - `New(WithLimits(...))` bounds how long a decode may run and how much PCM
     86   it may produce. Malformed audio can otherwise make a native decoder
     87   grind indefinitely, so the default budget is finite. Untrusted input
     88   should set its own.
     89 - `FFmpegLoad`, `FFmpegDecode` and `FFmpegStream` shell out to ffmpeg
     90   regardless of platform, and need no `Decoder`.
     91 - The `pcm` subpackage adapts the output to other audio stacks.
     92   `pcm.NewStreamer` presents s16le as pairs of float64 samples, which
     93   satisfies beep's `Streamer` and `StreamCloser` without importing beep,
     94   since Go interfaces are structural. It takes an `io.Reader`, so it works
     95   over a `Stream` or over buffered PCM.
     96 - The `play` subpackage plays PCM through oto. The core package builds
     97   for every target Go supports; `play` is bounded by its audio backend,
     98   which needs cgo on Linux and has no FreeBSD support. Its
     99   context is fixed to the first file's sample rate and channel count for
    100   the life of the process.
    101 
    102 ## Versions
    103 
    104 `v1.1.0` is the first release to use. It replaced the package-level
    105 `Start`, `End`, `Load`, `Decode` and `Play` with a `Decoder` value and the
    106 `play` subpackage, renamed `Format.BitDepth` to `BytesPerSample`, made the
    107 Media Foundation bindings internal, and added streaming decodes, bounded
    108 decodes and the `pcm` adapters.
    109 
    110 `v1.0.0` is retracted. It went out before the API had settled, and it
    111 decoded 24-bit sources to 24-bit rather than the s16le documented here.
    112 Nothing depended on it.
    113 
    114 ## Format support
    115 
    116 Measured by `TestFormatSupport`, which generates a file per format with
    117 ffmpeg and decodes it. CI runs it on each platform and publishes the
    118 result, so this table comes from machines rather than from vendor
    119 documentation.
    120 
    121 | Format | Windows | macOS | Linux and others |
    122 |---|---|---|---|
    123 | AAC-LC in MP4 | yes | yes | yes |
    124 | AAC-LC in ADTS | yes | yes | yes |
    125 | MP3 | yes | yes | yes |
    126 | WAV, 16-bit | yes | yes | yes |
    127 | WAV, 24-bit | yes | yes | yes |
    128 | FLAC | yes | yes | yes |
    129 | ALAC in MP4 | yes | yes | yes |
    130 | Opus in Ogg | yes | yes | yes |
    131 | Vorbis in Ogg | yes | not measured | yes |
    132 | WMA v2 | yes | no | yes |
    133 
    134 Both native backends decode far more than the AAC this package was
    135 written for, so it is useful well beyond that. The only gap either of
    136 them has is WMA on macOS, which AudioToolbox rejects outright rather
    137 than mangling. Vorbis on macOS is unmeasured because the ffmpeg build
    138 there could not produce a file to try.
    139 
    140 Output is s16le whatever goes in, including from a 24-bit source.
    141 Decoded lengths differ slightly between backends for codecs that carry
    142 priming samples, which is why the tests compare against a per-backend
    143 reference rather than one golden file.
    144 
    145 ### AAC profiles
    146 
    147 These deserve separate billing, because a decoder that handles only the
    148 base profile does not fail on the others. High efficiency AAC is an
    149 AAC-LC core plus an extension that older decoders are meant to ignore,
    150 and ignoring it yields audio at half the sample rate with the treble
    151 missing and no error raised. A backend that supports only AAC-LC is
    152 therefore worse than one that rejects the file.
    153 
    154 This table is from vendor documentation rather than measured, because
    155 generating the high efficiency profiles needs an encoder ffmpeg does not
    156 ship by default.
    157 
    158 | Profile | Windows | macOS | ffmpeg |
    159 |---|---|---|---|
    160 | AAC-LC | yes, multichannel | yes | yes |
    161 | HE-AAC v1 | yes, multichannel | yes | yes |
    162 | HE-AAC v2 | yes, stereo | yes | yes |
    163 | xHE-AAC | yes, Windows 11 | yes | partial |
    164 | AAC-LD and AAC-ELD | no | yes | yes |