Add the v1 audio signature to the pipeline (IR-004, IR-005, IR-007, IR-008)
Implements the content-derived spectral-peak signature from JRay-public-server/SPEC.md §3 so a truth file is self-identifying: 120 s window centred on the media midpoint, mono at 11025 Hz, 4096/1024 Hann STFT, 32 log-spaced bins over 300-3000 Hz, one byte per frame (5-bit peak band + 2-bit energy class), base64, `v1:` prefix. Audio decode is a second stream from the FFmpeg libraries the pipeline already links for video; libswresample is added to the existing ffmpeg_libs interface target. The FFT is written out rather than pulled from a library for the same reason the plugin vendors one: the output has to be bit-identical across two languages, so a dependency whose version could change the numerics is a liability. The server spec fixes the geometry but not enough to reproduce a byte stream — Hann periodicity, band aggregation, the energy-class definition, tie-breaking and the base64 alphabet are all unconstrained by it. Those are pinned in audio_signature.hpp and mirrored in the golden fixture, so the plugin can be implemented from the fixture alone. IR-005: tests/fixtures/audio/ carries a deterministic 120 s tone (FLAC — lossless, so identical PCM to the WAV make_fixture.py emits, and 3.5x smaller in git) plus the signature it must produce, the decoded-PCM checksum and the full parameter contract. That directory is the artefact shared with the plugin repo; the PCM checksum is separate from the signature so a codec-level difference is distinguishable from a DSP one. IR-007: media under 120 s emits no signature. Same for a file with no audio stream or one that will not open — UR-9 is an enhancement and must never be able to break a fetch. Verified against an independent Python reference implementation: same bytes. All 32 bands and all 4 energy classes appear in the golden vector, and the window-centring test wraps the fixture in 90 s of silence either side and requires the golden value back. Not wired into the truth-file output yet — that is the schema_version bump under IR-002/IR-003 and is deliberately out of scope here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,158 @@
|
||||
#pragma once
|
||||
// ── JRay audio signature, v1 ─────────────────────────────────────────────────
|
||||
//
|
||||
/// TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003
|
||||
//
|
||||
// A content-derived spectral-peak signature taken from the *centre* of the
|
||||
// media, so a truth file is self-identifying: a consumer can tell whether a
|
||||
// local file is the same cut as the one a manifest describes, and recover the
|
||||
// frame offset when it is the same cut trimmed differently.
|
||||
//
|
||||
// The construction is owned by `JRay-public-server/SPEC.md` §3 and is
|
||||
// reproduced by the JRay Jellyfin plugin in C#. **The two implementations must
|
||||
// agree byte for byte** — a signature that differs in any parameter simply does
|
||||
// not match, which defeats the entire point. Every deviation is therefore a
|
||||
// breaking change and must go through the `v1:` prefix (see kVersionPrefix).
|
||||
//
|
||||
// Server spec §3, restated:
|
||||
//
|
||||
// 1. Decode a 120 s window centred on the midpoint (runtime/2 ± 60 s).
|
||||
// 2. Downmix to mono, resample to 11025 Hz.
|
||||
// 3. STFT: 4096-sample frame, 1024-sample hop, Hann window (~1290 frames).
|
||||
// 4. Per frame, log-magnitude spectrum over 300–3000 Hz.
|
||||
// 5. 32 logarithmically spaced bins; peak bin index + coarse 2-bit energy
|
||||
// class.
|
||||
// 6. Pack one byte per frame; base64-encode.
|
||||
// 7. Prefix `v1:`.
|
||||
//
|
||||
// ── Details the server spec leaves open, pinned here for v1 ──────────────────
|
||||
//
|
||||
// The prose above is not sufficient to reproduce a byte stream, so the choices
|
||||
// below are the contract. They are mirrored in
|
||||
// `tests/fixtures/audio/jray_audio_v1_golden.json`, which is the artefact
|
||||
// shared with the plugin repo (IR-005).
|
||||
//
|
||||
// Arithmetic All DSP in IEEE-754 **double**. float32 is not sufficient:
|
||||
// the golden fixture has frames whose two strongest bands are
|
||||
// within 1.3% of each other, which double resolves identically
|
||||
// everywhere and float32 does not.
|
||||
// Sample scale FFmpeg's native s16→flt conversion, x * (1/32768), then
|
||||
// widened to double. Values in [-1, 1).
|
||||
// Framing Only whole frames: n_frames = 1 + (n_samples - 4096) / 1024,
|
||||
// integer division, 0 when n_samples < 4096. A 120.000 s
|
||||
// window is 1 323 000 samples → **1288 frames**.
|
||||
// ("~1290" in the spec; the server accepts a tolerance.)
|
||||
// Window Hann, **periodic**: w[n] = 0.5 * (1 - cos(2*pi*n/4096)).
|
||||
// Not the symmetric (N-1) variant.
|
||||
// Transform Plain radix-2 decimation-in-time complex FFT over 4096 real
|
||||
// samples (imag = 0), no normalisation. Magnitude is
|
||||
// sqrt(re² + im²). Twiddles from cos/sin of
|
||||
// -2*pi*k/len computed in double.
|
||||
// Band edges edge[b] = 300 * (3000/300)^(b/32), b = 0..32. Band b spans
|
||||
// FFT bins [k_lo[b], k_lo[b+1]) with
|
||||
// k_lo[b] = ceil(edge[b] * 4096 / 11025) — i.e. bins 112..1114
|
||||
// inclusive, 8 bins in the narrowest band. Precomputed as an
|
||||
// integer table so no float comparison decides membership.
|
||||
// Band value **Mean** of the linear magnitudes in the band. Mean, not
|
||||
// sum, so a wide high band is not favoured over a narrow low
|
||||
// one; magnitude, not power, because it is an energy proxy and
|
||||
// more codec-robust than a single bin's peak.
|
||||
// Peak bin argmax over the 32 band values; ties resolve to the **lowest
|
||||
// index**. The log of step 4 is a monotone squash and so
|
||||
// cannot change an argmax — it is applied only where it is
|
||||
// observable, in the energy class below.
|
||||
// Energy class The spec says "coarse 2-bit energy class" and no more. v1
|
||||
// defines it as the frame's band-limited energy relative to
|
||||
// the window, which is invariant to gain (loudness
|
||||
// normalisation must not change a signature) and robust to
|
||||
// trimming (the median barely moves):
|
||||
// E_f = mean magnitude over *all* FFT bins 112..1114
|
||||
// Eref = median over frames of E_f, taken as the upper
|
||||
// median sorted[n/2] — no averaging of the two middle
|
||||
// values, so the reference is always an actual
|
||||
// observed value and is bit-reproducible
|
||||
// r = log10((E_f + 1e-12) / (Eref + 1e-12))
|
||||
// class = 0 if r < -0.6, 1 if r < -0.2, 2 if r < 0.2, else 3
|
||||
// The thresholds deliberately straddle r = 0 rather than sit
|
||||
// on it, so the median frame itself is not on a boundary.
|
||||
// Byte layout bit 7 = 0 (reserved), bits 6..2 = 5-bit band index,
|
||||
// bits 1..0 = 2-bit energy class:
|
||||
// byte = (band << 2) | class → always 0..127
|
||||
// This is the structural constraint the server validates on
|
||||
// upload (§3 "Validation and abuse").
|
||||
// Base64 Standard alphabet A–Za–z0–9+/ with '=' padding.
|
||||
//
|
||||
// ── Short media (IR-007) ─────────────────────────────────────────────────────
|
||||
//
|
||||
// `runtime/2 ± 60 s` underflows below 120 s, so **no signature is emitted** and
|
||||
// no sync offset is applied downstream. Both producers apply the identical
|
||||
// rule; diverging here would break exactly the short items most likely to be
|
||||
// misidentified. `compute_signature` returns `std::nullopt`.
|
||||
//
|
||||
// The same nullopt is returned for a file with no audio stream, an unopenable
|
||||
// file, or an unknown duration. UR-9 is an enhancement and must never be able
|
||||
// to break a fetch — degradation, not failure.
|
||||
|
||||
#include <array>
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <optional>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
namespace sae::audio {
|
||||
|
||||
// ── Contract constants — changing any of these is a `v1:` bump ───────────────
|
||||
inline constexpr int kSampleRate = 11025;
|
||||
inline constexpr int kFrameSize = 4096;
|
||||
inline constexpr int kHopSize = 1024;
|
||||
inline constexpr int kNumBands = 32;
|
||||
inline constexpr double kBandLoHz = 300.0;
|
||||
inline constexpr double kBandHiHz = 3000.0;
|
||||
inline constexpr double kWindowSec = 120.0;
|
||||
inline constexpr double kEnergyEps = 1e-12;
|
||||
// Class thresholds on log10(E_frame / E_median); see the header comment.
|
||||
inline constexpr double kEnergyClassEdges[3] = {-0.6, -0.2, 0.2};
|
||||
// 120.000 s at 11025 Hz. The decoded window is truncated to exactly this so the
|
||||
// frame count does not wobble with seek granularity or resampler tail.
|
||||
inline constexpr std::size_t kWindowSamples =
|
||||
static_cast<std::size_t>(kWindowSec * kSampleRate); // 1 323 000
|
||||
inline constexpr std::size_t kExpectedFrames =
|
||||
1 + (kWindowSamples - kFrameSize) / kHopSize; // 1288
|
||||
static_assert(kWindowSamples == 1323000, "120 s at 11025 Hz");
|
||||
static_assert(kExpectedFrames == 1288, "server spec's ~1290 frames");
|
||||
|
||||
/// The version prefix is the signature's own, separate from `schema_version`:
|
||||
/// a future change to the DSP chain must be *detectable* rather than silently
|
||||
/// producing non-matching signatures (IR-008).
|
||||
inline constexpr const char* kVersionPrefix = "v1:";
|
||||
|
||||
/// FFT bin range [first, last) for each of the 32 log-spaced bands.
|
||||
/// Computed once from the constants above; exposed so the golden fixture can
|
||||
/// assert the table itself, not merely the signature it produces.
|
||||
const std::array<std::pair<int, int>, kNumBands>& band_fft_bins();
|
||||
|
||||
/// Decode the centre window of `path` as mono float PCM at 11025 Hz.
|
||||
/// nullopt when the media is shorter than 120 s (IR-007), has no audio stream,
|
||||
/// or cannot be opened. Never throws.
|
||||
std::optional<std::vector<float>> decode_centre_window(const std::string& path);
|
||||
|
||||
/// One packed byte per whole STFT frame. Empty when `mono` is shorter than one
|
||||
/// frame. This is the payload that gets base64-encoded.
|
||||
std::vector<std::uint8_t> pack_frames(const std::vector<float>& mono);
|
||||
|
||||
/// `v1:` + base64(pack_frames(mono)). nullopt when no whole frame fits.
|
||||
std::optional<std::string> signature_from_mono(const std::vector<float>& mono);
|
||||
|
||||
/// Decode + sign. The one call the pipeline makes. nullopt per IR-007 and on
|
||||
/// any decode failure — degradation, not failure.
|
||||
std::optional<std::string> compute_signature(const std::string& path);
|
||||
|
||||
// ── Small utilities, exposed for the golden-fixture test ────────────────────
|
||||
std::string base64_encode(const std::uint8_t* data, std::size_t n);
|
||||
/// FNV-1a 64. Used only to pin the *decoded PCM* in the golden fixture, so a
|
||||
/// codec-level difference is distinguishable from a DSP-level one.
|
||||
std::uint64_t fnv1a64(const void* data, std::size_t n);
|
||||
|
||||
} // namespace sae::audio
|
||||
Reference in New Issue
Block a user