feat(audio): align a fetched manifest to the local file before storing

The signature had a producer and a reader but no consumer, so nothing
ever fingerprinted anything. `ManifestAligner` runs on the fetch path,
before the windows are stored.

A local alignment supersedes the server's offset. The server has never
seen this file — its offset is a runtime-difference inference at best,
while the local comparison is against the media the windows will actually
be drawn over. It also needs no round trip, so no signature leaves the
instance. This is what jRay's spec already meant by matching being a
consumer concern: the server never rewrites a manifest, so one stored
manifest serves every trim of the same cut.

The offset has two terms and only one is in the server's pseudocode. Both
windows are centred on their own file's midpoint, so unequal runtimes
start them at different absolute times; a release with 40 s of extra head
material recovers 20 s from the slide and 20 s from the anchor
difference. Using the slide alone is wrong by half the runtime difference
on every shifted release.

Degradation, never failure. Signatures off, no manifest signature, media
under the window, a `v2:` producer, a missing binary, a decode error —
each applies the server's offset rather than refusing, because a
signature is an enhancement to cut matching and must never break a fetch.

"Un-comparable" and "does not match" are kept distinct, which a test
caught: `Compare` returns null for both, and conflating them would report
a 90-second extra as content disagreeing with its own manifest. A genuine
disagreement is stored anyway — the audio may legitimately differ, a
different language track being the obvious case — and surfaced as a
caveat that outranks the tier's, since it is the stronger statement.

The applied offset, score, slide and the local file's own signature are
written beside the truth file: the offset is otherwise unrecoverable once
the windows are shifted, and the stored signature lets a later fetch
align without decoding again. Provenance is never injected into the truth
file, so the bytes served back stay the producer's (JR-004).

`docs/audio-alignment.md` documents the mechanism end to end.

TRACES: JR-047 | SR-003
This commit is contained in:
2026-07-31 16:52:21 +02:00
parent e8ce779ad3
commit 1e247c4c7d
12 changed files with 981 additions and 29 deletions
+13 -1
View File
@@ -72,6 +72,11 @@ Tag code with `// TRACES: JR-012 | SR-002`.
| UT-050 | Identical signatures score 1.0 at offset 0 and reach the `audio` tier | JR-044 | **Passing** |
| UT-051 | **A shifted release recovers its offset** rather than failing to match — the case the feature exists for | JR-044 | **Passing** |
| UT-052 | Unrelated content yields **no match at all**, and runtime skew contributes its window-anchor term to the offset | JR-044 | **Passing** |
| UT-053 | **A local alignment supersedes the server's offset**, and the server's claim is retained rather than overwritten | JR-047 | **Passing** |
| UT-054 | The local signature is recorded, so a later fetch aligns without decoding the media again | JR-047 | **Passing** |
| UT-055 | Every unavailable local path — off, no manifest signature, neither, short media, **and a `v2:` producer** — falls back to the server rather than refusing | JR-047 | **Passing** |
| UT-056 | Two signatures that genuinely disagree are recorded as a mismatch and **still do not break the fetch** | JR-047 | **Passing** |
| UT-057 | A mismatch **outranks the tier** in the caveat shown to the user — a `runtime` match would otherwise show nothing at all | JR-047 | **Passing** |
All execute and pass. UT-043 and UT-044 are the two that need an FFmpeg binary,
which the plugin gets from Jellyfin at run time and a bare CI container may not
@@ -216,18 +221,24 @@ preserved.
| JR-040 | The config page states plainly that **each configured server multiplies the exposure** | **PR-005** | Medium | Planned |
| JR-041 | The plugin never fetches, stores, or transmits gallery data — reference faces or embeddings. It has no gallery code path at all | **SR-005** | High | Done |
## Audio signature (JR-042 … JR-045)
## Audio signature (JR-042 … JR-045, JR-047)
Mirror-image of extraction `IR-004`/`IR-005`/`IR-007`/`IR-008`. Both producers
must agree **bit-for-bit**, so each obligation is stated on both sides rather
than assumed to be inherited.
JR-042 … JR-045 are the signature itself, produced and read. **JR-047 is what
uses it**: without a consumer on the fetch path the other four are a fingerprint
nothing ever fingerprints. See [`audio-alignment.md`](audio-alignment.md) for the
end-to-end mechanism.
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| JR-042 | Compute the signature **exactly** per server spec §3, using the FFmpeg binary Jellyfin already ships via `IMediaEncoder.EncoderPath` — no new dependency | SR-003 | Medium | **Done** (UT-039…044) — `AudioSignature` is the DSP, `AudioSignatureService` the decode; FFmpeg is invoked as a child process for decode, downmix and resample, and nothing was added to the project's dependencies |
| JR-043 | Golden-vector fixture **shared with the extraction repo**, proving the two implementations are bit-exact | SR-003 | High | **Done** (UT-038, UT-039) — `fixtures/audio/` holds the extraction repo's three files byte-identically; the C# signature equals the recorded vector exactly |
| JR-044 | Media shorter than 120 s: emit no signature and apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** (UT-045, UT-046, UT-050…052) — the producer half was already in `AudioSignatureService`; the consumer half needed a reader, so `AudioSignatureMatcher` implements the §3 slide and declines an offset outright below the window. The boundary is asserted on one file at 119.999 s and 120.000 s |
| JR-045 | Emit and honour the signature's own `v1:` prefix, so a DSP change is detectable rather than silently non-matching | SR-003 | Low | **Done** (UT-047…049) — `AudioSignatureMatcher.TryParseFrames` refuses any prefix but `v1:`, and refuses malformed or structurally invalid payloads, so a future producer's `v2:` drops the item to the runtime tier instead of scoring as if it were understood |
| JR-047 | **A fetched manifest is aligned against the local file before its windows are stored**, and the alignment is recorded beside the truth data | SR-003 | High | **Done** (UT-053…057) — `ManifestAligner` runs on the fetch path. A local alignment supersedes the server's offset, since the server has never seen this file; every unavailable path degrades to the server's offset rather than refusing, and a genuine signature disagreement is recorded and surfaced as a caveat without failing the fetch |
## Human-in-the-loop association (JR-046)
@@ -355,6 +366,7 @@ framework reference and leans on `RollForward` to reach the 10.0 runtime.
| JR-043 | **T1** | Signature matches the shared golden vector **bit-for-bit** | The fixture PCM is regenerated from `make_fixture.py` and checked against the recorded decode checksums first, so the check binds on a host with no codec and a decode divergence is distinguishable from a DSP one |
| JR-044 | T1 | Media < 120 s yields no signature and no offset | Exactly 120 s — the boundary both repos must agree on |
| JR-045 | T1 | `v1:` emitted; an unknown prefix is refused, not parsed | `v2:` signature from a future producer |
| JR-047 | **T1** | A fetched manifest is aligned locally before storage, and the alignment is recorded | **Every way the local path can be unavailable must degrade to the server's offset, never refuse** — signatures off, no manifest signature, media under the window, a `v2:` producer. Distinguish those from a genuine mismatch: a 90 s extra is not content that disagrees with its manifest, and telling a user it is would be worse than saying nothing |
| JR-046 | T2 + **T4** | *Assertions deferred* — recording an association and persisting it is T2; the review UI itself is T4 | Cannot be written until the truth-file interface for unidentified presence is settled (system open question 2) and AR-021/AR-022 land |
Three are worth singling out. **JR-021** and **JR-041** are static checks because