Author SHA1 Message Date
dtourolle 55b4633324 chore(traces): regenerate the traceability report 2026-08-04 14:16:25 +02:00
dtourolle 8c5fb5950d docs: the matcher implements a superseded version of §3
JRay-public-server UR-009 changed the §3 score to count a frame as
agreeing within ±1 frame, on scene-actor-extraction VR-014's measurement:
over 40 correctly recovered offsets on real film audio the exact-frame
rule scored 27 of them below 0.85 and demoted them to `loose`, because
the two windows are cut on their own file's frame grid and those grids
do not coincide.

`AudioSignatureMatcher` still implements the exact rule. Nothing is
misaligned by the divergence — the recovered offset is unaffected, and
JR-047 makes the local answer supersede the server's — but the plugin
will show a caveat for a match that is not in doubt. Recorded against
JR-044 rather than fixed here, so the gap is findable.

TRACES: JR-044 | SR-003
2026-07-31 22:43:37 +02:00
dtourolle 1e247c4c7d feat(audio): align a fetched manifest to the local file before storing
The signature had a producer and a reader but no consumer, so nothing
ever fingerprinted anything. `ManifestAligner` runs on the fetch path,
before the windows are stored.

A local alignment supersedes the server's offset. The server has never
seen this file — its offset is a runtime-difference inference at best,
while the local comparison is against the media the windows will actually
be drawn over. It also needs no round trip, so no signature leaves the
instance. This is what jRay's spec already meant by matching being a
consumer concern: the server never rewrites a manifest, so one stored
manifest serves every trim of the same cut.

The offset has two terms and only one is in the server's pseudocode. Both
windows are centred on their own file's midpoint, so unequal runtimes
start them at different absolute times; a release with 40 s of extra head
material recovers 20 s from the slide and 20 s from the anchor
difference. Using the slide alone is wrong by half the runtime difference
on every shifted release.

Degradation, never failure. Signatures off, no manifest signature, media
under the window, a `v2:` producer, a missing binary, a decode error —
each applies the server's offset rather than refusing, because a
signature is an enhancement to cut matching and must never break a fetch.

"Un-comparable" and "does not match" are kept distinct, which a test
caught: `Compare` returns null for both, and conflating them would report
a 90-second extra as content disagreeing with its own manifest. A genuine
disagreement is stored anyway — the audio may legitimately differ, a
different language track being the obvious case — and surfaced as a
caveat that outranks the tier's, since it is the stronger statement.

The applied offset, score, slide and the local file's own signature are
written beside the truth file: the offset is otherwise unrecoverable once
the windows are shifted, and the stored signature lets a later fetch
align without decoding again. Provenance is never injected into the truth
file, so the bytes served back stay the producer's (JR-004).

`docs/audio-alignment.md` documents the mechanism end to end.

TRACES: JR-047 | SR-003
2026-07-31 16:52:21 +02:00
dtourolle e8ce779ad3 docs: land the registers for schema v2 and the audio signature
Status and verification rows for the six requirements landed in this
branch, plus UT-029 … UT-052, and the generated traceability matrix that
`docs/traceability.md` holds in the other two components but was missing
here. `traces-report.json` is gitignored to match the server: the matrix
is committed, the JSON report is not, since nothing reads it back.

Two corrections rather than additions:

`AudioSignatureTests` was tagged UT-029 … UT-035, IDs the register had
already assigned to the schema tests. Renumbered to UT-038 … UT-044,
matching the register, which was right — duplicate IDs defeat the point
of IDs being permanent.

`ReadCappedAsync` implements JR-028's response cap and carried no tag,
so the requirement read as uncovered.

The gate reports 0 orphan tags.

TRACES: JR-002, JR-003, JR-028, JR-041, JR-042, JR-043, JR-044, JR-045 | SR-003
2026-07-31 16:24:40 +02:00
dtourolle 8b430d53c0 test(gallery): verify the no-gallery-data prohibition by absence
JR-041 is a requirement to *not* do something, so like JR-021 it can only
be verified by absence. Mirrors the server's UR-012, which is preserved
the same way.

The check looks for any *parse* of gallery data — a field name, a
property, a type — not merely for network calls: an embedding arriving
here would mean SR-005 had already been breached upstream.

TRACES: JR-041 | SR-005
2026-07-31 16:24:28 +02:00
dtourolle c863fe85f5 feat(audio): signature reader, offset recovery, and version refusal
Closes the consumer halves of JR-044 and JR-045, which were blocked on
there being no reader at all. `AudioSignatureMatcher` implements the
specification's slide — ±600 frames, scoring the fraction of overlapping
frames whose peak band agrees — and returns the tier and offset.

JR-045: `TryParseFrames` refuses any prefix but `v1:`. A `v2:` signature
from a future producer describes a DSP chain this build does not
implement, so scoring it as v1 would be a confident wrong answer where
declining is a correct one — the item drops to the runtime tier, which is
the entire reason the prefix is separate from `schema_version`.

JR-044: a runtime under 120 s yields no match and therefore no offset,
read off the runtime rather than inferred from a missing string, because
the runtime is what both producers test. The boundary is asserted on one
file at 119.999 s and 120.000 s, so a null cannot be blamed on the decode.

The offset has two terms, which is easy to miss: the recovered slide, and
the difference between where the two windows are anchored, since both are
centred on their own file's midpoint. A release carrying 40 s of extra
head material recovers 20 s from each.

One parameter is not from the specification and is marked as such in the
code: an alignment must overlap by at least 64 frames before its score
counts, or the extreme offsets compare a handful of frames where a chance
agreement scores 1.0 and beats the true alignment.

TRACES: JR-044, JR-045 | SR-003
2026-07-31 16:24:23 +02:00
dtourolle 17be9bcc4e feat(audio): v1 signature producer, bit-exact with the pipeline
`AudioSignature` is the DSP — band table, periodic Hann, radix-2 FFT,
band-mean peak, energy class, packing — and `AudioSignatureService` the
decode, running the FFmpeg binary `IMediaEncoder.EncoderPath` names. The
plugin gained no dependency.

The server specification's prose does not determine a byte stream, so the
parameters it leaves open are pinned by the fixture shared with the
extraction repo and restated at the top of `AudioSignature`: double
throughout, whole frames only, periodic Hann, unnormalised FFT, band mean
rather than sum, argmax ties to the lowest index.

`fixtures/audio/` holds the extraction repo's three files byte-identically
and the computed signature equals the recorded vector exactly. The binding
check regenerates the fixture PCM from `make_fixture.py`'s arithmetic and
verifies it against the recorded decode checksums, so it runs on a host
with no codec at all and a decode divergence stays distinguishable from a
DSP one; the two tests that drive real FFmpeg self-skip without a binary.

The workflow named "Test Plugin" until now only compiled one. A test that
is built and never run is not evidence, and a golden vector shared across
two repos exists precisely so CI fails when they drift.

TRACES: JR-042, JR-043 | SR-003
2026-07-31 16:24:11 +02:00
dtourolle 19aecee646 feat(truth): schema_version 2 read path, v2 only
Replaces the v1 shape rather than accepting both. `anneal_sec` and the
top-level `sample_fps` are deleted, not zeroed; `extraction` and `cut`
blocks arrive; `scenes` become objects carrying belief and route, so a
window records how far to trust it instead of being a bare float pair.

`TruthSchema.IsSupported` is the single gate and is applied on all four
read paths — sidecar, managed store load, managed PUT, and converted
manifest. Previously only the controller checked, so the version the
plugin claimed to require and the one it would actually parse were free
to drift. Rejections name the file and the version found, so an item that
looks empty is distinguishable from one that was refused.

`ManifestConverter` carries belief, route and both provenance blocks
through: dropping them would silently downgrade every fetched manifest
against a locally extracted one.

TRACES: JR-002, JR-003 | SR-003
2026-07-31 16:23:58 +02:00
40 changed files with 4434 additions and 85 deletions
+10
View File
@@ -42,6 +42,16 @@ jobs:
working-directory: test-${{ github.run_id }} working-directory: test-${{ github.run_id }}
run: dotnet build Jellyfin.Plugin.JRay.sln --configuration Debug --no-restore --no-self-contained /m:1 run: dotnet build Jellyfin.Plugin.JRay.sln --configuration Debug --no-restore --no-self-contained /m:1
# The workflow is named "Test Plugin" and until now only compiled one. A
# test that is built but never run is not evidence, and JR-043 is the case
# that makes it matter: the point of a golden vector shared with the
# extraction repo is that CI fails when the two implementations drift.
# T1 needs no ASP.NET runtime and no FFmpeg — the audio golden check
# regenerates its own fixture PCM.
- name: Run tests
working-directory: test-${{ github.run_id }}
run: dotnet test Jellyfin.Plugin.JRay.sln --configuration Debug --no-restore --no-build
- name: Cleanup - name: Cleanup
if: always() if: always()
run: rm -rf test-${{ github.run_id }} run: rm -rf test-${{ github.run_id }}
+9
View File
@@ -3,3 +3,12 @@ obj/
.vs/ .vs/
.idea/ .idea/
artifacts artifacts
# Python artefacts from the traceability tooling
__pycache__/
*.pyc
# Generated traceability output — regenerate with the gate, never hand-edit.
# `docs/traceability.md` is committed, as in the other two components; the JSON
# report is not, since nothing reads it back.
traces-report.json
@@ -0,0 +1,270 @@
using System;
using System.IO;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Plugin.JRay.Configuration;
using Jellyfin.Plugin.JRay.Services;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Jellyfin.Plugin.JRay.Tests;
/// <summary>
/// JR-044 (media shorter than 120 s emits no signature and takes no sync offset)
/// and JR-045 (the `v1:` prefix is emitted *and honoured*).
///
/// Both requirements have a producer half and a consumer half. The producer
/// halves live with <see cref="AudioSignature"/> and were closed by
/// UT-038 … UT-044; these are the consumer halves, which needed a reader —
/// <see cref="AudioSignatureMatcher"/> — before they could be closed at all.
///
/// The distinction that matters throughout: **refusing is a correct answer, and
/// a silently wrong one is not.** An unreadable or out-of-version signature must
/// drop the item to the runtime tier, never score as if it were understood.
///
/// TRACES: UT-045, UT-046, UT-047, UT-048, UT-049, UT-050, UT-051, UT-052 | JR-044, JR-045
/// </summary>
public class AudioSignatureMatcherTests
{
private static readonly string FixtureDir =
Path.Combine(AppContext.BaseDirectory, "fixtures", "audio");
private static readonly Lazy<string> GoldenSignature = new(() =>
JsonDocument.Parse(File.ReadAllText(Path.Combine(FixtureDir, "jray_audio_v1_golden.json")))
.RootElement.GetProperty("signature").GetString()!);
private static readonly Lazy<byte[]> GoldenFrames = new(() =>
AudioSignatureMatcher.TryParseFrames(GoldenSignature.Value)!);
// A feature-length runtime, so the window-anchor term is exercised at a
// realistic scale rather than at the 120 s boundary.
private const double FeatureRuntime = 7200.0;
// UT-045 — JR-044, the producer boundary.
[Fact]
public async Task ShortMedia_YieldsNoSignature_AndExactly120sDoes()
{
// Decisive because it is the *same file* either side of the boundary:
// only the runtime differs, so a null cannot be blamed on the decode.
// The fixture is exactly 120.000 s, which is the boundary itself.
var fixture = Path.Combine(FixtureDir, "jray_audio_v1_tone.flac");
// Below the window: refused before the encoder is ever consulted, which
// is what the deliberately invalid path proves.
Assert.Null(await AudioSignatureService.ComputeWithEncoderAsync(
"/nonexistent/ffmpeg",
fixture,
AudioSignature.WindowSec - 0.001,
NullLogger.Instance,
CancellationToken.None).ConfigureAwait(true));
var ffmpeg = FindFfmpeg();
if (ffmpeg is null)
{
return;
}
Assert.Null(await AudioSignatureService.ComputeWithEncoderAsync(
ffmpeg,
fixture,
AudioSignature.WindowSec - 0.001,
NullLogger.Instance,
CancellationToken.None).ConfigureAwait(true));
// Exactly at the boundary the window fits, so a signature is emitted.
// Both producers must agree here or they diverge on precisely the short
// items most likely to be misidentified.
Assert.Equal(
GoldenSignature.Value,
await AudioSignatureService.ComputeWithEncoderAsync(
ffmpeg,
fixture,
AudioSignature.WindowSec,
NullLogger.Instance,
CancellationToken.None).ConfigureAwait(true));
}
// UT-046 — JR-044, the consumer half.
[Fact]
public void ShortMedia_TakesNoOffset_EvenWithTwoValidSignatures()
{
var signature = GoldenSignature.Value;
// Two identical, perfectly valid signatures — the strongest possible
// match — still yield nothing when either side is under the window. The
// rule is checked on the runtime, not inferred from a missing string,
// because the runtime is what both producers test.
Assert.Null(AudioSignatureMatcher.Compare(
signature, signature, AudioSignature.WindowSec - 0.001, FeatureRuntime));
Assert.Null(AudioSignatureMatcher.Compare(
signature, signature, FeatureRuntime, AudioSignature.WindowSec - 0.001));
// At exactly the boundary it matches, so the refusal above is the
// threshold and not a blanket refusal.
Assert.NotNull(AudioSignatureMatcher.Compare(
signature, signature, AudioSignature.WindowSec, AudioSignature.WindowSec));
}
// UT-047 — JR-045, the requirement's whole point.
[Fact]
public void UnknownVersionPrefix_IsRefused_NotParsed()
{
// A v2 signature from a future producer, whose payload is byte-identical
// to a valid v1 one. Parsing it as v1 would yield a confident, plausible,
// wrong score; refusing drops the item to the runtime tier, which is
// correct. This is the entire reason the prefix is separate from
// schema_version.
var payload = GoldenSignature.Value[AudioSignature.VersionPrefix.Length..];
Assert.Null(AudioSignatureMatcher.TryParseFrames("v2:" + payload));
Assert.Null(AudioSignatureMatcher.TryParseFrames("v10:" + payload));
Assert.Null(AudioSignatureMatcher.TryParseFrames(payload)); // no prefix at all
Assert.Null(AudioSignatureMatcher.TryParseFrames("V1:" + payload)); // case is not cosmetic
// And it must be refused by the matcher too, not merely by the parser —
// a v2 signature produces no match and therefore no offset.
Assert.Null(AudioSignatureMatcher.Compare(
GoldenSignature.Value, "v2:" + payload, FeatureRuntime, FeatureRuntime));
Assert.Null(AudioSignatureMatcher.Compare(
"v2:" + payload, GoldenSignature.Value, FeatureRuntime, FeatureRuntime));
}
// UT-048 — JR-045, the accepting side.
[Fact]
public void V1Signature_ParsesToExactlyTheProducedFrames()
{
// The reader is the inverse of the producer, checked against the golden
// vector rather than against the producer's own output, so the two are
// pinned to the fixture and not merely to each other.
var frames = AudioSignatureMatcher.TryParseFrames(GoldenSignature.Value);
Assert.NotNull(frames);
Assert.Equal(AudioSignature.ExpectedFrames, frames!.Length);
Assert.Equal(GoldenSignature.Value, AudioSignature.VersionPrefix + Convert.ToBase64String(frames));
}
// UT-049 — JR-045, structural refusal.
[Fact]
public void MalformedSignatures_AreRefused_WithoutThrowing()
{
Assert.Null(AudioSignatureMatcher.TryParseFrames(null));
Assert.Null(AudioSignatureMatcher.TryParseFrames(string.Empty));
Assert.Null(AudioSignatureMatcher.TryParseFrames("v1:")); // empty payload
Assert.Null(AudioSignatureMatcher.TryParseFrames("v1:not!base64!"));
// Bit 7 is reserved by the packing — a byte with it set is not a frame.
// The server refuses this on upload; the client must not accept what the
// server would have rejected.
Assert.Null(AudioSignatureMatcher.TryParseFrames(
AudioSignature.VersionPrefix + Convert.ToBase64String(new byte[] { 0x04, 0x80, 0x08 })));
}
// UT-050 — the aligned case.
[Fact]
public void IdenticalSignatures_ScorePerfectly_AtZeroOffset()
{
var match = AudioSignatureMatcher.Compare(
GoldenSignature.Value, GoldenSignature.Value, FeatureRuntime, FeatureRuntime);
Assert.NotNull(match);
Assert.Equal(1.0, match!.Value.Score);
Assert.Equal(0, match.Value.OffsetFrames);
Assert.Equal(0.0, match.Value.OffsetSec);
Assert.Equal(MatchTier.Audio, match.Value.Tier);
}
// UT-051 — the case the feature exists for.
[Fact]
public void AShiftedRelease_RecoversTheOffset_RatherThanFailingToMatch()
{
// A release trimmed differently from the one the manifest was built on:
// the same cut, sampled at a different point. Before offset recovery this
// failed the runtime tier outright; the recovered shift is what makes one
// stored manifest serve every trim.
const int Shift = 100;
const int Span = 1000;
const int RemoteStart = 144;
var source = GoldenFrames.Value;
var remote = Signature(source, RemoteStart, Span);
var local = Signature(source, RemoteStart - Shift, Span);
var match = AudioSignatureMatcher.Compare(local, remote, FeatureRuntime, FeatureRuntime);
Assert.NotNull(match);
Assert.Equal(1.0, match!.Value.Score);
Assert.Equal(Shift, match.Value.OffsetFrames);
Assert.Equal(MatchTier.Audio, match.Value.Tier);
// Equal runtimes, so the window-anchor term vanishes and the offset is
// purely the recovered slide.
Assert.Equal(Shift * AudioSignatureMatcher.FrameSeconds, match.Value.OffsetSec, 9);
}
// UT-052 — the rejecting case, and the anchor term.
[Fact]
public void UnrelatedContent_DoesNotMatch_AndRuntimeSkewShiftsTheOffset()
{
// Two independent band sequences agree about 1 frame in 32, far below the
// loose floor. A matcher that returned its argmax regardless would hand
// back a confident alignment for unrelated films.
Assert.Null(AudioSignatureMatcher.Compare(
GoldenSignature.Value, PseudoRandomSignature(1288, seed: 12345), FeatureRuntime, FeatureRuntime));
// Both windows are centred on their own file's midpoint, so when the
// runtimes differ the windows start at different absolute times and that
// difference is part of the offset. Without this term the offset would be
// wrong by half the runtime difference on every shifted release.
var skew = 40.0;
var match = AudioSignatureMatcher.Compare(
GoldenSignature.Value,
GoldenSignature.Value,
FeatureRuntime + skew,
FeatureRuntime);
Assert.NotNull(match);
Assert.Equal(0, match!.Value.OffsetFrames);
Assert.Equal(skew / 2.0, match.Value.OffsetSec, 9);
}
private static string Signature(byte[] source, int start, int count)
=> AudioSignature.VersionPrefix + Convert.ToBase64String(source, start, count);
private static string PseudoRandomSignature(int frames, int seed)
{
var bytes = new byte[frames];
var state = (uint)seed;
for (var i = 0; i < frames; i++)
{
// Deterministic LCG — a fixed sequence, so a failure here is
// reproducible rather than flaky.
state = (state * 1664525u) + 1013904223u;
bytes[i] = (byte)((((state >> 16) % AudioSignature.NumBands) << 2) | ((state >> 8) & 0x03));
}
return AudioSignature.VersionPrefix + Convert.ToBase64String(bytes);
}
private static string? FindFfmpeg()
{
var configured = Environment.GetEnvironmentVariable("JRAY_TEST_FFMPEG");
if (!string.IsNullOrEmpty(configured))
{
return File.Exists(configured) ? configured : null;
}
var name = OperatingSystem.IsWindows() ? "ffmpeg.exe" : "ffmpeg";
foreach (var dir in (Environment.GetEnvironmentVariable("PATH") ?? string.Empty)
.Split(Path.PathSeparator, StringSplitOptions.RemoveEmptyEntries))
{
var candidate = Path.Combine(dir, name);
if (File.Exists(candidate))
{
return candidate;
}
}
return null;
}
}
@@ -0,0 +1,366 @@
using System;
using System.Buffers.Binary;
using System.Globalization;
using System.IO;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Plugin.JRay.Services;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Jellyfin.Plugin.JRay.Tests;
/// <summary>
/// JR-042 (the signature is computed exactly per server specification §3) and
/// JR-043 (the golden-vector fixture shared with the extraction repo).
///
/// The headline claim is cross-repo: the C++ pipeline and this plugin are two
/// independent implementations of one fingerprint, and two fingerprints that
/// differ in any parameter simply do not match. That claim is only worth
/// anything if it is checked, so `fixtures/audio/` holds the same three files
/// the extraction repo holds — byte-identical — and these tests assert against
/// the values recorded in them, never against each other.
///
/// The binding check needs no FFmpeg. `make_fixture.py` generates the fixture
/// media from plain arithmetic, so UT-038 regenerates that PCM here and proves
/// it is byte-identical to what the pipeline decoded, using the checksums the
/// fixture records. Everything after that is pure DSP, which is what lets this
/// run on any CI host — the extraction repo's counterpart (UT-101) drives the
/// same vector through libavcodec, and both must land on the same string.
///
/// TRACES: UT-038, UT-039, UT-040, UT-041, UT-042, UT-043, UT-044 | JR-042, JR-043
/// </summary>
public class AudioSignatureTests
{
private static readonly string FixtureDir =
Path.Combine(AppContext.BaseDirectory, "fixtures", "audio");
private static readonly Lazy<JsonDocument> GoldenDoc = new(() =>
JsonDocument.Parse(File.ReadAllText(Path.Combine(FixtureDir, "jray_audio_v1_golden.json"))));
private static readonly Lazy<short[]> FixturePcm = new(GenerateFixturePcm);
private static readonly Lazy<float[]> FixtureSamples = new(() =>
{
var pcm = FixturePcm.Value;
var samples = new float[pcm.Length];
for (var i = 0; i < pcm.Length; i++)
{
// FFmpeg's native s16 -> flt conversion. 1/32768 is a power of two,
// so this is exact rather than merely close.
samples[i] = pcm[i] / 32768f;
}
return samples;
});
private static JsonElement Golden => GoldenDoc.Value.RootElement;
private static string GoldenSignature => Golden.GetProperty("signature").GetString()!;
// UT-038
[Fact]
public void RegeneratedFixturePcm_MatchesTheRecordedDecodedWindow()
{
// Checked before the signature, and separately from it, so a mismatch
// is diagnosable: if this passes and UT-039 fails, the DSP diverged; if
// this fails, the input did, and the signature comparison would only
// have told you "different" without saying where.
var pcm = FixturePcm.Value;
var decoded = Golden.GetProperty("decoded_window");
Assert.Equal(decoded.GetProperty("samples").GetInt32(), pcm.Length);
var s16 = new byte[pcm.Length * sizeof(short)];
for (var i = 0; i < pcm.Length; i++)
{
BinaryPrimitives.WriteInt16LittleEndian(s16.AsSpan(i * sizeof(short)), pcm[i]);
}
Assert.Equal(Hex64(decoded.GetProperty("s16le_fnv1a64").GetString()!), Fnv1a64(s16));
var f32 = new byte[FixtureSamples.Value.Length * sizeof(float)];
for (var i = 0; i < FixtureSamples.Value.Length; i++)
{
BinaryPrimitives.WriteSingleLittleEndian(f32.AsSpan(i * sizeof(float)), FixtureSamples.Value[i]);
}
Assert.Equal(Hex64(decoded.GetProperty("f32le_fnv1a64").GetString()!), Fnv1a64(f32));
}
// UT-039
[Fact]
public void Signature_OfTheGoldenFixture_MatchesTheRecordedValueExactly()
{
// The cross-repo check. Not "close", not "matches to within a tier" —
// the same string the C++ producer emits for the same audio.
Assert.Equal(GoldenSignature, AudioSignature.FromMonoSamples(FixtureSamples.Value));
}
// UT-040
[Fact]
public void BandTable_MatchesTheRecordedOne_AndTilesTheRangeExactly()
{
// The band-to-FFT-bin table is the part of the construction most likely
// to drift between two implementations — an off-by-one in a ceiling, a
// half-open range read as closed — so it is pinned independently of the
// signature it produces.
var table = AudioSignature.BandFftBins();
var want = Golden.GetProperty("band_fft_bins");
Assert.Equal(want.GetArrayLength(), table.Count);
for (var b = 0; b < table.Count; b++)
{
Assert.Equal(want[b][0].GetInt32(), table[b].Low);
Assert.Equal(want[b][1].GetInt32(), table[b].High);
Assert.True(table[b].High > table[b].Low, $"band {b} is empty");
if (b > 0)
{
// Contiguous, so the frame energy really is the sum of the band
// sums — no gap, no bin counted twice.
Assert.Equal(table[b - 1].High, table[b].Low);
}
}
}
// UT-041
[Fact]
public void Signature_IsWellFormed_PrefixFrameCountAndStructuralBytes()
{
var signature = AudioSignature.FromMonoSamples(FixtureSamples.Value);
Assert.NotNull(signature);
// JR-045 — the signature carries its own version, separate from
// schema_version, so a future DSP change is detectable rather than
// silently producing signatures that no longer match.
Assert.StartsWith(AudioSignature.VersionPrefix, signature, StringComparison.Ordinal);
var bytes = Convert.FromBase64String(signature[AudioSignature.VersionPrefix.Length..]);
Assert.Equal(Golden.GetProperty("frame_count").GetInt32(), bytes.Length);
Assert.Equal(AudioSignature.ExpectedFrames, bytes.Length);
// The server validates this structure on upload: each byte is a 5-bit
// band index plus a 2-bit energy class, so bit 7 is always clear and an
// arbitrary byte is not a valid signature. That is what keeps the field
// from being usable as a payload channel.
var bandsSeen = new bool[AudioSignature.NumBands];
var classesSeen = new bool[4];
foreach (var b in bytes)
{
Assert.Equal(0, b & 0x80);
bandsSeen[(b >> 2) & 0x1F] = true;
classesSeen[b & 0x03] = true;
}
// The fixture is built to exercise the whole output alphabet. If it ever
// stops doing so, the golden vector has become a weaker check than it
// looks — so that property is asserted rather than assumed.
Assert.All(bandsSeen, Assert.True);
Assert.All(classesSeen, Assert.True);
}
// UT-042
[Fact]
public void PackFrames_UsesWholeFramesOnly()
{
Assert.Empty(AudioSignature.PackFrames(new float[AudioSignature.FrameSize - 1]));
Assert.Single(AudioSignature.PackFrames(new float[AudioSignature.FrameSize]));
Assert.Single(AudioSignature.PackFrames(
new float[AudioSignature.FrameSize + AudioSignature.HopSize - 1]));
Assert.Equal(2, AudioSignature.PackFrames(
new float[AudioSignature.FrameSize + AudioSignature.HopSize].AsSpan()).Length);
// A partial frame is not a signature: below one frame there is nothing
// to emit, and emitting a padded frame would be a different fingerprint
// from the pipeline's.
Assert.Null(AudioSignature.FromMonoSamples(new float[AudioSignature.FrameSize - 1]));
// The full window is 1288 frames — asserted as a constant rather than by
// running the DSP over 1.3M zeros, which is the same claim for free.
Assert.Equal(1323000, AudioSignature.WindowSamples);
Assert.Equal(
1 + ((AudioSignature.WindowSamples - AudioSignature.FrameSize) / AudioSignature.HopSize),
AudioSignature.ExpectedFrames);
}
// UT-043
[Fact]
public async Task Decode_ThroughFfmpeg_ReproducesTheGoldenSignature()
{
// The one test that exercises the real decode — the command line, the
// stream selection, the downmix and resample — rather than the DSP
// alone. It needs an FFmpeg binary, which the plugin gets from Jellyfin
// at run time and which a bare CI container may not have; UT-038 and
// UT-039 are what make the cross-repo claim binding without one.
var ffmpeg = FindFfmpeg();
if (ffmpeg is null)
{
return;
}
var signature = await AudioSignatureService.ComputeWithEncoderAsync(
ffmpeg,
Path.Combine(FixtureDir, "jray_audio_v1_tone.flac"),
AudioSignature.WindowSec,
NullLogger.Instance,
CancellationToken.None).ConfigureAwait(true);
Assert.Equal(GoldenSignature, signature);
}
// UT-044
[Fact]
public async Task Decode_TakesTheWindowFromTheCentre_NotTheHead()
{
// Sampling from the centre is the whole reason the construction avoids
// the head and tail — logos and cold opens at one end, credits at the
// other — so it needs its own check. Nothing else here pins the seek: a
// head-anchored window passes every other test in this file.
var ffmpeg = FindFfmpeg();
if (ffmpeg is null)
{
return;
}
var pad = 90 * AudioSignature.SampleRate;
var padded = new short[(pad * 2) + FixturePcm.Value.Length];
FixturePcm.Value.CopyTo(padded, pad);
var wav = Path.Combine(Path.GetTempPath(), $"jray_audio_centre_{Environment.ProcessId}.wav");
try
{
WriteWav(wav, padded);
var signature = await AudioSignatureService.ComputeWithEncoderAsync(
ffmpeg,
wav,
padded.Length / (double)AudioSignature.SampleRate,
NullLogger.Instance,
CancellationToken.None).ConfigureAwait(true);
Assert.Equal(GoldenSignature, signature);
}
finally
{
File.Delete(wav);
}
}
/// <summary>
/// Regenerates the fixture media's PCM, as <c>make_fixture.py</c> defines
/// it: 120 s of tones stepping through all 32 log-bands, amplitudes walking
/// a golden-ratio sequence so all four energy classes appear, over a quiet
/// constant 777 Hz bed so no frame is degenerate.
/// </summary>
/// <remarks>
/// A port, not a re-derivation — the point is that a repo can rebuild the
/// input from scratch and check the result against the recorded checksums,
/// which is what UT-038 does. The closest sample to a quantisation boundary
/// sits 3.7e-7 away from one, so the result does not depend on which libm
/// rounds the sine.
/// </remarks>
private static short[] GenerateFixturePcm()
{
const int Segment = 32768;
const int BandStride = 7;
const double AmpLogMin = -1.55;
const double AmpLogSpan = 1.53;
const double PhiFrac = 0.6180339887498949;
const double BackgroundHz = 777.0;
const double BackgroundAmp = 0.004;
var n = (int)Math.Round(AudioSignature.SampleRate * AudioSignature.WindowSec);
var samples = new short[n];
var twoPi = 2.0 * Math.PI;
var phase = 0.0;
for (var start = 0; start < n; start += Segment)
{
var s = start / Segment;
var end = Math.Min(n, start + Segment);
var band = (s * BandStride) % AudioSignature.NumBands;
var freq = AudioSignature.BandLoHz * Math.Pow(
AudioSignature.BandHiHz / AudioSignature.BandLoHz,
(band + 0.5) / AudioSignature.NumBands);
var amp = Math.Pow(10.0, AmpLogMin + (AmpLogSpan * ((s * PhiFrac) % 1.0)));
var step = twoPi * freq / AudioSignature.SampleRate;
for (var k = 0; k < end - start; k++)
{
var i = start + k;
var x = amp * Math.Sin(phase + (step * k));
x += BackgroundAmp * Math.Sin(twoPi * BackgroundHz * i / AudioSignature.SampleRate);
x = Math.Clamp(x, -1.0, 1.0);
samples[i] = (short)Math.Floor((x * 32767.0) + 0.5);
}
phase = (phase + (step * (end - start))) % twoPi;
}
return samples;
}
private static void WriteWav(string path, short[] samples)
{
var dataBytes = samples.Length * sizeof(short);
using var stream = File.Create(path);
using var writer = new BinaryWriter(stream);
writer.Write("RIFF"u8);
writer.Write(36 + dataBytes);
writer.Write("WAVE"u8);
writer.Write("fmt "u8);
writer.Write(16); // PCM header size
writer.Write((short)1); // PCM
writer.Write((short)1); // mono
writer.Write(AudioSignature.SampleRate);
writer.Write(AudioSignature.SampleRate * sizeof(short));
writer.Write((short)sizeof(short)); // block align
writer.Write((short)16); // bits per sample
writer.Write("data"u8);
writer.Write(dataBytes);
foreach (var sample in samples)
{
writer.Write(sample);
}
}
private static string? FindFfmpeg()
{
var configured = Environment.GetEnvironmentVariable("JRAY_TEST_FFMPEG");
if (!string.IsNullOrEmpty(configured))
{
return File.Exists(configured) ? configured : null;
}
var name = OperatingSystem.IsWindows() ? "ffmpeg.exe" : "ffmpeg";
foreach (var dir in (Environment.GetEnvironmentVariable("PATH") ?? string.Empty)
.Split(Path.PathSeparator, StringSplitOptions.RemoveEmptyEntries))
{
var candidate = Path.Combine(dir, name);
if (File.Exists(candidate))
{
return candidate;
}
}
return null;
}
private static ulong Fnv1a64(ReadOnlySpan<byte> data)
{
var hash = 0xcbf29ce484222325UL;
foreach (var b in data)
{
hash ^= b;
hash *= 0x100000001b3UL;
}
return hash;
}
private static ulong Hex64(string value)
{
var digits = value.StartsWith("0x", StringComparison.OrdinalIgnoreCase) ? value[2..] : value;
return ulong.Parse(digits, NumberStyles.HexNumber, CultureInfo.InvariantCulture);
}
}
@@ -53,4 +53,16 @@
<ProjectReference Include="..\Jellyfin.Plugin.JRay\Jellyfin.Plugin.JRay.csproj" /> <ProjectReference Include="..\Jellyfin.Plugin.JRay\Jellyfin.Plugin.JRay.csproj" />
</ItemGroup> </ItemGroup>
<ItemGroup>
<!--
The audio-signature golden vector, shared verbatim with the extraction
repo (JR-043): the same three files, byte for byte, in both repos. Two
independent implementations of one fingerprint are only useful if they
agree exactly, and this is what makes that a checked claim.
-->
<Content Include="fixtures\**\*">
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
</Content>
</ItemGroup>
</Project> </Project>
@@ -43,9 +43,9 @@ public class ManagedTruthStoreTests : IDisposable
private static TruthFile Truth() private static TruthFile Truth()
{ {
var truth = new TruthFile { SchemaVersion = 1, Movie = "/m.mkv" }; var truth = new TruthFile { SchemaVersion = TruthSchema.SupportedVersion, Movie = "/m.mkv" };
var actor = new TruthActor { Name = "A", TmdbId = "884" }; var actor = new TruthActor { Name = "A", TmdbId = "884" };
actor.Scenes.Add([1.0, 2.0]); actor.Scenes.Add(new TruthScene { Start = 1.0, End = 2.0 });
truth.Actors.Add(actor); truth.Actors.Add(actor);
return truth; return truth;
} }
@@ -0,0 +1,214 @@
using System;
using System.IO;
using System.Text.Json;
using Jellyfin.Plugin.JRay.Configuration;
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Xunit;
namespace Jellyfin.Plugin.JRay.Tests;
/// <summary>
/// JR-047 — a fetched manifest is aligned against the local file before its
/// windows are stored, and how that was decided is recorded.
///
/// The claim under test is that **the local alignment supersedes the server's
/// offset**. The server has never seen this file; its offset can only be a
/// runtime-difference inference, while a local alignment compares the manifest's
/// own signature against the media the windows will be drawn over.
///
/// The counterweight is that a signature must never break a fetch. Every way the
/// local path can fail — switched off, no manifest signature, short media, a
/// failed decode, or two signatures that do not match — has to fall back to the
/// server's offset rather than refusing. Both halves are asserted here.
///
/// `Resolve` is the decision, split out from the decode so it can be driven
/// without FFmpeg or a media file.
///
/// TRACES: UT-053, UT-054, UT-055, UT-056, UT-057 | JR-047
/// </summary>
public class ManifestAlignerTests
{
private const double FeatureRuntime = 7200.0;
private const double ServerOffset = 3.5;
private static readonly string FixtureDir =
Path.Combine(AppContext.BaseDirectory, "fixtures", "audio");
private static readonly Lazy<string> GoldenSignature = new(() =>
JsonDocument.Parse(File.ReadAllText(Path.Combine(FixtureDir, "jray_audio_v1_golden.json")))
.RootElement.GetProperty("signature").GetString()!);
private static readonly Lazy<byte[]> GoldenFrames = new(() =>
AudioSignatureMatcher.TryParseFrames(GoldenSignature.Value)!);
// UT-053
[Fact]
public void ALocalAlignment_SupersedesTheServersOffset()
{
// Same cut, sampled 100 frames apart — a differently trimmed release.
// The server offered 3.5 s from a runtime comparison; the local audio
// says otherwise, and the local answer is the one that gets applied.
const int Shift = 100;
var source = GoldenFrames.Value;
var alignment = ManifestAligner.Resolve(
Signature(source, 44, 1000),
Signature(source, 144, 1000),
FeatureRuntime,
FeatureRuntime,
MatchTier.Runtime,
ServerOffset);
Assert.Equal(AlignmentSource.Local, alignment.Source);
Assert.Equal(MatchTier.Audio, alignment.Tier);
Assert.Equal(Shift, alignment.OffsetFrames);
Assert.Equal(1.0, alignment.Score!.Value);
Assert.Equal(Shift * AudioSignatureMatcher.FrameSeconds, alignment.OffsetSec, 9);
// The server's claim is kept rather than overwritten: the applied offset
// is otherwise unrecoverable once the windows are shifted, and the two
// disagreeing is exactly what someone debugging would need to see.
Assert.Equal(ServerOffset, alignment.ServerOffsetSec);
Assert.Equal(MatchTier.Runtime, alignment.ServerTier);
}
// UT-054
[Fact]
public void TheLocalSignature_IsRecorded_SoALaterFetchNeedNotDecodeAgain()
{
// The decode is the expensive half and the reason signatures are opt-in.
// Keeping the local one beside the truth file is what lets a second
// manifest be aligned for free.
var alignment = ManifestAligner.Resolve(
GoldenSignature.Value,
GoldenSignature.Value,
FeatureRuntime,
FeatureRuntime,
MatchTier.Runtime,
ServerOffset);
Assert.Equal(GoldenSignature.Value, alignment.LocalSignature);
Assert.Equal(AlignmentSource.Local, alignment.Source);
Assert.Equal(0.0, alignment.OffsetSec);
}
// UT-055
[Fact]
public void EveryUnavailableLocalPath_FallsBackToTheServer_RatherThanRefusing()
{
// A signature is an enhancement to cut matching. A missing one costs a
// tier; it must never be able to break a fetch, so each of these stores
// the manifest on the server's terms.
var cases = new (string? Local, string? Manifest, string Why)[]
{
(null, GoldenSignature.Value, "signatures off, or the decode failed"),
(GoldenSignature.Value, null, "the manifest carried no signature"),
(null, null, "neither side has one"),
};
foreach (var (local, manifest, why) in cases)
{
var alignment = ManifestAligner.Resolve(
local, manifest, FeatureRuntime, FeatureRuntime, MatchTier.Runtime, ServerOffset);
Assert.Equal(AlignmentSource.Server, alignment.Source);
Assert.Equal(ServerOffset, alignment.OffsetSec);
Assert.Equal(MatchTier.Runtime, alignment.Tier);
Assert.Null(alignment.Score);
Assert.Null(alignment.OffsetFrames);
}
// Short media is the same fallback, reached through JR-044 rather than
// through a missing string: both sides have a perfectly valid signature
// and it is still the runtime that decides.
var shortMedia = ManifestAligner.Resolve(
GoldenSignature.Value,
GoldenSignature.Value,
119.0,
FeatureRuntime,
MatchTier.Runtime,
ServerOffset);
Assert.Equal(AlignmentSource.Server, shortMedia.Source);
Assert.Equal(ServerOffset, shortMedia.OffsetSec);
// A `v2:` signature from a future producer is *un-comparable*, not a
// mismatch (JR-045). Reporting it as one would tell the user their audio
// disagrees with the manifest when all that happened is the producer
// moved ahead of this build.
var futureProducer = ManifestAligner.Resolve(
GoldenSignature.Value,
"v2:" + GoldenSignature.Value[AudioSignature.VersionPrefix.Length..],
FeatureRuntime,
FeatureRuntime,
MatchTier.Runtime,
ServerOffset);
Assert.Equal(AlignmentSource.Server, futureProducer.Source);
Assert.Equal(ServerOffset, futureProducer.OffsetSec);
}
// UT-056
[Fact]
public void TwoSignaturesThatDoNotMatch_AreRecorded_ButStillDoNotBreakTheFetch()
{
// The strongest available hint that a manifest describes different
// content. It is not treated as a failure — the audio may legitimately
// differ, a different language track being the obvious case — but it is
// not discarded either.
var alignment = ManifestAligner.Resolve(
GoldenSignature.Value,
PseudoRandomSignature(1288, seed: 4242),
FeatureRuntime,
FeatureRuntime,
MatchTier.Runtime,
ServerOffset);
Assert.Equal(AlignmentSource.LocalMismatch, alignment.Source);
// The manifest is still stored, on the server's terms.
Assert.Equal(ServerOffset, alignment.OffsetSec);
Assert.Equal(MatchTier.Runtime, alignment.Tier);
}
// UT-057
[Fact]
public void AMismatchOutranksTheTier_InTheCaveatShownToTheUser()
{
// A `runtime`-tier match normally needs no caveat at all, so without
// this the strongest warning available would be the one never shown.
var mismatch = new TruthAlignment { Source = AlignmentSource.LocalMismatch };
var caveat = ManifestConverter.DescribeCaveat(MatchTier.Runtime, 0.0, mismatch);
Assert.NotNull(caveat);
Assert.Contains("does not match", caveat, StringComparison.Ordinal);
// It outranks `loose` too, which would otherwise have claimed the slot
// with the weaker statement — that the runtimes differ, not the audio.
var looseCaveat = ManifestConverter.DescribeCaveat(MatchTier.Loose, 0.0, mismatch);
Assert.Contains("does not match", looseCaveat!, StringComparison.Ordinal);
// A clean local alignment keeps the existing behaviour: a shift is
// explained, an aligned match needs nothing.
var local = new TruthAlignment { Source = AlignmentSource.Local };
Assert.Contains("shifted by", ManifestConverter.DescribeCaveat(MatchTier.Audio, 9.29, local)!, StringComparison.Ordinal);
Assert.Null(ManifestConverter.DescribeCaveat(MatchTier.Audio, 0.0, local));
}
private static string Signature(byte[] source, int start, int count)
=> AudioSignature.VersionPrefix + Convert.ToBase64String(source, start, count);
private static string PseudoRandomSignature(int frames, int seed)
{
var bytes = new byte[frames];
var state = (uint)seed;
for (var i = 0; i < frames; i++)
{
state = (state * 1664525u) + 1013904223u;
bytes[i] = (byte)((((state >> 16) % AudioSignature.NumBands) << 2) | ((state >> 8) & 0x03));
}
return AudioSignature.VersionPrefix + Convert.ToBase64String(bytes);
}
}
@@ -270,8 +270,8 @@ public class ManifestExchangeTests
var truth = ManifestConverter.ToTruthFile(m, 40, "/media/film.mkv"); var truth = ManifestConverter.ToTruthFile(m, 40, "/media/film.mkv");
Assert.Equal(new[] { 140.0, 160.0 }, truth.Actors[0].Scenes[0]); Assert.Equal((140.0, 160.0), (truth.Actors[0].Scenes[0].Start, truth.Actors[0].Scenes[0].End));
Assert.Equal(new[] { 240.0, 260.0 }, truth.Actors[0].Scenes[1]); Assert.Equal((240.0, 260.0), (truth.Actors[0].Scenes[1].Start, truth.Actors[0].Scenes[1].End));
} }
[Fact] [Fact]
@@ -284,8 +284,8 @@ public class ManifestExchangeTests
var truth = ManifestConverter.ToTruthFile(m, -40, "/media/film.mkv"); var truth = ManifestConverter.ToTruthFile(m, -40, "/media/film.mkv");
Assert.Equal(0.0, truth.Actors[0].Scenes[0][0]); Assert.Equal(0.0, truth.Actors[0].Scenes[0].Start);
Assert.True(truth.Actors[0].Scenes[0][1] >= truth.Actors[0].Scenes[0][0]); Assert.True(truth.Actors[0].Scenes[0].End >= truth.Actors[0].Scenes[0].Start);
} }
[Fact] [Fact]
@@ -302,8 +302,8 @@ public class ManifestExchangeTests
var truth = ManifestConverter.ToTruthFile(m, 0, "/media/film.mkv"); var truth = ManifestConverter.ToTruthFile(m, 0, "/media/film.mkv");
Assert.Equal(2, truth.Actors[0].Scenes.Count); Assert.Equal(2, truth.Actors[0].Scenes.Count);
Assert.Equal(new[] { 10.0, 20.0 }, truth.Actors[0].Scenes[0]); Assert.Equal((10.0, 20.0), (truth.Actors[0].Scenes[0].Start, truth.Actors[0].Scenes[0].End));
Assert.Equal(new[] { 20.0, 30.0 }, truth.Actors[0].Scenes[1]); Assert.Equal((20.0, 30.0), (truth.Actors[0].Scenes[1].Start, truth.Actors[0].Scenes[1].End));
} }
[Fact] [Fact]
@@ -25,7 +25,7 @@ public class PresenceLookupTests
var actor = new TruthActor { Name = "Steve Buscemi", TmdbId = "884" }; var actor = new TruthActor { Name = "Steve Buscemi", TmdbId = "884" };
foreach (var w in windows) foreach (var w in windows)
{ {
actor.Scenes.Add(w); actor.Scenes.Add(new TruthScene { Start = w[0], End = w[1] });
} }
return actor; return actor;
@@ -102,18 +102,22 @@ public class PresenceLookupTests
// JR-004: served exactly as given. A round trip through the serializer // JR-004: served exactly as given. A round trip through the serializer
// is where a silent normalisation would show up. // is where a silent normalisation would show up.
const string Json = """ const string Json = """
{"schema_version":1,"movie":"/m.mkv","sample_fps":1,"anneal_sec":2, {"schema_version":2,"movie":"/m.mkv",
"extraction":{"sample_fps":1,"extinction_sec":12},
"cut":{"runtime_sec":6420.5},
"actors":[{"name":"A","imdb_id":"","tmdb_id":"884","jellyfin_id":"", "actors":[{"name":"A","imdb_id":"","tmdb_id":"884","jellyfin_id":"",
"scenes":[[0.0,10.0],[10.0,20.0],[30.0,30.0]]}]} "scenes":[{"start":0.0,"end":10.0},
{"start":10.0,"end":20.0},
{"start":30.0,"end":30.0}]}]}
"""; """;
var parsed = JsonSerializer.Deserialize<TruthFile>(Json, new JsonSerializerOptions(JsonSerializerDefaults.Web))!; var parsed = JsonSerializer.Deserialize<TruthFile>(Json, new JsonSerializerOptions(JsonSerializerDefaults.Web))!;
var windows = parsed.Actors[0].Scenes; var windows = parsed.Actors[0].Scenes;
Assert.Equal(3, windows.Count); Assert.Equal(3, windows.Count);
Assert.Equal([0.0, 10.0], windows[0]); Assert.Equal((0.0, 10.0), (windows[0].Start, windows[0].End));
Assert.Equal([10.0, 20.0], windows[1]); Assert.Equal((10.0, 20.0), (windows[1].Start, windows[1].End));
Assert.Equal([30.0, 30.0], windows[2]); Assert.Equal((30.0, 30.0), (windows[2].Start, windows[2].End));
} }
// UT-023 // UT-023
@@ -129,7 +133,7 @@ public class PresenceLookupTests
var actor = Actor(); var actor = Actor();
for (var w = 0; w < 1000; w++) for (var w = 0; w < 1000; w++)
{ {
actor.Scenes.Add([w * 10.0, (w * 10.0) + 4.0]); actor.Scenes.Add(new TruthScene { Start = w * 10.0, End = (w * 10.0) + 4.0 });
} }
truth.Actors.Add(actor); truth.Actors.Add(actor);
@@ -0,0 +1,229 @@
using System.Text.Json;
using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Xunit;
namespace Jellyfin.Plugin.JRay.Tests;
/// <summary>
/// JR-002 (the <c>schema_version</c> 2 shape) and JR-003 (an unknown version is
/// refused, never guessed).
///
/// These are the flag-day tests. v1 is gone rather than deprecated, so what has
/// to be pinned is not only that v2 parses but that v1 does *not* quietly
/// half-parse into something a reader would treat as real.
///
/// TRACES: UT-029, UT-030, UT-031, UT-032, UT-033, UT-034, UT-035, UT-036, UT-037 | JR-002, JR-003
/// </summary>
public class TruthSchemaTests
{
private static readonly JsonSerializerOptions Options = new(JsonSerializerDefaults.Web);
private const string V2 = """
{"schema_version":2,"movie":"/data/movies/Movie.mkv",
"extraction":{"sample_fps":5,"extinction_sec":12,"gallery_size":1820,
"gallery_scope":"global","pipeline_version":"scene-actor-extraction 0.4.1"},
"cut":{"runtime_sec":6420.5,"audio_signature":"v1:v7fA3k"},
"actors":[{"name":"Steve Buscemi","imdb_id":"nm0000114","tmdb_id":"884",
"jellyfin_id":"abc123-guid",
"scenes":[{"start":191.6,"end":209.2,"belief":0.98,"route":"live"},
{"start":438.2,"end":465.6,"belief":0.81,"route":"deferred"}]}]}
""";
// UT-029
[Fact]
public void AV2FileRoundTripsWithProvenanceAndCutIntact()
{
var parsed = JsonSerializer.Deserialize<TruthFile>(V2, Options)!;
Assert.Equal(2, parsed.SchemaVersion);
Assert.Equal("/data/movies/Movie.mkv", parsed.Movie);
// sample_fps moved into `extraction` in the bump. Reading it from the
// top level would silently yield zero.
Assert.Equal(5, parsed.Extraction!.SampleFps);
Assert.Equal(12, parsed.Extraction.ExtinctionSec);
Assert.Equal(1820, parsed.Extraction.GallerySize);
Assert.Equal("global", parsed.Extraction.GalleryScope);
Assert.Equal(6420.5, parsed.Cut!.RuntimeSec);
Assert.Equal("v1:v7fA3k", parsed.Cut.AudioSignature);
}
// UT-030
[Fact]
public void ScenesAreObjectsThatRetainBeliefAndRoute()
{
// The reason `scenes` stopped being float pairs. A window that loses its
// belief and route is indistinguishable from a v1 window, which is
// exactly the regression this pins.
var parsed = JsonSerializer.Deserialize<TruthFile>(V2, Options)!;
var windows = parsed.Actors[0].Scenes;
Assert.Equal(2, windows.Count);
Assert.Equal(0.98, windows[0].Belief);
Assert.Equal("live", windows[0].Route);
Assert.Equal(0.81, windows[1].Belief);
Assert.Equal("deferred", windows[1].Route);
}
// UT-031
[Theory]
[InlineData("live")]
[InlineData("deferred")]
[InlineData("pooled")]
public void AllThreeRoutesSurviveARoundTrip(string route)
{
// All three are named by extraction AR-017. A serializer that dropped an
// unrecognised one would make `pooled` claims look live.
var truth = new TruthFile { SchemaVersion = TruthSchema.SupportedVersion };
var actor = new TruthActor { Name = "A", TmdbId = "884" };
actor.Scenes.Add(new TruthScene { Start = 1, End = 2, Belief = 0.5, Route = route });
truth.Actors.Add(actor);
var round = JsonSerializer.Deserialize<TruthFile>(JsonSerializer.Serialize(truth, Options), Options)!;
Assert.Equal(route, round.Actors[0].Scenes[0].Route);
}
// UT-032
[Fact]
public void AWindowWithoutBeliefIsNullRatherThanZero()
{
// Absent and "believed with probability zero" are different statements.
// Defaulting to 0.0 would make an unannotated window look maximally
// untrustworthy, and a consumer ranking on belief would discard it.
const string NoBelief = """
{"schema_version":2,"movie":"/m.mkv",
"actors":[{"name":"A","tmdb_id":"884","scenes":[{"start":1,"end":2}]}]}
""";
var parsed = JsonSerializer.Deserialize<TruthFile>(NoBelief, Options)!;
Assert.Null(parsed.Actors[0].Scenes[0].Belief);
Assert.Null(parsed.Actors[0].Scenes[0].Route);
}
// UT-033 — JR-003, the flag day itself.
[Theory]
[InlineData(1)]
[InlineData(3)]
[InlineData(0)]
public void AnUnsupportedVersionIsRefused(int version)
{
// Both directions matter. v1 is the version that exists in the wild, and
// v3 is a future producer this build cannot know the shape of — guessing
// at either is what JR-003 forbids.
var truth = new TruthFile { SchemaVersion = version };
Assert.False(TruthSchema.IsSupported(truth));
Assert.Contains(version.ToString(System.Globalization.CultureInfo.InvariantCulture), TruthSchema.DescribeRejection(version), System.StringComparison.Ordinal);
}
// UT-034
[Fact]
public void AMissingSchemaVersionIsRefusedRatherThanAssumedCurrent()
{
// An absent field deserialises to 0. Treating that as "probably the
// current version" is the single most tempting mistake here, and it
// would accept any malformed document that happened to parse.
const string NoVersion = """
{"movie":"/m.mkv","actors":[]}
""";
var parsed = JsonSerializer.Deserialize<TruthFile>(NoVersion, Options);
Assert.False(TruthSchema.IsSupported(parsed));
}
// UT-035
[Fact]
public void ANullTruthFileIsRefusedWithoutThrowing()
{
// `null` reaches this from a file containing the literal `null`, which
// parses successfully. The check must reject it rather than dereference.
Assert.False(TruthSchema.IsSupported(null));
}
// UT-037
[Fact]
public void TheProducersActualOutputParses()
{
// Byte-for-byte the shape `scene-actor-extraction` writes today
// (`src/nodes/result_sink_node.hpp`, IR-002) — *not* the fully populated
// example from the spec. It omits `cut` entirely and carries only three
// of the five `extraction` fields.
//
// This is the test that would have caught the break: the plugin read v1
// while the pipeline had already moved to v2, so nothing the pipeline
// produced could be read at all. A round-trip test written against the
// spec's example alone would have passed throughout.
const string AsProduced = """
{
"schema_version": 2,
"movie": "/data/movies/Film.mkv",
"extraction": {
"sample_fps": 5.0,
"extinction_sec": 12.0,
"gallery_scope": "global"
},
"actors": [
{
"name": "Steve Buscemi",
"imdb_id": "nm0000114",
"tmdb_id": "884",
"jellyfin_id": "",
"scenes": [
{ "start": 191.6, "end": 209.2, "belief": 0.98, "route": "live" }
]
}
]
}
""";
var parsed = JsonSerializer.Deserialize<TruthFile>(AsProduced, Options);
Assert.True(TruthSchema.IsSupported(parsed));
Assert.Equal(5.0, parsed!.Extraction!.SampleFps);
Assert.Equal(12.0, parsed.Extraction.ExtinctionSec);
Assert.Equal("global", parsed.Extraction.GalleryScope);
// Absent blocks and fields are null, not defaults that would read as data.
Assert.Null(parsed.Cut);
Assert.Null(parsed.Extraction.GallerySize);
Assert.Null(parsed.Extraction.PipelineVersion);
var window = Assert.Single(parsed.Actors[0].Scenes);
Assert.Equal((191.6, 209.2), (window.Start, window.End));
Assert.Equal(0.98, window.Belief);
Assert.Equal("live", window.Route);
}
// UT-036
[Fact]
public void AV1FileDoesNotHalfParseIntoUsableWindows()
{
// The load-bearing claim of the flag day. v1 `scenes` were float pairs,
// so a v1 file either fails to deserialise or produces windows that are
// not usable — what must never happen is silent success with windows at
// 0,0, which would report actors present at the start of every film.
const string V1 = """
{"schema_version":1,"movie":"/m.mkv","sample_fps":1,"anneal_sec":2,
"actors":[{"name":"A","tmdb_id":"884","scenes":[[0.0,10.0],[10.0,20.0]]}]}
""";
TruthFile? parsed = null;
try
{
parsed = JsonSerializer.Deserialize<TruthFile>(V1, Options);
}
catch (JsonException)
{
// The expected path: a float pair is not an object.
return;
}
// If it did parse, the version gate is what stops it being used.
Assert.False(TruthSchema.IsSupported(parsed));
}
}
@@ -0,0 +1,194 @@
{
"_": "Golden vector for the JRay v1 audio signature (JRay-public-server SPEC.md \u00a73). Shared verbatim between scene-actor-extraction (C++) and the jRay Jellyfin plugin (C#) so the two implementations can be proven bit-identical. IR-004, IR-005, IR-007, IR-008.",
"version": "v1",
"signature": "v1:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAeHx8eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh4eHh8fHzk5OTk5OTk5OTk5OTk5OTk5OTk5OTk5OTk5OTk5V1dXV1dXV1dXV1dXV1dXV1dXV1dXV1dXV1dXV1dXV1dXV1dycnJycnJycnJycnJycnJycnJycnJycnJycnJycnMPDgwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKytFRUVFRUVFRUVFRUVFRUVFRUVFRUVFRUVFRUVFRWNjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2NjY2Njfn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5/GxoZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGRkZGTc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3Nzc3UlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSU1JsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbGxsbAoLCwoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCgoKCwsLJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUlJSVDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ15eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eX19eeXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXl5eXkXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFzExMTExMTExMTExMTExMTExMTExMTExMTExMTExT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09qampqampqampqampqampqampqampqampqampqamsHBwUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyM+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4+Pj4/PlhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYd3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3ExMRERERERERERERERERERERERERERERERERERERES8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpKSkpLS0plZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZQMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDHR0dHR0dHR0dHR0dHR0dHR0dHR0dHR0dHR0dHR0eHh44ODg4ODg4ODg4ODg4ODg4ODg4ODg4ODg4ODg4OFdXV1ZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWVlZWV1dXcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXFxcXEPDw8PDw8PDw8PDw8PDw8PDw8PDw8PDw8PDw8PDw8PDw8PDyoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKysqRERERERERERERERERERERERERERERERERERERERjY2NjY2NjY2NjYw==",
"frame_count": 1288,
"media": {
"file": "jray_audio_v1_tone.flac",
"generator": "make_fixture.py",
"container": "FLAC (lossless \u2014 decodes to exactly the PCM make_fixture.py emits)",
"duration_sec": 120.0,
"sample_rate": 11025,
"channels": 1,
"sample_format": "s16",
"sha256": "912ecd426cd426dccb37753e0249694227619c701cb9f533502b37da0fbe8096",
"bytes": 585142
},
"decoded_window": {
"_": "Checksums of the 120 s centre window after downmix to mono and resample to 11025 Hz, i.e. exactly the stream `ffmpeg -ss <mid-60> -t 120 -i <file> -vn -ac 1 -ar 11025 -f f32le -` produces. Check these first: a mismatch here is a decode problem, not a DSP one.",
"samples": 1323000,
"f32le_fnv1a64": "0x1ef7899cd4d12662",
"s16le_fnv1a64": "0xf824fa56f125c0dc"
},
"params": {
"window_sec": 120.0,
"window_centre": "runtime/2, i.e. samples from runtime/2 - 60 s; truncated to exactly 1323000 samples",
"min_duration_sec": 120.0,
"min_duration_rule": "IR-007 \u2014 below this emit NO signature and apply no sync offset",
"sample_rate": 11025,
"channels": 1,
"arithmetic": "IEEE-754 double throughout; float32 is not sufficient",
"sample_scale": "s16 * (1/32768), FFmpeg's native s16->flt",
"frame_size": 4096,
"hop_size": 1024,
"frame_count_rule": "1 + (n_samples - 4096) / 1024, integer division; whole frames only",
"window_fn": "Hann, PERIODIC: w[n] = 0.5 * (1 - cos(2*pi*n/4096))",
"transform": "radix-2 DIT complex FFT over the 4096 real samples (imag=0), no normalisation",
"magnitude": "sqrt(re^2 + im^2), linear",
"band_lo_hz": 300.0,
"band_hi_hz": 3000.0,
"num_bands": 32,
"band_edges": "edge[b] = 300 * (3000/300)^(b/32), b = 0..32",
"band_bins": "band b owns FFT bins [k_lo[b], k_lo[b+1]) with k_lo[b] = ceil(edge[b] * 4096 / 11025); see band_fft_bins",
"band_value": "MEAN of the linear magnitudes in the band (not sum, not max)",
"peak_bin": "argmax over the 32 band values; ties resolve to the LOWEST index",
"energy_metric": "E = mean magnitude over all FFT bins 112..1114, i.e. the whole 300-3000 Hz band",
"energy_reference": "upper median of E over all frames: sorted[n/2], no averaging of the two middle values",
"energy_ratio": "r = log10((E + 1e-12) / (E_ref + 1e-12))",
"energy_class_edges": [
-0.6,
-0.2,
0.2
],
"energy_class": "0 if r < -0.6, 1 if r < -0.2, 2 if r < 0.2, else 3",
"byte_layout": "bit7 = 0 (reserved), bits6..2 = 5-bit band index, bits1..0 = 2-bit energy class; byte = (band << 2) | class",
"base64": "standard alphabet A-Za-z0-9+/ with '=' padding",
"prefix": "v1:"
},
"band_fft_bins": [
[
112,
120
],
[
120,
129
],
[
129,
139
],
[
139,
149
],
[
149,
160
],
[
160,
172
],
[
172,
185
],
[
185,
199
],
[
199,
213
],
[
213,
229
],
[
229,
246
],
[
246,
265
],
[
265,
285
],
[
285,
306
],
[
306,
328
],
[
328,
353
],
[
353,
379
],
[
379,
408
],
[
408,
438
],
[
438,
471
],
[
471,
506
],
[
506,
543
],
[
543,
584
],
[
584,
627
],
[
627,
674
],
[
674,
724
],
[
724,
778
],
[
778,
836
],
[
836,
899
],
[
899,
966
],
[
966,
1038
],
[
1038,
1115
]
],
"notes": [
"The server spec fixes the window, rate, STFT geometry, band and the 5+2 bit packing. Everything under params beyond that (Hann periodicity, band aggregation, the energy-class definition, tie-breaking, base64 alphabet) is pinned HERE for v1 \u2014 the spec does not constrain it, and two implementations that guess differently produce non-matching signatures.",
"Decision margins on this fixture: the two strongest bands are within 1.3% on the closest frame, and the closest frame to an energy-class edge is 3.6e-3 away in log10. Both are many orders of magnitude above double-precision FFT differences, so any two correct double- precision implementations agree; a float32 implementation is not guaranteed to.",
"Coverage: all 32 bands and all 4 energy classes appear in the golden signature.",
"Robustness observed on this fixture: identical peak-bin sequence after a stereo/44100 Hz round trip and after AAC 128 kbit/s re-encoding."
]
}
@@ -0,0 +1,114 @@
#!/usr/bin/env python3
"""Regenerate the JRay audio-signature golden fixture.
python3 make_fixture.py # writes jray_audio_v1_tone.flac here
This is the *source of truth* for the fixture media: `jray_audio_v1_tone.flac`
is a lossless FLAC encoding of exactly the PCM this script emits, so any repo
that wants to check its own audio-signature implementation against the golden
vector in `jray_audio_v1_golden.json` can regenerate the input from scratch and
confirm it is byte-identical (the golden file records `pcm_fnv1a64`, a hash of
the decoded 16-bit samples).
Deliberately dependency-free (no numpy) and written in plain arithmetic so it
ports to any language in ~20 lines.
Signal — 120.000 s, mono, 11025 Hz, 16-bit signed PCM:
* split into segments of 32768 samples (~2.97 s), 40.4 segments in total;
* segment `s` carries one sine at the geometric centre of log-band
`(s * 7) mod 32` of the 300-3000 Hz band, so all 32 bands are exercised;
* its amplitude walks a golden-ratio low-discrepancy sequence over
[10^-1.55, 10^-0.02] so frame energies spread continuously across ~1.5
decades and all four energy classes are exercised, without a dense cluster
of frames sitting on a class boundary;
* phase is carried across segment boundaries (no clicks);
* a constant, far quieter 777 Hz tone sits underneath so no frame is
degenerate;
* samples are quantised with floor(x * 32767 + 0.5).
Why FLAC and not WAV: 120 s of 11025 Hz 16-bit PCM is 2.6 MB and does not
compress in git. FLAC is lossless — FFmpeg decodes it to exactly the PCM
written here — and is ~3.5x smaller. `--wav` writes the uncompressed original
if you want to diff it.
"""
import math
import struct
import subprocess
import sys
import os
SAMPLE_RATE = 11025
DURATION_SEC = 120.0
SEGMENT = 32768 # samples per tone segment
BAND_STRIDE = 7 # coprime with 32 -> visits every band
BAND_LO_HZ = 300.0
BAND_HI_HZ = 3000.0
NUM_BANDS = 32
AMP_LOG_MIN = -1.55 # 10^-1.55 ~= 0.028
AMP_LOG_SPAN = 1.53 # up to 10^-0.02 ~= 0.955
PHI_FRAC = 0.6180339887498949
BG_HZ = 777.0
BG_AMP = 0.004
OUT_FLAC = "jray_audio_v1_tone.flac"
OUT_WAV = "jray_audio_v1_tone.wav"
def generate():
"""Return the 120 s signal as a list of int16 sample values."""
n = int(round(SAMPLE_RATE * DURATION_SEC))
out = [0] * n
phase = 0.0
two_pi = 2.0 * math.pi
for start in range(0, n, SEGMENT):
s = start // SEGMENT
end = min(n, start + SEGMENT)
band = (s * BAND_STRIDE) % NUM_BANDS
# geometric centre of log-band `band`
freq = BAND_LO_HZ * (BAND_HI_HZ / BAND_LO_HZ) ** ((band + 0.5) / NUM_BANDS)
amp = 10.0 ** (AMP_LOG_MIN + AMP_LOG_SPAN * ((s * PHI_FRAC) % 1.0))
step = two_pi * freq / SAMPLE_RATE
for k in range(end - start):
i = start + k
x = amp * math.sin(phase + step * k)
x += BG_AMP * math.sin(two_pi * BG_HZ * i / SAMPLE_RATE)
if x > 1.0:
x = 1.0
elif x < -1.0:
x = -1.0
out[i] = int(math.floor(x * 32767.0 + 0.5))
phase = (phase + step * (end - start)) % two_pi
return out
def write_wav(path, samples):
data = struct.pack("<%dh" % len(samples), *samples)
hdr = b"RIFF" + struct.pack("<I", 36 + len(data)) + b"WAVE"
hdr += b"fmt " + struct.pack("<IHHIIHH", 16, 1, 1, SAMPLE_RATE,
SAMPLE_RATE * 2, 2, 16)
hdr += b"data" + struct.pack("<I", len(data))
with open(path, "wb") as fh:
fh.write(hdr + data)
def main():
here = os.path.dirname(os.path.abspath(__file__))
samples = generate()
wav = os.path.join(here, OUT_WAV)
write_wav(wav, samples)
if "--wav" in sys.argv:
print("wrote", wav)
return
flac = os.path.join(here, OUT_FLAC)
# -compression_level 12 is deterministic for a given libFLAC/ffmpeg build;
# only the container bytes vary, never the decoded PCM.
subprocess.run(["ffmpeg", "-nostdin", "-v", "error", "-y", "-i", wav,
"-c:a", "flac", "-compression_level", "12", flac],
check=True)
os.remove(wav)
print("wrote", flac)
if __name__ == "__main__":
main()
@@ -31,6 +31,7 @@ public class ManifestController : ControllerBase
private readonly ILibraryManager _libraryManager; private readonly ILibraryManager _libraryManager;
private readonly IManifestExchangeClient _exchange; private readonly IManifestExchangeClient _exchange;
private readonly IManagedTruthStore _truthStore; private readonly IManagedTruthStore _truthStore;
private readonly ManifestAligner _aligner;
private readonly ILogger<ManifestController> _logger; private readonly ILogger<ManifestController> _logger;
/// <summary> /// <summary>
@@ -39,16 +40,19 @@ public class ManifestController : ControllerBase
/// <param name="libraryManager">Library manager.</param> /// <param name="libraryManager">Library manager.</param>
/// <param name="exchange">Manifest exchange client.</param> /// <param name="exchange">Manifest exchange client.</param>
/// <param name="truthStore">Managed truth store.</param> /// <param name="truthStore">Managed truth store.</param>
/// <param name="aligner">Aligns a fetched manifest to the local file.</param>
/// <param name="logger">Logger.</param> /// <param name="logger">Logger.</param>
public ManifestController( public ManifestController(
ILibraryManager libraryManager, ILibraryManager libraryManager,
IManifestExchangeClient exchange, IManifestExchangeClient exchange,
IManagedTruthStore truthStore, IManagedTruthStore truthStore,
ManifestAligner aligner,
ILogger<ManifestController> logger) ILogger<ManifestController> logger)
{ {
_libraryManager = libraryManager; _libraryManager = libraryManager;
_exchange = exchange; _exchange = exchange;
_truthStore = truthStore; _truthStore = truthStore;
_aligner = aligner;
_logger = logger; _logger = logger;
} }
@@ -102,35 +106,53 @@ public class ManifestController : ControllerBase
return NotFound(new { error = "no configured server had an acceptable manifest" }); return NotFound(new { error = "no configured server had an acceptable manifest" });
} }
// Align before storing. The server has never seen this file, so where a
// local audio alignment is possible it supersedes the offset the server
// sent — and it needs no round trip, so no signature leaves the
// instance (JR-047).
var alignment = await _aligner.AlignAsync(
item.Path ?? string.Empty,
item.RunTimeTicks is { } ticks ? TimeSpan.FromTicks(ticks).TotalSeconds : 0.0,
outcome.Manifest,
outcome.Tier,
outcome.OffsetSec,
config.ComputeAudioSignatures,
cancellationToken)
.ConfigureAwait(false);
var caveat = ManifestConverter.DescribeCaveat(alignment.Tier, alignment.OffsetSec, alignment);
// The offset is applied here, once, so the stored truth is always in the // The offset is applied here, once, so the stored truth is always in the
// local file's own timebase and no reader needs offset awareness. // local file's own timebase and no reader needs offset awareness.
var truth = ManifestConverter.ToTruthFile(outcome.Manifest, outcome.OffsetSec, item.Path ?? string.Empty); var truth = ManifestConverter.ToTruthFile(outcome.Manifest, alignment.OffsetSec, item.Path ?? string.Empty);
var provenance = new TruthProvenance var provenance = new TruthProvenance
{ {
Source = TruthSource.Fetched, Source = TruthSource.Fetched,
ServerUrl = outcome.ServerUrl, ServerUrl = outcome.ServerUrl,
MatchTier = outcome.Tier, MatchTier = alignment.Tier,
OffsetSec = outcome.OffsetSec, OffsetSec = alignment.OffsetSec,
Caveat = ManifestConverter.DescribeCaveat(outcome.Tier, outcome.OffsetSec), Alignment = alignment,
Caveat = caveat,
RecordedAt = DateTime.UtcNow, RecordedAt = DateTime.UtcNow,
}; };
await _truthStore.SaveAsync(itemId, truth, provenance, cancellationToken).ConfigureAwait(false); await _truthStore.SaveAsync(itemId, truth, provenance, cancellationToken).ConfigureAwait(false);
_logger.LogInformation( _logger.LogInformation(
"Stored manifest for {ItemId} from {Server} at tier {Tier} (offset {Offset}s)", "Stored manifest for {ItemId} from {Server} at tier {Tier} (offset {Offset}s, aligned by {Source})",
itemId, itemId,
outcome.ServerUrl, outcome.ServerUrl,
outcome.Tier, alignment.Tier,
outcome.OffsetSec); alignment.OffsetSec,
alignment.Source);
return Ok(new ManifestFetchResult return Ok(new ManifestFetchResult
{ {
ServerUrl = outcome.ServerUrl, ServerUrl = outcome.ServerUrl,
Match = outcome.Tier.ToString().ToLowerInvariant(), Match = alignment.Tier.ToString().ToLowerInvariant(),
OffsetSec = outcome.OffsetSec, OffsetSec = alignment.OffsetSec,
ActorCount = truth.Actors.Count, ActorCount = truth.Actors.Count,
Caveat = ManifestConverter.DescribeCaveat(outcome.Tier, outcome.OffsetSec), Caveat = caveat,
}); });
} }
@@ -2,6 +2,7 @@ using System;
using System.Threading; using System.Threading;
using System.Threading.Tasks; using System.Threading.Tasks;
using Jellyfin.Plugin.JRay.Models; using Jellyfin.Plugin.JRay.Models;
using Jellyfin.Plugin.JRay.Services;
using Jellyfin.Plugin.JRay.Services.Interfaces; using Jellyfin.Plugin.JRay.Services.Interfaces;
using Microsoft.AspNetCore.Authorization; using Microsoft.AspNetCore.Authorization;
using Microsoft.AspNetCore.Http; using Microsoft.AspNetCore.Http;
@@ -15,10 +16,11 @@ namespace Jellyfin.Plugin.JRay.Controllers;
/// locally. See SPEC.md §2. /// locally. See SPEC.md §2.
/// </summary> /// </summary>
/// <remarks> /// <remarks>
/// The <c>schema_version</c> check here refuses an unrecognised version rather /// The <c>schema_version</c> check refuses an unrecognised version rather than
/// than guessing at its shape. It is currently the only source that checks — /// guessing at its shape. It defers to <see cref="TruthSchema"/> rather than
/// sidecar reads do not — which JR-003 requires be fixed by moving the check /// holding its own constant: this used to be the only source that checked while
/// into the shared read path. /// sidecar reads did not, so the version the plugin claimed to require and the
/// one it would actually parse could drift apart.
/// </remarks> /// </remarks>
[ApiController] [ApiController]
[Route("Plugins/JRay/Items/{itemId}/Truth")] [Route("Plugins/JRay/Items/{itemId}/Truth")]
@@ -26,8 +28,6 @@ namespace Jellyfin.Plugin.JRay.Controllers;
// TRACES: JR-003, JR-009, JR-014 | SR-003 // TRACES: JR-003, JR-009, JR-014 | SR-003
public class TruthController : ControllerBase public class TruthController : ControllerBase
{ {
private const int SupportedSchemaVersion = 1;
private readonly IManagedTruthStore _managedTruthStore; private readonly IManagedTruthStore _managedTruthStore;
private readonly ITruthDataService _truthDataService; private readonly ITruthDataService _truthDataService;
@@ -54,9 +54,9 @@ public class TruthController : ControllerBase
[ProducesResponseType(StatusCodes.Status400BadRequest)] [ProducesResponseType(StatusCodes.Status400BadRequest)]
public async Task<IActionResult> PutTruth(Guid itemId, [FromBody] TruthFile truth, CancellationToken cancellationToken) public async Task<IActionResult> PutTruth(Guid itemId, [FromBody] TruthFile truth, CancellationToken cancellationToken)
{ {
if (truth.SchemaVersion != SupportedSchemaVersion) if (!TruthSchema.IsSupported(truth))
{ {
return BadRequest($"Unsupported schema_version {truth.SchemaVersion}; expected {SupportedSchemaVersion}."); return BadRequest(TruthSchema.DescribeRejection(truth.SchemaVersion));
} }
var provenance = TruthProvenance.Local(TruthSource.Pushed, DateTime.UtcNow); var provenance = TruthProvenance.Local(TruthSource.Pushed, DateTime.UtcNow);
@@ -0,0 +1,42 @@
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// Which comparison produced the offset that was applied to a fetched manifest.
/// </summary>
/// <remarks>
/// Recorded because the two are not equally strong evidence. The server has
/// never seen the local file, so its offset is at best a runtime-difference
/// inference; a local alignment compares the manifest's own audio signature
/// against the file the windows will actually be drawn over.
/// </remarks>
public enum AlignmentSource
{
/// <summary>
/// The server's offset was applied — no local alignment was possible.
/// </summary>
/// <remarks>
/// Either signatures are switched off, the item is under the 120 s window,
/// the manifest carried no signature, or the decode failed. A signature is
/// an enhancement, so every one of those degrades to this rather than
/// failing the fetch.
/// </remarks>
Server = 0,
/// <summary>
/// A local audio alignment was computed and its offset was applied.
/// </summary>
Local = 1,
/// <summary>
/// A local alignment was attempted and the two signatures did not match at
/// any tier; the server's offset was applied and the disagreement recorded.
/// </summary>
/// <remarks>
/// Deliberately not a failure. The audio may legitimately differ — a
/// different language track, a heavy re-encode — and a signature must never
/// be able to break a fetch. But it is the strongest available hint that a
/// manifest describes different content, so it is surfaced as a caveat
/// rather than discarded.
/// </remarks>
LocalMismatch = 2,
}
+6 -2
View File
@@ -45,9 +45,13 @@ public class TruthActor
public string JellyfinId { get; set; } = string.Empty; public string JellyfinId { get; set; } = string.Empty;
/// <summary> /// <summary>
/// Gets the list of [start_sec, end_sec] windows during which the actor is in the scene. /// Gets the windows during which the actor is in the scene.
/// </summary> /// </summary>
/// <remarks>
/// Objects since <c>schema_version</c> 2, not the float pairs v1 used, so a
/// window can carry the belief and route behind the claim.
/// </remarks>
[JsonPropertyName("scenes")] [JsonPropertyName("scenes")]
[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)] [JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public Collection<double[]> Scenes { get; } = new(); public Collection<TruthScene> Scenes { get; } = new();
} }
@@ -0,0 +1,74 @@
using System.Text.Json.Serialization;
using Jellyfin.Plugin.JRay.Configuration;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// How a fetched manifest was aligned to the local file, recorded alongside the
/// truth data.
/// </summary>
/// <remarks>
/// The offset is applied once, at store time (JR-030), after which the stored
/// windows look native and nothing would say they had been shifted. This block
/// is what makes that reconstructable: which comparison produced the offset, how
/// strong it was, and the local file's own signature, so a later fetch can
/// re-align without decoding the media again.
/// </remarks>
// TRACES: JR-047 | SR-003
public class TruthAlignment
{
/// <summary>Gets or sets which comparison produced the applied offset.</summary>
[JsonPropertyName("source")]
public AlignmentSource Source { get; set; }
/// <summary>Gets or sets the tier that was applied.</summary>
[JsonPropertyName("tier")]
public MatchTier Tier { get; set; }
/// <summary>Gets or sets the offset applied to every window, in seconds.</summary>
[JsonPropertyName("offset_sec")]
public double OffsetSec { get; set; }
/// <summary>
/// Gets or sets the local alignment's score, or <c>null</c> when none was
/// computed.
/// </summary>
[JsonPropertyName("score")]
public double? Score { get; set; }
/// <summary>
/// Gets or sets the recovered slide between the two analysis windows, in
/// frames, or <c>null</c> when no local alignment was computed.
/// </summary>
/// <remarks>
/// Only part of <see cref="OffsetSec"/>: the rest comes from the two windows
/// being anchored at different points when the runtimes differ.
/// </remarks>
[JsonPropertyName("offset_frames")]
public int? OffsetFrames { get; set; }
/// <summary>
/// Gets or sets the local file's own <c>v1:</c> audio signature, or
/// <c>null</c> when none was computed.
/// </summary>
/// <remarks>
/// Kept so a later fetch can align against a new manifest without running
/// FFmpeg over the media again — the expensive half of the operation, and
/// the reason signatures are opt-in. It never leaves the instance: it is
/// stored beside the truth file, and contribution strips provenance
/// entirely (JR-034).
/// </remarks>
[JsonPropertyName("local_signature")]
public string? LocalSignature { get; set; }
/// <summary>
/// Gets or sets the offset the server claimed, retained even when a local
/// alignment superseded it.
/// </summary>
[JsonPropertyName("server_offset_sec")]
public double ServerOffsetSec { get; set; }
/// <summary>Gets or sets the tier the server claimed.</summary>
[JsonPropertyName("server_tier")]
public MatchTier ServerTier { get; set; }
}
+35
View File
@@ -0,0 +1,35 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// Which encode the timings in a <see cref="TruthFile"/> apply to (SPEC.md §1,
/// JR-002).
/// </summary>
/// <remarks>
/// Timings are only meaningful against a particular cut. Recording the runtime
/// they were measured from is what lets a consumer notice that a file has been
/// re-encoded, re-trimmed, or replaced with a different release — rather than
/// silently showing an actor twenty seconds late.
/// </remarks>
// TRACES: JR-002 | SR-003
public class TruthCut
{
/// <summary>
/// Gets or sets the decoded duration the timings came from, in seconds.
/// </summary>
[JsonPropertyName("runtime_sec")]
public double? RuntimeSec { get; set; }
/// <summary>
/// Gets or sets the version-prefixed spectral-peak audio signature, or
/// <c>null</c> when the producer emitted none.
/// </summary>
/// <remarks>
/// Carries its own <c>v1:</c> prefix so a DSP change is detectable rather
/// than silently non-matching (JR-045). Media shorter than 120 s carries no
/// signature at all (JR-044).
/// </remarks>
[JsonPropertyName("audio_signature")]
public string? AudioSignature { get; set; }
}
@@ -0,0 +1,53 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// Extraction provenance for a <see cref="TruthFile"/> (SPEC.md §1, JR-002).
/// </summary>
/// <remarks>
/// These fields moved here from the top level in the SR-003 bump so that the
/// truth file and the Jmanifest's <c>extraction</c> block have the same shape.
/// They differed for no reason, and two nearly-identical shapes are what makes a
/// converter quietly drop a field.
/// <para>
/// <b>There is no <c>anneal_sec</c>.</b> It was withdrawn rather than retained
/// as a vestigial zero: presence now follows track extent, so a track survives
/// its own gaps and there is nothing to anneal (extraction AR-012/AR-013). A
/// field naming a mechanism the pipeline no longer has is actively misleading,
/// and would outlive everyone who remembers why it reads zero.
/// </para>
/// </remarks>
// TRACES: JR-002 | SR-003
public class TruthExtraction
{
/// <summary>Gets or sets the sampling rate used during extraction.</summary>
[JsonPropertyName("sample_fps")]
public double? SampleFps { get; set; }
/// <summary>
/// Gets or sets the re-acquisition timeout that shapes window extent, in
/// seconds. Successor to the withdrawn <c>anneal_sec</c>.
/// </summary>
/// <remarks>
/// This is what a consumer needs in order to interpret a window: it bounds
/// how long an actor could be unseen without the window being closed.
/// </remarks>
[JsonPropertyName("extinction_sec")]
public double? ExtinctionSec { get; set; }
/// <summary>Gets or sets the producing pipeline's version string.</summary>
[JsonPropertyName("pipeline_version")]
public string? PipelineVersion { get; set; }
/// <summary>Gets or sets how many references the gallery held.</summary>
[JsonPropertyName("gallery_size")]
public int? GallerySize { get; set; }
/// <summary>
/// Gets or sets <c>global</c> or <c>limited</c> — the strongest single
/// quality signal when two manifests compete for one cut.
/// </summary>
[JsonPropertyName("gallery_scope")]
public string? GalleryScope { get; set; }
}
+24 -11
View File
@@ -5,7 +5,7 @@ namespace Jellyfin.Plugin.JRay.Models;
/// <summary> /// <summary>
/// Root object of a scene-actor-extraction "truth" file /// Root object of a scene-actor-extraction "truth" file
/// (schema_version 1, minimal verbosity). See SPEC.md §1. /// (<c>schema_version</c> 2). See SPEC.md §1.
/// </summary> /// </summary>
/// <remarks> /// <remarks>
/// JRay <b>owns</b> this format; the extraction pipeline is its producer and the /// JRay <b>owns</b> this format; the extraction pipeline is its producer and the
@@ -13,15 +13,22 @@ namespace Jellyfin.Plugin.JRay.Models;
/// independently, breaking changes are batched into one coordinated /// independently, breaking changes are batched into one coordinated
/// <c>schema_version</c> bump rather than made piecemeal. /// <c>schema_version</c> bump rather than made piecemeal.
/// ///
/// This type is still the v1 shape. JR-002 replaces it: <c>anneal_sec</c> out, /// <para>
/// an <c>extraction</c> provenance block and a <c>cut</c> block in, and /// <b>This is the v2 shape, and v1 is gone rather than deprecated.</b> The bump
/// <c>scenes</c> becoming objects that carry belief and identification route. /// removed <c>anneal_sec</c>, moved <c>sample_fps</c> into
/// <see cref="TruthExtraction"/>, added <see cref="TruthCut"/>, and turned
/// <c>scenes</c> from float pairs into <see cref="TruthScene"/> objects carrying
/// belief and route. Nothing here reads a v1 file: see
/// <see cref="Services.TruthSchema"/> for why that is a decision rather than an
/// omission.
/// </para>
/// </remarks> /// </remarks>
// TRACES: JR-001, JR-002 | SR-003 // TRACES: JR-001, JR-002 | SR-003
public class TruthFile public class TruthFile
{ {
/// <summary> /// <summary>
/// Gets or sets the schema version of this file. /// Gets or sets the schema version of this file. Only
/// <see cref="Services.TruthSchema.SupportedVersion"/> is accepted.
/// </summary> /// </summary>
[JsonPropertyName("schema_version")] [JsonPropertyName("schema_version")]
public int SchemaVersion { get; set; } public int SchemaVersion { get; set; }
@@ -29,20 +36,26 @@ public class TruthFile
/// <summary> /// <summary>
/// Gets or sets the source media path at extraction time (informational). /// Gets or sets the source media path at extraction time (informational).
/// </summary> /// </summary>
/// <remarks>
/// Stripped on contribution (JR-034): it is a contributor's directory
/// layout, which is nobody else's business and identifies them.
/// </remarks>
[JsonPropertyName("movie")] [JsonPropertyName("movie")]
public string Movie { get; set; } = string.Empty; public string Movie { get; set; } = string.Empty;
/// <summary> /// <summary>
/// Gets or sets the sampling rate (frames per second) used during extraction. /// Gets or sets extraction provenance, or <c>null</c> when the producer
/// recorded none.
/// </summary> /// </summary>
[JsonPropertyName("sample_fps")] [JsonPropertyName("extraction")]
public double SampleFps { get; set; } public TruthExtraction? Extraction { get; set; }
/// <summary> /// <summary>
/// Gets or sets the gap (seconds) below which consecutive detections were merged into one scene. /// Gets or sets which encode the timings apply to, or <c>null</c> when the
/// producer recorded none.
/// </summary> /// </summary>
[JsonPropertyName("anneal_sec")] [JsonPropertyName("cut")]
public double AnnealSec { get; set; } public TruthCut? Cut { get; set; }
/// <summary> /// <summary>
/// Gets the list of actors in the film, each with their scene-presence windows. /// Gets the list of actors in the film, each with their scene-presence windows.
@@ -74,6 +74,18 @@ public class TruthProvenance
[JsonPropertyName("offset_sec")] [JsonPropertyName("offset_sec")]
public double OffsetSec { get; set; } public double OffsetSec { get; set; }
/// <summary>
/// Gets or sets how the applied offset was arrived at, or null for local
/// sources and for fetches made before alignment was recorded.
/// </summary>
/// <remarks>
/// <see cref="OffsetSec"/> says what was applied; this says why, and keeps
/// the local file's own signature so a later fetch can re-align without
/// decoding the media again (JR-047).
/// </remarks>
[JsonPropertyName("alignment")]
public TruthAlignment? Alignment { get; set; }
/// <summary> /// <summary>
/// Gets or sets a human-readable caveat to surface with the overlay, or /// Gets or sets a human-readable caveat to surface with the overlay, or
/// null when the claim needs none. /// null when the claim needs none.
+50
View File
@@ -0,0 +1,50 @@
using System.Text.Json.Serialization;
namespace Jellyfin.Plugin.JRay.Models;
/// <summary>
/// One presence window in a <see cref="TruthFile"/>.
/// </summary>
/// <remarks>
/// <b>A window is a claim about scene membership, not a recognition event</b>
/// (SR-002). An actor who turns away, is occluded, or is off-camera while the
/// shot cuts to whoever they are speaking to is still present — so a consumer
/// must never read a boundary as "the face was detected here", and must not
/// merge, split, trim or reorder windows.
/// <para>
/// In <c>schema_version</c> 1 this was a bare <c>[start, end]</c> float pair. It
/// became an object in the SR-003 bump so a window can carry the evidence behind
/// it: a consumer that shows presence should be able to say how strongly it is
/// believed and how it was arrived at, which a pair of numbers cannot express.
/// </para>
/// </remarks>
// TRACES: JR-002, JR-004 | SR-002, SR-003
public class TruthScene
{
/// <summary>Gets or sets the window start, in seconds, inclusive.</summary>
[JsonPropertyName("start")]
public double Start { get; set; }
/// <summary>Gets or sets the window end, in seconds, inclusive.</summary>
[JsonPropertyName("end")]
public double End { get; set; }
/// <summary>
/// Gets or sets the accumulated posterior that justified this claim, in
/// <c>[0, 1]</c>, or <c>null</c> when the producer did not record one.
/// </summary>
/// <remarks>
/// Optional rather than defaulted to zero: absent and "believed with
/// probability zero" are different statements, and a claim nobody believes
/// would not have been written.
/// </remarks>
[JsonPropertyName("belief")]
public double? Belief { get; set; }
/// <summary>
/// Gets or sets how the actor was identified: <c>live</c>, <c>deferred</c>
/// or <c>pooled</c> (extraction AR-017).
/// </summary>
[JsonPropertyName("route")]
public string? Route { get; set; }
}
@@ -22,5 +22,13 @@ public class ServiceRegistrator : IPluginServiceRegistrator
// server that is down should be skipped for the whole sweep, not // server that is down should be skipped for the whole sweep, not
// retried once per item (JR-037). // retried once per item (JR-037).
serviceCollection.AddSingleton<IManifestExchangeClient, ManifestExchangeClient>(); serviceCollection.AddSingleton<IManifestExchangeClient, ManifestExchangeClient>();
// Registered concretely: there is no second implementation for an
// interface to abstract over and nothing to gain from inventing one
// (JR-042). The aligner consumes the signature on the fetch path, so a
// manifest is checked against the local file before its windows are
// stored (JR-047).
serviceCollection.AddSingleton<AudioSignatureService>();
serviceCollection.AddSingleton<ManifestAligner>();
} }
} }
@@ -0,0 +1,373 @@
using System;
using System.Collections.Generic;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// The JRay v1 content-derived audio signature: spectral peak bins taken from
/// the centre of a media file, so a truth file is self-identifying.
/// </summary>
/// <remarks>
/// The construction is owned by the public server specification §3 and is
/// implemented a second time, in C++, by the extraction pipeline
/// (<c>src/audio_signature.*</c>, extraction <c>IR-004</c>). <b>The two must
/// agree byte for byte</b> — a signature that differs in any parameter simply
/// does not match, which defeats the entire point of having one. Every constant
/// below is therefore load-bearing, and any change to one is a change to the
/// <c>v1:</c> prefix as well.
/// <para>
/// The server specification's prose is not sufficient to reproduce a byte
/// stream, so the details it leaves open are pinned by the golden fixture
/// shared with the extraction repo (JR-043,
/// <c>fixtures/audio/jray_audio_v1_golden.json</c>), and restated here:
/// </para>
/// <list type="bullet">
/// <item>Arithmetic is IEEE-754 <c>double</c> throughout. <c>float</c> is not
/// sufficient: the fixture has frames whose two strongest bands are within 1.3%
/// of each other.</item>
/// <item>Samples arrive as FFmpeg's native <c>s16 -&gt; flt</c> conversion,
/// <c>x * (1/32768)</c>, widened to double here.</item>
/// <item>Whole frames only:
/// <c>n_frames = 1 + (n_samples - 4096) / 1024</c>, integer division.</item>
/// <item>Hann window, <b>periodic</b>: <c>0.5 * (1 - cos(2*pi*n/4096))</c>, not
/// the symmetric <c>N-1</c> variant.</item>
/// <item>Plain radix-2 FFT, no normalisation; magnitude is
/// <c>sqrt(re^2 + im^2)</c>.</item>
/// <item>A band's value is the <b>mean</b> of the linear magnitudes in it, so a
/// wide high band is not favoured over a narrow low one.</item>
/// <item>The peak is the <c>argmax</c> over the 32 bands, ties to the lowest
/// index. The specification's log is a monotone squash and so cannot change an
/// argmax; it is applied only where it is observable, in the energy class.</item>
/// </list>
/// <para>
/// The FFT is written out here rather than taken from a library for the same
/// reason the pipeline writes its own: it is a fixed, fully specified transform,
/// and a dependency whose version could change the numerics is a liability when
/// the output has to be identical across two languages.
/// </para>
/// </remarks>
// TRACES: JR-042 | SR-003
public static class AudioSignature
{
/// <summary>Sample rate the signature is computed at, in Hz.</summary>
public const int SampleRate = 11025;
/// <summary>STFT frame size, in samples.</summary>
public const int FrameSize = 4096;
/// <summary>STFT hop size, in samples (~93 ms).</summary>
public const int HopSize = 1024;
/// <summary>Number of logarithmically spaced bands.</summary>
public const int NumBands = 32;
/// <summary>Low edge of the analysed band, in Hz.</summary>
public const double BandLoHz = 300.0;
/// <summary>High edge of the analysed band, in Hz.</summary>
public const double BandHiHz = 3000.0;
/// <summary>Length of the analysed window, in seconds.</summary>
public const double WindowSec = 120.0;
/// <summary>
/// Length of the analysed window, in samples (120.000 s at 11025 Hz).
/// </summary>
/// <remarks>
/// The decoded window is truncated to exactly this, so the frame count is
/// the same for every input and does not wobble with seek granularity or a
/// resampler tail.
/// </remarks>
public const int WindowSamples = 1323000;
/// <summary>Frames a full window yields: <c>1 + (1323000 - 4096) / 1024</c>.</summary>
/// <remarks>
/// The server specification says "~1290" and accepts a tolerance; the exact
/// count follows from the framing rule and is 1288.
/// </remarks>
public const int ExpectedFrames = 1288;
/// <summary>Guard added to both sides of the energy ratio.</summary>
public const double EnergyEps = 1e-12;
/// <summary>
/// The signature's own version prefix, separate from <c>schema_version</c>.
/// </summary>
/// <remarks>
/// A future change to the DSP chain must be <i>detectable</i> rather than
/// silently producing signatures that no longer match (JR-045).
/// </remarks>
public const string VersionPrefix = "v1:";
// Class thresholds on log10(E_frame / E_median). They deliberately straddle
// r = 0 rather than sit on it, so the median frame itself is never on a
// boundary.
private static readonly double[] EnergyClassEdges = [-0.6, -0.2, 0.2];
private static readonly (int Low, int High)[] Bands = BuildBandTable();
private static readonly double[] Window = BuildHannWindow();
private static readonly int[] BitReversal = BuildBitReversal();
private static readonly double[][] TwiddleReal = BuildTwiddles(cosine: true);
private static readonly double[][] TwiddleImag = BuildTwiddles(cosine: false);
/// <summary>
/// Gets the half-open FFT bin range <c>[Low, High)</c> owned by each of the
/// 32 log-spaced bands.
/// </summary>
/// <remarks>
/// Exposed so the golden fixture can assert the table itself rather than
/// only the signature it produces: the band table is the part of the
/// construction most likely to drift between two implementations.
/// </remarks>
/// <returns>One range per band, contiguous and non-overlapping.</returns>
public static IReadOnlyList<(int Low, int High)> BandFftBins() => Bands;
/// <summary>
/// Packs one byte per whole STFT frame: a 5-bit peak band index and a 2-bit
/// energy class.
/// </summary>
/// <remarks>
/// The byte layout is <c>(band &lt;&lt; 2) | class</c>, so bit 7 is always
/// clear and an arbitrary byte is not a valid signature. That structural
/// constraint is what the server validates on upload, and what keeps the
/// field from being usable as a payload channel.
/// </remarks>
/// <param name="mono">Mono samples at <see cref="SampleRate"/>, in [-1, 1).</param>
/// <returns>One byte per frame; empty when not even one frame fits.</returns>
public static byte[] PackFrames(ReadOnlySpan<float> mono)
{
if (mono.Length < FrameSize)
{
return [];
}
var frames = 1 + ((mono.Length - FrameSize) / HopSize);
var binLow = Bands[0].Low;
var binHigh = Bands[NumBands - 1].High; // exclusive
double binCount = binHigh - binLow;
var re = new double[FrameSize];
var im = new double[FrameSize];
var peak = new byte[frames];
var energy = new double[frames];
for (var f = 0; f < frames; f++)
{
var src = mono.Slice(f * HopSize, FrameSize);
for (var n = 0; n < FrameSize; n++)
{
re[n] = src[n] * Window[n];
im[n] = 0.0;
}
Fft(re, im);
// Per-band mean magnitude. The bands tile 300-3000 Hz with no gaps
// and no overlaps, so the frame's band-limited energy is the sum of
// the band sums — accumulated in band order, because the order of a
// floating-point summation is part of the contract.
var best = -1.0;
var bestBand = 0;
var total = 0.0;
for (var b = 0; b < NumBands; b++)
{
var (low, high) = Bands[b];
var sum = 0.0;
for (var k = low; k < high; k++)
{
sum += Math.Sqrt((re[k] * re[k]) + (im[k] * im[k]));
}
total += sum;
var mean = sum / (high - low);
if (mean > best)
{
best = mean; // ties -> lowest index
bestBand = b;
}
}
peak[f] = (byte)bestBand;
energy[f] = total / binCount;
}
// The reference is the upper median of the frame energies: an actually
// observed value rather than the average of the two middle ones, so it
// is bit-reproducible. It is also gain-invariant — loudness
// normalisation must not change a signature — and barely moves when the
// window is trimmed.
var sorted = (double[])energy.Clone();
Array.Sort(sorted);
var reference = sorted[sorted.Length / 2];
var packed = new byte[frames];
for (var f = 0; f < frames; f++)
{
var r = Math.Log10((energy[f] + EnergyEps) / (reference + EnergyEps));
packed[f] = (byte)(((peak[f] & 0x1F) << 2) | (EnergyClass(r) & 0x03));
}
return packed;
}
/// <summary>
/// Computes the full signature string for a decoded centre window.
/// </summary>
/// <param name="mono">Mono samples at <see cref="SampleRate"/>, in [-1, 1).</param>
/// <returns>
/// <c>v1:</c> followed by the base64 of <see cref="PackFrames"/>, or
/// <c>null</c> when not even one frame fits.
/// </returns>
public static string? FromMonoSamples(ReadOnlySpan<float> mono)
{
var packed = PackFrames(mono);
if (packed.Length == 0)
{
return null;
}
// Standard alphabet with '=' padding, which is what the server and the
// pipeline both emit.
return VersionPrefix + Convert.ToBase64String(packed);
}
private static int EnergyClass(double ratio)
{
if (ratio < EnergyClassEdges[0])
{
return 0;
}
if (ratio < EnergyClassEdges[1])
{
return 1;
}
if (ratio < EnergyClassEdges[2])
{
return 2;
}
return 3;
}
// edge[b] = 300 * (3000/300)^(b/32); band b owns FFT bins
// [k_lo[b], k_lo[b+1]) with k_lo[b] = ceil(edge[b] / hz_per_bin). Taking the
// ceiling once, into an integer table, means membership is never decided by
// a float comparison per bin per frame — which is where two implementations
// would otherwise be free to disagree.
private static (int Low, int High)[] BuildBandTable()
{
var hzPerBin = (double)SampleRate / FrameSize;
var edges = new int[NumBands + 1];
for (var b = 0; b <= NumBands; b++)
{
var hz = BandLoHz * Math.Pow(BandHiHz / BandLoHz, (double)b / NumBands);
edges[b] = (int)Math.Ceiling(hz / hzPerBin);
}
var table = new (int Low, int High)[NumBands];
for (var b = 0; b < NumBands; b++)
{
table[b] = (edges[b], edges[b + 1]);
}
return table;
}
private static double[] BuildHannWindow()
{
var w = new double[FrameSize];
for (var n = 0; n < FrameSize; n++)
{
w[n] = 0.5 * (1.0 - Math.Cos(2.0 * Math.PI * n / FrameSize));
}
return w;
}
private static int[] BuildBitReversal()
{
var bits = 0;
while ((1 << bits) < FrameSize)
{
bits++;
}
var rev = new int[FrameSize];
for (var i = 0; i < FrameSize; i++)
{
var r = 0;
for (var b = 0; b < bits; b++)
{
if ((i & (1 << b)) != 0)
{
r |= 1 << (bits - 1 - b);
}
}
rev[i] = r;
}
return rev;
}
// Twiddles are precomputed per stage from cos/sin of -2*pi*j/len, so the
// angle is an exactly reproducible double in either language and only the
// library's own rounding of cos/sin (<= 1 ulp) can differ — orders of
// magnitude below the decision margins the golden fixture records.
private static double[][] BuildTwiddles(bool cosine)
{
var stages = new List<double[]>();
for (var len = 2; len <= FrameSize; len <<= 1)
{
var half = len / 2;
var stage = new double[half];
for (var j = 0; j < half; j++)
{
var angle = -2.0 * Math.PI * j / len;
stage[j] = cosine ? Math.Cos(angle) : Math.Sin(angle);
}
stages.Add(stage);
}
return [.. stages];
}
// Radix-2 decimation-in-time complex FFT, in place, no normalisation.
private static void Fft(double[] re, double[] im)
{
for (var i = 0; i < FrameSize; i++)
{
var j = BitReversal[i];
if (i < j)
{
(re[i], re[j]) = (re[j], re[i]);
(im[i], im[j]) = (im[j], im[i]);
}
}
var stage = 0;
for (var len = 2; len <= FrameSize; len <<= 1, stage++)
{
var half = len / 2;
var wr = TwiddleReal[stage];
var wi = TwiddleImag[stage];
for (var start = 0; start < FrameSize; start += len)
{
for (var j = 0; j < half; j++)
{
var a = start + j;
var b = a + half;
var tr = (re[b] * wr[j]) - (im[b] * wi[j]);
var ti = (re[b] * wi[j]) + (im[b] * wr[j]);
re[b] = re[a] - tr;
im[b] = im[a] - ti;
re[a] += tr;
im[a] += ti;
}
}
}
}
}
@@ -0,0 +1,70 @@
using System;
using Jellyfin.Plugin.JRay.Configuration;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// The outcome of comparing a local audio signature against a remote one.
/// </summary>
/// <remarks>
/// Produced only when the two signatures actually align; a comparison that
/// reaches no tier yields no result at all rather than a zero-scored one, so a
/// caller cannot mistake "did not match" for "matched at the bottom".
/// </remarks>
public readonly struct AudioSignatureMatch : IEquatable<AudioSignatureMatch>
{
/// <summary>
/// Gets the fraction of overlapping frames whose peak band agreed, in [0, 1].
/// </summary>
public double Score { get; init; }
/// <summary>
/// Gets the best-scoring alignment, in frames, between the two analysis
/// windows.
/// </summary>
/// <remarks>
/// Positive means the local window lags the remote one. This is a
/// window-relative quantity and is <b>not</b> the offset to apply to
/// timings — see <see cref="OffsetSec"/>, which additionally accounts for
/// the two windows being anchored at different points in their files.
/// </remarks>
public int OffsetFrames { get; init; }
/// <summary>
/// Gets the seconds to add to every remote window to bring it into the local
/// file's timebase.
/// </summary>
/// <remarks>
/// This is the quantity JR-030 applies at store time, and the one
/// <see cref="ManifestConverter.ToTruthFile"/> takes.
/// </remarks>
public double OffsetSec { get; init; }
/// <summary>Gets the tier this score earns.</summary>
public MatchTier Tier { get; init; }
/// <summary>Compares two matches for equality.</summary>
/// <param name="left">Left operand.</param>
/// <param name="right">Right operand.</param>
/// <returns><c>true</c> when the two are equal.</returns>
public static bool operator ==(AudioSignatureMatch left, AudioSignatureMatch right) => left.Equals(right);
/// <summary>Compares two matches for inequality.</summary>
/// <param name="left">Left operand.</param>
/// <param name="right">Right operand.</param>
/// <returns><c>true</c> when the two differ.</returns>
public static bool operator !=(AudioSignatureMatch left, AudioSignatureMatch right) => !left.Equals(right);
/// <inheritdoc/>
public bool Equals(AudioSignatureMatch other)
=> Score.Equals(other.Score)
&& OffsetFrames == other.OffsetFrames
&& OffsetSec.Equals(other.OffsetSec)
&& Tier == other.Tier;
/// <inheritdoc/>
public override bool Equals(object? obj) => obj is AudioSignatureMatch other && Equals(other);
/// <inheritdoc/>
public override int GetHashCode() => HashCode.Combine(Score, OffsetFrames, OffsetSec, Tier);
}
@@ -0,0 +1,207 @@
using System;
using Jellyfin.Plugin.JRay.Configuration;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Reads v1 audio signatures and aligns two of them, recovering the time offset
/// between differently trimmed releases of one cut.
/// </summary>
/// <remarks>
/// The <b>consumer</b> half of the signature feature: <see cref="AudioSignature"/>
/// produces, this reads. Matching belongs to the plugin rather than the server
/// because the offset is applied client-side (JR-030) and manifests are never
/// rewritten — one stored manifest serves every trim of the same cut.
/// <para>
/// The construction is public server specification §3 "Matching and offset
/// recovery": slide one signature against the other over ±600 frames, score the
/// fraction of overlapping frames whose <b>peak band</b> agrees, and take the
/// argmax. The energy class is deliberately not scored — it is the coarser and
/// less re-encoding-stable of the two fields, and the specification's rule names
/// the peak bin alone.
/// </para>
/// </remarks>
// TRACES: JR-044, JR-045 | SR-003
public static class AudioSignatureMatcher
{
/// <summary>Widest alignment searched, in frames (±56 s).</summary>
/// <remarks>
/// Covers realistic trim differences. A release that differs in *speed*
/// (a PAL 4% speed-up) is not a constant offset and is correctly rejected by
/// the score threshold rather than mis-aligned by this search.
/// </remarks>
public const int MaxOffsetFrames = 600;
/// <summary>Score at or above which two signatures are the same cut.</summary>
public const double AudioThreshold = 0.85;
/// <summary>Score at or above which two signatures are possibly the same cut.</summary>
public const double LooseThreshold = 0.60;
/// <summary>
/// Fewest overlapping frames an alignment must have before its score counts.
/// </summary>
/// <remarks>
/// <b>Not from the specification</b> — a guard this implementation adds. The
/// slide is defined over overlapping frames, and without a floor the extreme
/// offsets compare a handful of frames, where a chance agreement scores 1.0
/// and beats the true alignment. 64 frames is ~6 s. It only ever excludes
/// alignments near the ±600 limit: two full-length signatures overlap by 688
/// frames even at the widest offset, so this never binds on the case the
/// feature exists for.
/// </remarks>
public const int MinOverlapFrames = 64;
/// <summary>Gets the duration one STFT frame advances, in seconds.</summary>
public static double FrameSeconds => (double)AudioSignature.HopSize / AudioSignature.SampleRate;
/// <summary>
/// Parses a signature string into its per-frame bytes, refusing anything that
/// is not a well-formed <c>v1:</c> signature.
/// </summary>
/// <remarks>
/// JR-045. An unknown prefix is <b>refused, not parsed</b>: a <c>v2:</c>
/// signature from a future producer describes a DSP chain this build does not
/// implement, so scoring it as v1 would silently produce a wrong answer where
/// declining produces a correct one — the item simply falls to the runtime
/// tier. That is the entire purpose of the prefix being separate from
/// <c>schema_version</c>.
/// <para>
/// Structure is checked too, matching what the server validates on upload:
/// every byte is a 5-bit band index and a 2-bit energy class, so bit 7 is
/// always clear.
/// </para>
/// </remarks>
/// <param name="signature">A signature string, or <c>null</c>.</param>
/// <returns>The frame bytes, or <c>null</c> if this is not a v1 signature.</returns>
public static byte[]? TryParseFrames(string? signature)
{
if (string.IsNullOrEmpty(signature)
|| !signature.StartsWith(AudioSignature.VersionPrefix, StringComparison.Ordinal))
{
return null;
}
var payload = signature[AudioSignature.VersionPrefix.Length..];
Span<byte> decoded = new byte[((payload.Length / 4) + 1) * 3];
if (!Convert.TryFromBase64String(payload, decoded, out var written) || written == 0)
{
return null;
}
var frames = decoded[..written].ToArray();
foreach (var frame in frames)
{
if ((frame & 0x80) != 0)
{
return null;
}
}
return frames;
}
/// <summary>
/// Aligns a local signature against a remote one and reports the tier and
/// offset it earns.
/// </summary>
/// <remarks>
/// Returns <c>null</c> — no match, and therefore <b>no offset</b> — when
/// either signature is absent or is not a v1 signature (JR-045), when either
/// item is shorter than the analysis window (JR-044), or when the best
/// alignment scores below <see cref="LooseThreshold"/>.
/// <para>
/// The short-media rule is checked on the runtimes rather than inferred from
/// a missing signature, because the two producers must apply the identical
/// rule and the runtime is what both of them test. An item under 120 s falls
/// back to the runtime tier, which is adequate: a sub-two-minute item is
/// rarely the ambiguous-providence case the signature exists to solve.
/// </para>
/// </remarks>
/// <param name="localSignature">Signature computed from the local file.</param>
/// <param name="remoteSignature">Signature carried by the fetched manifest.</param>
/// <param name="localRuntimeSec">Runtime of the local file, in seconds.</param>
/// <param name="remoteRuntimeSec">Runtime the manifest records, in seconds.</param>
/// <returns>The match, or <c>null</c> when the two do not align.</returns>
public static AudioSignatureMatch? Compare(
string? localSignature,
string? remoteSignature,
double localRuntimeSec,
double remoteRuntimeSec)
{
// JR-044 — below the window there is no signature to trust and no offset
// to apply, whatever the strings happen to contain.
if (localRuntimeSec < AudioSignature.WindowSec || remoteRuntimeSec < AudioSignature.WindowSec)
{
return null;
}
var local = TryParseFrames(localSignature);
var remote = TryParseFrames(remoteSignature);
if (local is null || remote is null)
{
return null;
}
var bestScore = -1.0;
var bestOffset = 0;
var found = false;
for (var d = -MaxOffsetFrames; d <= MaxOffsetFrames; d++)
{
// i indexes the remote signature; the local frame it is compared
// against is i + d, which must fall inside the local signature.
var first = Math.Max(0, -d);
var last = Math.Min(remote.Length, local.Length - d);
var overlap = last - first;
if (overlap < MinOverlapFrames)
{
continue;
}
var agreed = 0;
for (var i = first; i < last; i++)
{
if (((local[i + d] >> 2) & 0x1F) == ((remote[i] >> 2) & 0x1F))
{
agreed++;
}
}
var score = (double)agreed / overlap;
// Ties go to the alignment closest to zero: when a signature is
// degenerate enough that several offsets score alike, "not shifted"
// is the reading that does least damage, and the choice must be
// deterministic rather than an artefact of iteration order.
if (!found || score > bestScore || (score == bestScore && Math.Abs(d) < Math.Abs(bestOffset)))
{
bestScore = score;
bestOffset = d;
found = true;
}
}
if (!found || bestScore < LooseThreshold)
{
return null;
}
// Both windows are centred on their own file's midpoint, so an offset
// between the two windows is only part of the answer: the windows start
// at different absolute times whenever the runtimes differ, and that
// difference is the rest of it. Runtimes are >= WindowSec here, so
// neither start clamps at zero — the same condition under which
// AudioSignatureService does not clamp either.
var localStart = (localRuntimeSec / 2.0) - (AudioSignature.WindowSec / 2.0);
var remoteStart = (remoteRuntimeSec / 2.0) - (AudioSignature.WindowSec / 2.0);
return new AudioSignatureMatch
{
Score = bestScore,
OffsetFrames = bestOffset,
OffsetSec = (localStart - remoteStart) + (bestOffset * FrameSeconds),
Tier = bestScore >= AudioThreshold ? MatchTier.Audio : MatchTier.Loose,
};
}
}
@@ -0,0 +1,285 @@
using System;
using System.Buffers.Binary;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.MediaEncoding;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Computes the v1 audio signature for a media file, decoding its centre window
/// with the FFmpeg binary Jellyfin already ships.
/// </summary>
/// <remarks>
/// <b>No new dependency.</b> FFmpeg performs decode, downmix and resample; the
/// plugin adds only the fixed FFT and bin-peak extraction in
/// <see cref="AudioSignature"/>. The binary is reached through
/// <see cref="IMediaEncoder.EncoderPath"/>, so an installation that can
/// transcode can compute signatures, with nothing further to install and no
/// second copy of FFmpeg to keep in step.
/// <para>
/// The pipeline computes the same signature for files it processes locally
/// (extraction <c>IR-004</c>); this exists for the files it never sees. Both
/// producers must therefore agree exactly, including on the decode: the command
/// below is the CLI spelling of what the pipeline asks libswresample for — best
/// audio stream, mono, 11025 Hz, 32-bit float — and the golden fixture pins the
/// decoded PCM as well as the signature, so a codec-level divergence is
/// distinguishable from a DSP-level one (JR-043).
/// </para>
/// <para>
/// <b>Every failure degrades to no signature rather than to an error.</b> A
/// signature is an enhancement to cut matching; a missing one costs a tier, and
/// must never be able to break a fetch.
/// </para>
/// </remarks>
// TRACES: JR-042 | SR-003
public class AudioSignatureService
{
private readonly IMediaEncoder _encoder;
private readonly ILogger<AudioSignatureService> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="AudioSignatureService"/> class.
/// </summary>
/// <param name="encoder">Supplies the path of the FFmpeg binary Jellyfin ships.</param>
/// <param name="logger">Logger.</param>
public AudioSignatureService(IMediaEncoder encoder, ILogger<AudioSignatureService> logger)
{
_encoder = encoder;
_logger = logger;
}
/// <summary>
/// Computes the signature of the 120 s window centred on the media's
/// midpoint.
/// </summary>
/// <remarks>
/// The centre is used because the head and tail are the least
/// content-specific parts of a release: logos and cold opens at one end,
/// credits at the other.
/// <para>
/// Media shorter than <see cref="AudioSignature.WindowSec"/> yields
/// <c>null</c>: the window underflows, so there is no signature — the
/// identical rule the extraction producer applies, since diverging here
/// would break exactly the short items most likely to be misidentified
/// (JR-044, whose remaining half — applying no sync offset — belongs with
/// signature matching).
/// </para>
/// </remarks>
/// <param name="path">Path of the media file.</param>
/// <param name="runtimeSeconds">The item's runtime, as Jellyfin knows it.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>
/// The <c>v1:</c>-prefixed signature, or <c>null</c> for short media, media
/// with no usable audio, and any decode failure.
/// </returns>
public Task<string?> ComputeAsync(
string path,
double runtimeSeconds,
CancellationToken cancellationToken)
=> ComputeWithEncoderAsync(_encoder?.EncoderPath, path, runtimeSeconds, _logger, cancellationToken);
/// <summary>
/// <see cref="ComputeAsync"/> with the FFmpeg binary named explicitly.
/// </summary>
/// <remarks>
/// Internal so the golden-fixture test can drive the real decode path with
/// whatever FFmpeg the machine has, rather than standing up a fake
/// <see cref="IMediaEncoder"/> — a stub of a thirty-member interface would
/// be the larger risk of the two, and it is the decode that is under test.
/// </remarks>
/// <param name="encoderPath">Path of the FFmpeg binary to run.</param>
/// <param name="path">Path of the media file.</param>
/// <param name="runtimeSeconds">The item's runtime, as Jellyfin knows it.</param>
/// <param name="logger">Logger.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>The <c>v1:</c>-prefixed signature, or <c>null</c>.</returns>
internal static async Task<string?> ComputeWithEncoderAsync(
string? encoderPath,
string path,
double runtimeSeconds,
ILogger logger,
CancellationToken cancellationToken)
{
if (string.IsNullOrEmpty(path) || runtimeSeconds < AudioSignature.WindowSec)
{
return null;
}
if (string.IsNullOrEmpty(encoderPath))
{
logger.LogDebug("No FFmpeg binary available; skipping the audio signature for {Path}", path);
return null;
}
try
{
var samples = await DecodeCentreWindowAsync(encoderPath, path, runtimeSeconds, logger, cancellationToken)
.ConfigureAwait(false);
if (samples is null)
{
return null;
}
return AudioSignature.FromMonoSamples(samples);
}
catch (OperationCanceledException)
{
throw;
}
catch (Exception ex)
{
logger.LogDebug(ex, "Audio signature failed for {Path}; continuing without one", path);
return null;
}
}
/// <summary>
/// Decodes the centre window as mono 32-bit float PCM at 11025 Hz.
/// </summary>
/// <remarks>
/// The stream is truncated to exactly
/// <see cref="AudioSignature.WindowSamples"/> samples, so the frame count is
/// the same for every input rather than wobbling with seek granularity or a
/// resampler tail.
/// <para>
/// No <c>-map</c> is given: FFmpeg's default audio selection is the same
/// "best stream" choice the pipeline makes with
/// <c>av_find_best_stream</c>, and naming <c>0:a:0</c> instead would pick a
/// different track from the pipeline's on any file whose first audio stream
/// is not its main one — a commentary track, say.
/// </para>
/// </remarks>
private static async Task<float[]?> DecodeCentreWindowAsync(
string encoderPath,
string path,
double runtimeSeconds,
ILogger logger,
CancellationToken cancellationToken)
{
var start = (runtimeSeconds / 2.0) - (AudioSignature.WindowSec / 2.0);
if (start < 0.0)
{
start = 0.0;
}
var startArgument = start.ToString("0.000", CultureInfo.InvariantCulture);
var windowArgument = AudioSignature.WindowSec.ToString("0.000", CultureInfo.InvariantCulture);
var rateArgument = AudioSignature.SampleRate.ToString(CultureInfo.InvariantCulture);
var startInfo = new ProcessStartInfo
{
FileName = encoderPath,
RedirectStandardOutput = true,
RedirectStandardError = true,
RedirectStandardInput = false,
UseShellExecute = false,
CreateNoWindow = true,
};
startInfo.ArgumentList.Add("-nostdin");
startInfo.ArgumentList.Add("-v");
startInfo.ArgumentList.Add("error");
// Input seeking, so FFmpeg does not decode the whole file to reach the
// middle of it. Accurate by default: it seeks to the preceding keyframe
// and discards the excess, which is what the pipeline does by hand.
startInfo.ArgumentList.Add("-ss");
startInfo.ArgumentList.Add(startArgument);
startInfo.ArgumentList.Add("-i");
startInfo.ArgumentList.Add(path);
startInfo.ArgumentList.Add("-t");
startInfo.ArgumentList.Add(windowArgument);
startInfo.ArgumentList.Add("-vn");
startInfo.ArgumentList.Add("-sn");
startInfo.ArgumentList.Add("-dn");
startInfo.ArgumentList.Add("-ac");
startInfo.ArgumentList.Add("1");
startInfo.ArgumentList.Add("-ar");
startInfo.ArgumentList.Add(rateArgument);
startInfo.ArgumentList.Add("-f");
startInfo.ArgumentList.Add("f32le");
startInfo.ArgumentList.Add("-");
using var process = new Process { StartInfo = startInfo };
if (!process.Start())
{
return null;
}
try
{
// Read stderr concurrently: it is redirected, so leaving it unread
// would deadlock the moment FFmpeg filled the pipe.
var errorTask = process.StandardError.ReadToEndAsync(cancellationToken);
var bytes = await ReadWindowAsync(process.StandardOutput.BaseStream, cancellationToken)
.ConfigureAwait(false);
var error = await errorTask.ConfigureAwait(false);
if (bytes.Length < AudioSignature.FrameSize * sizeof(float))
{
// No audio stream, an unreadable file, or a runtime Jellyfin
// knows but the container does not support seeking into.
logger.LogDebug(
"FFmpeg returned {Bytes} bytes of audio for {Path}: {Error}",
bytes.Length,
path,
error);
return null;
}
var samples = new float[bytes.Length / sizeof(float)];
for (var i = 0; i < samples.Length; i++)
{
samples[i] = BinaryPrimitives.ReadSingleLittleEndian(bytes.AsSpan(i * sizeof(float)));
}
return samples;
}
finally
{
if (!process.HasExited)
{
try
{
process.Kill(entireProcessTree: true);
}
catch (InvalidOperationException)
{
// Exited between the check and the kill.
}
}
}
}
private static async Task<byte[]> ReadWindowAsync(Stream stream, CancellationToken cancellationToken)
{
var wanted = AudioSignature.WindowSamples * sizeof(float);
var buffer = new byte[wanted];
var filled = 0;
while (filled < wanted)
{
var read = await stream.ReadAsync(buffer.AsMemory(filled, wanted - filled), cancellationToken)
.ConfigureAwait(false);
if (read == 0)
{
break;
}
filled += read;
}
if (filled == wanted)
{
return buffer;
}
var truncated = new byte[filled];
Array.Copy(buffer, truncated, filled);
return truncated;
}
}
@@ -49,12 +49,30 @@ public sealed class ManagedTruthStore : IManagedTruthStore
try try
{ {
using var stream = File.OpenRead(path); using var stream = File.OpenRead(path);
return await JsonSerializer.DeserializeAsync<TruthFile>(stream, JsonOptions, cancellationToken) var truth = await JsonSerializer.DeserializeAsync<TruthFile>(stream, JsonOptions, cancellationToken)
.ConfigureAwait(false); .ConfigureAwait(false);
// Managed truth is checked on the way out as well as on the way in.
// Data written by an earlier plugin version is already on disk, and
// it did not pass today's PUT.
if (!TruthSchema.IsSupported(truth))
{
_logger.LogWarning(
"JRay: ignoring managed truth {Path} — schema_version {Found}, expected {Expected}. Re-push or re-extract this item.",
path,
truth?.SchemaVersion ?? 0,
TruthSchema.SupportedVersion);
return null;
}
return truth;
} }
catch (Exception ex) when (ex is IOException or JsonException) catch (Exception ex) when (ex is IOException or JsonException)
{ {
_logger.LogWarning(ex, "JRay: failed to read managed truth file {Path}", path); _logger.LogWarning(
ex,
"JRay: failed to read managed truth file {Path}. If this is v1 data, re-push it — v1 is no longer read.",
path);
return null; return null;
} }
} }
@@ -0,0 +1,205 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Jellyfin.Plugin.JRay.Configuration;
using Jellyfin.Plugin.JRay.Models;
using Microsoft.Extensions.Logging;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Aligns a freshly fetched manifest to the local file before its windows are
/// stored.
/// </summary>
/// <remarks>
/// <b>Why the client re-derives an offset the server already sent.</b> The
/// server has never seen the local file. Its offset is a claim about a runtime
/// it was told, so it can only ever be a runtime-difference inference. A local
/// alignment compares the manifest's own audio signature against the file the
/// windows will actually be drawn over, which is the authoritative comparison —
/// and it needs no round trip, so no signature leaves the instance. Where the
/// two disagree, the local one wins.
/// <para>
/// This is what jRay's specification means by matching being "a consumer
/// concern": the server never rewrites a manifest, so one stored manifest serves
/// every trim of the same cut, and each client shifts it to its own timebase
/// (JR-030).
/// </para>
/// <para>
/// <b>Degradation, not failure.</b> Signatures switched off, an item under the
/// 120 s window, a manifest with no signature, a missing FFmpeg binary, a decode
/// error, or two signatures that simply do not match — every one of these falls
/// back to the server's offset. A signature is an enhancement to cut matching; a
/// missing one costs a tier and must never be able to break a fetch.
/// </para>
/// </remarks>
// TRACES: JR-047 | SR-003
public class ManifestAligner
{
private readonly AudioSignatureService _signatures;
private readonly ILogger<ManifestAligner> _logger;
/// <summary>
/// Initializes a new instance of the <see cref="ManifestAligner"/> class.
/// </summary>
/// <param name="signatures">Computes the local file's audio signature.</param>
/// <param name="logger">Logger.</param>
public ManifestAligner(AudioSignatureService signatures, ILogger<ManifestAligner> logger)
{
_signatures = signatures;
_logger = logger;
}
/// <summary>
/// Decides which offset to apply, given a local signature that has already
/// been computed.
/// </summary>
/// <remarks>
/// Split from <see cref="AlignAsync"/> so the decision is testable without
/// an FFmpeg binary or a media file: everything interesting happens here,
/// and the caller only supplies the two strings.
/// </remarks>
/// <param name="localSignature">The local file's signature, or <c>null</c>.</param>
/// <param name="manifestSignature">The manifest's signature, or <c>null</c>.</param>
/// <param name="localRuntimeSec">Local file runtime, in seconds.</param>
/// <param name="manifestRuntimeSec">Runtime the manifest records, in seconds.</param>
/// <param name="serverTier">The tier the server reported.</param>
/// <param name="serverOffsetSec">The offset the server reported.</param>
/// <returns>What to apply, and how it was decided.</returns>
public static TruthAlignment Resolve(
string? localSignature,
string? manifestSignature,
double localRuntimeSec,
double manifestRuntimeSec,
MatchTier serverTier,
double serverOffsetSec)
{
var alignment = new TruthAlignment
{
Source = AlignmentSource.Server,
Tier = serverTier,
OffsetSec = serverOffsetSec,
ServerTier = serverTier,
ServerOffsetSec = serverOffsetSec,
LocalSignature = localSignature,
};
// "No alignment was possible" and "the audio does not match" are very
// different things to tell someone, and `Compare` returns null for both.
// A comparison is only possible when both items clear the 120 s window
// (JR-044) and both signatures parse as v1 (JR-045) — a `v2:` signature
// from a future producer is un-comparable, not a mismatch. Separating
// them here is what keeps a 90-second extra from being reported as
// content that disagrees with its own manifest.
var comparable = localSignature is not null
&& manifestSignature is not null
&& localRuntimeSec >= AudioSignature.WindowSec
&& manifestRuntimeSec >= AudioSignature.WindowSec
&& AudioSignatureMatcher.TryParseFrames(localSignature) is not null
&& AudioSignatureMatcher.TryParseFrames(manifestSignature) is not null;
if (!comparable)
{
return alignment;
}
var match = AudioSignatureMatcher.Compare(
localSignature, manifestSignature, localRuntimeSec, manifestRuntimeSec);
if (match is null)
{
// Both sides had a signature and they did not align at any tier. The
// server's offset still applies — the audio may legitimately differ,
// and a signature must not break a fetch — but this is the strongest
// available hint that the manifest describes different content, so
// it is recorded rather than silently discarded.
alignment.Source = AlignmentSource.LocalMismatch;
return alignment;
}
alignment.Source = AlignmentSource.Local;
alignment.Tier = match.Value.Tier;
alignment.OffsetSec = match.Value.OffsetSec;
alignment.Score = match.Value.Score;
alignment.OffsetFrames = match.Value.OffsetFrames;
return alignment;
}
/// <summary>
/// Computes the local file's signature if it can, then resolves the
/// alignment.
/// </summary>
/// <param name="mediaPath">Path of the local media file.</param>
/// <param name="localRuntimeSec">Local file runtime, in seconds.</param>
/// <param name="manifest">The fetched manifest.</param>
/// <param name="serverTier">The tier the server reported.</param>
/// <param name="serverOffsetSec">The offset the server reported.</param>
/// <param name="computeSignatures">Whether signatures are enabled in configuration.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>What to apply, and how it was decided.</returns>
public async Task<TruthAlignment> AlignAsync(
string mediaPath,
double localRuntimeSec,
Jmanifest manifest,
MatchTier serverTier,
double serverOffsetSec,
bool computeSignatures,
CancellationToken cancellationToken)
{
ArgumentNullException.ThrowIfNull(manifest);
var manifestSignature = manifest.Cut?.AudioSignature;
string? localSignature = null;
// The decode is the expensive half, so it is skipped outright when it
// could not change the answer: no manifest signature to compare against
// means no local alignment is possible.
if (computeSignatures && manifestSignature is not null && !string.IsNullOrEmpty(mediaPath))
{
localSignature = await _signatures
.ComputeAsync(mediaPath, localRuntimeSec, cancellationToken)
.ConfigureAwait(false);
}
var alignment = Resolve(
localSignature,
manifestSignature,
localRuntimeSec,
manifest.Cut?.RuntimeSec ?? 0.0,
serverTier,
serverOffsetSec);
switch (alignment.Source)
{
case AlignmentSource.Local:
_logger.LogInformation(
"Local audio alignment for {Path}: score {Score:F3}, {Frames} frames, offset {Offset:F3}s at tier {Tier} (server said {ServerOffset:F3}s at {ServerTier})",
mediaPath,
alignment.Score,
alignment.OffsetFrames,
alignment.OffsetSec,
alignment.Tier,
alignment.ServerOffsetSec,
alignment.ServerTier);
break;
case AlignmentSource.LocalMismatch:
_logger.LogWarning(
"Audio signatures for {Path} did not align with the fetched manifest; applying the server's {Offset:F3}s at {Tier}. This may be a different cut, a different language track, or a heavy re-encode.",
mediaPath,
alignment.OffsetSec,
alignment.Tier);
break;
default:
_logger.LogDebug(
"No local alignment for {Path}; applying the server's {Offset:F3}s at {Tier}",
mediaPath,
alignment.OffsetSec,
alignment.Tier);
break;
}
return alignment;
}
}
@@ -39,11 +39,38 @@ public static class ManifestConverter
var truth = new TruthFile var truth = new TruthFile
{ {
SchemaVersion = 1, SchemaVersion = TruthSchema.SupportedVersion,
Movie = mediaPath ?? string.Empty, Movie = mediaPath ?? string.Empty,
SampleFps = manifest.Extraction?.SampleFps ?? 0,
}; };
// Provenance is carried across rather than flattened. The two blocks
// have the same shape by design (JR-002), so anything the server knew
// about how a manifest was produced survives into the stored truth.
if (manifest.Extraction is { } extraction)
{
truth.Extraction = new TruthExtraction
{
SampleFps = extraction.SampleFps,
ExtinctionSec = extraction.ExtinctionSec,
PipelineVersion = extraction.PipelineVersion,
GallerySize = extraction.GallerySize,
GalleryScope = extraction.GalleryScope,
};
}
// The cut is the *manifest's*, not the local file's: it records the
// encode the timings were measured against, which is what makes the
// applied offset interpretable later. Recording the local runtime here
// instead would erase the very discrepancy the offset corrects.
if (manifest.Cut is { } cut)
{
truth.Cut = new TruthCut
{
RuntimeSec = cut.RuntimeSec,
AudioSignature = cut.AudioSignature,
};
}
foreach (var actor in manifest.Actors) foreach (var actor in manifest.Actors)
{ {
var converted = new TruthActor var converted = new TruthActor
@@ -60,7 +87,18 @@ public static class ManifestConverter
// reader can index. // reader can index.
var start = Math.Max(0, scene.Start + offsetSec); var start = Math.Max(0, scene.Start + offsetSec);
var end = Math.Max(start, scene.End + offsetSec); var end = Math.Max(start, scene.End + offsetSec);
converted.Scenes.Add(new[] { start, end }); converted.Scenes.Add(new TruthScene
{
Start = start,
End = end,
// Belief and route survive the conversion. They are what a
// consumer needs to know how far to trust a window, and
// dropping them here would silently downgrade every fetched
// manifest against a locally extracted one.
Belief = scene.Belief,
Route = scene.Route,
});
} }
truth.Actors.Add(converted); truth.Actors.Add(converted);
@@ -80,9 +118,19 @@ public static class ManifestConverter
/// </remarks> /// </remarks>
/// <param name="tier">The tier achieved.</param> /// <param name="tier">The tier achieved.</param>
/// <param name="offsetSec">The offset applied.</param> /// <param name="offsetSec">The offset applied.</param>
/// <param name="alignment">How the offset was arrived at, when known.</param>
/// <returns>A caveat string, or null when the match needs no explanation.</returns> /// <returns>A caveat string, or null when the match needs no explanation.</returns>
public static string? DescribeCaveat(MatchTier tier, double offsetSec) public static string? DescribeCaveat(MatchTier tier, double offsetSec, TruthAlignment? alignment = null)
{ {
// Ranked before the tier, because it is the stronger statement: the
// server's tier says the runtimes are compatible, while a signature
// mismatch says the audio itself is not. The second outranks the first.
if (alignment?.Source == AlignmentSource.LocalMismatch)
{
return "This file's audio does not match the fetched manifest — it may be a different cut, "
+ "a different language track, or a heavy re-encode. Timings may be wrong.";
}
if (tier == MatchTier.Loose) if (tier == MatchTier.Loose)
{ {
return "Matched loosely — the runtime differs from this server's copy, so timings may drift."; return "Matched loosely — the runtime differs from this server's copy, so timings may drift.";
@@ -279,6 +279,7 @@ public class ManifestExchangeClient : IManifestExchangeClient, IDisposable
/// completion and then measuring is exactly the denial-of-service primitive /// completion and then measuring is exactly the denial-of-service primitive
/// the cap exists to prevent. /// the cap exists to prevent.
/// </remarks> /// </remarks>
// TRACES: JR-028 | SR-004
private static async Task<string?> ReadCappedAsync( private static async Task<string?> ReadCappedAsync(
HttpResponseMessage response, HttpResponseMessage response,
long cap, long cap,
@@ -48,7 +48,7 @@ public static class PresenceLookup
// scale (see JR-006). // scale (see JR-006).
foreach (var window in actor.Scenes) foreach (var window in actor.Scenes)
{ {
if (window.Length == 2 && window[0] <= t && t <= window[1]) if (window is not null && window.Start <= t && t <= window.End)
{ {
return true; return true;
} }
@@ -101,17 +101,17 @@ public static class PresenceLookup
double previousStart = double.NegativeInfinity; double previousStart = double.NegativeInfinity;
foreach (var window in actor.Scenes) foreach (var window in actor.Scenes)
{ {
if (window.Length != 2) if (window is null)
{ {
continue; continue;
} }
if (window[0] < previousStart) if (window.Start < previousStart)
{ {
return false; return false;
} }
previousStart = window[0]; previousStart = window.Start;
} }
return true; return true;
@@ -83,12 +83,36 @@ public sealed class TruthDataService : ITruthDataService
using var stream = File.OpenRead(truthPath); using var stream = File.OpenRead(truthPath);
var truth = await JsonSerializer.DeserializeAsync<TruthFile>(stream, JsonOptions, cancellationToken) var truth = await JsonSerializer.DeserializeAsync<TruthFile>(stream, JsonOptions, cancellationToken)
.ConfigureAwait(false); .ConfigureAwait(false);
if (!TruthSchema.IsSupported(truth))
{
// Named, not silent. A stale v1 sidecar makes an item look
// un-extracted, and re-extracting a library that has already
// been processed is the most expensive mistake this plugin can
// cause a user to make. The log line is what distinguishes the
// two states.
_logger.LogWarning(
"JRay: ignoring truth file {TruthPath} — schema_version {Found}, expected {Expected}. Re-extract this item.",
truthPath,
truth?.SchemaVersion ?? 0,
TruthSchema.SupportedVersion);
_cache[itemId] = new CacheEntry(null, DateTime.UtcNow);
return null;
}
_cache[itemId] = new CacheEntry(truth, DateTime.UtcNow); _cache[itemId] = new CacheEntry(truth, DateTime.UtcNow);
return truth; return truth;
} }
catch (Exception ex) when (ex is IOException or JsonException) catch (Exception ex) when (ex is IOException or JsonException)
{ {
_logger.LogWarning(ex, "JRay: failed to read truth file {TruthPath}", truthPath); // A v1 file also lands here rather than above: `scenes` was a float
// pair in v1 and is an object in v2, so it fails to deserialise
// before the version can be inspected. Both routes must therefore
// name the file, which is why the message below says the same thing.
_logger.LogWarning(
ex,
"JRay: failed to read truth file {TruthPath}. If this is a v1 file, re-extract the item — v1 is no longer read.",
truthPath);
return null; return null;
} }
} }
@@ -0,0 +1,69 @@
using System.Globalization;
using Jellyfin.Plugin.JRay.Models;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// The one place that decides whether a truth file speaks a version this plugin
/// understands.
/// </summary>
/// <remarks>
/// <b>Flag day, not dual-accept.</b> Only <see cref="SupportedVersion"/> is
/// accepted; every other value is refused on every path — sidecar read, managed
/// <c>PUT</c>, managed store load, and converted manifest. There is no
/// transitional v1 read path.
/// <para>
/// All three components are pre-release and move together, and the alternative
/// carries a cost that outlasts the transition: a v1 read path is the one nobody
/// exercises, so it is the one that rots, and it would have to be dragged
/// through every subsequent change to the reader.
/// </para>
/// <para>
/// <b>The consequence is stated rather than discovered:</b> v1 sidecar files
/// already on disk stop being read at the bump and stay dark until the library
/// is re-extracted. Callers log the rejection naming the file and the version
/// found, because an item that looks un-extracted when it is merely stale is the
/// failure mode that wastes a user's compute.
/// </para>
/// <para>
/// This exists as a shared unit because the check used to live only in the
/// <c>PUT</c> controller while sidecar reads did not check at all — so the
/// format the plugin claimed to require and the format it would actually parse
/// were different things.
/// </para>
/// </remarks>
// TRACES: JR-003 | SR-003
public static class TruthSchema
{
/// <summary>
/// The only <c>schema_version</c> this plugin reads or writes.
/// </summary>
/// <remarks>
/// System-level (SR-003): the same number means the same format in all three
/// repos, and is incremented once per breaking change across all of them.
/// </remarks>
public const int SupportedVersion = 2;
/// <summary>
/// Determines whether a truth file speaks the supported version.
/// </summary>
/// <param name="truth">The parsed truth file, which may be <c>null</c>.</param>
/// <returns><c>true</c> only when the version matches exactly.</returns>
public static bool IsSupported(TruthFile? truth)
=> truth is not null && truth.SchemaVersion == SupportedVersion;
/// <summary>
/// Builds the message describing why a truth file was refused.
/// </summary>
/// <remarks>
/// Names both the version found and the version expected. A rejection that
/// says only "unsupported" leaves the reader unable to tell a stale file
/// from a corrupt one.
/// </remarks>
/// <param name="found">The version encountered.</param>
/// <returns>A message naming both versions.</returns>
public static string DescribeRejection(int found)
=> string.Create(
CultureInfo.InvariantCulture,
$"Unsupported schema_version {found}; expected {SupportedVersion}. Re-extract the item — v1 truth data is no longer read.");
}
+93 -11
View File
@@ -55,9 +55,14 @@ The truth file is **not** the Jmanifest. It carries installation-local fields
identity block the exchange adds. See §6. identity block the exchange adds. See §6.
**Current:** [`Models/TruthFile.cs`](Jellyfin.Plugin.JRay/Models/TruthFile.cs), **Current:** [`Models/TruthFile.cs`](Jellyfin.Plugin.JRay/Models/TruthFile.cs),
`schema_version: 1`. **Gap:** the schema below is not implemented, and the other `schema_version: 2` the schema below is implemented, with
two specs describe the pending bump in more detail than this one does — the [`Models/TruthScene.cs`](Jellyfin.Plugin.JRay/Models/TruthScene.cs),
ownership is stated but not yet exercised. [`Models/TruthExtraction.cs`](Jellyfin.Plugin.JRay/Models/TruthExtraction.cs) and
[`Models/TruthCut.cs`](Jellyfin.Plugin.JRay/Models/TruthCut.cs).
**Gap:** the ownership claim is still not *enforced* — nothing checks that the
other two repos reference this section rather than restating the schema. JR-001's
verification tier is `static` for that reason, and no such check exists yet.
### JR-002 — `schema_version: 2` ### JR-002 — `schema_version: 2`
@@ -118,9 +123,12 @@ same upload must agree on its `content_id`, and belief is a producer-side
estimate that may legitimately differ between pipeline versions for identical estimate that may legitimately differ between pipeline versions for identical
timings. It replicates the way `audio_signature` does — see the server spec §9a. timings. It replicates the way `audio_signature` does — see the server spec §9a.
**Gap:** entire requirement. The changes are all breaking and ship as **one** **Current:** implemented. `scenes` are `TruthScene` objects carrying `belief` and
bump (SR-003), together with extraction `IR-002` and the server's acceptance of `route`; `extraction.*` and `cut.*` are their own types; `anneal_sec` and the
the new shape. top-level `sample_fps` are **deleted, not zeroed**. `ManifestConverter` carries
belief, route and both blocks through from a fetched manifest rather than
flattening them. **Gap:** none — this shipped with extraction `IR-002` and the
server's `jmanifest_version: 2` as the one coordinated SR-003 bump.
### JR-003 — Unknown `schema_version` is refused, never guessed ### JR-003 — Unknown `schema_version` is refused, never guessed
@@ -140,9 +148,20 @@ version found, rather than silently reporting no coverage — an item that looks
un-extracted when it was merely stale is the failure mode that wastes a user's un-extracted when it was merely stale is the failure mode that wastes a user's
compute. compute.
**Current:** `PUT` rejects `schema_version != 1` with `400`; sidecar reads do not **Current:** implemented.
check the version at all. **Gap:** the version check must move into the shared [`Services/TruthSchema.cs`](Jellyfin.Plugin.JRay/Services/TruthSchema.cs) holds
read path so all three sources are covered, and the target becomes `2`. the single `SupportedVersion = 2` and is consulted on **all four** paths — sidecar
read, managed store load, managed `PUT`, and the converter that writes a fetched
manifest. Rejections name the file and the version found, per the paragraph above.
Managed truth is checked on load as well as on `PUT`, because data written by an
earlier plugin build is already on disk and never passed today's `PUT`.
One consequence is worth stating: a v1 file usually fails to *deserialise* before
its version can be read, since `scenes` was a float pair and is now an object. So
both the parse-failure path and the version-gate path must name the file — a
rejection the user cannot attribute to a stale sidecar is the failure mode JR-003
exists to prevent. **Gap:** none.
### JR-004 — A window is a scene-membership claim ### JR-004 — A window is a scene-membership claim
@@ -765,8 +784,71 @@ frames whose peak bin matches — is a **consumer** concern and belongs to this
plugin. Offsets are applied client-side per JR-030; manifests are never plugin. Offsets are applied client-side per JR-030; manifests are never
rewritten. rewritten.
**Current:** `ComputeAudioSignatures` exists as a configuration switch. **Gap:** **Current:** JR-042 and JR-043 hold. `AudioSignature` implements the construction
entire requirement, both computation and matching. above — band table, periodic Hann, radix-2 FFT, band-mean peak, energy class,
packing — and `AudioSignatureService` decodes the centre window by running the
FFmpeg binary `IMediaEncoder.EncoderPath` names, so the plugin gained no
dependency. `fixtures/audio/` holds the extraction repo's three golden-vector
files byte-identically, and the computed signature equals the recorded vector
exactly (UT-038 … UT-044). The binding check regenerates the fixture PCM from
`make_fixture.py`'s arithmetic and verifies it against the recorded decode
checksums, so it runs on a CI host with no codec at all; the two tests that
exercise the real FFmpeg decode self-skip without a binary.
JR-044 and JR-045 hold as well, both halves. `AudioSignatureMatcher` is the
reader they were waiting on: it implements the ±600-frame slide above, scoring
the fraction of overlapping frames whose peak band agrees, and returns the tier
and the offset. `TryParseFrames` refuses any prefix but `v1:`, so a future
producer's `v2:` drops the item to the runtime tier rather than being scored as
if it were understood; and a runtime under 120 s yields no match and therefore no
offset, read off the runtime rather than inferred from a missing string, because
the runtime is what both producers test (UT-045 … UT-052).
The offset has **two terms**, which is easy to miss: the recovered slide, and the
difference between where the two windows are anchored. Both windows are centred
on their own file's midpoint, so unequal runtimes start them at different
absolute times. A release carrying 40 s of extra head material recovers 20 s from
each term.
One parameter is **not** from the specification: an alignment must overlap by at
least 64 frames (~6 s) before its score counts. Without a floor the extreme
offsets compare a handful of frames, where a chance agreement scores 1.0 and
beats the true alignment. It never binds on the real case — two full-length
signatures still overlap by 688 frames at the widest offset.
JR-047 connects it. `ManifestAligner` runs on the fetch path, before anything is
stored: it computes the local file's signature, compares it against the one the
manifest carries, and **applies the local offset in preference to the server's**.
The server has never seen this file — its offset is a runtime-difference
inference at best — and the comparison needs no round trip, so no signature
leaves the instance. `ComputeAudioSignatures` now gates that decode.
Everything that can go wrong degrades to the server's offset rather than
refusing: signatures off, no manifest signature, media under the window, a `v2:`
producer, a missing binary, a decode error. A signature is an enhancement to cut
matching and must never be able to break a fetch. A *genuine* disagreement —
both signatures valid, both items long enough, best alignment still under 0.60 —
is recorded and surfaced as a caveat that outranks the tier's own, since "the
audio does not match" is a stronger statement than "the runtimes differ"; the
manifest is still stored, because the audio may legitimately differ.
"Un-comparable" and "does not match" are kept distinct. A 90-second extra is not
content disagreeing with its manifest, and reporting it as such would be worse
than saying nothing.
The applied offset, the score, the recovered slide, and the local file's own
signature are written beside the truth file (`TruthAlignment`) — the offset is
otherwise unrecoverable once the windows are shifted, and the stored signature
lets a later fetch align without decoding again.
See [`docs/audio-alignment.md`](docs/audio-alignment.md) for the mechanism end to
end.
**Gap:** contribution does not yet attach a signature (JR-034), and the stored
`local_signature` is written but not read back, so today every fetch decodes.
Server-side catalogue matching — `POST /manifests/search` and the `audio` tier —
is the server's UR-009 and is deliberately sequenced after signature coverage
accumulates; nothing here depends on it.
--- ---
+288
View File
@@ -0,0 +1,288 @@
# Audio signature and alignment — how it works
How a manifest fetched from a public server is checked against, and shifted onto,
your own copy of a film.
Requirements: `JR-042``JR-045`, `JR-047`. Construction is owned by
[`JRay-public-server/SPEC.md` §3](../../JRay-public-server/SPEC.md); this document
is the mechanism end to end.
---
## The problem
A manifest says "Peter Capaldi is on screen from 42:10 to 42:38". Your file may
not agree, because releases of the same film are trimmed differently — a
distributor logo here, a longer certificate card there. Thirty seconds of extra
head material makes every window in the manifest wrong by thirty seconds.
The old defence was the runtime tier: accept a manifest only if the runtimes
agree within ±2 s. That rejects exactly the releases it should be fixing, and
accepts anything that happens to be the same length.
A **content-derived signature** answers the question the exchange actually needs
*do these timings apply to this media?* — and, when the answer is "yes but
shifted", says by how much.
Deliberately, it does not answer *what file is this?* There is no file hash
anywhere in the system; the tier was withdrawn on legal grounds. The signature
identifies a **cut**, so two different encodes of the same edit agree.
---
## The signature
120 seconds from the **centre** of the media — the head and tail are the least
content-specific parts of a release, being logos and credits.
```
decode centre window runtime/2 ± 60 s
downmix mono
resample 11025 Hz
STFT 4096-sample frame, 1024-sample hop (~93 ms), Hann
band 3003000 Hz, 32 logarithmically spaced bands
per frame peak band index (5 bits) + energy class (2 bits)
-> one byte, bit 7 always clear
result 1288 bytes, base64, prefixed "v1:"
```
Peak bins are used because they survive lossy re-encoding, loudness
normalisation and channel-layout differences, where absolute magnitudes do not.
Same principle as Chromaprint/AcoustID, but self-contained: no external service
is queried, so no lookup leaks which titles an instance holds.
Bit 7 being always clear is not decoration. It is what makes an arbitrary byte
string *not* a valid signature, which is what the server validates on upload, and
what stops the field being usable as a payload channel. The length is fixed by
the construction rather than merely bounded, for the same reason — a caller
cannot choose it, so it cannot become a variable-size container.
### Why the parameters are written down twice
The specification's prose does not determine a byte stream. Two people
implementing "32 log-spaced bands, take the peak" will disagree on at least:
- `float` or `double` — the fixture has frames whose two strongest bands are
within 1.3% of each other, so `float` is not sufficient;
- periodic Hann (`/N`) or symmetric (`/(N-1)`);
- whether a band's value is the **mean** or the **sum** of its magnitudes (sum
favours wide high bands over narrow low ones);
- which way an `argmax` breaks ties;
- whether the frame count is `1 + (n - 4096)/1024` or something that wobbles
with the resampler tail.
Every one of those is pinned at the top of
[`AudioSignature.cs`](../Jellyfin.Plugin.JRay/Services/AudioSignature.cs) and
matched in the C++ producer. A signature that differs in any parameter simply
does not match, which defeats the entire point of having one.
### Two implementations, proven equal
The extraction pipeline (C++, `IR-004`) computes this for files it processes
locally. The plugin (C#, `JR-042`) computes it for the files the pipeline never
sees. Both must agree exactly.
That is a *checked* claim, not an aspiration. `fixtures/audio/` holds three files
byte-identical to the extraction repo's copies:
| File | What it pins |
|---|---|
| `jray_audio_v1_tone.flac` | 120 s of tones stepping through all 32 bands, amplitudes walking a golden-ratio sequence so all four energy classes appear |
| `jray_audio_v1_golden.json` | The expected signature, the band→FFT-bin table, and checksums of the decoded PCM |
| `make_fixture.py` | Regenerates the media from plain arithmetic — no numpy, ports to any language in ~20 lines |
The binding check regenerates the fixture PCM from `make_fixture.py`'s arithmetic
and verifies it against the recorded `s16le`/`f32le` checksums **before** making
any DSP claim. So it runs on a CI host with no codec at all, and a decode
divergence stays distinguishable from a DSP one. The two tests that drive real
FFmpeg self-skip without a binary — a check that skips is not a check, so it is
never the only cover for a claim.
---
## Matching, and the offset
Two signatures are compared by sliding one against the other:
```
for offset in -600 .. +600 frames: # ±56 s
score(offset) = fraction of overlapping frames whose peak band matches
best = argmax score
```
| Score | Meaning |
|---|---|
| `≥ 0.85` | Same cut. `audio` tier. The offset applies |
| `0.60 0.85` | Possibly the same cut, degraded audio. `loose` tier, surfaced as a caveat |
| `< 0.60` | Different content. No match |
Only the **peak band** is scored. The energy class is the coarser and less
re-encoding-stable of the two fields, and the specification's rule names the peak
bin alone.
Speed-differing releases (a PAL 4% speed-up) are not a constant offset and are
correctly rejected by the score threshold rather than mis-aligned.
### The offset has two terms
This is the part that is easy to get wrong. Both windows are centred on **their
own file's** midpoint, so when the runtimes differ the two windows do not start
at the same point in the content:
```
offset = (local_window_start - manifest_window_start) + slide × 1024/11025
└────────── anchor difference ──────────┘ └──── recovered ────┘
```
A release carrying 40 s of extra head material recovers **20 s from each term**.
Using the slide alone would be wrong by half the runtime difference on every
shifted release. The specification's pseudocode describes only the slide, because
it is written from the server's position, where both signatures are being
compared against one stored manifest.
### One parameter that is not from the specification
An alignment must overlap by at least **64 frames** (~6 s) before its score
counts. Without a floor the extreme offsets compare a handful of frames, where a
chance agreement scores 1.0 and beats the true alignment. It never binds on the
real case: two full-length signatures still overlap by 688 frames at the widest
offset. It is marked as a local addition in the code.
---
## What happens when a manifest arrives
`ManifestController.FetchItem``ManifestAligner.AlignAsync`, before anything is
stored:
```
1. Server returns a manifest, a tier, and an offset.
2. Does the manifest carry cut.audio_signature? no -> use the server's offset
Are signatures enabled in configuration? no -> use the server's offset
3. Decode this file's centre window, compute its signature.
4. Compare the two.
match -> apply the LOCAL offset and tier
no match -> apply the server's offset, record the disagreement
not comparable -> apply the server's offset
5. Apply the offset to every window, once, at store time.
6. Write the truth file, and the alignment beside it.
```
### Why the local answer wins
**The server has never seen your file.** Its offset is a claim about a runtime it
was told — at best a runtime-difference inference. A local alignment compares the
manifest's own signature against the media the windows will actually be drawn
over, which is the authoritative comparison. It also needs no round trip, so no
signature ever leaves the instance.
This is what the specification means by matching being "a consumer concern": the
server never rewrites a manifest, so **one stored manifest serves every trim of
the same cut**, and each client shifts it onto its own timebase.
### Degradation, never failure
A signature is an enhancement to cut matching. A missing one costs a tier and
**must never be able to break a fetch**. Every one of these stores the manifest
on the server's terms:
- signatures switched off in configuration (they are opt-in — the decode costs a
second or two of I/O per item);
- the item is under 120 s, so the window underflows and there is no signature to
compute (`JR-044`);
- the manifest carried no signature;
- the manifest's signature is `v2:` from a future producer — **refused, not
parsed** (`JR-045`), because scoring an unknown DSP chain as v1 would be a
confident wrong answer where declining is a correct one;
- no FFmpeg binary, no audio stream, or a decode error.
### "Un-comparable" and "does not match" are different
The distinction matters more than it looks. A 90-second extra is not content that
disagrees with its manifest — it is content that could not be compared. Reporting
the first as the second would show a user a scary warning about the wrong thing.
A genuine mismatch — both signatures present, both valid, both items long enough,
and the best alignment still below 0.60 — is the strongest available hint that a
manifest describes different content. It is **still not a failure**: the audio may
legitimately differ, a different language track being the obvious case. So the
manifest is stored on the server's terms and the disagreement is surfaced as a
caveat, which outranks the tier's own caveat because it is the stronger
statement.
---
## What gets stored
The offset is applied **once**, at store time, so the stored windows are always
in your file's own timebase and no read path needs offset awareness (`JR-030`).
That makes the offset unrecoverable afterwards — the windows look native — which
is why the alignment is recorded beside the truth file:
```json
"alignment": {
"source": "Local",
"tier": "Audio",
"offset_sec": 40.0,
"score": 0.97,
"offset_frames": 215,
"local_signature": "v1:AAAAA…",
"server_offset_sec": 0.0,
"server_tier": "Runtime"
}
```
`local_signature` is kept so a **later fetch aligns for free** — the decode is
the expensive half and the reason signatures are opt-in. It never leaves the
instance: provenance is stored beside the truth file, not inside it, and
contribution strips provenance entirely (`JR-034`).
The truth file itself is untouched by any of this. Injecting fields would mean
the bytes served back are not the bytes the producer wrote, which is the property
`JR-004` turns on.
---
## Where the code is
| Concern | File |
|---|---|
| The DSP | [`Services/AudioSignature.cs`](../Jellyfin.Plugin.JRay/Services/AudioSignature.cs) |
| The decode | [`Services/AudioSignatureService.cs`](../Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs) |
| Reading and matching | [`Services/AudioSignatureMatcher.cs`](../Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs) |
| The fetch-path decision | [`Services/ManifestAligner.cs`](../Jellyfin.Plugin.JRay/Services/ManifestAligner.cs) |
| What is recorded | [`Models/TruthAlignment.cs`](../Jellyfin.Plugin.JRay/Models/TruthAlignment.cs) |
| Server-side validation | `JRay-public-server/src/validate.rs`, `validate_audio_signature` |
| C++ producer | `scene-actor-extraction/src/audio_signature.{hpp,cpp}` |
Tests: `AudioSignatureTests` (UT-038 … UT-044), `AudioSignatureMatcherTests`
(UT-045 … UT-052), `ManifestAlignerTests` (UT-053 … UT-057).
## Verifying the cross-repo claim by hand
```sh
# C# side
cd jRay && dotnet test Jellyfin.Plugin.JRay.Tests/Jellyfin.Plugin.JRay.Tests.csproj
# C++ side, against the same fixture
cd scene-actor-extraction/build && python3 -c "
import json, sae_audio
g = json.load(open('../tests/fixtures/audio/jray_audio_v1_golden.json'))
print(sae_audio.compute_signature('../tests/fixtures/audio/jray_audio_v1_tone.flac') == g['signature'])
"
# and that the fixtures really are the same bytes
md5sum jRay/Jellyfin.Plugin.JRay.Tests/fixtures/audio/jray_audio_v1_golden.json \
scene-actor-extraction/tests/fixtures/audio/jray_audio_v1_golden.json
```
## Not built yet
- **Matching against a server's catalogue.** `POST /manifests/search` and the
`audio` tier server-side are `UR-009`, still in progress. The sequencing is
deliberate — accumulate signatures first, enable matching once coverage is
useful. Nothing here depends on it: alignment works from the signature the
manifest already carries.
- **Contribution.** Uploads do not yet attach a signature (`JR-034`).
- **Re-using the stored signature.** It is written but not yet read back on a
second fetch, so today every fetch decodes.
+93 -17
View File
@@ -48,10 +48,50 @@ Tag code with `// TRACES: JR-012 | SR-002`.
| UT-026 | Provenance is **not** written into the truth file | JR-010, JR-004 | **Passing** | | UT-026 | Provenance is **not** written into the truth file | JR-010, JR-004 | **Passing** |
| UT-027 | `Delete` removes provenance too — no record outliving its claim | JR-010 | **Passing** | | UT-027 | `Delete` removes provenance too — no record outliving its claim | JR-010 | **Passing** |
| UT-028 | Unknown item yields null rather than a fabricated record | JR-010 | **Passing** | | UT-028 | Unknown item yields null rather than a fabricated record | JR-010 | **Passing** |
| UT-029 | A v2 file round-trips; `extraction.*` and `cut.*` survive intact — `sample_fps` read from the block, not the top level | JR-002 | **Passing** |
| UT-030 | `scenes` objects retain **belief and route** — a window that loses them is indistinguishable from v1 | JR-002 | **Passing** |
| UT-031 | All three routes (`live`, `deferred`, `pooled`) survive a round trip | JR-002 | **Passing** |
| UT-032 | A window without belief reads `null`, **not `0.0`** — absent and disbelieved are different claims | JR-002 | **Passing** |
| UT-033 | `schema_version` 1, 3 and 0 are all **refused**, and the message names the version found | JR-003 | **Passing** |
| UT-034 | A **missing** `schema_version` is refused, never assumed current | JR-003 | **Passing** |
| UT-035 | A `null` truth file is refused without throwing | JR-003 | **Passing** |
| UT-036 | **A v1 file never half-parses into usable windows** — it either fails to deserialise or is stopped by the gate | JR-003 | **Passing** |
| UT-037 | **The producer's actual output parses** — the exact shape `result_sink_node.hpp` writes, omitting `cut` and two `extraction` fields, not the spec's fully populated example | JR-002 | **Passing** |
| UT-038 | The **regenerated** fixture PCM matches the recorded `s16le` and `f32le` checksums — the input is proven identical before any DSP claim is made | JR-043 | **Passing** |
| UT-039 | **Signature equals the shared golden vector, byte for byte** — the same string the C++ producer emits | JR-042, JR-043 | **Passing** |
| UT-040 | Band → FFT-bin table matches the recorded one, and tiles 3003000 Hz contiguously with no empty band | JR-042 | **Passing** |
| UT-041 | Well-formed: `v1:` prefix, 1288 frames, bit 7 always clear, **and the fixture still exercises all 32 bands and all 4 energy classes** | JR-042 | **Passing** |
| UT-042 | Whole frames only — 4095 samples yield nothing, 5120 yield two; a partial frame is never padded into a signature | JR-042 | **Passing** |
| UT-043 | The **real FFmpeg decode** of the fixture reproduces the golden signature | JR-042 | **Passing** (needs a binary) |
| UT-044 | **The window is taken from the centre**: the fixture wrapped in 90 s of silence either side signs identically | JR-042 | **Passing** (needs a binary) |
| UT-045 | **The 120 s boundary, on one file**: runtime 119.999 emits nothing, runtime 120.000 emits the golden signature — only the runtime differs, so a null cannot be blamed on the decode | JR-044 | **Passing** (boundary needs a binary) |
| UT-046 | Two identical, perfectly valid signatures still yield **no match and no offset** when either runtime is under the window — the rule is read off the runtime, not inferred from a missing string | JR-044 | **Passing** |
| UT-047 | A `v2:` signature whose payload is byte-identical to a valid v1 one is **refused, not parsed** — by the matcher as well as the parser | JR-045 | **Passing** |
| UT-048 | A v1 signature parses to exactly the produced frames, checked against the **golden vector** rather than against the producer's own output | JR-045 | **Passing** |
| UT-049 | Missing prefix, empty payload, invalid base64 and a **set reserved bit** are each refused without throwing — the client never accepts what the server would reject | JR-045 | **Passing** |
| UT-050 | Identical signatures score 1.0 at offset 0 and reach the `audio` tier | JR-044 | **Passing** |
| UT-051 | **A shifted release recovers its offset** rather than failing to match — the case the feature exists for | JR-044 | **Passing** |
| UT-052 | Unrelated content yields **no match at all**, and runtime skew contributes its window-anchor term to the offset | JR-044 | **Passing** |
| UT-053 | **A local alignment supersedes the server's offset**, and the server's claim is retained rather than overwritten | JR-047 | **Passing** |
| UT-054 | The local signature is recorded, so a later fetch aligns without decoding the media again | JR-047 | **Passing** |
| UT-055 | Every unavailable local path — off, no manifest signature, neither, short media, **and a `v2:` producer** — falls back to the server rather than refusing | JR-047 | **Passing** |
| UT-056 | Two signatures that genuinely disagree are recorded as a mismatch and **still do not break the fetch** | JR-047 | **Passing** |
| UT-057 | A mismatch **outranks the tier** in the caveat shown to the user — a `runtime` match would otherwise show nothing at all | JR-047 | **Passing** |
All execute and pass. The suite is also checked to **fail** on deliberate All execute and pass. UT-043 and UT-044 are the two that need an FFmpeg binary,
mutations, because a suite that has only ever passed is not evidence that it which the plugin gets from Jellyfin at run time and a bare CI container may not
tests anything. Three so far, each restored and re-verified afterwards: have; they self-skip without one. That is why the cross-repo claim rests on
UT-038 and UT-039, which regenerate the fixture PCM from `make_fixture.py`'s
arithmetic and need no codec at all — a check that skips is not a check.
UT-045 follows the same rule: its *below-the-boundary* half is codec-free and
always binds, because the runtime check short-circuits before the encoder is
consulted — asserted with a deliberately invalid encoder path, so passing proves
the short-circuit rather than merely a failed decode.
The suite is also checked to **fail** on deliberate mutations, because a suite
that has only ever passed is not evidence that it tests anything. Each was
restored and re-verified afterwards:
| Mutation | Fails | Blast radius | | Mutation | Fails | Blast radius |
|---|---|---| |---|---|---|
@@ -59,10 +99,23 @@ tests anything. Three so far, each restored and re-verified afterwards:
| Downgrade the missing-dependency warning to `Information` | UT-013 | 1 test | | Downgrade the missing-dependency warning to `Information` | UT-013 | 1 test |
| Make the window end bound exclusive (`t < end`) | UT-016, UT-018 | 2 tests | | Make the window end bound exclusive (`t < end`) | UT-016, UT-018 | 2 tests |
| Stop `Delete` removing provenance | UT-027 | 1 test | | Stop `Delete` removing provenance | UT-027 | 1 test |
| Make `TruthSchema.IsSupported` accept any version | UT-033, UT-034, UT-035, UT-036 | 4 tests |
| Emit `v2:` as the signature prefix | UT-039, UT-043, UT-044 | 3 tests |
| Aggregate a band by **sum** instead of mean | UT-039, UT-043, UT-044 | 3 tests |
| Anchor the decode window at the head instead of the centre | UT-044 | 1 test |
The third is the one worth keeping: a single character turns an inclusive window Two further mutations were tried and **did not fail**, which is worth recording
into a half-open one, which would drop an actor at exactly the moment a scene rather than hiding: the symmetric `N-1` Hann window in place of the periodic one,
ends — and nothing else in the suite would have noticed. and the lower median in place of the upper as the energy reference. Both are
pinned by prose in the shared fixture, and on this synthetic vector neither moves
a peak bin or crosses an energy-class edge. They are conventions the golden
vector does not police, so a second implementation could get either wrong and
still pass — the fixture would need frames sitting nearer those boundaries to
catch it.
The window-bound mutation is the one worth keeping: a single character turns an
inclusive window into a half-open one, which would drop an actor at exactly the
moment a scene ends — and nothing else in the suite would have noticed.
`JR` is flat rather than split by theme. The plugin is one deployable with one `JR` is flat rather than split by theme. The plugin is one deployable with one
audience, and the thematic grouping lives in the section headings below, where it audience, and the thematic grouping lives in the section headings below, where it
@@ -81,8 +134,8 @@ coordinated `schema_version` bumps (SR-003).
| ID | Requirement | Traces to | Priority | Status | | ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---| |---|---|---|---|---|
| JR-001 | The truth-file format is normatively defined here; other repos reference it rather than restating it | SR-003 | High | In Progress | | JR-001 | The truth-file format is normatively defined here; other repos reference it rather than restating it | SR-003 | High | In Progress |
| JR-002 | `schema_version: 2` shape — `extraction.*` provenance block, `cut.*` block, `scenes` as objects carrying belief and route | SR-003 | High | Planned | | JR-002 | `schema_version: 2` shape — `extraction.*` provenance block, `cut.*` block, `scenes` as objects carrying belief and route | SR-003 | High | **Done** (UT-029…032) — `TruthScene`, `TruthExtraction`, `TruthCut`; `anneal_sec` and top-level `sample_fps` deleted, not zeroed. `ManifestConverter` carries belief, route and both blocks through |
| JR-003 | Reject an unknown `schema_version`, never guess. **Flag day: v2 only**, no dual-accept | SR-003 | High | Planned | | JR-003 | Reject an unknown `schema_version`, never guess. **Flag day: v2 only**, no dual-accept | SR-003 | High | **Done** (UT-033…036) — `TruthSchema.IsSupported` is the single gate, applied on all four paths: sidecar read, managed store load, managed `PUT`, converted manifest. Rejections name the file and the version found |
| JR-004 | A window is a **scene-membership claim**, not a recognition event — never reinterpreted, merged, split or trimmed | **SR-002** | High | **Done** (UT-021, UT-022) | | JR-004 | A window is a **scene-membership claim**, not a recognition event — never reinterpreted, merged, split or trimmed | **SR-002** | High | **Done** (UT-021, UT-022) |
| JR-005 | Query semantics: actor present at `t` if any window contains `t`; presentation must not assert instantaneous visibility | **SR-002** | High | **Done** (UT-016…019) | | JR-005 | Query semantics: actor present at `t` if any window contains `t`; presentation must not assert instantaneous visibility | **SR-002** | High | **Done** (UT-016…019) |
| JR-006 | Read path holds up under **numerous** windows — no assumption of a handful of long ones | SR-002 | Medium | **Done** (UT-020, UT-023) | | JR-006 | Read path holds up under **numerous** windows — no assumption of a handful of long ones | SR-002 | Medium | **Done** (UT-020, UT-023) |
@@ -168,18 +221,40 @@ preserved.
| JR-040 | The config page states plainly that **each configured server multiplies the exposure** | **PR-005** | Medium | Planned | | JR-040 | The config page states plainly that **each configured server multiplies the exposure** | **PR-005** | Medium | Planned |
| JR-041 | The plugin never fetches, stores, or transmits gallery data — reference faces or embeddings. It has no gallery code path at all | **SR-005** | High | Done | | JR-041 | The plugin never fetches, stores, or transmits gallery data — reference faces or embeddings. It has no gallery code path at all | **SR-005** | High | Done |
## Audio signature (JR-042 … JR-045) ## Audio signature (JR-042 … JR-045, JR-047)
Mirror-image of extraction `IR-004`/`IR-005`/`IR-007`/`IR-008`. Both producers Mirror-image of extraction `IR-004`/`IR-005`/`IR-007`/`IR-008`. Both producers
must agree **bit-for-bit**, so each obligation is stated on both sides rather must agree **bit-for-bit**, so each obligation is stated on both sides rather
than assumed to be inherited. than assumed to be inherited.
JR-042 … JR-045 are the signature itself, produced and read. **JR-047 is what
uses it**: without a consumer on the fetch path the other four are a fingerprint
nothing ever fingerprints. See [`audio-alignment.md`](audio-alignment.md) for the
end-to-end mechanism.
| ID | Requirement | Traces to | Priority | Status | | ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---| |---|---|---|---|---|
| JR-042 | Compute the signature **exactly** per server spec §3, using the FFmpeg binary Jellyfin already ships via `IMediaEncoder.EncoderPath` — no new dependency | SR-003 | Medium | Planned | | JR-042 | Compute the signature **exactly** per server spec §3, using the FFmpeg binary Jellyfin already ships via `IMediaEncoder.EncoderPath` — no new dependency | SR-003 | Medium | **Done** (UT-039…044) — `AudioSignature` is the DSP, `AudioSignatureService` the decode; FFmpeg is invoked as a child process for decode, downmix and resample, and nothing was added to the project's dependencies |
| JR-043 | Golden-vector fixture **shared with the extraction repo**, proving the two implementations are bit-exact | SR-003 | High | Planned | | JR-043 | Golden-vector fixture **shared with the extraction repo**, proving the two implementations are bit-exact | SR-003 | High | **Done** (UT-038, UT-039) — `fixtures/audio/` holds the extraction repo's three files byte-identically; the C# signature equals the recorded vector exactly |
| JR-044 | Media shorter than 120 s: emit no signature and apply no sync offset — identical rule in both producers | SR-003 | Low | Planned | | JR-044 | Media shorter than 120 s: emit no signature and apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** (UT-045, UT-046, UT-050…052) — the producer half was already in `AudioSignatureService`; the consumer half needed a reader, so `AudioSignatureMatcher` implements the §3 slide and declines an offset outright below the window. The boundary is asserted on one file at 119.999 s and 120.000 s |
| JR-045 | Emit and honour the signature's own `v1:` prefix, so a DSP change is detectable rather than silently non-matching | SR-003 | Low | Planned | | JR-045 | Emit and honour the signature's own `v1:` prefix, so a DSP change is detectable rather than silently non-matching | SR-003 | Low | **Done** (UT-047…049) — `AudioSignatureMatcher.TryParseFrames` refuses any prefix but `v1:`, and refuses malformed or structurally invalid payloads, so a future producer's `v2:` drops the item to the runtime tier instead of scoring as if it were understood |
| JR-047 | **A fetched manifest is aligned against the local file before its windows are stored**, and the alignment is recorded beside the truth data | SR-003 | High | **Done** (UT-053…057) — `ManifestAligner` runs on the fetch path. A local alignment supersedes the server's offset, since the server has never seen this file; every unavailable path degrades to the server's offset rather than refusing, and a genuine signature disagreement is recorded and surfaced as a caveat without failing the fetch |
> **Outstanding: §3's score gained ±1 frame of tolerance and this matcher has
> not.** `AudioSignatureMatcher` implements the exact-frame rule §3 carried
> until `JRay-public-server` UR-009 landed. The change was
> `scene-actor-extraction` VR-014's measurement: over 40 correctly recovered
> offsets on real film audio the exact rule scored 27 of them below 0.85 and
> demoted them to `loose`, because the two windows are cut on their own file's
> frame grid and those grids do not coincide. With ±1 frame all 40 reach
> `audio` and the strongest false match is unmoved at 0.16.
>
> **Nothing is misaligned by the divergence** — the offset the matcher recovers
> is unaffected, and JR-047 makes the local answer supersede the server's — but
> the plugin will label as `loose` alignments the server calls `audio`, which is
> a caveat shown to a user for a match that is not in doubt. JR-044's matcher
> and [`audio-alignment.md`](audio-alignment.md)'s scoring table both need the
> revised rule. Until then this repo implements a superseded version of §3.
## Human-in-the-loop association (JR-046) ## Human-in-the-loop association (JR-046)
@@ -263,8 +338,8 @@ framework reference and leans on `RollForward` to reach the 10.0 runtime.
| ID | Tier | Test asserts | Edge cases to cover | | ID | Tier | Test asserts | Edge cases to cover |
|---|---|---|---| |---|---|---|---|
| JR-001 | static | Other repos' specs link here rather than restating the schema | A second copy of the schema anywhere is the failure | | JR-001 | static | Other repos' specs link here rather than restating the schema | A second copy of the schema anywhere is the failure |
| JR-002 | T1 | A v2 file round-trips; `scenes` objects retain belief and route | Window with belief exactly at the ownership threshold; all three route values | | JR-002 | T1 | A v2 file round-trips; `scenes` objects retain belief and route | All three route values; belief absent reads `null` not `0.0`; **the producer's real output**, which omits `cut` and two `extraction` fields — a test written only against the spec's populated example would have passed throughout the break |
| JR-003 | **T1** | `schema_version` 1 and 3 are both **rejected**, not coerced | Missing field entirely; non-integer value | | JR-003 | **T1** | `schema_version` 1 and 3 are both **rejected**, not coerced | Missing field entirely; `null` document; **a v1 file must not half-parse into usable windows** |
| JR-004 | T1 | Windows are stored and served byte-identical to input | Adjacent windows that "look" mergeable must **not** merge | | JR-004 | T1 | Windows are stored and served byte-identical to input | Adjacent windows that "look" mergeable must **not** merge |
| JR-005 | T1 | `t` exactly on `start` and on `end` are both present | Zero-length window; overlapping windows for one actor | | JR-005 | T1 | `t` exactly on `start` and on `end` are both present | Zero-length window; overlapping windows for one actor |
| JR-006 | T1 | Response bounded by actor count, not window count; lookup not quadratic | 50 × 1000 windows; **unsorted input still resolves** — sortedness is a producer guarantee, never a correctness dependency | | JR-006 | T1 | Response bounded by actor count, not window count; lookup not quadratic | 50 × 1000 windows; **unsorted input still resolves** — sortedness is a producer guarantee, never a correctness dependency |
@@ -303,10 +378,11 @@ framework reference and leans on `RollForward` to reach the 10.0 runtime.
| JR-039 | T1 | Batch never exceeds 100 items | Library of 10⁴ items produces a paced sweep | | JR-039 | T1 | Batch never exceeds 100 items | Library of 10⁴ items produces a paced sweep |
| JR-040 | **T4** | Config page states the per-server exposure | Manual review of copy | | JR-040 | **T4** | Config page states the per-server exposure | Manual review of copy |
| JR-041 | **static** | No embedding or image field is parsed or stored | Grep-based, mirroring the server's UR-012 | | JR-041 | **static** | No embedding or image field is parsed or stored | Grep-based, mirroring the server's UR-012 |
| JR-042 | T1 | DSP chain matches the specified parameters exactly | Window, hop, band, bin count each asserted individually | | JR-042 | T1 | DSP chain matches the specified parameters exactly | Window, hop, band, bin count each asserted individually; the decode itself is covered only where an FFmpeg binary exists, so it must not be the only cover for any claim |
| JR-043 | **T1** | Signature matches the shared golden vector **bit-for-bit** | Media < 120 s → no signature; identical result in both repos | | JR-043 | **T1** | Signature matches the shared golden vector **bit-for-bit** | The fixture PCM is regenerated from `make_fixture.py` and checked against the recorded decode checksums first, so the check binds on a host with no codec and a decode divergence is distinguishable from a DSP one |
| JR-044 | T1 | Media < 120 s yields no signature and no offset | Exactly 120 s — the boundary both repos must agree on | | JR-044 | T1 | Media < 120 s yields no signature and no offset | Exactly 120 s — the boundary both repos must agree on |
| JR-045 | T1 | `v1:` emitted; an unknown prefix is refused, not parsed | `v2:` signature from a future producer | | JR-045 | T1 | `v1:` emitted; an unknown prefix is refused, not parsed | `v2:` signature from a future producer |
| JR-047 | **T1** | A fetched manifest is aligned locally before storage, and the alignment is recorded | **Every way the local path can be unavailable must degrade to the server's offset, never refuse** — signatures off, no manifest signature, media under the window, a `v2:` producer. Distinguish those from a genuine mismatch: a 90 s extra is not content that disagrees with its manifest, and telling a user it is would be worse than saying nothing |
| JR-046 | T2 + **T4** | *Assertions deferred* — recording an association and persisting it is T2; the review UI itself is T4 | Cannot be written until the truth-file interface for unidentified presence is settled (system open question 2) and AR-021/AR-022 land | | JR-046 | T2 + **T4** | *Assertions deferred* — recording an association and persisting it is T2; the review UI itself is T4 | Cannot be written until the truth-file interface for unidentified presence is settled (system open question 2) and AR-021/AR-022 land |
Three are worth singling out. **JR-021** and **JR-041** are static checks because Three are worth singling out. **JR-021** and **JR-041** are static checks because
+826
View File
@@ -0,0 +1,826 @@
# Requirements traceability matrix
<!-- GENERATED FILE - do not edit by hand. -->
<!-- Regenerate: scripts/traceability/traceability-gate.sh -->
**Generated:** 2026-07-31T15:05:02+00:00
Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, static`).
## Summary
| Metric | Value |
|---|---|
| Source files scanned | 78 |
| TRACES tags found | 48 |
| EXCEPTION tags found | 0 |
| Requirements defined | 47 |
| Requirements covered | 38 |
| **Coverage** | **80.9%** (38/47) |
| Coverage of CI-executable scope | 84.4% (38/45) |
| Tagged but unexecuted in CI | 1 |
| Orphan tags | 0 |
### By type
| Type | Covered | Tagged but unexecuted | Defined |
|---|---|---|---|
| JR | 38 | 1 | 47 |
- **UT** tags present (separate taxonomy, not counted in coverage): UT-001, UT-002, UT-003, UT-004, UT-005, UT-007, UT-008, UT-009, UT-010, UT-011, UT-012, UT-013, UT-014, UT-015, UT-016, UT-017, UT-018, UT-019, UT-020, UT-021, UT-022, UT-023, UT-024, UT-025, UT-026, UT-027, UT-028, UT-029, UT-030, UT-031, UT-032, UT-033, UT-034, UT-035, UT-036, UT-037, UT-038, UT-039, UT-040, UT-041, UT-042, UT-043, UT-044, UT-045, UT-046, UT-047, UT-048, UT-049, UT-050, UT-051, UT-052, UT-053, UT-054, UT-055, UT-056, UT-057
- **PR** tags present (separate taxonomy, not counted in coverage): PR-001, PR-003, PR-004, PR-005, PR-006
- **SR** tags present (separate taxonomy, not counted in coverage): SR-001, SR-002, SR-003, SR-004, SR-005
## Not executable in CI
These requirements have no verification tier this repo's CI host can run, so a tag on them is evidence of *intent*, not of verification. They are never counted as covered.
| ID | Tiers | Tagged in source | Requirement |
|---|---|---|---|
| JR-020 | T4 | yes | Pause overlay: injected client script queries `jray?t=` and renders t… |
| JR-040 | T4 | no | The config page states plainly that **each configured server multipli… |
**Tagged but unexecuted:** JR-020 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately.
## Orphan tags
A tag naming an ID `requirements.md` does not define. This is what renumbering produces, and what a typo produces.
_None._
## Requirements tracing up to nothing
A register row whose `Traces to` cell names no parent. Work serving no stated goal is how scope creeps in, and it is invisible unless something looks.
_None._
## Recorded exceptions
Deliberate, documented departures from an invariant (`EXCEPTION: XX-nnn <reason>`). Reported separately and never counted as coverage — an exception is a decision to be reviewed, not evidence a requirement is met.
_None._
## Register
| ID | Status | Tier | Traces to | Trace state | Tagged in | Requirement |
|---|---|---|---|---|---|---|
| JR-001 | In Progress | static | SR-003 | covered | `Jellyfin.Plugin.JRay/Models/TruthFile.cs` | The truth-file format is normatively defined here; other repos refere… |
| JR-002 | **Done** (UT-029…03… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs`, `Jellyfin.Plugin.JRay/Models/TruthCut.cs`, `Jellyfin.Plugin.JRay/Models/TruthExtraction.cs`, `Jellyfin.Plugin.JRay/Models/TruthFile.cs`, `Jellyfin.Plugin.JRay/Models/TruthScene.cs` | `schema_version: 2` shape — `extraction.*` provenance block, `cut.*` … |
| JR-003 | **Done** (UT-033…03… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs`, `Jellyfin.Plugin.JRay/Controllers/TruthController.cs`, `Jellyfin.Plugin.JRay/Services/TruthSchema.cs` | Reject an unknown `schema_version`, never guess. **Flag day: v2 only*… |
| JR-004 | **Done** (UT-021, U… | T1 | **SR-002** | covered | `Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs`, `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs`, `Jellyfin.Plugin.JRay/Models/TruthActor.cs`, `Jellyfin.Plugin.JRay/Models/TruthScene.cs`, `Jellyfin.Plugin.JRay/Services/PresenceLookup.cs` | A window is a **scene-membership claim**, not a recognition event — n… |
| JR-005 | **Done** (UT-016…01… | T1 | **SR-002** | covered | `Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs`, `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs`, `Jellyfin.Plugin.JRay/Models/ActorInScene.cs`, `Jellyfin.Plugin.JRay/Services/PresenceLookup.cs`, `Jellyfin.Plugin.JRay/Web/jray-overlay.js` | Query semantics: actor present at `t` if any window contains `t`; pre… |
| JR-006 | **Done** (UT-020, U… | T1 | SR-002 | covered | `Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs`, `Jellyfin.Plugin.JRay/Services/PresenceLookup.cs` | Read path holds up under **numerous** windows — no assumption of a ha… |
| JR-007 | Done | T1 | SR-001 | covered | `Jellyfin.Plugin.JRay/Models/TruthActor.cs` | Identity is public identifiers: prefer `jellyfin_id` locally, else re… |
| JR-008 | Done | T1 | PR-001 | covered | `Jellyfin.Plugin.JRay/Services/TruthDataService.cs` | Discover a sidecar truth file beside the media, by configurable suffix |
| JR-009 | Done | T2 | PR-004 | covered | `Jellyfin.Plugin.JRay/Controllers/TruthController.cs`, `Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs` | Accept truth data pushed by a remote worker (`PUT`/`DELETE`), admin k… |
| JR-010 | **Done** (UT-024…02… | T1 | PR-001 | covered | `Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs`, `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs`, `Jellyfin.Plugin.JRay/Models/TruthProvenance.cs`, `Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs`, `Jellyfin.Plugin.JRay/Services/TruthDataService.cs` | Precedence: managed truth (pushed **or** fetched) overrides a sidecar… |
| JR-011 | Done | T1 | PR-001 | covered | `Jellyfin.Plugin.JRay/Services/TruthDataService.cs` | Loaded truth is cached; any write invalidates the item's cache entry … |
| JR-012 | Done | T2 | PR-001 | covered | `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs` | `GET .../Timeline` returns the full truth file for an item |
| JR-013 | Done | T2 | PR-001 | covered | `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs` | `GET .../jray?t=` returns an **extensible** context envelope; consume… |
| JR-014 | Done | T2 | PR-004 | covered | `Jellyfin.Plugin.JRay/Controllers/ActorsController.cs`, `Jellyfin.Plugin.JRay/Controllers/PolicyController.cs`, `Jellyfin.Plugin.JRay/Controllers/TruthController.cs`, `Jellyfin.Plugin.JRay/Controllers/WebController.cs` | Authorisation: reads need an authenticated user, admin routes need th… |
| JR-015 | Done | T2 | PR-003 | covered | `Jellyfin.Plugin.JRay/Controllers/TasksController.cs` | `Tasks/Pending` serves a random sample of items with no truth data, s… |
| JR-016 | **Done** (UT-007…01… | T1 | PR-003 | covered | `Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs`, `Jellyfin.Plugin.JRay/Controllers/PolicyController.cs`, `Jellyfin.Plugin.JRay/Services/MediaPolicyStore.cs`, `Jellyfin.Plugin.JRay/Services/PolicyResolver.cs` | Prioritise/ignore rules scoped `Genre` / `Series` / `Item`; **most sp… |
| JR-017 | Done | T1 | PR-003 | covered | `Jellyfin.Plugin.JRay/Controllers/TasksController.cs` | Rules steer **work discovery only** — never the overlay or the read e… |
| JR-018 | Done | T1 | PR-003 | covered | `Jellyfin.Plugin.JRay/Controllers/CoverageController.cs` | Coverage report by media type and genre; ignored items leave the perc… |
| JR-019 | Done | T2 | PR-003 | covered | `Jellyfin.Plugin.JRay/Controllers/CoverageController.cs` | Picker endpoints (genres, series, item search) populate the rule edit… |
| JR-020 | Done | T4 | **PR-001** | tagged, unexecuted | `Jellyfin.Plugin.JRay/Controllers/WebController.cs`, `Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs`, `Jellyfin.Plugin.JRay/Web/jray-overlay.js` | Pause overlay: injected client script queries `jray?t=` and renders t… |
| JR-021 | **Done** | static | PR-004 | covered | `Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs`, `scripts/checks/no-index-injection.sh` | **jRay never injects into `index.html` on disk.** File Transformation… |
| JR-022 | **Done** (UT-001…00… | T1 | PR-004 | covered | `Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs`, `Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs` | Migration: remove any on-disk patch left by an earlier jRay, identifi… |
| JR-023 | **Done** (UT-012…01… | T1, T4 | PR-004 | covered | `Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs`, `Jellyfin.Plugin.JRay/Controllers/StatusController.cs`, `Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs` | Absent the dependency, disable **only** the overlay and say so in the… |
| JR-024 | Done | T1 | SR-004 | covered | `Jellyfin.Plugin.JRay/Web/jray-overlay.js` | Actor names and all server-supplied strings render as **text, never m… |
| JR-025 | Done | T1 | PR-006 | covered | `Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs`, `Jellyfin.Plugin.JRay/Controllers/ManifestController.cs`, `Jellyfin.Plugin.JRay/Models/Jmanifest.cs`, `Jellyfin.Plugin.JRay/Services/Interfaces/IManifestExchangeClient.cs`, `Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs` | Query an **ordered list** of servers; first result clearing the confi… |
| JR-026 | Planned | T1 | PR-006 | untagged | - | For a series, first-match applies per **episode** — later servers are… |
| JR-027 | Done | T1 | SR-004 | covered | `Jellyfin.Plugin.JRay/Models/Jmanifest.cs`, `Jellyfin.Plugin.JRay/Services/ManifestValidator.cs` | Treat **every** server as untrusted, including the default: re-valida… |
| JR-028 | Done | T1 | SR-004 | covered | `Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs` | Enforce response size caps **while streaming** — 2 MiB single, 25 MiB… |
| JR-029 | Done | T1 | SR-004 | covered | `Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs` | HTTPS required for non-loopback servers; certificate validation must … |
| JR-030 | Done | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay/Services/ManifestConverter.cs`, `Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs` | Apply an `audio`-tier `offset` to **every** window before storing — s… |
| JR-031 | Planned | T2 | PR-006 | covered | `Jellyfin.Plugin.JRay/Controllers/ManifestController.cs` | Fetch endpoints: item fetch, series bundle fetch, per-server status, … |
| JR-032 | Planned | T1 | PR-006 | untagged | - | Identify is **never automatic** — storing a candidate is a separate c… |
| JR-033 | Planned | T1 | PR-006 | untagged | - | Scheduled sweep over items lacking truth data, using the **batch** `e… |
| JR-034 | Planned | T1 | PR-005 | untagged | - | Contribution strips `movie` and `jellyfin_id`, attaches identity from… |
| JR-035 | Planned | T1 | PR-006 | untagged | - | Uploads set `Expect: 100-continue`, so a rejection lands before a bun… |
| JR-036 | In Progress | T1 | PR-006 | covered | `Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs`, `Jellyfin.Plugin.JRay/Models/TruthProvenance.cs` | Minimum accepted match tier is configurable; a `loose` match surfaces… |
| JR-037 | Done | T1 | PR-006 | covered | `Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs` | A server that is unreachable or failing is skipped on a short timeout… |
| JR-038 | Done | T1 | **PR-005** | covered | `Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs` | Every exchange feature is **opt-in and off by default**, including th… |
| JR-039 | Planned | T1 | **PR-005** | untagged | - | No library-wide inventory in one request: batch `exists` capped at 10… |
| JR-040 | Planned | T4 | **PR-005** | untagged | - | The config page states plainly that **each configured server multipli… |
| JR-041 | Done | static | **SR-005** | covered | `scripts/checks/no-gallery-data.sh` | The plugin never fetches, stores, or transmits gallery data — referen… |
| JR-042 | **Done** (UT-039…04… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs`, `Jellyfin.Plugin.JRay/Services/AudioSignature.cs`, `Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs` | Compute the signature **exactly** per server spec §3, using the FFmpe… |
| JR-043 | **Done** (UT-038, U… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs` | Golden-vector fixture **shared with the extraction repo**, proving th… |
| JR-044 | **Done** (UT-045, U… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs`, `Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs` | Media shorter than 120 s: emit no signature and apply no sync offset … |
| JR-045 | **Done** (UT-047…04… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs`, `Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs` | Emit and honour the signature's own `v1:` prefix, so a DSP change is … |
| JR-046 | **TBD** | T2, T4 | [system §4](../scripts/vendor/jray-proj… | untagged | - | Review UI for unidentified track clusters: show context crops, pick f… |
| JR-047 | **Done** (UT-053…05… | T1 | SR-003 | covered | `Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs`, `Jellyfin.Plugin.JRay/Models/TruthAlignment.cs`, `Jellyfin.Plugin.JRay/Services/ManifestAligner.cs` | **A fetched manifest is aligned against the local file before its win… |
## Detailed mapping
### JR-001
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Models/TruthFile.cs:26`](../Jellyfin.Plugin.JRay/Models/TruthFile.cs#L26) — `public class TruthFile`
### JR-002
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Models/TruthCut.cs:15`](../Jellyfin.Plugin.JRay/Models/TruthCut.cs#L15) — `public class TruthCut`
- [`Jellyfin.Plugin.JRay/Models/TruthExtraction.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthExtraction.cs#L21) — `public class TruthExtraction`
- [`Jellyfin.Plugin.JRay/Models/TruthFile.cs:26`](../Jellyfin.Plugin.JRay/Models/TruthFile.cs#L26) — `public class TruthFile`
- [`Jellyfin.Plugin.JRay/Models/TruthScene.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthScene.cs#L21) — `public class TruthScene`
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### JR-003
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Controllers/TruthController.cs:28`](../Jellyfin.Plugin.JRay/Controllers/TruthController.cs#L28) — `public class TruthController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/TruthSchema.cs:35`](../Jellyfin.Plugin.JRay/Services/TruthSchema.cs#L35) — `public static class TruthSchema`
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### JR-004
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/TruthActor.cs:20`](../Jellyfin.Plugin.JRay/Models/TruthActor.cs#L20) — `public class TruthActor`
- [`Jellyfin.Plugin.JRay/Models/TruthScene.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthScene.cs#L21) — `public class TruthScene`
- [`Jellyfin.Plugin.JRay/Services/PresenceLookup.cs:27`](../Jellyfin.Plugin.JRay/Services/PresenceLookup.cs#L27) — `public static class PresenceLookup`
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### JR-005
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/ActorInScene.cs:15`](../Jellyfin.Plugin.JRay/Models/ActorInScene.cs#L15) — `public class ActorInScene`
- [`Jellyfin.Plugin.JRay/Services/PresenceLookup.cs:27`](../Jellyfin.Plugin.JRay/Services/PresenceLookup.cs#L27) — `public static class PresenceLookup`
- [`Jellyfin.Plugin.JRay/Web/jray-overlay.js:14`](../Jellyfin.Plugin.JRay/Web/jray-overlay.js#L14) — `function truncate(text, maxLength)`
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### JR-006
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/PresenceLookup.cs:27`](../Jellyfin.Plugin.JRay/Services/PresenceLookup.cs#L27) — `public static class PresenceLookup`
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### JR-007
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Models/TruthActor.cs:20`](../Jellyfin.Plugin.JRay/Models/TruthActor.cs#L20) — `public class TruthActor`
### JR-008
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Services/TruthDataService.cs:25`](../Jellyfin.Plugin.JRay/Services/TruthDataService.cs#L25) — `public sealed class TruthDataService : ITruthDataService`
### JR-009
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Controllers/TruthController.cs:28`](../Jellyfin.Plugin.JRay/Controllers/TruthController.cs#L28) — `public class TruthController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs:21`](../Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs#L21) — `public sealed class ManagedTruthStore : IManagedTruthStore`
### JR-010
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/TruthProvenance.cs:38`](../Jellyfin.Plugin.JRay/Models/TruthProvenance.cs#L38) — `public class TruthProvenance`
- [`Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs:21`](../Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs#L21) — `public sealed class ManagedTruthStore : IManagedTruthStore`
- [`Jellyfin.Plugin.JRay/Services/TruthDataService.cs:25`](../Jellyfin.Plugin.JRay/Services/TruthDataService.cs#L25) — `public sealed class TruthDataService : ITruthDataService`
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### JR-011
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Services/TruthDataService.cs:25`](../Jellyfin.Plugin.JRay/Services/TruthDataService.cs#L25) — `public sealed class TruthDataService : ITruthDataService`
### JR-012
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
### JR-013
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
### JR-014
**Locations:** 4
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Controllers/PolicyController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/PolicyController.cs#L24) — `public class PolicyController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Controllers/TruthController.cs:28`](../Jellyfin.Plugin.JRay/Controllers/TruthController.cs#L28) — `public class TruthController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Controllers/WebController.cs:21`](../Jellyfin.Plugin.JRay/Controllers/WebController.cs#L21) — `public class WebController : ControllerBase`
### JR-015
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/TasksController.cs:34`](../Jellyfin.Plugin.JRay/Controllers/TasksController.cs#L34) — `public class TasksController : ControllerBase`
### JR-016
**Locations:** 4
- [`Jellyfin.Plugin.JRay/Controllers/PolicyController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/PolicyController.cs#L24) — `public class PolicyController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/MediaPolicyStore.cs:20`](../Jellyfin.Plugin.JRay/Services/MediaPolicyStore.cs#L20) — `public sealed class MediaPolicyStore : IMediaPolicyStore`
- [`Jellyfin.Plugin.JRay/Services/PolicyResolver.cs:14`](../Jellyfin.Plugin.JRay/Services/PolicyResolver.cs#L14) — `public static class PolicyResolver`
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### JR-017
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/TasksController.cs:34`](../Jellyfin.Plugin.JRay/Controllers/TasksController.cs#L34) — `public class TasksController : ControllerBase`
### JR-018
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/CoverageController.cs:30`](../Jellyfin.Plugin.JRay/Controllers/CoverageController.cs#L30) — `public class CoverageController : ControllerBase`
### JR-019
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/CoverageController.cs:30`](../Jellyfin.Plugin.JRay/Controllers/CoverageController.cs#L30) — `public class CoverageController : ControllerBase`
### JR-020
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Controllers/WebController.cs:21`](../Jellyfin.Plugin.JRay/Controllers/WebController.cs#L21) — `public class WebController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs:31`](../Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs#L31) — `public static class FileTransformationRegistration`
- [`Jellyfin.Plugin.JRay/Web/jray-overlay.js:14`](../Jellyfin.Plugin.JRay/Web/jray-overlay.js#L14) — `function truncate(text, maxLength)`
### JR-021
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs:26`](../Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs#L26) — `public static class WebClientPatchService`
- [`scripts/checks/no-index-injection.sh:11`](../scripts/checks/no-index-injection.sh#L11) — `Unknown`
### JR-022
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs:26`](../Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs#L26) — `public static class WebClientPatchService`
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### JR-023
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Controllers/StatusController.cs:20`](../Jellyfin.Plugin.JRay/Controllers/StatusController.cs#L20) — `public class StatusController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs:31`](../Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs#L31) — `public static class FileTransformationRegistration`
- [`Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs:20`](../Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs#L20) — `public class FileTransformationRegistrationTests`
### JR-024
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Web/jray-overlay.js:14`](../Jellyfin.Plugin.JRay/Web/jray-overlay.js#L14) — `function truncate(text, maxLength)`
### JR-025
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs:75`](../Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs#L75) — `public class ManifestServer`
- [`Jellyfin.Plugin.JRay/Controllers/ManifestController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/ManifestController.cs#L24) — `public class ManifestController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/Jmanifest.cs:21`](../Jellyfin.Plugin.JRay/Models/Jmanifest.cs#L21) — `public class Jmanifest`
- [`Jellyfin.Plugin.JRay/Services/Interfaces/IManifestExchangeClient.cs:16`](../Jellyfin.Plugin.JRay/Services/Interfaces/IManifestExchangeClient.cs#L16) — `public interface IManifestExchangeClient`
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### JR-027
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Models/Jmanifest.cs:21`](../Jellyfin.Plugin.JRay/Models/Jmanifest.cs#L21) — `public class Jmanifest`
- [`Jellyfin.Plugin.JRay/Services/ManifestValidator.cs:26`](../Jellyfin.Plugin.JRay/Services/ManifestValidator.cs#L26) — `public static class ManifestValidator`
### JR-028
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:282`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L282) — `private static async Task<string?> ReadCappedAsync(`
### JR-029
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### JR-030
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/ManifestConverter.cs:11`](../Jellyfin.Plugin.JRay/Services/ManifestConverter.cs#L11) — `public static class ManifestConverter`
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### JR-031
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Controllers/ManifestController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/ManifestController.cs#L24) — `public class ManifestController : ControllerBase`
### JR-036
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs:16`](../Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs#L16) — `public class PluginConfiguration : BasePluginConfiguration`
- [`Jellyfin.Plugin.JRay/Models/TruthProvenance.cs:38`](../Jellyfin.Plugin.JRay/Models/TruthProvenance.cs#L38) — `public class TruthProvenance`
### JR-037
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### JR-038
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs:16`](../Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs#L16) — `public class PluginConfiguration : BasePluginConfiguration`
### JR-041
**Locations:** 1
- [`scripts/checks/no-gallery-data.sh:14`](../scripts/checks/no-gallery-data.sh#L14) — `Unknown`
### JR-042
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Services/AudioSignature.cs:49`](../Jellyfin.Plugin.JRay/Services/AudioSignature.cs#L49) — `public static class AudioSignature`
- [`Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs:39`](../Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs#L39) — `public class AudioSignatureService`
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### JR-043
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### JR-044
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs:24`](../Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs#L24) — `public static class AudioSignatureMatcher`
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### JR-045
**Locations:** 2
- [`Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs:24`](../Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs#L24) — `public static class AudioSignatureMatcher`
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### JR-047
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Models/TruthAlignment.cs:17`](../Jellyfin.Plugin.JRay/Models/TruthAlignment.cs#L17) — `public class TruthAlignment`
- [`Jellyfin.Plugin.JRay/Services/ManifestAligner.cs:36`](../Jellyfin.Plugin.JRay/Services/ManifestAligner.cs#L36) — `public class ManifestAligner`
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
### PR-001
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Controllers/WebController.cs:21`](../Jellyfin.Plugin.JRay/Controllers/WebController.cs#L21) — `public class WebController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/TruthProvenance.cs:38`](../Jellyfin.Plugin.JRay/Models/TruthProvenance.cs#L38) — `public class TruthProvenance`
- [`Jellyfin.Plugin.JRay/Services/TruthDataService.cs:25`](../Jellyfin.Plugin.JRay/Services/TruthDataService.cs#L25) — `public sealed class TruthDataService : ITruthDataService`
### PR-003
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Controllers/CoverageController.cs:30`](../Jellyfin.Plugin.JRay/Controllers/CoverageController.cs#L30) — `public class CoverageController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Controllers/PolicyController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/PolicyController.cs#L24) — `public class PolicyController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Controllers/TasksController.cs:34`](../Jellyfin.Plugin.JRay/Controllers/TasksController.cs#L34) — `public class TasksController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/MediaPolicyStore.cs:20`](../Jellyfin.Plugin.JRay/Services/MediaPolicyStore.cs#L20) — `public sealed class MediaPolicyStore : IMediaPolicyStore`
- [`Jellyfin.Plugin.JRay/Services/PolicyResolver.cs:14`](../Jellyfin.Plugin.JRay/Services/PolicyResolver.cs#L14) — `public static class PolicyResolver`
### PR-004
**Locations:** 5
- [`Jellyfin.Plugin.JRay/Controllers/StatusController.cs:20`](../Jellyfin.Plugin.JRay/Controllers/StatusController.cs#L20) — `public class StatusController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs:31`](../Jellyfin.Plugin.JRay/Services/FileTransformationRegistration.cs#L31) — `public static class FileTransformationRegistration`
- [`Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs:21`](../Jellyfin.Plugin.JRay/Services/ManagedTruthStore.cs#L21) — `public sealed class ManagedTruthStore : IManagedTruthStore`
- [`Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs:26`](../Jellyfin.Plugin.JRay/Services/WebClientPatchService.cs#L26) — `public static class WebClientPatchService`
- [`scripts/checks/no-index-injection.sh:11`](../scripts/checks/no-index-injection.sh#L11) — `Unknown`
### PR-005
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs:75`](../Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs#L75) — `public class ManifestServer`
- [`Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs:16`](../Jellyfin.Plugin.JRay/Configuration/PluginConfiguration.cs#L16) — `public class PluginConfiguration : BasePluginConfiguration`
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### PR-006
**Locations:** 4
- [`Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs:75`](../Jellyfin.Plugin.JRay/Configuration/ManifestServer.cs#L75) — `public class ManifestServer`
- [`Jellyfin.Plugin.JRay/Controllers/ManifestController.cs:24`](../Jellyfin.Plugin.JRay/Controllers/ManifestController.cs#L24) — `public class ManifestController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Services/Interfaces/IManifestExchangeClient.cs:16`](../Jellyfin.Plugin.JRay/Services/Interfaces/IManifestExchangeClient.cs#L16) — `public interface IManifestExchangeClient`
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:32`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L32) — `public class ManifestExchangeClient : IManifestExchangeClient, IDisposable`
### SR-001
**Locations:** 1
- [`Jellyfin.Plugin.JRay/Models/TruthActor.cs:20`](../Jellyfin.Plugin.JRay/Models/TruthActor.cs#L20) — `public class TruthActor`
### SR-002
**Locations:** 7
- [`Jellyfin.Plugin.JRay/Controllers/ActorsController.cs:26`](../Jellyfin.Plugin.JRay/Controllers/ActorsController.cs#L26) — `public class ActorsController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/ActorInScene.cs:15`](../Jellyfin.Plugin.JRay/Models/ActorInScene.cs#L15) — `public class ActorInScene`
- [`Jellyfin.Plugin.JRay/Models/TruthActor.cs:20`](../Jellyfin.Plugin.JRay/Models/TruthActor.cs#L20) — `public class TruthActor`
- [`Jellyfin.Plugin.JRay/Models/TruthScene.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthScene.cs#L21) — `public class TruthScene`
- [`Jellyfin.Plugin.JRay/Services/ManifestConverter.cs:11`](../Jellyfin.Plugin.JRay/Services/ManifestConverter.cs#L11) — `public static class ManifestConverter`
- [`Jellyfin.Plugin.JRay/Services/PresenceLookup.cs:27`](../Jellyfin.Plugin.JRay/Services/PresenceLookup.cs#L27) — `public static class PresenceLookup`
- [`Jellyfin.Plugin.JRay/Web/jray-overlay.js:14`](../Jellyfin.Plugin.JRay/Web/jray-overlay.js#L14) — `function truncate(text, maxLength)`
### SR-003
**Locations:** 13
- [`Jellyfin.Plugin.JRay/Controllers/TruthController.cs:28`](../Jellyfin.Plugin.JRay/Controllers/TruthController.cs#L28) — `public class TruthController : ControllerBase`
- [`Jellyfin.Plugin.JRay/Models/Jmanifest.cs:21`](../Jellyfin.Plugin.JRay/Models/Jmanifest.cs#L21) — `public class Jmanifest`
- [`Jellyfin.Plugin.JRay/Models/TruthAlignment.cs:17`](../Jellyfin.Plugin.JRay/Models/TruthAlignment.cs#L17) — `public class TruthAlignment`
- [`Jellyfin.Plugin.JRay/Models/TruthCut.cs:15`](../Jellyfin.Plugin.JRay/Models/TruthCut.cs#L15) — `public class TruthCut`
- [`Jellyfin.Plugin.JRay/Models/TruthExtraction.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthExtraction.cs#L21) — `public class TruthExtraction`
- [`Jellyfin.Plugin.JRay/Models/TruthFile.cs:26`](../Jellyfin.Plugin.JRay/Models/TruthFile.cs#L26) — `public class TruthFile`
- [`Jellyfin.Plugin.JRay/Models/TruthScene.cs:21`](../Jellyfin.Plugin.JRay/Models/TruthScene.cs#L21) — `public class TruthScene`
- [`Jellyfin.Plugin.JRay/Services/AudioSignature.cs:49`](../Jellyfin.Plugin.JRay/Services/AudioSignature.cs#L49) — `public static class AudioSignature`
- [`Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs:24`](../Jellyfin.Plugin.JRay/Services/AudioSignatureMatcher.cs#L24) — `public static class AudioSignatureMatcher`
- [`Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs:39`](../Jellyfin.Plugin.JRay/Services/AudioSignatureService.cs#L39) — `public class AudioSignatureService`
- [`Jellyfin.Plugin.JRay/Services/ManifestAligner.cs:36`](../Jellyfin.Plugin.JRay/Services/ManifestAligner.cs#L36) — `public class ManifestAligner`
- [`Jellyfin.Plugin.JRay/Services/ManifestConverter.cs:11`](../Jellyfin.Plugin.JRay/Services/ManifestConverter.cs#L11) — `public static class ManifestConverter`
- [`Jellyfin.Plugin.JRay/Services/TruthSchema.cs:35`](../Jellyfin.Plugin.JRay/Services/TruthSchema.cs#L35) — `public static class TruthSchema`
### SR-004
**Locations:** 3
- [`Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs:282`](../Jellyfin.Plugin.JRay/Services/ManifestExchangeClient.cs#L282) — `private static async Task<string?> ReadCappedAsync(`
- [`Jellyfin.Plugin.JRay/Services/ManifestValidator.cs:26`](../Jellyfin.Plugin.JRay/Services/ManifestValidator.cs#L26) — `public static class ManifestValidator`
- [`Jellyfin.Plugin.JRay/Web/jray-overlay.js:14`](../Jellyfin.Plugin.JRay/Web/jray-overlay.js#L14) — `function truncate(text, maxLength)`
### SR-005
**Locations:** 1
- [`scripts/checks/no-gallery-data.sh:14`](../scripts/checks/no-gallery-data.sh#L14) — `Unknown`
### UT-001
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### UT-002
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### UT-003
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### UT-004
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### UT-005
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/WebClientPatchServiceTests.cs#L16) — `public class WebClientPatchServiceTests`
### UT-007
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### UT-008
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### UT-009
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### UT-010
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### UT-011
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs:14`](../Jellyfin.Plugin.JRay.Tests/PolicyResolverTests.cs#L14) — `public class PolicyResolverTests`
### UT-012
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs:20`](../Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs#L20) — `public class FileTransformationRegistrationTests`
### UT-013
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs:20`](../Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs#L20) — `public class FileTransformationRegistrationTests`
### UT-014
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs:20`](../Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs#L20) — `public class FileTransformationRegistrationTests`
### UT-015
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs:20`](../Jellyfin.Plugin.JRay.Tests/FileTransformationRegistrationTests.cs#L20) — `public class FileTransformationRegistrationTests`
### UT-016
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-017
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-018
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-019
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-020
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-021
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-022
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-023
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs:19`](../Jellyfin.Plugin.JRay.Tests/PresenceLookupTests.cs#L19) — `public class PresenceLookupTests`
### UT-024
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### UT-025
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### UT-026
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### UT-027
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### UT-028
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs:21`](../Jellyfin.Plugin.JRay.Tests/ManagedTruthStoreTests.cs#L21) — `public class ManagedTruthStoreTests : IDisposable`
### UT-029
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-030
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-031
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-032
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-033
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-034
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-035
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-036
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-037
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs:16`](../Jellyfin.Plugin.JRay.Tests/TruthSchemaTests.cs#L16) — `public class TruthSchemaTests`
### UT-038
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-039
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-040
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-041
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-042
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-043
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-044
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs:32`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureTests.cs#L32) — `public class AudioSignatureTests`
### UT-045
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-046
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-047
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-048
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-049
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-050
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-051
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-052
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs:26`](../Jellyfin.Plugin.JRay.Tests/AudioSignatureMatcherTests.cs#L26) — `public class AudioSignatureMatcherTests`
### UT-053
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
### UT-054
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
### UT-055
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
### UT-056
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
### UT-057
**Locations:** 1
- [`Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs:28`](../Jellyfin.Plugin.JRay.Tests/ManifestAlignerTests.cs#L28) — `public class ManifestAlignerTests`
+42
View File
@@ -0,0 +1,42 @@
#!/usr/bin/env bash
# JR-041 — the plugin never fetches, stores, or transmits gallery data.
#
# Like JR-021 this is a requirement to *not do* something, so it is verified by
# absence. It mirrors the server's UR-012, which is preserved the same way: there
# is no field capable of carrying an embedding or a crop, and no code that would
# read one.
#
# The plugin's whole contact with identity is public identifiers (JR-007) and
# scene windows (JR-004). A reference face or a 512-d vector arriving here would
# mean SR-005 had been breached upstream, so the check is for any *parse* of one
# — a field name, a property, a type — not merely for network calls.
#
# TRACES: JR-041 | SR-005
set -euo pipefail
cd "$(dirname "$0")/../.."
roots=("Jellyfin.Plugin.JRay")
status=0
# Field and property names that would carry gallery data. Matched as whole words
# so that unrelated identifiers containing them are not false positives.
banned='\b(embedding|embeddings|face_crop|faceCrop|FaceCrop|reference_face|referenceFace|ReferenceFace|mugshot|Mugshot|gallery_vector|galleryVector|descriptor512|Embedding)\b'
if grep -rn --include='*.cs' --include='*.js' -E "$banned" "${roots[@]}"; then
echo "FAIL (JR-041): a gallery-data field or type is referenced in the plugin." >&2
status=1
fi
# A base64 blob is the other shape this could arrive in. The plugin has no
# legitimate reason to decode one: every string it handles is a name, an id, or
# a URL.
if grep -rn --include='*.cs' -E '\bConvert\.FromBase64String\b' "${roots[@]}"; then
echo "FAIL (JR-041): base64 decoding found — the plugin handles no binary payloads." >&2
status=1
fi
if [ "$status" -eq 0 ]; then
echo "OK (JR-041): no gallery-data code path."
fi
exit "$status"