Files
jRay/Jellyfin.Plugin.JRay/Services/ManifestConverter.cs
T
dtourolle 1e247c4c7d feat(audio): align a fetched manifest to the local file before storing
The signature had a producer and a reader but no consumer, so nothing
ever fingerprinted anything. `ManifestAligner` runs on the fetch path,
before the windows are stored.

A local alignment supersedes the server's offset. The server has never
seen this file — its offset is a runtime-difference inference at best,
while the local comparison is against the media the windows will actually
be drawn over. It also needs no round trip, so no signature leaves the
instance. This is what jRay's spec already meant by matching being a
consumer concern: the server never rewrites a manifest, so one stored
manifest serves every trim of the same cut.

The offset has two terms and only one is in the server's pseudocode. Both
windows are centred on their own file's midpoint, so unequal runtimes
start them at different absolute times; a release with 40 s of extra head
material recovers 20 s from the slide and 20 s from the anchor
difference. Using the slide alone is wrong by half the runtime difference
on every shifted release.

Degradation, never failure. Signatures off, no manifest signature, media
under the window, a `v2:` producer, a missing binary, a decode error —
each applies the server's offset rather than refusing, because a
signature is an enhancement to cut matching and must never break a fetch.

"Un-comparable" and "does not match" are kept distinct, which a test
caught: `Compare` returns null for both, and conflating them would report
a 90-second extra as content disagreeing with its own manifest. A genuine
disagreement is stored anyway — the audio may legitimately differ, a
different language track being the obvious case — and surfaced as a
caveat that outranks the tier's, since it is the stronger statement.

The applied offset, score, slide and the local file's own signature are
written beside the truth file: the offset is otherwise unrecoverable once
the windows are shifted, and the stored signature lets a later fetch
align without decoding again. Provenance is never injected into the truth
file, so the bytes served back stay the producer's (JR-004).

`docs/audio-alignment.md` documents the mechanism end to end.

TRACES: JR-047 | SR-003
2026-07-31 16:52:21 +02:00

149 lines
6.1 KiB
C#

using System;
using System.Globalization;
using Jellyfin.Plugin.JRay.Configuration;
using Jellyfin.Plugin.JRay.Models;
namespace Jellyfin.Plugin.JRay.Services;
/// <summary>
/// Converts a fetched manifest into the truth file the plugin stores.
/// </summary>
// TRACES: JR-030 | SR-002, SR-003
public static class ManifestConverter
{
/// <summary>
/// Builds a truth file from a manifest, shifting every window by
/// <paramref name="offsetSec"/>.
/// </summary>
/// <remarks>
/// <b>The offset is applied here, once, at store time.</b> The server returns
/// it and the client applies it, so a single stored manifest serves every
/// trim of the same cut without ever being rewritten upstream. Applying it on
/// the way in means the stored truth is always in the local file's own
/// timebase, so the overlay and the <c>jray?t=</c> query need no offset
/// awareness at read time — the alternative would put the same correction in
/// every reader, forever, and one of them would eventually forget.
/// <para>
/// Windows are shifted, never reshaped: a window is a claim about scene
/// membership (SR-002), so merging or trimming would answer a different
/// question than the one the extraction pipeline answered.
/// </para>
/// </remarks>
/// <param name="manifest">The validated manifest.</param>
/// <param name="offsetSec">Seconds to add to every window.</param>
/// <param name="mediaPath">Local media path, recorded informationally.</param>
/// <returns>The truth file to store.</returns>
public static TruthFile ToTruthFile(Jmanifest manifest, double offsetSec, string mediaPath)
{
ArgumentNullException.ThrowIfNull(manifest);
var truth = new TruthFile
{
SchemaVersion = TruthSchema.SupportedVersion,
Movie = mediaPath ?? string.Empty,
};
// Provenance is carried across rather than flattened. The two blocks
// have the same shape by design (JR-002), so anything the server knew
// about how a manifest was produced survives into the stored truth.
if (manifest.Extraction is { } extraction)
{
truth.Extraction = new TruthExtraction
{
SampleFps = extraction.SampleFps,
ExtinctionSec = extraction.ExtinctionSec,
PipelineVersion = extraction.PipelineVersion,
GallerySize = extraction.GallerySize,
GalleryScope = extraction.GalleryScope,
};
}
// The cut is the *manifest's*, not the local file's: it records the
// encode the timings were measured against, which is what makes the
// applied offset interpretable later. Recording the local runtime here
// instead would erase the very discrepancy the offset corrects.
if (manifest.Cut is { } cut)
{
truth.Cut = new TruthCut
{
RuntimeSec = cut.RuntimeSec,
AudioSignature = cut.AudioSignature,
};
}
foreach (var actor in manifest.Actors)
{
var converted = new TruthActor
{
Name = actor.Name ?? string.Empty,
ImdbId = actor.ImdbId ?? string.Empty,
TmdbId = actor.TmdbId ?? string.Empty,
};
foreach (var scene in actor.Scenes)
{
// Clamped at zero: a negative offset on an early window would
// otherwise produce a start before the file begins, which no
// reader can index.
var start = Math.Max(0, scene.Start + offsetSec);
var end = Math.Max(start, scene.End + offsetSec);
converted.Scenes.Add(new TruthScene
{
Start = start,
End = end,
// Belief and route survive the conversion. They are what a
// consumer needs to know how far to trust a window, and
// dropping them here would silently downgrade every fetched
// manifest against a locally extracted one.
Belief = scene.Belief,
Route = scene.Route,
});
}
truth.Actors.Add(converted);
}
return truth;
}
/// <summary>
/// A short, human-readable description of how a manifest matched, for the
/// UI to show as a caveat.
/// </summary>
/// <remarks>
/// A <c>loose</c> match should surface as a caveat rather than being applied
/// silently: it means the runtimes differ by up to 30 seconds, which is
/// usually a different trim of the same cut but is not guaranteed to be.
/// </remarks>
/// <param name="tier">The tier achieved.</param>
/// <param name="offsetSec">The offset applied.</param>
/// <param name="alignment">How the offset was arrived at, when known.</param>
/// <returns>A caveat string, or null when the match needs no explanation.</returns>
public static string? DescribeCaveat(MatchTier tier, double offsetSec, TruthAlignment? alignment = null)
{
// Ranked before the tier, because it is the stronger statement: the
// server's tier says the runtimes are compatible, while a signature
// mismatch says the audio itself is not. The second outranks the first.
if (alignment?.Source == AlignmentSource.LocalMismatch)
{
return "This file's audio does not match the fetched manifest — it may be a different cut, "
+ "a different language track, or a heavy re-encode. Timings may be wrong.";
}
if (tier == MatchTier.Loose)
{
return "Matched loosely — the runtime differs from this server's copy, so timings may drift.";
}
if (Math.Abs(offsetSec) > 0.001)
{
return string.Create(
CultureInfo.InvariantCulture,
$"Matched by audio content and shifted by {offsetSec:0.##}s to align with this file.");
}
return null;
}
}