Files
JRay-public-server/src/validate.rs
T
dtourolleandClaude Opus 5 88c7264094
CI / fmt, clippy, test (push) Failing after 1m22s
CI / static musl binary (push) Has been skipped
CI / advisories and licences (push) Successful in 26s
Ship the SR-003 schema bump: jmanifest_version 2
Moves the exchange envelope to version 2 in lockstep with the truth file's
schema_version, per SR-003's requirement that breaking changes be batched and
ship together rather than piecemeal. The plugin had already moved to
schema_version 2; the server declaring 1 while accepting the new fields
defeated the point of having a version at all.

Flag day, not dual-accept (JR-003): version 1 is now rejected outright. All
three components are pre-release, and a v1 read path would be the one nobody
exercises, so it is the one that would rot while being dragged through every
later change to the reader. A pipeline still emitting v1 is incompatible until
updated — stated plainly rather than papered over with a shim nobody tests.

scenes become objects carrying belief and route (extraction AR-017) instead of
float pairs. Belief is bounded to [0, 1] rather than merely stored: §5a's
Threat 1 argument rests on every accepted value being bounded, and an unbounded
float is a 64-bit channel however harmless it looks. route is a closed enum, so
an invented value cannot be stored.

UR-018 is the requirement with the trap in it, and the reason content_id.rs is
untouched by this commit: belief is a producer-side estimate that may
legitimately differ between pipeline versions for identical timings, so
including it in the canonical form would give two servers different ids for the
same content — the exact failure mode §9a quantises centiseconds to avoid,
reintroduced one field along. It replicates as an attribute, exactly as
audio_signature does. The golden vector still passes unchanged, which is the
evidence rather than the claim.

191 tests. UR-015..018 move from Planned to Done; coverage 24/32 (75%).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: UR-014, UR-015, UR-016, UR-017, UR-018 | SR-003
2026-07-31 09:13:14 +02:00

1119 lines
43 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! §6 stage 2 semantic validation, and the §5a character-class constraint.
//!
//! Shape is already enforced by `deny_unknown_fields` at parse time
//! ([`crate::model`]); everything here is *content*. Rejections name the
//! offending field, per §6.
use unicode_general_category::{get_general_category, GeneralCategory};
use unicode_normalization::{is_nfc, UnicodeNormalization};
use crate::model::{
Actor, Identity, IdentityType, Jmanifest, Route, SeriesBundle, JMANIFEST_VERSION,
};
/// §6 stage 1 caps. Body-size limits are applied as a layer (see [`crate::app`]);
/// these are the structural counts the schema layer enforces.
pub mod limits {
pub const MAX_ACTORS: usize = 500;
pub const MAX_SCENES_PER_ACTOR: usize = 2000;
pub const MAX_TOTAL_SCENES: usize = 20_000;
pub const MAX_NAME_CHARS: usize = 200;
pub const MAX_TITLE_CHARS: usize = 300;
/// §2 bundle caps.
pub const MAX_BUNDLE_EPISODES: usize = 500;
/// Times beyond `runtime_sec` + this tolerance are rejected (§6).
pub const RUNTIME_TOLERANCE_SEC: f64 = 5.0;
/// §6 stage 1 body caps, in bytes.
pub const BODY_LIMIT_MANIFEST: usize = 2 * 1024 * 1024;
pub const BODY_LIMIT_BUNDLE: usize = 25 * 1024 * 1024;
/// §3: fixed signature length. The tolerance covers seek and encoder
/// differences at the window edges — it is not a licence to vary the length.
pub const AUDIO_SIG_FRAMES: usize = 1290;
pub const AUDIO_SIG_TOLERANCE: usize = 32;
}
#[derive(Debug, thiserror::Error)]
#[error("{field}: {reason}")]
pub struct ValidationError {
pub field: String,
pub reason: String,
}
fn err(field: impl Into<String>, reason: impl Into<String>) -> ValidationError {
ValidationError { field: field.into(), reason: reason.into() }
}
type VResult<T> = Result<T, ValidationError>;
/// §3 / IR-007: the analysis window is `runtime/2 ± 60 s`, so below 120 s it
/// underflows and **no signature is emitted**. The rule is identical in both
/// producers and here; a rule that differs between them yields signatures that
/// never match.
pub const AUDIO_SIG_MIN_RUNTIME_SEC: f64 = 120.0;
/// A manifest that has passed §6 stage 2. Carries the normalised forms so
/// downstream stages do not re-derive them.
#[derive(Debug, Clone)]
pub struct ValidManifest {
pub manifest: Jmanifest,
/// Scene windows quantised to integer centiseconds (§7, §9a) — the same
/// quantisation used for `content_id`, so stored and hashed values cannot
/// diverge.
pub actor_scenes_cs: Vec<ActorScenes>,
}
/// One validated window, quantised and carrying its provenance.
///
/// `belief` and `route` ride alongside `start_cs`/`end_cs` rather than being
/// folded into them, because §9a hashes only the timings: belief is a
/// producer-side estimate that may differ between pipeline versions for
/// identical content, so it is replicated as an attribute, never as identity.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct SceneCs {
pub start_cs: i64,
pub end_cs: i64,
pub belief: Option<f64>,
pub route: Option<Route>,
}
impl SceneCs {
/// A window carrying timings only — the shape a producer that has not yet
/// adopted per-window belief emits, and the one `content_id` hashes.
pub fn plain(start_cs: i64, end_cs: i64) -> Self {
Self { start_cs, end_cs, belief: None, route: None }
}
}
#[derive(Debug, Clone)]
pub struct ActorScenes {
/// NFC-normalised name, used only for matching in stage 3 and then dropped.
pub name: Option<String>,
pub tmdb_id: Option<u64>,
pub imdb_id: Option<String>,
pub scenes_cs: Vec<SceneCs>,
}
/// Quantises seconds to whole centiseconds (§9a).
///
/// Integer centiseconds remove the float-canonicalisation failure mode rather
/// than dodging it: pipeline timings are *derived* by accumulating `1/fps`, so
/// they carry accumulated error, and any value near a rounding boundary would
/// otherwise hash differently on two servers.
/// TRACES: DR-011 | SR-003
pub fn to_centiseconds(secs: f64) -> i64 {
(secs * 100.0).round() as i64
}
// ---------------------------------------------------------------------------
// Identifier formats (§6 stage 2)
// ---------------------------------------------------------------------------
/// `^tt\d{7,8}$`
fn is_title_imdb_id(s: &str) -> bool {
let Some(digits) = s.strip_prefix("tt") else { return false };
matches!(digits.len(), 7 | 8) && digits.bytes().all(|b| b.is_ascii_digit())
}
/// `^nm\d{7,8}$`
fn is_person_imdb_id(s: &str) -> bool {
let Some(digits) = s.strip_prefix("nm") else { return false };
matches!(digits.len(), 7 | 8) && digits.bytes().all(|b| b.is_ascii_digit())
}
/// `^\d{1,9}$`
fn is_tmdb_id(s: &str) -> bool {
!s.is_empty() && s.len() <= 9 && s.bytes().all(|b| b.is_ascii_digit())
}
// ---------------------------------------------------------------------------
// §5a free-text constraints
// ---------------------------------------------------------------------------
/// §5a character class: Unicode letters, marks, spaces, and `. ' - ,` only.
///
/// No digits and no `/ + =`, which is what **defeats base64/hex smuggling**. No
/// control characters, and no zero-width or bidi-control codepoints.
/// TRACES: UR-011 | SR-004
fn is_allowed_text_char(c: char) -> bool {
if matches!(c, '.' | '\'' | '-' | ',' | ' ') {
return true;
}
// Explicitly excluded regardless of category: zero-width and bidi controls.
if matches!(c, '\u{200B}'..='\u{200F}' | '\u{202A}'..='\u{202E}'
| '\u{2060}'..='\u{2064}' | '\u{2066}'..='\u{2069}' | '\u{FEFF}')
{
return false;
}
if !matches!(
get_general_category(c),
GeneralCategory::UppercaseLetter
| GeneralCategory::LowercaseLetter
| GeneralCategory::TitlecaseLetter
| GeneralCategory::ModifierLetter
| GeneralCategory::OtherLetter
| GeneralCategory::NonspacingMark
| GeneralCategory::SpacingMark
| GeneralCategory::EnclosingMark
) {
return false;
}
// Reject *compatibility* variants of otherwise-allowed letters — mathematical
// bold (`𝐒`), fullwidth (`A`), enclosed and other presentation forms.
//
// These are genuine letters by category, so the check above admits them, and
// NFC does not fold them (only NFKC would). They matter for two reasons:
// homoglyph spoofing of a real person's name, and the fact that a
// fullwidth-digit alphabet would reopen the very encoding channel §5a's "no
// digits" rule closes. A character that NFKC would rewrite is not the
// character it appears to be, so it is not accepted.
//
// Names are stored as TMDB references anyway (§5a), so the cost of being
// strict here is nil: a real TMDB name is already in normal form.
!is_compatibility_variant(c)
}
/// True when NFKC rewrites `c` into something other than itself.
fn is_compatibility_variant(c: char) -> bool {
let mut it = c.nfkc();
match (it.next(), it.next()) {
(Some(first), None) => first != c,
// Decomposes to several characters, so it is certainly not canonical.
(Some(_), Some(_)) => true,
(None, _) => true,
}
}
/// Validates a free-text field against §5a's permissive-but-closed pattern and
/// returns it NFC-normalised.
fn check_text(field: &str, value: &str, max_chars: usize) -> VResult<String> {
if value.chars().count() > max_chars {
return Err(err(field, format!("longer than {max_chars} characters")));
}
let normalised: String = if is_nfc(value) { value.to_string() } else { value.nfc().collect() };
if normalised.chars().count() > max_chars {
return Err(err(field, format!("longer than {max_chars} characters after NFC")));
}
if let Some(bad) = normalised.chars().find(|c| !is_allowed_text_char(*c)) {
return Err(err(field, format!("contains disallowed character U+{:04X}", bad as u32)));
}
Ok(normalised)
}
/// §6: reject any string anywhere that looks like an absolute filesystem path
/// or a `file://` URI.
///
/// `movie` itself is already a parse error via `deny_unknown_fields`; this
/// closes the same leak arriving through a field that *is* allowed.
fn check_not_path_shaped(field: &str, value: &str) -> VResult<()> {
let v = value.trim();
let looks_like_path = v.starts_with('/')
|| v.starts_with("\\\\")
|| v.to_ascii_lowercase().starts_with("file://")
|| (v.len() >= 3
&& v.as_bytes()[0].is_ascii_alphabetic()
&& v.as_bytes()[1] == b':'
&& matches!(v.as_bytes()[2], b'\\' | b'/'));
if looks_like_path {
return Err(err(field, "looks like a filesystem path or file:// URI"));
}
Ok(())
}
// ---------------------------------------------------------------------------
// Manifest validation
// ---------------------------------------------------------------------------
/// TRACES: UR-003, UR-014 | SR-003, SR-004
pub fn validate_manifest(mut m: Jmanifest) -> VResult<ValidManifest> {
if m.jmanifest_version != JMANIFEST_VERSION {
return Err(err(
"jmanifest_version",
format!("unsupported version {}, expected {JMANIFEST_VERSION}", m.jmanifest_version),
));
}
validate_identity(&mut m.identity)?;
validate_cut(&m)?;
if let Some(ex) = &m.extraction {
if let Some(pv) = &ex.pipeline_version {
check_not_path_shaped("extraction.pipeline_version", pv)?;
if pv.chars().count() > limits::MAX_TITLE_CHARS {
return Err(err("extraction.pipeline_version", "too long"));
}
}
if let Some(fps) = ex.sample_fps {
if !fps.is_finite() || fps <= 0.0 {
return Err(err("extraction.sample_fps", "must be a positive finite number"));
}
}
if let Some(a) = ex.extinction_sec {
if !a.is_finite() || a < 0.0 {
return Err(err(
"extraction.extinction_sec",
"must be a non-negative finite number",
));
}
}
}
let actor_scenes_cs = validate_actors(&m)?;
Ok(ValidManifest { manifest: m, actor_scenes_cs })
}
fn validate_identity(id: &mut Identity) -> VResult<()> {
match id.kind {
IdentityType::Movie => {
if id.series_tmdb_id.is_some() || id.series_imdb_id.is_some() {
return Err(err("identity.series_tmdb_id", "not valid for type=movie"));
}
if id.season.is_some() || id.episode.is_some() {
return Err(err("identity.season", "not valid for type=movie"));
}
// §6: neither `tmdb_id` nor `imdb_id` in `identity`.
if id.tmdb_id.is_none() && id.imdb_id.is_none() {
return Err(err("identity", "requires at least one of tmdb_id or imdb_id"));
}
if let Some(t) = &id.tmdb_id {
if !is_tmdb_id(t) {
return Err(err("identity.tmdb_id", "must match ^\\d{1,9}$"));
}
}
if let Some(i) = &id.imdb_id {
if !is_title_imdb_id(i) {
return Err(err("identity.imdb_id", "must match ^tt\\d{7,8}$"));
}
}
}
IdentityType::Episode => {
if id.tmdb_id.is_some() || id.imdb_id.is_some() {
return Err(err(
"identity.tmdb_id",
"use series_tmdb_id / series_imdb_id for type=episode",
));
}
if id.series_tmdb_id.is_none() && id.series_imdb_id.is_none() {
return Err(err(
"identity",
"requires at least one of series_tmdb_id or series_imdb_id",
));
}
if let Some(t) = &id.series_tmdb_id {
if !is_tmdb_id(t) {
return Err(err("identity.series_tmdb_id", "must match ^\\d{1,9}$"));
}
}
if let Some(i) = &id.series_imdb_id {
if !is_title_imdb_id(i) {
return Err(err("identity.series_imdb_id", "must match ^tt\\d{7,8}$"));
}
}
// Bounded integers (§5a).
match id.season {
Some(s) if (0..=1000).contains(&s) => {}
Some(_) => return Err(err("identity.season", "out of range 0..=1000")),
None => return Err(err("identity.season", "required for type=episode")),
}
match id.episode {
Some(e) if (0..=10_000).contains(&e) => {}
Some(_) => return Err(err("identity.episode", "out of range 0..=10000")),
None => return Err(err("identity.episode", "required for type=episode")),
}
}
}
if let Some(y) = id.year {
if !(1870..=2200).contains(&y) {
return Err(err("identity.year", "out of range 1870..=2200"));
}
}
if let Some(t) = &id.title {
check_not_path_shaped("identity.title", t)?;
id.title = Some(check_text("identity.title", t, limits::MAX_TITLE_CHARS)?);
}
Ok(())
}
fn validate_cut(m: &Jmanifest) -> VResult<()> {
let rt = m.cut.runtime_sec;
// §2: `cut.runtime_sec` is required — absence is already a parse error, so
// what remains is range.
if !rt.is_finite() || rt <= 0.0 || rt > 200_000.0 {
return Err(err("cut.runtime_sec", "must be a finite duration in (0, 200000]"));
}
if let Some(d) = m.cut.container_duration_sec {
if !d.is_finite() || d <= 0.0 || d > 200_000.0 {
return Err(err("cut.container_duration_sec", "must be a finite duration"));
}
}
if let Some(h) = &m.cut.video_hash {
validate_video_hash(h)?;
}
if let Some(sig) = &m.cut.audio_signature {
validate_audio_signature(sig, rt)?;
}
Ok(())
}
/// §3: the OpenSubtitles hash, in the fixed `opensubtitles:<16 hex>` form.
fn validate_video_hash(h: &str) -> VResult<()> {
let Some(hex) = h.strip_prefix("opensubtitles:") else {
return Err(err("cut.video_hash", "must be prefixed 'opensubtitles:'"));
};
if hex.len() != 16 || !hex.bytes().all(|b| b.is_ascii_hexdigit()) {
return Err(err("cut.video_hash", "expected 16 hex digits after the prefix"));
}
Ok(())
}
/// §3 "Validation and abuse": fixed length, base64, and each byte structurally
/// constrained (5-bit bin index + 2-bit energy class).
///
/// A variable-length blob would be a payload channel — precisely what §5a
/// closes — so length is checked, not merely bounded.
///
/// `runtime_sec` is needed because §3 shortens the window for very short items;
/// see [`expected_min_frames`].
/// TRACES: UR-009, UR-011 | SR-003, SR-004
pub fn validate_audio_signature(sig: &str, runtime_sec: f64) -> VResult<()> {
// IR-007: media shorter than the window emits **no signature**, and no sync
// offset is applied to it. A signature present on such an item did not come
// from the specified construction, so it is rejected rather than stored —
// whatever it is, it is not the thing this field is for.
if runtime_sec < AUDIO_SIG_MIN_RUNTIME_SEC {
return Err(err(
"cut.audio_signature",
format!(
"must not be present for media shorter than {AUDIO_SIG_MIN_RUNTIME_SEC:.0}s — \
the {AUDIO_SIG_MIN_RUNTIME_SEC:.0}s analysis window underflows"
),
));
}
let Some(payload) = sig.strip_prefix("v1:") else {
return Err(err("cut.audio_signature", "must be version-prefixed 'v1:'"));
};
let bytes = base64_decode(payload)
.map_err(|e| err("cut.audio_signature", format!("invalid base64: {e}")))?;
// §3, and `scene-actor-extraction` IR-007: the length is **fixed by the
// construction**, not merely bounded. A 120 s window at a 1024-sample hop and
// 11025 Hz yields ~1290 frames, and an item too short for that window emits
// no signature at all — the window `runtime/2 ± 60 s` underflows below 120 s,
// so there is nothing to shorten.
//
// That makes the length non-negotiable, which is what keeps the field inside
// SR-004: a caller cannot choose it, so it cannot be used as a variable-size
// container. The tolerance covers seek and encoder differences at the window
// edges, nothing more.
let lo = limits::AUDIO_SIG_FRAMES - limits::AUDIO_SIG_TOLERANCE;
let hi = limits::AUDIO_SIG_FRAMES + limits::AUDIO_SIG_TOLERANCE;
if bytes.len() < lo || bytes.len() > hi {
return Err(err(
"cut.audio_signature",
format!("decoded length {} outside the fixed {lo}..={hi} frames", bytes.len()),
));
}
// 5-bit bin index (0..=31) + 2-bit energy class => bit 7 must be clear.
// Arbitrary bytes are therefore invalid, keeping §5a's "no free-form
// storage" property intact.
if let Some(pos) = bytes.iter().position(|b| b & 0x80 != 0) {
return Err(err("cut.audio_signature", format!("frame {pos} has reserved high bit set")));
}
Ok(())
}
fn validate_actors(m: &Jmanifest) -> VResult<Vec<ActorScenes>> {
if m.actors.len() > limits::MAX_ACTORS {
return Err(err("actors", format!("more than {} entries", limits::MAX_ACTORS)));
}
// §6 small-|M| handling: `|M| == 0` is rejected. 15 of the 331 corpus files
// have empty actor lists — extraction failures, not contributions.
if m.actors.is_empty() {
return Err(err("actors", "empty actor list is an extraction failure, not a contribution"));
}
let mut out = Vec::with_capacity(m.actors.len());
let mut total_scenes = 0usize;
let mut seen_tmdb: Vec<u64> = Vec::new();
let mut seen_imdb: Vec<String> = Vec::new();
for (i, a) in m.actors.iter().enumerate() {
let scenes_cs = validate_scenes(i, a, m.cut.runtime_sec)?;
total_scenes += scenes_cs.len();
if total_scenes > limits::MAX_TOTAL_SCENES {
return Err(err(
"actors",
format!("more than {} scene windows in total", limits::MAX_TOTAL_SCENES),
));
}
let tmdb_id = match &a.tmdb_id {
Some(t) if !t.is_empty() => {
if !is_tmdb_id(t) {
return Err(err(format!("actors[{i}].tmdb_id"), "must match ^\\d{1,9}$"));
}
Some(t.parse::<u64>().map_err(|_| {
err(format!("actors[{i}].tmdb_id"), "not a representable integer")
})?)
}
_ => None,
};
let imdb_id = match &a.imdb_id {
Some(v) if !v.is_empty() => {
if !is_person_imdb_id(v) {
return Err(err(format!("actors[{i}].imdb_id"), "must match ^nm\\d{7,8}$"));
}
Some(v.clone())
}
_ => None,
};
if tmdb_id.is_none() && imdb_id.is_none() && a.name.as_deref().unwrap_or("").is_empty() {
return Err(err(
format!("actors[{i}]"),
"requires at least one of tmdb_id, imdb_id or name",
));
}
// §6: duplicate actors within one manifest.
if let Some(t) = tmdb_id {
if seen_tmdb.contains(&t) {
return Err(err(format!("actors[{i}].tmdb_id"), "duplicate actor in manifest"));
}
seen_tmdb.push(t);
}
if let Some(v) = &imdb_id {
if seen_imdb.contains(v) {
return Err(err(format!("actors[{i}].imdb_id"), "duplicate actor in manifest"));
}
seen_imdb.push(v.clone());
}
let name = match &a.name {
Some(n) if !n.is_empty() => {
check_not_path_shaped(&format!("actors[{i}].name"), n)?;
Some(check_text(&format!("actors[{i}].name"), n, limits::MAX_NAME_CHARS)?)
}
_ => None,
};
out.push(ActorScenes { name, tmdb_id, imdb_id, scenes_cs });
}
Ok(out)
}
/// TRACES: UR-013 | SR-002
fn validate_scenes(idx: usize, a: &Actor, runtime_sec: f64) -> VResult<Vec<SceneCs>> {
if a.scenes.len() > limits::MAX_SCENES_PER_ACTOR {
return Err(err(
format!("actors[{idx}].scenes"),
format!("more than {} entries", limits::MAX_SCENES_PER_ACTOR),
));
}
let max_t = runtime_sec + limits::RUNTIME_TOLERANCE_SEC;
let mut out = Vec::with_capacity(a.scenes.len());
for (j, scene) in a.scenes.iter().copied().enumerate() {
let field = format!("actors[{idx}].scenes[{j}]");
let (start, end) = (scene.start, scene.end);
// §6: non-finite values (NaN/Infinity), negative times, `end < start`,
// or times beyond `runtime_sec` + tolerance.
if !start.is_finite() || !end.is_finite() {
return Err(err(field, "non-finite value"));
}
if start < 0.0 || end < 0.0 {
return Err(err(field, "negative time"));
}
if end < start {
return Err(err(field, "end before start"));
}
if end > max_t {
return Err(err(
field,
format!(
"end {end} beyond runtime_sec + {}s tolerance",
limits::RUNTIME_TOLERANCE_SEC
),
));
}
// A posterior outside scene(0, 1) is not a probability. Bounded here rather
// than merely stored, because §5a's Threat 1 argument rests on every
// accepted value being a *bounded* number — an unbounded float is a
// 64-bit channel, however harmless it looks.
if let Some(belief) = scene.belief {
if !belief.is_finite() || !(0.0..=1.0).contains(&belief) {
return Err(err(
format!("actors[{idx}].scenes[{j}].belief"),
"must be a finite probability in scene(0, 1)",
));
}
}
// `route` is a closed enum, so an unrecognised value is already a parse
// error — nothing to check here.
out.push(SceneCs {
start_cs: to_centiseconds(start),
end_cs: to_centiseconds(end),
belief: scene.belief,
route: scene.route,
});
}
// §2: scenes are sorted. Checked on the quantised values so the stored form
// is the one guaranteed ordered.
if out.windows(2).any(|w| w[1].start_cs < w[0].start_cs) {
return Err(err(format!("actors[{idx}].scenes"), "windows must be sorted by start time"));
}
Ok(out)
}
// ---------------------------------------------------------------------------
// Bundle validation
// ---------------------------------------------------------------------------
/// Validates the bundle *envelope* only.
///
/// §4: a malformed envelope is a whole-request `400`, whereas individual bad
/// episodes are reported in the per-episode results list — the bundle is not
/// atomic, because all-or-nothing would let one bad episode discard an entire
/// season's compute (§2).
/// TRACES: UR-006 | PR-006
pub fn validate_bundle_envelope(b: &SeriesBundle) -> VResult<()> {
if b.jmanifest_version != JMANIFEST_VERSION {
return Err(err(
"jmanifest_version",
format!("unsupported version {}, expected {JMANIFEST_VERSION}", b.jmanifest_version),
));
}
if b.series.series_tmdb_id.is_none() && b.series.series_imdb_id.is_none() {
return Err(err("series", "requires at least one of series_tmdb_id or series_imdb_id"));
}
if let Some(t) = &b.series.series_tmdb_id {
if !is_tmdb_id(t) {
return Err(err("series.series_tmdb_id", "must match ^\\d{1,9}$"));
}
}
if let Some(i) = &b.series.series_imdb_id {
if !is_title_imdb_id(i) {
return Err(err("series.series_imdb_id", "must match ^tt\\d{7,8}$"));
}
}
if let Some(t) = &b.series.title {
check_not_path_shaped("series.title", t)?;
check_text("series.title", t, limits::MAX_TITLE_CHARS)?;
}
if b.episodes.is_empty() {
return Err(err("episodes", "bundle contains no episodes"));
}
if b.episodes.len() > limits::MAX_BUNDLE_EPISODES {
return Err(err("episodes", format!("more than {} episodes", limits::MAX_BUNDLE_EPISODES)));
}
Ok(())
}
// ---------------------------------------------------------------------------
// base64 (standard alphabet, padded)
// ---------------------------------------------------------------------------
/// Minimal standard-alphabet base64 decoder.
///
/// Vendored rather than pulled in as a dependency: the only base64 in this
/// service is the fixed-format audio signature, and §3 argues for keeping the
/// dependency surface small on the same grounds as the plugin's FFT.
fn base64_decode(s: &str) -> Result<Vec<u8>, &'static str> {
fn val(b: u8) -> Result<u8, &'static str> {
match b {
b'A'..=b'Z' => Ok(b - b'A'),
b'a'..=b'z' => Ok(b - b'a' + 26),
b'0'..=b'9' => Ok(b - b'0' + 52),
b'+' => Ok(62),
b'/' => Ok(63),
_ => Err("character outside the base64 alphabet"),
}
}
let bytes = s.as_bytes();
if bytes.len() % 4 != 0 {
return Err("length is not a multiple of 4");
}
if bytes.is_empty() {
return Ok(Vec::new());
}
let mut out = Vec::with_capacity(bytes.len() / 4 * 3);
for (i, chunk) in bytes.chunks(4).enumerate() {
let last = i == bytes.len() / 4 - 1;
let pad = if last {
chunk.iter().filter(|&&b| b == b'=').count()
} else {
if chunk.contains(&b'=') {
return Err("padding before the final chunk");
}
0
};
if pad > 2 {
return Err("more than two padding characters");
}
let mut acc = 0u32;
for (k, &b) in chunk.iter().enumerate() {
let v = if b == b'=' {
if k < 4 - pad {
return Err("padding in a data position");
}
0
} else {
val(b)?
};
acc = (acc << 6) | v as u32;
}
let triple = acc.to_be_bytes();
out.push(triple[1]);
if pad < 2 {
out.push(triple[2]);
}
if pad < 1 {
out.push(triple[3]);
}
}
Ok(out)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::model::Scene;
/// A window with timings only — belief and route are exercised separately.
fn scene(start: f64, end: f64) -> Scene {
Scene { start, end, belief: None, route: None }
}
fn base_manifest() -> Jmanifest {
serde_json::from_str(
r#"{"jmanifest_version":2,
"identity":{"type":"movie","tmdb_id":"504172","title":"The Death of Stalin"},
"cut":{"runtime_sec":6420.5},
"actors":[{"name":"Steve Buscemi","tmdb_id":"884","scenes":[{"start":191.6,"end":209.2}]}]}"#,
)
.unwrap()
}
#[test]
fn accepts_a_realistic_manifest() {
let v = validate_manifest(base_manifest()).unwrap();
assert_eq!(v.actor_scenes_cs.len(), 1);
assert_eq!(v.actor_scenes_cs[0].scenes_cs, vec![SceneCs::plain(19160, 20920)]);
}
#[test]
fn windows_are_never_reshaped() {
// UR-013 / SR-002: a window is a claim about *scene membership*, not a
// recognition event. The server therefore stores what it was given —
// quantised, but never merged, split or trimmed.
//
// The adjacent-window case is the one that matters: a naive
// implementation might "tidy" two windows that touch into one, which
// would destroy the distinction SR-002 draws between an actor who turned
// away (one window, gap absorbed by the producer) and one who genuinely
// left and returned (two windows).
let mut m = base_manifest();
m.actors[0].scenes = vec![
scene(10.0, 20.0),
scene(20.0, 30.0), // exactly adjacent — must stay separate
scene(30.01, 40.0), // a hair's gap — likewise
scene(100.0, 100.0), // zero-length — a real producer emits these
];
let v = validate_manifest(m).unwrap();
assert_eq!(
v.actor_scenes_cs[0].scenes_cs,
vec![
SceneCs::plain(1000, 2000),
SceneCs::plain(2000, 3000),
SceneCs::plain(3001, 4000),
SceneCs::plain(10000, 10000)
],
"windows must survive validation unchanged apart from quantisation"
);
}
#[test]
fn extinction_sec_replaces_anneal_sec() {
// The SR-003 withdrawal. `anneal_sec` cannot even be constructed here —
// it is not a field on `Extraction` — so this asserts the successor is
// accepted and range-checked; `tests/api.rs` covers the wire rejection.
let mut m = base_manifest();
m.extraction = Some(crate::model::Extraction {
sample_fps: Some(5.0),
extinction_sec: Some(12.0),
pipeline_version: Some("test 0.1".into()),
gallery_size: Some(1820),
gallery_scope: Some(crate::model::GalleryScope::Global),
});
assert!(validate_manifest(m).is_ok());
let mut m = base_manifest();
m.extraction = Some(crate::model::Extraction {
sample_fps: None,
extinction_sec: Some(-1.0),
pipeline_version: None,
gallery_size: None,
gallery_scope: None,
});
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "extraction.extinction_sec");
}
#[test]
fn rejects_empty_actor_list() {
let mut m = base_manifest();
m.actors.clear();
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "actors");
}
#[test]
fn rejects_scene_beyond_runtime_tolerance() {
let mut m = base_manifest();
m.actors[0].scenes = vec![scene(10.0, 6500.0)];
let e = validate_manifest(m).unwrap_err();
assert!(e.field.starts_with("actors[0].scenes"), "got {}", e.field);
}
#[test]
fn accepts_scene_within_runtime_tolerance() {
let mut m = base_manifest();
m.actors[0].scenes = vec![scene(10.0, 6424.0)];
assert!(validate_manifest(m).is_ok());
}
#[test]
fn rejects_end_before_start_and_negative_and_nonfinite() {
for scenes in [
vec![scene(50.0, 10.0)],
vec![scene(-1.0, 10.0)],
vec![scene(f64::NAN, 10.0)],
vec![scene(0.0, f64::INFINITY)],
] {
let mut m = base_manifest();
m.actors[0].scenes = scenes;
assert!(validate_manifest(m).is_err());
}
}
#[test]
fn rejects_unsorted_scenes() {
let mut m = base_manifest();
m.actors[0].scenes = vec![scene(100.0, 120.0), scene(10.0, 20.0)];
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "actors[0].scenes");
}
#[test]
fn rejects_duplicate_actor() {
let mut m = base_manifest();
m.actors.push(m.actors[0].clone());
let e = validate_manifest(m).unwrap_err();
assert!(e.reason.contains("duplicate"), "got {}", e.reason);
}
#[test]
fn rejects_bad_identifier_formats() {
let mut m = base_manifest();
m.identity.imdb_id = Some("tt123".into());
assert!(validate_manifest(m).is_err());
let mut m = base_manifest();
m.identity.tmdb_id = Some("504172x".into());
assert!(validate_manifest(m).is_err());
let mut m = base_manifest();
m.actors[0].imdb_id = Some("tt0000114".into()); // title id in a person field
assert!(validate_manifest(m).is_err());
}
#[test]
fn requires_an_identity_key() {
let mut m = base_manifest();
m.identity.tmdb_id = None;
m.identity.imdb_id = None;
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "identity");
}
#[test]
fn episode_identity_requires_season_and_episode() {
let json = r#"{"jmanifest_version":2,
"identity":{"type":"episode","series_tmdb_id":"1396","title":"Breaking Bad"},
"cut":{"runtime_sec":2820.0},
"actors":[{"name":"Bryan Cranston","tmdb_id":"17419","scenes":[{"start":10.0,"end":20.0}]}]}"#;
let m: Jmanifest = serde_json::from_str(json).unwrap();
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "identity.season");
}
#[test]
fn valid_episode_identity_is_accepted() {
let json = r#"{"jmanifest_version":2,
"identity":{"type":"episode","series_tmdb_id":"1396","series_imdb_id":"tt0903747",
"title":"Breaking Bad","season":2,"episode":5},
"cut":{"runtime_sec":2820.0},
"actors":[{"name":"Bryan Cranston","tmdb_id":"17419","scenes":[{"start":10.0,"end":20.0}]}]}"#;
let m: Jmanifest = serde_json::from_str(json).unwrap();
assert!(validate_manifest(m).is_ok());
}
#[test]
fn movie_identity_rejects_episode_coordinates() {
let mut m = base_manifest();
m.identity.season = Some(1);
assert!(validate_manifest(m).is_err());
}
// §5a: the character class alone must defeat base64/hex smuggling, which
// needs digits and padding characters.
#[test]
fn name_character_class_rejects_smuggling() {
for name in [
"SGVsbG8gd29ybGQ=", // base64
"deadbeef1234", // hex
"Steve/Buscemi",
"Steve+Buscemi",
"Actor 2", // digits
"Steve\u{200B}Buscemi", // zero-width space
"Steve\u{202E}imecsuB", // bidi override
"Steve\u{0007}Buscemi", // control character
"<script>x</script>",
] {
let mut m = base_manifest();
m.actors[0].name = Some(name.to_string());
assert!(
validate_manifest(m).is_err(),
"name {name:?} should be rejected by the §5a character class"
);
}
}
#[test]
fn name_character_class_rejects_compatibility_homoglyphs() {
// Another gap an injection test caught. These are letters by Unicode
// category, so a category-only check admits them, and NFC does not fold
// them — only NFKC would. Two problems: they spoof a real person's name,
// and a fullwidth-digit alphabet would reopen the encoding channel that
// §5a's "no digits" rule exists to close.
for name in [
"𝐒𝐭𝐞𝐯𝐞 𝐁𝐮𝐬𝐜𝐞𝐦𝐢", // mathematical bold
"Steve", // fullwidth
"ⓈⓉⒺⓋⒺ", // enclosed alphanumerics
"STEVE 123", // fullwidth with digits
"film", // ligature
"Ⅻ", // Roman numeral
] {
let mut m = base_manifest();
m.actors[0].name = Some(name.to_string());
assert!(
validate_manifest(m).is_err(),
"compatibility homoglyph {name:?} should be rejected"
);
}
}
#[test]
fn name_character_class_accepts_real_names() {
for name in [
"Steve Buscemi",
"Michael Palin",
"Jean-Luc Picard",
"Renée Zellweger",
"Hayao Miyazaki",
"宮崎 駿",
"Miloš Forman",
"O'Brien",
"Sammy Davis, Jr.",
] {
let mut m = base_manifest();
m.actors[0].name = Some(name.to_string());
assert!(validate_manifest(m).is_ok(), "name {name:?} should be accepted");
}
}
#[test]
fn rejects_path_shaped_strings_in_allowed_fields() {
for probe in ["/data/movies/x.mkv", "C:\\media\\x.mkv", "\\\\nas\\media", "file:///x"] {
let mut m = base_manifest();
m.identity.title = Some(probe.to_string());
assert!(validate_manifest(m).is_err(), "{probe} should be rejected");
}
}
#[test]
fn rejects_overlong_name() {
let mut m = base_manifest();
m.actors[0].name = Some("a".repeat(limits::MAX_NAME_CHARS + 1));
assert!(validate_manifest(m).is_err());
}
#[test]
fn rejects_too_many_actors() {
let mut m = base_manifest();
let a = m.actors[0].clone();
m.actors = (0..=limits::MAX_ACTORS)
.map(|i| {
let mut c = a.clone();
c.tmdb_id = Some((1000 + i).to_string());
c
})
.collect();
let e = validate_manifest(m).unwrap_err();
assert_eq!(e.field, "actors");
}
#[test]
fn video_hash_format_is_enforced() {
let mut m = base_manifest();
m.cut.video_hash = Some("opensubtitles:8e245d9679d31e12".into());
assert!(validate_manifest(m).is_ok());
for bad in ["8e245d9679d31e12", "opensubtitles:xyz", "opensubtitles:8e245d9679d31e1"] {
let mut m = base_manifest();
m.cut.video_hash = Some(bad.into());
assert!(validate_manifest(m).is_err(), "{bad} should be rejected");
}
}
#[test]
fn centisecond_quantisation_is_stable_for_accumulated_float_error() {
// §9a: real corpus values look like 8045.066666660665.
assert_eq!(to_centiseconds(8045.066666660665), 804507);
assert_eq!(to_centiseconds(8045.066666666), 804507);
assert_eq!(to_centiseconds(0.0), 0);
}
#[test]
fn base64_roundtrip() {
// "Man" => "TWFu"; padding variants.
assert_eq!(base64_decode("TWFu").unwrap(), b"Man");
assert_eq!(base64_decode("TWE=").unwrap(), b"Ma");
assert_eq!(base64_decode("TQ==").unwrap(), b"M");
assert!(base64_decode("TWF").is_err());
assert!(base64_decode("TW$u").is_err());
assert!(base64_decode("T=Fu").is_err());
}
/// A feature-length runtime, so the full window applies.
const FEATURE_RUNTIME: f64 = 6420.5;
#[test]
fn audio_signature_validation() {
// 1290 frames with the high bit clear, base64-encoded.
let frames = vec![0x3Fu8; limits::AUDIO_SIG_FRAMES];
let sig = format!("v1:{}", base64_encode_for_test(&frames));
assert!(validate_audio_signature(&sig, FEATURE_RUNTIME).is_ok());
// Missing version prefix.
assert!(
validate_audio_signature(&base64_encode_for_test(&frames), FEATURE_RUNTIME).is_err()
);
// High bit set is structurally invalid, so arbitrary bytes cannot ride
// along in this field (§3, §5a).
let mut bad = frames.clone();
bad[7] = 0xFF;
let sig = format!("v1:{}", base64_encode_for_test(&bad));
assert!(validate_audio_signature(&sig, FEATURE_RUNTIME).is_err());
// Oversized blob would be a payload channel.
let huge = vec![0x01u8; limits::AUDIO_SIG_FRAMES + limits::AUDIO_SIG_TOLERANCE + 1];
let sig = format!("v1:{}", base64_encode_for_test(&huge));
assert!(validate_audio_signature(&sig, FEATURE_RUNTIME).is_err());
}
#[test]
fn media_below_the_window_must_carry_no_signature() {
// IR-007, reconciled with server §3: the window `runtime/2 ± 60 s`
// underflows below 120 s, so no signature exists to send. One present on
// such an item did not come from the specified construction, whatever it
// is. An earlier draft of §3 allowed a shortened window here; that was
// the weaker rule, because a caller-varying length is exactly the
// property SR-004 forbids.
let frames = vec![0x3Fu8; limits::AUDIO_SIG_FRAMES];
let sig = format!("v1:{}", base64_encode_for_test(&frames));
for runtime in [1.0f64, 30.0, 119.0, 119.999] {
let e = validate_audio_signature(&sig, runtime).unwrap_err();
assert_eq!(e.field, "cut.audio_signature");
assert!(
e.reason.contains("shorter than"),
"a {runtime}s item should be refused on its runtime: {}",
e.reason
);
}
// At and above the window, the normal rules apply.
assert!(validate_audio_signature(&sig, 120.0).is_ok());
assert!(validate_audio_signature(&sig, FEATURE_RUNTIME).is_ok());
}
#[test]
fn the_signature_length_is_fixed_not_caller_chosen() {
// What keeps the field inside SR-004: the length is a property of the
// construction, so a caller cannot use it as a variable-size container.
for frames in [1usize, 16, 100, 900, 1200] {
let sig = format!("v1:{}", base64_encode_for_test(&vec![0x3Fu8; frames]));
assert!(
validate_audio_signature(&sig, FEATURE_RUNTIME).is_err(),
"{frames} frames should be rejected — the length is fixed"
);
}
// Only the construction's own length, within edge tolerance, is accepted.
for frames in [
limits::AUDIO_SIG_FRAMES - limits::AUDIO_SIG_TOLERANCE,
limits::AUDIO_SIG_FRAMES,
limits::AUDIO_SIG_FRAMES + limits::AUDIO_SIG_TOLERANCE,
] {
let sig = format!("v1:{}", base64_encode_for_test(&vec![0x3Fu8; frames]));
assert!(validate_audio_signature(&sig, FEATURE_RUNTIME).is_ok(), "{frames} frames");
}
}
fn base64_encode_for_test(data: &[u8]) -> String {
const A: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut out = String::new();
for chunk in data.chunks(3) {
let b = [chunk[0], *chunk.get(1).unwrap_or(&0), *chunk.get(2).unwrap_or(&0)];
let n = u32::from_be_bytes([0, b[0], b[1], b[2]]);
out.push(A[(n >> 18 & 63) as usize] as char);
out.push(A[(n >> 12 & 63) as usize] as char);
out.push(if chunk.len() > 1 { A[(n >> 6 & 63) as usize] as char } else { '=' });
out.push(if chunk.len() > 2 { A[(n & 63) as usize] as char } else { '=' });
}
out
}
#[test]
fn bundle_envelope_checks() {
let ok = r#"{"jmanifest_version":2,
"series":{"series_tmdb_id":"1396","title":"Breaking Bad"},
"episodes":[{"jmanifest_version":2,
"identity":{"type":"episode","series_tmdb_id":"1396","season":1,"episode":1},
"cut":{"runtime_sec":2820.0},
"actors":[{"tmdb_id":"17419","scenes":[{"start":1.0,"end":2.0}]}]}]}"#;
let b: SeriesBundle = serde_json::from_str(ok).unwrap();
assert!(validate_bundle_envelope(&b).is_ok());
let mut empty = b.clone();
empty.episodes.clear();
assert!(validate_bundle_envelope(&empty).is_err());
let mut no_id = b.clone();
no_id.series.series_tmdb_id = None;
no_id.series.series_imdb_id = None;
assert!(validate_bundle_envelope(&no_id).is_err());
}
}