AR-018 — an embedding joins a track's store only if its similarity to something already there falls inside a band, rather than merely being far from the gallery. Above the upper bound it is redundant: another look at a pose the store already covers, teaching the annex nothing while costing a slot a novel view could have used. Below the lower bound it is suspect: within one track every face is the same person by construction, so an embedding unlike everything else on the track is evidence that construction failed — a track-ID collision or a bad detection. Admitting it is exactly how an actor's annex gets poisoned with someone else's face. The old gate had only the upper half of that idea, expressed as a raw cosine against the gallery. Both bounds are now calibrated probabilities (AR-024), so the same number means the same thing here as in association and evidence weighting rather than three different things. This catches track-ID collisions EARLIER than the spread gate did — at the door rather than at promotion — so the buffer never becomes two-person in the first place. The spread gate stays as a second line for a track that drifts gradually instead of jumping. The existing test was asserting the mechanism rather than the outcome, so it was rewritten to assert what actually matters: whichever gate fires, the outsider must not reach the annex. Rejections are counted. A store that admits nothing is as broken as one that admits everything, and neither is visible otherwise. Band defaults 0.90-0.95 are working values pending VR-007; the two bounds fail in opposite directions and must be swept separately. Suite: 92 cases, 6133 assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-018, AR-024 | SR-005
379 lines
28 KiB
Markdown
379 lines
28 KiB
Markdown
# scene-actor-extraction — requirements register
|
||
|
||
Stable IDs for every requirement in [`SPEC.md`](SPEC.md), which holds the prose.
|
||
This file is the **authoritative list**; the CI gate reads its denominators from
|
||
here (see [`../../SPEC.md`](../../SPEC.md) §6).
|
||
|
||
**IDs are permanent.** A withdrawn requirement is marked `Withdrawn` and its
|
||
number is never reused — renumbering is what produces orphan TRACES tags. This
|
||
register replaces the earlier thematic `A1…E8` scheme, which had already produced
|
||
an `A1a` and an out-of-order `E6`.
|
||
|
||
Tag code with `// TRACES: AR-012 | SR-002`.
|
||
|
||
| Type | Scope |
|
||
|---|---|
|
||
| `AR` | Algorithm — the extraction pipeline itself |
|
||
| `DP` | Deployment — how it runs |
|
||
| `IR` | Integration — contracts with other components |
|
||
| `GR` | Gallery — building and maintaining actor references |
|
||
| `VR` | Validation — parameter studies and benchmarks |
|
||
| `UT` / `IT` | Unit / integration tests |
|
||
|
||
Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||
|
||
---
|
||
|
||
## Algorithm (AR)
|
||
|
||
| ID | Requirement | Traces to | Priority | Status |
|
||
|---|---|---|---|---|
|
||
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
|
||
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
|
||
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing |
|
||
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** — KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs |
|
||
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`; the RANSAC fit it replaces was RNG-driven and point-dropping. Galleries baked before the change want a rebuild |
|
||
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
|
||
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
|
||
| AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted |
|
||
| AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done |
|
||
| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Done** — decode butterfly joined via `SceneBoundaries`; the sampled branch waits for the detector's watermark. Frames past its last scored window are counted as unverified, never assumed boundary-free |
|
||
| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned |
|
||
| AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done** — `src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track |
|
||
| AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done** — `last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed |
|
||
| AR-014 | Belief swap A→B terminates the track and starts a new one | SR-002 | Medium | **Done** — swap closes at `last_seen` and opens a successor at the swap frame; counted |
|
||
| AR-015 | Two live tracks owned by one actor ⇒ treat as a detected cut, re-associate | SR-002 | Medium | **Done** — reverse index detects it on the causing update; counted |
|
||
| AR-016 | All tracks closed at EOF — a film ends with faces on screen | SR-002 | High | **Done** — `flush()`, idempotent, closes at last sighting or final tick |
|
||
| AR-017 | Every presence claim carries its belief and identification route | SR-002 | High | **Done** — `DeadTrack` carries belief and observation count |
|
||
| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space; replaces `expand_novelty_sim`. Rejections counted |
|
||
| AR-019 | Per-film gallery annex from owned tracks; acquires the non-frontal views TMDB lacks | SR-005 | Medium | In Progress |
|
||
| AR-020 | Deferred re-identification of unknown tracks against the final expanded gallery | SR-005 | High | Planned |
|
||
| AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned |
|
||
| AR-022 | Capture still-unidentified tracks: embeddings, metadata, **context crops** for human review | §4 | Medium | Planned |
|
||
| AR-023 | Fit sigmoid calibration from intra/inter similarity distributions | SR-002 | High | Done |
|
||
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association and accumulation both in probability space; `track_max_embed_dist`, `cut_revive_sim` retired |
|
||
| AR-025 | Per-track Bayesian accumulation in log-odds, with correlated-observation discounting | SR-002 | High | **Done** — log-odds accumulation with correlation discounting owned by the registry, `src/evidence_discount.hpp` |
|
||
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
|
||
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
|
||
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned |
|
||
| AR-029 | Sharpness measure on the **aligned crop** (scale-normalised, so it cannot re-measure size) | SR-002 | Medium | Planned |
|
||
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
|
||
|
||
## Deployment (DP)
|
||
|
||
| ID | Requirement | Traces to | Priority | Status |
|
||
|---|---|---|---|---|
|
||
| DP-001 | One analysis core; modes are front-ends and must not fork pipeline logic | PR-004 | High | Done |
|
||
| DP-002 | Batch CLI over one title | PR-004 | High | Done |
|
||
| DP-003 | On-demand resident service with bounded, observable queue | PR-004 | Medium | Planned |
|
||
| DP-004 | Opportunistic/idle mode: external trigger, hard stop, implicit re-queue | PR-004 | Medium | Planned |
|
||
| DP-005 | Native installer, no Docker; Fedora + Arch | PR-004 | Medium | Planned |
|
||
| DP-006 | Background incremental gallery refresh on a timer | PR-003 | Medium | Planned |
|
||
| DP-007 | CI builder image, CPU-only, pinned by tag in the Gitea container registry | PR-004 | High | Planned |
|
||
| DP-008 | Builder images + release jobs per backend (cpu / cuda / rocm); ship binaries, not engines | PR-004 | Medium | Planned |
|
||
|
||
## Integration (IR)
|
||
|
||
| ID | Requirement | Traces to | Priority | Status |
|
||
|---|---|---|---|---|
|
||
| IR-001 | Emit the JRay truth format as sibling `.jray.json` | SR-003 | High | Done |
|
||
| IR-002 | Windows carry belief + route; `extraction.*` carries `extinction_sec`, `gallery_scope` | SR-003 | High | **Done** — `schema_version: 2`; windows are objects with `belief` + `route`; `extraction.*` carries `extinction_sec` and `gallery_scope`; `anneal_sec` removed |
|
||
| IR-003 | Output written **after** the deferred pass, not at EOF | SR-003 | High | **In Progress** — sink builds windows from registry claims and flushes at EOF; the deferred pass (AR-020) does not exist yet, so output is still final at EOF |
|
||
| IR-004 | Compute the audio signature exactly per server spec §3 | SR-003 | Medium | **Done** — `src/audio_signature.*`; not yet emitted into the truth file (IR-002) |
|
||
| IR-005 | Golden-vector fixture shared with the plugin repo to prove bit-exactness | SR-003 | High | **Done** — `tests/fixtures/audio/`; v1 parameters now normative in server spec §3 |
|
||
| IR-007 | Media < 120 s: emit no signature, apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** |
|
||
| IR-008 | Emit and honour the signature's own `v1:` version prefix | SR-003 | Low | **Done** |
|
||
| IR-006 | Jellyfin round-trip: pull pending queue, push complete results only | SR-001 | High | Done |
|
||
|
||
## Gallery (GR)
|
||
|
||
| ID | Requirement | Traces to | Priority | Status |
|
||
|---|---|---|---|---|
|
||
| GR-001 | Build gallery from Jellyfin library cast, TMDB profile fallback | SR-001, SR-005 | High | Done |
|
||
| GR-002 | Incremental `--merge` refresh without re-embedding known actors | PR-003 | High | Done |
|
||
| GR-003 | Report coverage: zero-image actors, under-referenced actors, dedup, calibration PDFs | SR-001 | Medium | Planned |
|
||
| GR-004 | Stamp embedder identity into the gallery; **hard startup error** on mismatch | SR-001 | High | **Done** — basename + SHA-256 + `embed_dim`; mismatch fatal with no bypass, unstamped warns unless `--require-gallery-stamp`; `scripts/stamp_gallery.py` migrates in place |
|
||
| GR-005 | Gallery data never leaves the instance | **SR-005** | High | Done |
|
||
| GR-006 | Provenance tiers: baked / harvested / confirmed, distinguishable per embedding | SR-005 | High | Planned |
|
||
| GR-007 | Persist harvested embeddings **flagged and reviewable**, never silently equal to baked | SR-005 | Medium | Planned |
|
||
| GR-008 | Flag distributional outliers among an actor's references (poisoning guard) — `EXCEPTION: AR-024` | SR-005 | Medium | Planned |
|
||
| GR-009 | Human-confirmed associations persist and improve future extractions | §4 | Medium | TBD |
|
||
|
||
## Validation (VR)
|
||
|
||
| ID | Requirement | Traces to | Priority | Status |
|
||
|---|---|---|---|---|
|
||
| VR-001 | HDF5 post-inference dump at the embedded-frame boundary | PR-002 | High | Done |
|
||
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | **Done** — replay driven from committed fixtures in `tests/test_replay_fixtures.cpp`; determinism asserted |
|
||
| VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done |
|
||
| VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done |
|
||
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 24–32 px; 32 px gives 98.1% TPI, 0.0 FPI at every size |
|
||
| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned |
|
||
| VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned |
|
||
| VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned |
|
||
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
|
||
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | Planned |
|
||
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | Planned |
|
||
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | Planned |
|
||
|
||
---
|
||
|
||
## Verification strategy
|
||
|
||
**CI runs on an Intel N100 with no discrete GPU.** That is a hard constraint on
|
||
how each requirement can be verified, and it shapes the test design rather than
|
||
merely limiting it.
|
||
|
||
Four tiers, in decreasing order of preference:
|
||
|
||
| Tier | Runs in CI | What it covers |
|
||
|---|---|---|
|
||
| **T1 — Functor unit** | Yes | A KPN node's `operator()` driven directly with hand-built inputs |
|
||
| **T2 — Replay** | Yes | The composed pipeline driven from an HDF5 fixture — no GPU, no video |
|
||
| **T3 — CPU inference** | Yes, slowly | ORT CPU provider over a handful of frames; smoke tests only |
|
||
| **T4 — GPU** | **No** | Throughput, TRT engines, large-gallery GEMM |
|
||
|
||
### T1 is the primary tier, and KPN is why
|
||
|
||
**Node functors are plain callable structs, constructed independently of the
|
||
network that wraps them** (`main.cpp:186-207` builds them as stack objects;
|
||
`ObjectNode` merely adapts them). So a node is testable by constructing it and
|
||
calling `operator()` — no channels, no threads, no network, no fixture.
|
||
|
||
This is already the established pattern, not a proposal:
|
||
`tests/test_face_tracker.cpp` "drives the node's `operator()` with hand-built
|
||
`EmbeddedSceneFrame`s and inspects the emitted `track_ids`", and does so
|
||
"pure, GPU-free, model-free".
|
||
|
||
The consequence is that most of the redesign is verifiable **without any
|
||
fixture at all**: construct exactly the awkward state — a belief swap, two live
|
||
tracks converging on one actor, a film ending mid-track, a gap one frame under
|
||
the timeout — rather than hunting for a clip that happens to exhibit it.
|
||
|
||
Four hazards this removes outright:
|
||
|
||
- **No fixture-provenance risk** for these tests — the inputs are synthetic and
|
||
explicit.
|
||
- **No "fixture must be replayed from frame 0"** concern — state is constructed
|
||
directly.
|
||
- **No cross-test state leakage** (e.g. a tracker's `next_id_` persisting) — each
|
||
test constructs a fresh functor.
|
||
- **No replay-harness nondeterminism** — no channels, so no EOF-tail heuristics
|
||
or silent drops.
|
||
|
||
It also means **a dead upstream producer does not block testing a downstream
|
||
consumer.** `is_scene_boundary` currently has no producer (see AR-010), which
|
||
would make a *replay* test of the frame-dependent `track_alpha` pass vacuously —
|
||
but a T1 test simply constructs a frame with `is_scene_boundary = true` and
|
||
asserts the weighting changes. The producer gap is a pipeline defect to fix, not
|
||
a verification blocker.
|
||
|
||
### T2 covers what T1 cannot
|
||
|
||
Replay remains necessary for **composition** — that the nodes wired together
|
||
behave as the sum of their parts — and for realistic data at scale, which
|
||
synthetic inputs cannot honestly imitate. It is the tier that would catch a
|
||
wiring error, a channel-capacity problem, or an ordering assumption that only
|
||
appears under concurrency.
|
||
|
||
The HDF5 dump (VR-001) captures state after decode → detect → align → embed, so
|
||
replay needs no GPU and no video. That was built for the optimizer; it doubles as
|
||
CI, which is a strong argument for keeping the schema honest and for replay
|
||
driving the *real* nodes rather than a reimplementation (VR-002).
|
||
|
||
**Fixtures and studies are generated locally**, on the development machine where
|
||
the models, galleries and media already exist. CI consumes them; it never
|
||
produces them.
|
||
|
||
**Small committed fixtures are required.** A few HDF5 dumps covering the awkward
|
||
cases — a cut, a belief swap, two live tracks converging, a film ending
|
||
mid-track, an unknown track that only resolves after expansion — are worth more
|
||
than a large corpus, and they are small enough to commit.
|
||
|
||
**T4 requirements cannot pass in CI, and the gate must not pretend otherwise.**
|
||
For these, CI verifies that a test *exists and is tagged*, not that it passes;
|
||
the run happens on a GPU host, nightly or manually, and reports separately. A
|
||
requirement whose only evidence is a test that never executes should be visible
|
||
as such rather than counted as covered.
|
||
|
||
| Requirement | Tier | Note |
|
||
|---|---|---|
|
||
| AR-001, AR-005, AR-006 | T3 | Smoke only — correctness of detection/embedding is a model property, not ours |
|
||
| AR-002 | Minimum face size **32×32 px** (VR-005 measured), expressed in **original** resolution (decoupled from `dense_scale`) | Size filtering is arithmetic on dumped bboxes |
|
||
| AR-003, AR-004 | T1 + T4 | Backpressure logic is unit-testable; saturation behaviour needs real load |
|
||
| AR-007 … AR-017 | **T2** | The core of the redesign — fully replayable |
|
||
| AR-018 … AR-022 | **T2** | Expansion, deferred pass, clustering: all post-embedding |
|
||
| AR-023 … AR-025 | T1 | Calibration fit and log-odds accumulation are pure maths |
|
||
| AR-026, AR-027 | T4 | GEMM throughput and scaling — GPU host only |
|
||
| DP-* | T1 + manual | Lifecycle logic unit-tested; install paths are manual |
|
||
| IR-001 … IR-003 | T1 | Serialisation against a golden truth file |
|
||
| IR-004, IR-005 | **T1** | Audio signature is CPU DSP — the golden-vector fixture runs anywhere, which is precisely why it is the right cross-repo check |
|
||
| GR-001 … GR-005 | T1 + T3 | Gallery assembly is I/O and bookkeeping; embedding is T3 smoke |
|
||
| GR-006 … GR-008 | T1 | Tiering and outlier detection operate on stored embeddings |
|
||
| VR-* | Out of CI | Studies are run deliberately and their results committed as documents |
|
||
|
||
**One consequence worth stating:** AR-027 (arbitrary gallery scale) is
|
||
structurally unverifiable on the CI host. It needs a GPU host and a synthetic
|
||
large gallery, so it is the requirement most likely to silently regress. Its
|
||
benchmark (VR-008) should run on a schedule rather than on demand.
|
||
|
||
### CI never calls a model
|
||
|
||
**Not "should not" — cannot.** The N100 has no GPU, and even the ONNX Runtime CPU
|
||
provider is impractical: a measured run of the embedder on this hardware sits at
|
||
~930 ms per frame, so a 77 s clip at 5 fps would take roughly six minutes of
|
||
inference alone. Every model invocation therefore happens **locally, ahead of
|
||
time**, and CI consumes the result as data.
|
||
|
||
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
|
||
T2 are the only tiers that can exist in CI at all.
|
||
|
||
### Fixture corpus — `bali/`
|
||
|
||
Five clips of **Road to Bali (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
|
||
|
||
Public domain, and that is the reason to use it rather than a convenience:
|
||
**derived fixtures — dumps, crops, golden outputs — can be committed without the
|
||
rights question that rules out sharing gallery data (SR-005).** A fixture cut
|
||
from a copyrighted title could not live in the repository at all.
|
||
|
||
Two properties to design around rather than discover:
|
||
|
||
- **480×360 means small faces.** At this resolution a face is often 40–80 px, so
|
||
the AR-002 minimum of 66 px (original resolution) rejects much of what is
|
||
there. Fixture generation must set `--min-face-px` explicitly and record it,
|
||
or the dumps will be sparse for reasons unrelated to what is being tested.
|
||
- **77 s is short.** At 1 fps that is 77 frames — too thin to exercise an
|
||
extinction window measured in tens of seconds. Generate at 5 fps (≈385 frames,
|
||
~1 MB) and record the rate in provenance, since the behaviour under test
|
||
changes with it.
|
||
|
||
> **AR-004 blocks reproducible fixture generation.** A trial run of one clip
|
||
> produced 49 frames of an expected ~385, ending at 51 s of 77 s, with the
|
||
> diagnostics reporting 285 frames dropped at `camera_pos` and 51 at
|
||
> `face_aligner`. Channels overflow and **drop** rather than blocking, and what
|
||
> gets dropped depends on timing — so the same command run twice can produce
|
||
> different dumps. Golden fixtures cannot be built on that. AR-004 is therefore
|
||
> a prerequisite for VR-001 fixtures, not merely a throughput concern for crowd
|
||
> scenes.
|
||
|
||
### Fixtures — precomputed inference, pulled by CI
|
||
|
||
The N100 cannot run inference at any useful rate, so **inference output is
|
||
precomputed on a GPU host and consumed by CI as data.** This converts most of
|
||
what looks like GPU work into pure CPU replay.
|
||
|
||
| Fixture | Contents | Size | Storage |
|
||
|---|---|---|---|
|
||
| **Edge-case dumps** | ~6 short clips (30–60 s), one per awkward behaviour | ~0.1–1 MB each | **Committed in-repo** |
|
||
| **Corpus dumps** | Full-length titles from the validation corpus | ~21–38 MB each | **Gitea package registry**, pinned by version + checksum |
|
||
| **Synthetic gallery** | Random unit-norm embeddings, fixed seed | small | Generated at test time |
|
||
| **Golden truth files** | Expected output for each edge-case dump | KB | Committed |
|
||
| **Audio golden vectors** | FLAC + expected signature + parameter contract | ~600 KB | Committed, **shared with the plugin repo** |
|
||
|
||
Edge-case dumps are small enough to commit, and being in-repo means they version
|
||
with the code that reads them.
|
||
|
||
**Corpus dumps go to the Gitea package registry, not Git LFS.** Both are
|
||
available — the models already use LFS — but their fetch semantics differ in a
|
||
way that matters here. LFS objects are pulled on clone unless a developer
|
||
explicitly skips them, so ~38 MB per title behind LFS taxes everyone who clones,
|
||
forever, for data that only CI and the optimizer ever read. Registry artifacts
|
||
are fetched on demand by the job that needs them.
|
||
|
||
Rule of thumb: **LFS for what the build needs; the package registry for what a
|
||
particular job needs.** Models are the former; corpus dumps and the CI image
|
||
(DP-007) are the latter.
|
||
|
||
Pin by version and verify by checksum on fetch. A fixture that changes silently
|
||
under CI is worse than a missing one, because the failure presents as a code
|
||
regression.
|
||
|
||
**Generation must be reproducible and versioned.** A script, run on a GPU host,
|
||
regenerates every fixture from source clips; it is re-run when the VR-001 schema
|
||
version bumps. A fixture whose provenance is unknown is worse than no fixture,
|
||
because it will be trusted.
|
||
|
||
> **The limitation that must stay visible:** replay fixtures freeze upstream
|
||
> behaviour. A test driven from a dump verifies AR-007 onward *given those
|
||
> embeddings* — it cannot detect a regression in detection, alignment or
|
||
> embedding, because those produced the fixture. Nothing in CI can. That gap is
|
||
> covered only by the T3 smoke test and the scheduled GPU run, and it should not
|
||
> be papered over by a high replay-coverage number.
|
||
|
||
### Per-requirement verification plan
|
||
|
||
| ID | Tier | Test asserts | Edge cases to cover |
|
||
|---|---|---|---|
|
||
| AR-001 | T3 | Detector returns plausible boxes on a known frame | — smoke only |
|
||
| AR-002 | T2 | Faces below 32 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement |
|
||
| AR-003 | T2 | No cap applied; a 40-face frame yields 40 | Crowd frame |
|
||
| AR-004 | T1 | Saturated input blocks rather than drops or throws | Bounded queue at capacity; **byte-based** limit with large crops; SIGTERM mid-block |
|
||
| AR-005 | T1 | Known landmarks → expected 112×112 warp; the fit never mirrors | Landmarks near frame edge; degenerate/collinear points; a mirrored set — SVD returns a reflection unless the determinant guard rejects it |
|
||
| AR-006 | T3 | Embeddings are unit-norm | Batch smaller than, equal to, larger than `embed_batch_size` |
|
||
| AR-007 | T2 | Association picks the right track | Two faces crossing paths; one leaving frame as another enters |
|
||
| AR-008 | T2 | One pool; dormant tracks match on embedding, not IoU | Dormant track whose old bbox overlaps a *different* new face — must not match on position |
|
||
| AR-009/010 | T2 | Cut/boundary shifts weighting toward embedding | Cut with same people; cut with all-new people |
|
||
| AR-011 | T1 | TransNetV2 receives native-rate frames | Source at 24/25/30 fps — dedup window derived, not assumed |
|
||
| AR-012 | **T2** | Window spans full track extent, not first recognition | Actor recognised only at track end — window must still start at `first_seen` |
|
||
| AR-013 | **T2** | `last_seen` set/unset; window ends at last sighting | Gap just under vs just over timeout; reappearance after timeout → two windows |
|
||
| AR-014 | T2 | Belief swap closes one window, opens another | No blended window; no overlap at the swap frame |
|
||
| AR-015 | T2 | Two live tracks on one actor trigger re-association | Counter increments |
|
||
| AR-016 | **T2** | Every track closed at EOF | Film ending mid-shot — window ends at final frame, not dropped |
|
||
| AR-017 | T1 | Claim carries posterior and route | Deferred and pooled routes distinguishable |
|
||
| AR-018 | T1 | Band admits only within bounds | At each bound exactly; store never admits below lower bound |
|
||
| AR-019 | T2 | Promotion only when all three signals quiet | Cut mid-track blocks promotion |
|
||
| AR-020 | **T2** | Unknown resolved after expansion | Track failing at minute 12, resolved at EOF — the ordering-independence claim |
|
||
| AR-021 | T2 | Clustering merges same person, respects cannot-link | **Temporally overlapping tracks never merge**; measure how many merges the constraint rejects |
|
||
| AR-022 | T1 | Context crops retained, bounded per track | Track running for minutes |
|
||
| AR-023 | T1 | Sigmoid fit on synthetic separable data | Too few positive pairs → `valid=false`, fallback engages |
|
||
| AR-024 | **Static check** | No bare cosine outside a tagged `EXCEPTION` | Grep-based; this is the invariant's enforcement |
|
||
| AR-025 | T1 | Log-odds accumulate; correlated frames discounted | 30 identical frames must **not** reach the certainty of 30 diverse ones |
|
||
| AR-026 | T1 + T4 | GEMM path produces same result as reference loop | Equivalence on small input in CI; throughput on GPU host |
|
||
| AR-027 | **T4** | Throughput at 10²…10⁵ actors | Scheduled, not on-demand |
|
||
| AR-028 | **T2** | No embedding reaches the matcher unscored; the vector survives into the dump | Face failing exactly one axis; all three healthy; a face whose landmarks are degenerate — scored, not silently vanished |
|
||
| AR-029 | T1 | Synthetic blur ladder → monotonically falling sharpness | Gaussian vs motion blur; **small sharp face vs large soft one** — size must not leak into this axis |
|
||
| AR-030 | T1 | Alignment residual rises monotonically with foreshortening | **In-plane roll, scale and translation must leave it at zero** — the property that makes it a pose measure rather than a pose-and-everything-else measure; face size must not shift it; degenerate landmarks report not-ok rather than a number |
|
||
| VR-012 | **T4** | Knee located per axis on held-out films | Report each candidate threshold's cost in **lost true presence**, not only its gain in precision — a gate that improves misID by discarding half the cast has not helped |
|
||
| IR-001/002 | T1 | Serialised output matches golden file | Zero-length window; actor with many windows |
|
||
| IR-003 | T1 | Output written after deferred pass | Not at EOF |
|
||
| IR-004/005 | **T1** | Signature matches golden vector bit-for-bit | Identical result in both producer repos |
|
||
| IR-006 | T1 + manual | Queue pull and result push against a stubbed Jellyfin API | Partial result never pushed; push only after the deferred pass |
|
||
| IR-007 | **T1** | Media < 120 s emits no signature at all | Exactly 120 s; just under; zero-length audio. Must match the plugin's cutoff exactly — a caller-varying window length is what SR-004 forbids |
|
||
| IR-008 | T1 | `v1:` prefix emitted and honoured on read | Unknown prefix rejected, not guessed |
|
||
| GR-009 | T1 | Human-confirmed associations persist and are tier-tagged | Survives a gallery rebuild; distinguishable from baked and harvested |
|
||
| GR-004 | T1 | Mismatched embedder → hard startup error | Error names both sides; **unstamped warns, and errors under `SAE_REQUIRE_GALLERY_STAMP`**; same filename + different SHA-256 must still be a mismatch |
|
||
| GR-008 | T1 | Outlier flagged among an actor's references | Injected poisoned embedding detected |
|
||
| VR-009 | T1 | Posterior calibration holds | A 0.99 posterior is wrong ~1% of the time on held-out tracks |
|
||
|
||
Three of these are worth singling out because they verify claims that would
|
||
otherwise be assertions: **AR-012** (window starts at `first_seen` even when
|
||
recognition comes late) is the entire point of the redesign; **AR-020** (a track
|
||
failing mid-film resolves at EOF) is the claim that ordering stops mattering; and
|
||
**AR-025** (30 identical frames ≠ 30 diverse ones) is what stops the Bayesian
|
||
accumulation from being decoration.
|
||
|
||
---
|
||
|
||
## Withdrawn
|
||
|
||
| ID | Requirement | Reason |
|
||
|---|---|---|
|
||
| — | `anneal_sec` window merging | Superseded by AR-012/AR-013: a track survives its own gaps, so there is nothing to anneal |
|
||
| — | `extinction_sec` actor keep-alive | Superseded by AR-013: windows end at last sighting, which is what this over-claimed |
|
||
|
||
Both were deleted rather than retained at zero — a field naming a mechanism the
|
||
pipeline no longer has is actively misleading (see `SPEC.md` A6.6).
|
||
|
||
---
|
||
|
||
## Notes on coverage
|
||
|
||
- **VR-*** traces to PR-002 (scene-granularity answers) rather than to a system
|
||
requirement: parameter studies are single-repo work serving accuracy, and this
|
||
is correct rather than a gap.
|
||
- **PR-005** (leak nothing) has no `AR`/`DP` row. It is satisfied *structurally*
|
||
by SR-004 and GR-005 — the server holds no binary, the gallery never leaves the
|
||
instance — not by any component doing something. It cannot be verified by
|
||
pointing at code, and it dies the moment either prohibition is relaxed.
|