dtourolleandClaude Opus 5
d81fc59824
Gallery from one recording, probes from another, sweeping the probe's input resolution end to end. VR-005 asked the same question over gallery mugshots but degraded an already-aligned 112x112 crop with alignment held perfect, so it isolates the embedder. Here the whole frame is downscaled before the detector, so detection and landmark regression degrade with it — which is most of the difference. Corpus is two 4096x2160 clips of one shoot, four people, hand-sorted. Ground truth is sorted by hand and gated by verify_labels.py; labels carried down the scales geometrically by box position, never by embedding similarity, which would keep only the faces the embedder already gets right and drop the ones the sweep exists to find. Findings, all scored through the production gallery sigmoid at prob_threshold 0.754 — never a raw cosine: - Holding 90% of the plateau needs ~50 px end to end, against VR-005's ~22 px. min_face_px at 40 looks right; 32 would admit faces in the falling region. - FPI is 0.0% at every scale. Resolution loss goes entirely to TBI. - The ceiling is cross-view, not resolution: everyone matches themselves within a recording (0.55-0.85) and collapses across two (0.14-0.45, threshold 0.335). Only the subject with frontal *gallery* references identified reliably, whatever their probe pose — so the lever is gallery pose coverage, not a better landmark source. - Averaging SCRFD's overlapping detections instead of discarding them at NMS lifts cross-recording TPI 41% -> 49%, for one forward pass and no extra model. Four identities and one shoot, so the shape is the result and the absolute rates are not. Both clips contain all four people, so there is no out-of-gallery class and the 10x-weighted out-of-cast misID is untested here. Clips, frames, hand-sorted crops and results are gitignored and belong in the artifact registry — the sorting is human ground truth and expensive to redo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: VR-013 | AR-002, AR-005, AR-024
experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT /tmp scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
Layout
xray/— Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).dumps/— per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). Naming:<model>/dump_<Film>.h5. Regenerate withscene_analyze --dump-embeddings.galleries/— per-model galleries (gitignored JSON).gallery_<model>.json+ augmented variants. Regenerate with build_gallery / fetch_missing_actors.manifests/— film manifests (committed — small, and the Jellyfin ID join is the authoritative record of which films/paths/X-Ray-dirs were used).trajectories/— DE trajectories, one JSONL per run (committed — the evidence).results/— final per-run metrics + the model comparison table (committed).
Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? Method: optimize per model — each model gets its own dumps + gallery + full DE run, then compare each model at ITS OWN optimum (fairest — no model penalised by another's threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
|---|---|---|---|
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian.
Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a film he's not in) vs. matching against the whole 2418-actor gallery?
- full — match against the entire model gallery (2418 actors).
- restricted — per film, match only against its Jellyfin credited cast, filtered from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
LIMITATION — Jellyfin stores only ~15 actors per title. Jellyfin's People list is capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item /Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 top-billed leads, which caps its achievable recall at whatever fraction of on-screen actors are top-billed, but should drive out-of-cast misIDs toward zero. A production deployment wanting fuller cast restriction would need a richer cast source than Jellyfin (TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline galleries (no augmentation — avoids test-set leakage on either arm). Scored by the duration-weighted per-scene metric with the misID split (report_rates.py).