dtourolleandClaude Opus 5
d3ab598434
The cross-source study needs two 4K recordings and a hand-sorted set of face crops, neither of which belongs in git. Adds an xsource target to both artifact scripts. Push uploads the clips as-is (already compressed) and zips labelling/. Pull fetches both and regenerates frames with ffmpeg rather than downloading them: ~320 MB of PNG that is deterministic from the clips. The extraction settings are pinned in the script, not left to the caller, because the manifests key on frame filenames and on detection order within each frame — verify_labels.py runs afterwards and fails loudly if they drift. Pull refuses to overwrite an existing labelling/. It is human ground truth: somebody looked at 167 crops and placed each one, and silently replacing that with a remote copy would destroy the expensive half of the study. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: VR-013
experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT /tmp scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
Layout
xray/— Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).dumps/— per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). Naming:<model>/dump_<Film>.h5. Regenerate withscene_analyze --dump-embeddings.galleries/— per-model galleries (gitignored JSON).gallery_<model>.json+ augmented variants. Regenerate with build_gallery / fetch_missing_actors.manifests/— film manifests (committed — small, and the Jellyfin ID join is the authoritative record of which films/paths/X-Ray-dirs were used).trajectories/— DE trajectories, one JSONL per run (committed — the evidence).results/— final per-run metrics + the model comparison table (committed).
Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? Method: optimize per model — each model gets its own dumps + gallery + full DE run, then compare each model at ITS OWN optimum (fairest — no model penalised by another's threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
|---|---|---|---|
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian.
Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a film he's not in) vs. matching against the whole 2418-actor gallery?
- full — match against the entire model gallery (2418 actors).
- restricted — per film, match only against its Jellyfin credited cast, filtered from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
LIMITATION — Jellyfin stores only ~15 actors per title. Jellyfin's People list is capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item /Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 top-billed leads, which caps its achievable recall at whatever fraction of on-screen actors are top-billed, but should drive out-of-cast misIDs toward zero. A production deployment wanting fuller cast restriction would need a richer cast source than Jellyfin (TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline galleries (no augmentation — avoids test-set leakage on either arm). Scored by the duration-weighted per-scene metric with the misID split (report_rates.py).