dtourolle
b26c66dcce
make_figures.py gains the DE-landscape, calibration, and per-film leave-one-out holdout figures used by the experiment log. regen_frame_examples.sh replays all nine films with the shipped learned-boundary flood config and draws GT-aware TP/FP/FN frames, so every annotated image in the docs is reproducible.
experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT /tmp scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
Layout
xray/— Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).dumps/— per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). Naming:<model>/dump_<Film>.h5. Regenerate withscene_analyze --dump-embeddings.galleries/— per-model galleries (gitignored JSON).gallery_<model>.json+ augmented variants. Regenerate with build_gallery / fetch_missing_actors.manifests/— film manifests (committed — small, and the Jellyfin ID join is the authoritative record of which films/paths/X-Ray-dirs were used).trajectories/— DE trajectories, one JSONL per run (committed — the evidence).results/— final per-run metrics + the model comparison table (committed).
Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? Method: optimize per model — each model gets its own dumps + gallery + full DE run, then compare each model at ITS OWN optimum (fairest — no model penalised by another's threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
|---|---|---|---|
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian.
Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a film he's not in) vs. matching against the whole 2418-actor gallery?
- full — match against the entire model gallery (2418 actors).
- restricted — per film, match only against its Jellyfin credited cast, filtered from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
LIMITATION — Jellyfin stores only ~15 actors per title. Jellyfin's People list is capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item /Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 top-billed leads, which caps its achievable recall at whatever fraction of on-screen actors are top-billed, but should drive out-of-cast misIDs toward zero. A production deployment wanting fuller cast restriction would need a richer cast source than Jellyfin (TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline galleries (no augmentation — avoids test-set leakage on either arm). Scored by the duration-weighted per-scene metric with the misID split (report_rates.py).