# experiments/ — X-Ray validation & optimizer artifacts

Durable home (in the repo tree, NOT `/tmp` scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.

## Layout
- `xray/` — Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).
- `dumps/` — per-model embedding dumps, one HDF5 per (model, film). Gitignored (large).
  Naming: `<model>/dump_<Film>.h5`. Regenerate with `scene_analyze --dump-embeddings`.
- `galleries/` — per-model galleries (gitignored JSON). `gallery_<model>.json` +
  augmented variants. Regenerate with build_gallery / fetch_missing_actors.
- `manifests/` — film manifests (committed — small, and the Jellyfin ID join is the
  authoritative record of which films/paths/X-Ray-dirs were used).
- `trajectories/` — DE trajectories, one JSONL per run (committed — the evidence).
- `results/` — final per-run metrics + the model comparison table (committed).

## Embedding-model bake-off (July 2026)

Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest?
Method: **optimize per model** — each model gets its own dumps + gallery + full DE run,
then compare each model at ITS OWN optimum (fairest — no model penalised by another's
threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see
docs/optimizer-experiments.md).

Models:
| model | file | size | MovieNet rank-1 (prior) |
| ----- | ---- | ---- | ----------------------- |
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50  | models/arcface_w600k_r50.onnx  | 174 MB | 85.2% |
| ArcFace R18        | models/arcface_r18.onnx        | 48 MB  | 72.2% |
| ArcFace w600k MBF  | models/arcface_w600k_mbf.onnx  | 13 MB  | 83.3% |

9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society,
Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface,
Sound of Metal, Valerian.

## Gallery-mode bake-off (full vs cast-restricted)

Second axis alongside the model comparison: does restricting the matcher's candidate
set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a
film he's not in) vs. matching against the whole 2418-actor gallery?

- **full** — match against the entire model gallery (2418 actors).
- **restricted** — per film, match only against its Jellyfin credited cast, filtered
  from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.

**LIMITATION — Jellyfin stores only ~15 actors per title.** Jellyfin's People list is
capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface:
Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a
query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item
/Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15
top-billed leads, which caps its achievable recall at whatever fraction of on-screen
actors are top-billed, but should drive out-of-cast misIDs toward zero. A production
deployment wanting fuller cast restriction would need a richer cast source than Jellyfin
(TMDB/IMDb full credits).

Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline
galleries (no augmentation — avoids test-set leakage on either arm). Scored by the
duration-weighted per-scene metric with the misID split (report_rates.py).