dtourolle
d340da755a
docs/rep4-optimizer-results.md is the main deliverable: the model bake-off + threshold re-tune experiment log, including the ROCm teardown deadlock root cause and fix, DE concurrency tuning, the 16-combo results table, held-out validation against 5 films never seen by the optimizer (macro F1 67.4% vs. 75.3% training — a real generalization gap), the frozen-bbox "ghost track" failure mode found via annotated frame evidence, calibration curves per model, and an isolated-effects breakdown of gallery scope vs. pose expansion. MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/ pulls referenced images from the artifact registry and generates the calibration chart at build time (see the tooling commit) rather than committing images to the repo. experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories, manifests, results) moved to the Gitea package registry. film-lut.template.json is the committed placeholder for the gitignored file-lut.json (real local movie paths, never shared — some source filenames carry scene-release tags). Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for the new scene-detection path.
experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT /tmp scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
Layout
xray/— Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).dumps/— per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). Naming:<model>/dump_<Film>.h5. Regenerate withscene_analyze --dump-embeddings.galleries/— per-model galleries (gitignored JSON).gallery_<model>.json+ augmented variants. Regenerate with build_gallery / fetch_missing_actors.manifests/— film manifests (committed — small, and the Jellyfin ID join is the authoritative record of which films/paths/X-Ray-dirs were used).trajectories/— DE trajectories, one JSONL per run (committed — the evidence).results/— final per-run metrics + the model comparison table (committed).
Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? Method: optimize per model — each model gets its own dumps + gallery + full DE run, then compare each model at ITS OWN optimum (fairest — no model penalised by another's threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
|---|---|---|---|
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian.
Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a film he's not in) vs. matching against the whole 2418-actor gallery?
- full — match against the entire model gallery (2418 actors).
- restricted — per film, match only against its Jellyfin credited cast, filtered from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
LIMITATION — Jellyfin stores only ~15 actors per title. Jellyfin's People list is capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item /Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 top-billed leads, which caps its achievable recall at whatever fraction of on-screen actors are top-billed, but should drive out-of-cast misIDs toward zero. A production deployment wanting fuller cast restriction would need a richer cast source than Jellyfin (TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline galleries (no augmentation — avoids test-set leakage on either arm). Scored by the duration-weighted per-scene metric with the misID split (report_rates.py).