dtourolle
f891e579c5
The parser reads a tag up to end of line, so `# TRACES: GR-004 | SR-001 — prose` swallowed the prose into the tag and the row went unmatched. Splitting the comment leaves the tag greppable by the same pattern as the code tags and the commit trailers, which is the point of the house format. Mechanical throughout; no logic touched. The regenerated report reflects this session's new tags: 137 -> 148 found, and one more tagged-but-unexecuted, which is the SuperHero accuracy assertion that is documented but not yet a test.
experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT /tmp scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
Layout
xray/— Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).dumps/— per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). Naming:<model>/dump_<Film>.h5. Regenerate withscene_analyze --dump-embeddings.galleries/— per-model galleries (gitignored JSON).gallery_<model>.json+ augmented variants. Regenerate with build_gallery / fetch_missing_actors.manifests/— film manifests (committed — small, and the Jellyfin ID join is the authoritative record of which films/paths/X-Ray-dirs were used).trajectories/— DE trajectories, one JSONL per run (committed — the evidence).results/— final per-run metrics + the model comparison table (committed).
Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? Method: optimize per model — each model gets its own dumps + gallery + full DE run, then compare each model at ITS OWN optimum (fairest — no model penalised by another's threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
|---|---|---|---|
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian.
Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a film he's not in) vs. matching against the whole 2418-actor gallery?
- full — match against the entire model gallery (2418 actors).
- restricted — per film, match only against its Jellyfin credited cast, filtered from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
LIMITATION — Jellyfin stores only ~15 actors per title. Jellyfin's People list is capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item /Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 top-billed leads, which caps its achievable recall at whatever fraction of on-screen actors are top-billed, but should drive out-of-cast misIDs toward zero. A production deployment wanting fuller cast restriction would need a richer cast source than Jellyfin (TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline galleries (no augmentation — avoids test-set leakage on either arm). Scored by the duration-weighted per-scene metric with the misID split (report_rates.py).