refactor(bench): SuperHero replaces Road to Bali as the reference film

Bali was chosen because the TRECVID DVU set ships character mugshots, but
its reference crops are unusable at scale: median detected face 27 px
against a 69 px maximum, so every reference was upscaled 4x or more past
what the embedder was trained for (AR-011). A 66 px floor left 2 of 69
references; no threshold exists that both keeps the faces in distribution
and leaves enough of them to calibrate.

SuperHero is 69 px median and 241 px max. Its gallery builds at a 66 px
floor with 14 references over 5 characters, and calibrates on its own
(a=15.2867 b=-4.98633, 100% train accuracy) instead of borrowing constants.

Measured on the fused 17-minute film, one stream rather than per-scene
clips so presence windows cross real scene boundaries as SR-002 intends:
precision 1.00, recall 0.65, F1 0.79 — 13 true positives, 0 false
positives, 7 misses. Every out-of-gallery character was declined rather
than forced onto a nearest match. The misses are the short scenes (14 s,
38 s, 27 s), consistent with per-track accumulation needing sightings.

- build_gallery gains --min-face-px, filtering the *detected face* rather
  than the crop. The DVU images are scene crops, not mugshots, so crop
  dimensions say nothing about face scale. A poisoned reference is
  permanent in a way a bad frame is not: it corrupts every future match
  against that identity.
- scripts/fetch_dvu.sh fetches mugshots, scene graphs and segmentation for
  any DVU film. NIST names the same film three different ways, so KG_DIR
  and KG_FILE are overridable rather than derived. This exists as a script
  because the first copy of this data was assembled ad hoc in /tmp and was
  lost with it, taking the working gallery along.
- Replay fixtures move to the artifact registry: push/pull_artifacts.sh
  gain a replay-fixtures target, and tests/fixtures/dumps/.gitignore keeps
  them out of git. superhero.h5 is ~9 MB and regenerating it needs the
  film, the models and a GPU — none of which CI has. The gallery ships
  with the dumps, since a dump only replays against the gallery it was
  produced with.
- AR-012 and AR-013 coverage is ported onto the new fixture rather than
  dropped with the Bali cases: 12369 assertions, up from 7991, since the
  film is an order of magnitude larger than the clips.

Suite: 15679 assertions, 101 test cases.

TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
This commit is contained in:
2026-08-04 13:49:11 +02:00
parent eff696b49a
commit d98dc2855a
14 changed files with 244 additions and 87 deletions
+2 -2
View File
@@ -363,8 +363,8 @@ one identity out of the gallery would fix that.
thing measured is the thing that ships.
`scripts/validation/test_audio_offset.py` over
`tests/fixtures/audio/bali_offset_200s.flac`: 200 s of public-domain film audio
(the same Road to Bali clips the replay fixtures use), long enough for a 120 s
`tests/fixtures/audio/superhero_offset_200s.flac`: 200 s of public-domain film audio
(the same SuperHero clips the replay fixtures use), long enough for a 120 s
window to slide past the ±600-frame search cap. The slide itself is numpy here
on purpose — matching belongs to the consumer, so writing it out keeps this a
test of the signature rather than of somebody's matcher.
+2 -2
View File
@@ -231,9 +231,9 @@ time**, and CI consumes the result as data.
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
T2 are the only tiers that can exist in CI at all.
### Fixture corpus — `bali/`
### Fixture corpus — `hero/`
Five clips of **Road to Bali (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Five clips of **SuperHero (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Public domain, and that is the reason to use it rather than a convenience:
**derived fixtures — dumps, crops, golden outputs — can be committed without the