docs: SuperHero benchmark — how to reproduce it and what it scores

Records the reference film end to end: fetching the annotations and mugshots,
fusing the scene clips into one stream, building the gallery at the 66 px face
floor, and the measured result (precision 1.00, recall 0.65, F1 0.79).

Three things are written down because each cost time to discover:

- Why Bali was withdrawn. Its reference crops have a median detected face of
  27 px against a 69 px maximum, so every reference was upscaled past what the
  embedder was trained for (AR-011). No threshold fixed it — at 66 px, 2 of 69
  references survived. Any accuracy figure recorded against Bali measures
  upscaling artifacts as much as the pipeline.
- Run it as one film. Per-scene clips defeat per-film gallery expansion
  (AR-019) and pay model load ten times over.
- Check you are on the GPU. ORT's CUDA provider fails to load here and falls
  back to CPU silently, so a build-ort timing is a CPU number wearing a GPU
  label — a 15x error whose only symptom is a number with no baseline.

TRACES: AR-011, AR-019 | VR-001, VR-005 | SR-002
This commit is contained in:
2026-08-04 14:32:33 +02:00
parent 960a7c4eed
commit 22758da118
2 changed files with 187 additions and 0 deletions
+1
View File
@@ -35,6 +35,7 @@ extra_css:
nav:
- Home: index.md
- How We Score Against X-Ray: methodology.md
- Benchmark — SuperHero: benchmark.md
- Findings:
- Best Model: best-model.md
- Gallery Scope (Full vs. Limited): gallery-scope.md