refactor(bench): SuperHero replaces Road to Bali as the reference film

Bali was chosen because the TRECVID DVU set ships character mugshots, but
its reference crops are unusable at scale: median detected face 27 px
against a 69 px maximum, so every reference was upscaled 4x or more past
what the embedder was trained for (AR-011). A 66 px floor left 2 of 69
references; no threshold exists that both keeps the faces in distribution
and leaves enough of them to calibrate.

SuperHero is 69 px median and 241 px max. Its gallery builds at a 66 px
floor with 14 references over 5 characters, and calibrates on its own
(a=15.2867 b=-4.98633, 100% train accuracy) instead of borrowing constants.

Measured on the fused 17-minute film, one stream rather than per-scene
clips so presence windows cross real scene boundaries as SR-002 intends:
precision 1.00, recall 0.65, F1 0.79 — 13 true positives, 0 false
positives, 7 misses. Every out-of-gallery character was declined rather
than forced onto a nearest match. The misses are the short scenes (14 s,
38 s, 27 s), consistent with per-track accumulation needing sightings.

- build_gallery gains --min-face-px, filtering the *detected face* rather
  than the crop. The DVU images are scene crops, not mugshots, so crop
  dimensions say nothing about face scale. A poisoned reference is
  permanent in a way a bad frame is not: it corrupts every future match
  against that identity.
- scripts/fetch_dvu.sh fetches mugshots, scene graphs and segmentation for
  any DVU film. NIST names the same film three different ways, so KG_DIR
  and KG_FILE are overridable rather than derived. This exists as a script
  because the first copy of this data was assembled ad hoc in /tmp and was
  lost with it, taking the working gallery along.
- Replay fixtures move to the artifact registry: push/pull_artifacts.sh
  gain a replay-fixtures target, and tests/fixtures/dumps/.gitignore keeps
  them out of git. superhero.h5 is ~9 MB and regenerating it needs the
  film, the models and a GPU — none of which CI has. The gallery ships
  with the dumps, since a dump only replays against the gallery it was
  produced with.
- AR-012 and AR-013 coverage is ported onto the new fixture rather than
  dropped with the Bali cases: 12369 assertions, up from 7991, since the
  film is an order of magnitude larger than the clips.

Suite: 15679 assertions, 101 test cases.

TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002

scripts/artifacts/pull_artifacts.sh

@@ -10,6 +10,7 @@
 #   scripts/artifacts/pull_artifacts.sh galleries [version]
 #   scripts/artifacts/pull_artifacts.sh montage-frames <film-slug> [version]
 #   scripts/artifacts/pull_artifacts.sh experiment-data [version]
+#   scripts/artifacts/pull_artifacts.sh replay-fixtures [version]
 #   scripts/artifacts/pull_artifacts.sh report-highlights <name> [version]
 #   scripts/artifacts/pull_artifacts.sh xsource [version]
 # version defaults to "latest" (newest uploaded version, by created_at).
@@ -42,6 +43,22 @@ print(matches[-1]['version'])
 "
 }
 
+pull_replay_fixtures() {
+    local version="$1"
+    local dest="${REPO_ROOT}/tests/fixtures/dumps"
+    mkdir -p "$dest"
+    echo "=== replay-fixtures (version ${version}) ==="
+    local tmp; tmp="$(mktemp -d)"
+    if curl -sf "${DL_BASE}/generic/replay-fixtures/${version}/replay-fixtures.zip" \
+            -o "${tmp}/f.zip"; then
+        unzip -qo "${tmp}/f.zip" -d "$dest"
+        echo "  restored: $(ls "$dest" | wc -l) files into tests/fixtures/dumps/"
+    else
+        echo "    [warn] replay-fixtures.zip not found at version ${version}" >&2
+    fi
+    rm -rf "$tmp"
+}
+
 pull_galleries() {
     local version="$1"
     local dest="${REPO_ROOT}/experiments/galleries"
@@ -181,8 +198,13 @@ case "$TARGET" in
         [ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version xsource)"
         pull_xsource "$VERSION"
         ;;
+    replay-fixtures)
+        VERSION="${2:-latest}"
+        [ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version replay-fixtures)"
+        pull_replay_fixtures "$VERSION"
+        ;;
     *)
-        echo "unknown target: $TARGET (expected galleries, montage-frames, experiment-data, report-highlights, or xsource)" >&2
+        echo "unknown target: $TARGET (expected galleries, montage-frames, experiment-data, report-highlights, xsource, or replay-fixtures)" >&2
         exit 1
         ;;
 esac

scripts/artifacts/push_artifacts.sh

@@ -13,10 +13,12 @@
 #   scripts/artifacts/push_artifacts.sh experiment-data
 #   scripts/artifacts/push_artifacts.sh report-highlights
 #   scripts/artifacts/push_artifacts.sh xsource
+#   scripts/artifacts/push_artifacts.sh replay-fixtures
 #   scripts/artifacts/push_artifacts.sh galleries montage-frames experiment-data report-highlights
 #
 # Package layout (owner=dtourolle, repo=scene-actor-extraction):
 #   generic/galleries/<version>/gallery_<model>.h5        (one file per model)
+#   generic/replay-fixtures/<version>/replay-fixtures.zip (T2 dumps + their gallery)
 #   generic/montage-frames/<version>/<film-slug>.zip      (zipped per-film frames)
 #   generic/experiment-data/<version>/experiment-data.zip (manifests/trajectories/results)
 #   generic/report-highlights/<version>/<name>.jpg        (individual, hand-picked
@@ -50,6 +52,29 @@ upload() {
         -o /dev/null -w "    HTTP %{http_code}\n"
 }
 
+push_replay_fixtures() {
+    echo "=== replay-fixtures (version ${VERSION}) ==="
+    # T2 replay fixtures: per-frame detections, landmarks and embeddings dumped
+    # from a real run, so the tracker and identity stages can be replayed on CPU
+    # with no GPU, no models and no film. Too large for git (superhero.h5 alone
+    # is ~9 MB) and regenerating them needs the film plus a GPU, which CI has
+    # neither of — so they ship as artifacts and CI pulls them.
+    #
+    # The gallery travels with them: a dump replays against the gallery it was
+    # produced with, and pairing a dump with a different gallery silently
+    # changes every identity decision in it.
+    local dir="${REPO_ROOT}/tests/fixtures/dumps"
+    if [ ! -d "$dir" ]; then
+        echo "  no tests/fixtures/dumps dir, skipping" >&2
+        return
+    fi
+    local tmp
+    tmp="$(mktemp -d)"
+    ( cd "$dir" && zip -qr "$tmp/replay-fixtures.zip" . )
+    upload "replay-fixtures" "replay-fixtures.zip" "$tmp/replay-fixtures.zip"
+    rm -rf "$tmp"
+}
+
 push_galleries() {
     echo "=== galleries (version ${VERSION}) ==="
     local dir="${REPO_ROOT}/experiments/galleries"
@@ -149,7 +174,8 @@ for target in "$@"; do
         experiment-data)    push_experiment_data ;;
         report-highlights)  push_report_highlights ;;
         xsource)            push_xsource ;;
-        *) echo "unknown target: $target (expected galleries, montage-frames, experiment-data, report-highlights, or xsource)" >&2; exit 1 ;;
+        replay-fixtures)    push_replay_fixtures ;;
+        *) echo "unknown target: $target (expected galleries, montage-frames, experiment-data, report-highlights, xsource, or replay-fixtures)" >&2; exit 1 ;;
     esac
 done
 

scripts/fetch_dvu.sh

@@ -0,0 +1,98 @@
+#!/bin/bash
+# fetch_dvu.sh — pull one film's character mugshots and presence annotations from
+# the NIST TRECVID Deep Video Understanding development set.
+#
+# The DVU dev set is the reason Road to Bali is our benchmark film: it ships
+# 5-7 face crops per *character*, cut from the film itself, alongside
+# scene-scoped presence annotations. That matches SR-002 directly — presence is
+# per scene, not per frame — and it keeps ground truth in character space, so
+# scoring needs no actor->character mapping.
+#
+# This exists as a script, rather than as ad hoc commands, because the first
+# copy of this data lived in a temp directory and was lost to a /tmp wipe,
+# taking the working gallery with it.
+#
+# 14 films are asserted Creative Commons and need no data agreement (only the
+# 5 KinoLorber test films are gated).
+#
+# Usage:
+#   scripts/fetch_dvu.sh [film] [dest]
+#     film  default Road_To_Bali
+#     dest  default ./dvu
+set -euo pipefail
+
+BASE="https://www-nlpir.nist.gov/projects/trecvid/dvu/dvu.development.dataset"
+FILM="${1:-Road_To_Bali}"
+DEST="${2:-dvu}"
+
+mkdir -p "$DEST/images" "$DEST/scenes"
+
+echo "[dvu] $FILM -> $DEST"
+
+# Scene segmentation: start/end as HH:MM:SS. Note valkaama.csv line 38 carries a
+# shift-key typo (01:!4:00) — parse defensively if you extend this to that film.
+echo "[dvu] scene segmentation"
+curl -fsSL "$BASE/scene.segmentation.reference/${FILM}.csv" \
+    -o "$DEST/${FILM}.csv" || echo "  (missing: ${FILM}.csv)"
+
+# Entity types: which entities are Person vs Location/Concept. Only Person rows
+# become gallery identities — the images/ directory also holds Location and
+# Concept crops (bedroom, boat, ...), which must not enter a face gallery.
+#
+# Directory and file naming are inconsistent with the film slug used elsewhere:
+# the folder is Road_to_Bali (lowercase "to") while the entity file is
+# RoadToBali.entity.types.txt. Both are derived here rather than assumed.
+# NIST is inconsistent across all three axes, and not by a rule worth deriving:
+# Road to Bali is Road_To_Bali.csv / Road_to_Bali/ / RoadToBali.entity.types.txt,
+# while SuperHero is SuperHero.csv / superHero/ / superhero.entity.types.txt.
+# Defaults cover the Bali shape; override per film rather than guessing.
+#   KG_DIR=superHero KG_FILE=superhero scripts/fetch_dvu.sh SuperHero dvu-hero
+KG_DIR="${KG_DIR:-${FILM//_To_/_to_}}"
+KG_FILE="${KG_FILE:-$(echo "$FILM" | sed -E 's/_([a-z])/\U\1/g; s/_//g')}"
+
+echo "[dvu] entity types ($KG_DIR/$KG_FILE)"
+curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/${KG_FILE}.entity.types.txt" \
+    -o "$DEST/${FILM}.entity.types.txt" || echo "  (missing: entity types)"
+
+# Character face crops. Names are discovered from the directory listing rather
+# than probed as <Character>_N, since the crop count varies per character and
+# the listing is authoritative.
+echo "[dvu] character mugshots"
+PERSONS="$DEST/persons.txt"
+if [ -f "$DEST/${FILM}.entity.types.txt" ]; then
+    grep -iE "person" "$DEST/${FILM}.entity.types.txt" \
+        | sed -E 's/[[:space:]]*[:,].*$//' | tr -d '\r' \
+        | awk '{print tolower($1)}' | sort -u > "$PERSONS"
+fi
+
+curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/images/" 2>/dev/null \
+    | grep -oE 'href="[^"?/][^"]*\.png"' | sed -E 's/href="//; s/"//' | sort -u \
+    > "$DEST/all_images.txt"
+
+while read -r img; do
+    [ -z "$img" ] && continue
+    # Strip the trailing _N to recover the entity name.
+    who="$(echo "$img" | sed -E 's/_[0-9]+\.png$//' | awk '{print tolower($0)}')"
+    if [ -s "$PERSONS" ] && ! grep -qx "$who" "$PERSONS"; then
+        continue   # Location/Concept crop, not a face
+    fi
+    curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/images/${img}" \
+        -o "$DEST/images/${img}" 2>/dev/null || rm -f "$DEST/images/${img}"
+done < "$DEST/all_images.txt"
+
+# Per-scene knowledge graphs. A Person->Location edge means that person was
+# present for the whole scene. Some of these contain a stray ", ," that breaks
+# strict JSON parsers.
+echo "[dvu] scene graphs"
+for n in $(seq 1 60); do
+    curl -fsSL "$BASE/scenes_knowledge_graphs/${FILM//_/ }-${n}.json" \
+        -o "$DEST/scenes/${FILM}-${n}.json" 2>/dev/null \
+        || curl -fsSL "$BASE/scenes_knowledge_graphs/${FILM}-${n}.json" \
+            -o "$DEST/scenes/${FILM}-${n}.json" 2>/dev/null \
+            || rm -f "$DEST/scenes/${FILM}-${n}.json"
+done
+
+echo "[dvu] done:"
+echo "  mugshots: $(ls "$DEST/images" 2>/dev/null | wc -l)"
+echo "  scenes:   $(ls "$DEST/scenes" 2>/dev/null | wc -l)"
+echo "  csv:      $([ -f "$DEST/${FILM}.csv" ] && echo yes || echo no)"

scripts/make_fixtures.sh

@@ -19,13 +19,16 @@
 # on a full channel (AR-004). Before that fix the same command produced
 # different dumps run to run, since what got dropped depended on timing.
 #
-# Source: bali/ — Road to Bali (1952), public domain. That matters: derived
+# Source: hero/ — SuperHero, from the TRECVID DVU development set. Chosen over
+# SuperHero on face scale: Bali reference crops had a median detected face of
+# 27 px against a 69 px maximum, so every reference was upscaled far past what
+# the embedder was trained for. SuperHero is 69 px median, 241 px max. That matters: derived
 # fixtures can be committed, where anything cut from a copyrighted title could
 # not live in the repository at all.
 set -euo pipefail
 
 REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
-CLIPS="${CLIPS:-$REPO/../bali}"
+CLIPS="${CLIPS:-$REPO/../hero}"
 GALLERY="${GALLERY:-$REPO/gallery_lvface.h5}"
 BIN="${BIN:-$REPO/build/scene_analyze}"
 OUT="$REPO/tests/fixtures/dumps"
@@ -33,8 +36,12 @@ OUT="$REPO/tests/fixtures/dumps"
 # Pinned. Changing either invalidates every committed fixture.
 #   fps 5      — 1 fps over a 77 s clip is 77 frames, too thin to exercise an
 #                extinction window measured in tens of seconds.
-#   min-face   — 32 px, the VR-005 measured floor (98.1% TPI). The corpus is
-#                480x360, so a stricter value would reject most faces present.
+#   min-face   — 32 px. This is a *fixture* setting, deliberately below AR-002's
+#                production floor of 40 px (VR-013, measured end to end): the
+#                corpus is 480x360, where faces run 40-80 px, so pinning at 40
+#                would thin the dumps for reasons unrelated to what they test.
+#                32 px is where VR-005 still shows 98.1% TPI, so the faces kept
+#                are identifiable; it is not the threshold the pipeline ships.
 FPS=5
 MIN_FACE_PX=32
 
@@ -44,12 +51,12 @@ MIN_FACE_PX=32
 
 mkdir -p "$OUT"
 
-for clip in "$CLIPS"/Road_To_Bali-*.webm; do
+for clip in "$CLIPS"/SuperHero-*.webm; do
     n="$(basename "$clip" .webm)"; n="${n##*-}"
-    echo "── bali_$n"
+    echo "── superhero_$n"
     "$BIN" --movie "$clip" --gallery "$GALLERY" \
            --fps "$FPS" --min-face-px "$MIN_FACE_PX" \
-           --dump-embeddings "$OUT/bali_$n.h5" \
+           --dump-embeddings "$OUT/superhero_$n.h5" \
            --output /dev/null 2>&1 | grep -E "wrote|dropped" || true
 done
 

scripts/validation/test_audio_offset.py

@@ -51,7 +51,7 @@ sys.path.insert(0, str(BUILD))
 
 import sae_audio  # noqa: E402
 
-FIXTURE = REPO / "tests" / "fixtures" / "audio" / "bali_offset_200s.flac"
+FIXTURE = REPO / "tests" / "fixtures" / "audio" / "superhero_offset_200s.flac"
 TONE = REPO / "tests" / "fixtures" / "audio" / "jray_audio_v1_tone.flac"
 
 # Server SPEC.md section 3, "Matching and offset recovery". The cap is the