dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dtourolle/scene-actor-extraction:feature/quality-knee
Branches Tags
dtourolle/scene-actor-extraction:master dtourolle/scene-actor-extraction:ci/green-the-runner dtourolle/scene-actor-extraction:feature/opencv5 dtourolle/scene-actor-extraction:feature/quality-knee dtourolle/scene-actor-extraction:feature/traceability-tooling dtourolle/scene-actor-extraction:gitea-pages
dtourolle/scene-actor-extraction:v1.0.0
...
compare: dtourolle/scene-actor-extraction:079b490ede4506c648a3deb82e0bb584c32929d1
Branches Tags
dtourolle/scene-actor-extraction:master dtourolle/scene-actor-extraction:ci/green-the-runner dtourolle/scene-actor-extraction:feature/opencv5 dtourolle/scene-actor-extraction:feature/quality-knee dtourolle/scene-actor-extraction:feature/traceability-tooling dtourolle/scene-actor-extraction:gitea-pages
dtourolle/scene-actor-extraction:v1.0.0
11 Commits
feature/quality-knee ... 079b490ede
Author SHA1 Message Date
dtourolle 079b490ede Merge branch 'feature/ci-images' into feature/opencv5 2026-08-04 14:53:24 +02:00
dtourolle a3827646b9 feat(ci): CPU builder image for the unit-test workflow 
TRACES: DP-007 | PR-004

Pinned by tag rather than :latest, so a workflow run is reproducible against
the image it was written for.
 2026-08-04 14:53:12 +02:00
dtourolle 22758da118 docs: SuperHero benchmark — how to reproduce it and what it scores 
Records the reference film end to end: fetching the annotations and mugshots,
fusing the scene clips into one stream, building the gallery at the 66 px face
floor, and the measured result (precision 1.00, recall 0.65, F1 0.79).

Three things are written down because each cost time to discover:

- Why Bali was withdrawn. Its reference crops have a median detected face of
  27 px against a 69 px maximum, so every reference was upscaled past what the
  embedder was trained for (AR-011). No threshold fixed it — at 66 px, 2 of 69
  references survived. Any accuracy figure recorded against Bali measures
  upscaling artifacts as much as the pipeline.
- Run it as one film. Per-scene clips defeat per-film gallery expansion
  (AR-019) and pay model load ten times over.
- Check you are on the GPU. ORT's CUDA provider fails to load here and falls
  back to CPU silently, so a build-ort timing is a CPU number wearing a GPU
  label — a 15x error whose only symptom is a number with no baseline.

TRACES: AR-011, AR-019 | VR-001, VR-005 | SR-002
 2026-08-04 14:32:33 +02:00
dtourolle 960a7c4eed chore: bump KPN to 454f72c (ignore generated ORT cache) 2026-08-04 14:08:13 +02:00
dtourolle ae60ac7657 chore(models): track 2d106det and the larger SCRFD variants via LFS 
Detector variants used by the resolution and min-face studies. LFS per
.gitattributes, so the repo carries pointers rather than 20 MB of weights.
 2026-08-04 14:04:50 +02:00
dtourolle f891e579c5 chore(traces): put TRACES tags on their own line; regenerate the report 
The parser reads a tag up to end of line, so `# TRACES: GR-004 | SR-001 —
prose` swallowed the prose into the tag and the row went unmatched. Splitting
the comment leaves the tag greppable by the same pattern as the code tags and
the commit trailers, which is the point of the house format.

Mechanical throughout; no logic touched. The regenerated report reflects this
session's new tags: 137 -> 148 found, and one more tagged-but-unexecuted, which
is the SuperHero accuracy assertion that is documented but not yet a test.
 2026-08-04 14:04:21 +02:00
dtourolle d98dc2855a refactor(bench): SuperHero replaces Road to Bali as the reference film 
Bali was chosen because the TRECVID DVU set ships character mugshots, but
its reference crops are unusable at scale: median detected face 27 px
against a 69 px maximum, so every reference was upscaled 4x or more past
what the embedder was trained for (AR-011). A 66 px floor left 2 of 69
references; no threshold exists that both keeps the faces in distribution
and leaves enough of them to calibrate.

SuperHero is 69 px median and 241 px max. Its gallery builds at a 66 px
floor with 14 references over 5 characters, and calibrates on its own
(a=15.2867 b=-4.98633, 100% train accuracy) instead of borrowing constants.

Measured on the fused 17-minute film, one stream rather than per-scene
clips so presence windows cross real scene boundaries as SR-002 intends:
precision 1.00, recall 0.65, F1 0.79 — 13 true positives, 0 false
positives, 7 misses. Every out-of-gallery character was declined rather
than forced onto a nearest match. The misses are the short scenes (14 s,
38 s, 27 s), consistent with per-track accumulation needing sightings.

- build_gallery gains --min-face-px, filtering the *detected face* rather
  than the crop. The DVU images are scene crops, not mugshots, so crop
  dimensions say nothing about face scale. A poisoned reference is
  permanent in a way a bad frame is not: it corrupts every future match
  against that identity.
- scripts/fetch_dvu.sh fetches mugshots, scene graphs and segmentation for
  any DVU film. NIST names the same film three different ways, so KG_DIR
  and KG_FILE are overridable rather than derived. This exists as a script
  because the first copy of this data was assembled ad hoc in /tmp and was
  lost with it, taking the working gallery along.
- Replay fixtures move to the artifact registry: push/pull_artifacts.sh
  gain a replay-fixtures target, and tests/fixtures/dumps/.gitignore keeps
  them out of git. superhero.h5 is ~9 MB and regenerating it needs the
  film, the models and a GPU — none of which CI has. The gallery ships
  with the dumps, since a dump only replays against the gallery it was
  produced with.
- AR-012 and AR-013 coverage is ported onto the new fixture rather than
  dropped with the Bali cases: 12369 assertions, up from 7991, since the
  film is an order of magnitude larger than the clips.

Suite: 15679 assertions, 101 test cases.

TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
 2026-08-04 13:49:11 +02:00
dtourolleandClaude Opus 5 eff696b49a fix(expansion): finish AR-018, retiring the last two expansion cosines 
AR-018 was marked Done while the promotion path still ran on the
constants it was meant to replace. track_gallery.hpp rejected a track
when buffer_spread (1 minus the minimum pairwise cosine) exceeded
expand_track_spread_max, and skipped a view when its raw gal_sim cleared
expand_novelty_sim. Both were bare cosines with no recorded EXCEPTION,
so both were defects under the AR-024 invariant rather than tagging gaps.

The calibrated band was real but unreachable. expand_band_lo/hi were
declared in Config and read nowhere, and set_band() had no callers, so
the gate always ran at the hardcoded 0.90/0.95 while --expand-novelty-sim
and --expand-spread-max stayed live flags.

The spread gate becomes store_coherence: the band's lower bound asked of
every pair in the store, in probability space, rather than a second
constant. admit() compares a newcomer only against its nearest existing
member, so a gradually drifting track chains A to B to C with every step
inside the band while A and C are strangers — the shape a track-ID
collision takes over a slow pan. The bound is re-asked pairwise before
anything reaches an actor's annex.

The novelty gate is deleted rather than converted. SPEC section AR-018
contrasts the band with expand_novelty_sim as the thing it replaces, and
AR-019 requires only that the band is satisfied. Novelty-seeking now
lives entirely in the eviction ordering, which ranks by similarity to the
actor's references instead of cutting at a constant, so there is nothing
left to tune but the two bounds.

BufEntry stored a raw cosine and the eviction loop compared two of them.
The map is monotonic so the ranking was never wrong, but it left a bare
cosine as a decision variable; it now stores the calibrated probability.

The [AR-018] Catch2 tag previously sat on the spread gate, reporting the
replaced mechanism as verification of its replacement. It now sits on the
band: both bounds asserted exactly, since they are inclusive and an
off-by-one there is invisible anywhere else; refusal counted on each
side; and the config bounds driven away from the shipped defaults so a
hardcoded fallback fails. The case that carries the invariant is "band
thresholds probability, not cosine" — under a calibration shifted by
0.10, cosine 0.84 is admitted and cosine 0.92 refused, the opposite of
their raw verdicts. A raw-cosine gate passes an identity-calibrated test
by accident and cannot pass that one. 15 cases, 38 assertions, passing.

scene_preview.cpp takes the flag rename because it would otherwise
reference deleted Config fields. It still does not compile, for reasons
predating this change: it also reads track_max_embed_dist and
track_max_frames_missing, retired by the earlier AR-024 tracker work, and
constructs FaceTrackerFunc with one argument where the registry and
calibration are now required.

Two notes for anyone reading the chain. The main.cpp flag rename and the
AR-018/AR-024 register rows landed in 35e7033, whose trailer names AR-004
only, so git log --grep=AR-018 will not surface them. And
docs/traceability.md is left uncommitted on purpose: regenerating it now
would bake in VR-013 rows for two experiment scripts that are not yet
committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-018, AR-024 | SR-005
 2026-07-31 22:48:07 +02:00
dtourolleandClaude Opus 5 ffdad9873d test: tag the untagged suites; correct two stale headers 
Four test files and one node header carried no TRACES tag, so the
requirements they verify read as implemented-but-unverified. Tagging a
test is what distinguishes the two.

test_calibration.cpp is AR-023; its three [report] cases verify GR-003
and are tagged separately, since the report is fitted from the same
distributions but is its own requirement. test_similarity.cpp is the CI
half of AR-026 — equivalence against hand-computed dot products, where
throughput at scale is AR-027 and cannot run on this host.
test_face_tracker.cpp is AR-007 and AR-008.

Two headers described code that no longer exists. face_aligner_node.hpp
still documented the RANSAC fit AR-005 replaced with an Umeyama
least-squares fit over all five points — not merely out of date but the
opposite of what the file does, and it reads as a rationale for
discarding the landmarks AR-030 measures. test_face_tracker.cpp still
described the park/revive branch AR-008 deleted, and the raw-cosine
cut_revive_sim that guarded it, which AR-024 retired.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-005, AR-007, AR-008, AR-023, AR-026, AR-030 | GR-003 | SR-001, SR-002
 2026-07-31 22:47:59 +02:00
dtourolle 5c6603e63b fix(kpn): park on full outputs; surface node exceptions 
Adopts the KPN backpressure fix (28e0667) and registers the application
error listener it exposes.

`push_blocking` parked a scheduler worker inside the push. Each ObjectNode
owns a private single-thread pool, so the parked thread was the only one
that could drain that node's own input: under sustained backpressure
frame_source, camera_pos, face_detector and face_aligner all slept in
nanosleep at once and the pipeline stopped. Nodes now hold the value,
release the worker, and resume on a channel space-callback.

main.cpp registers set_error_handler so a node that throws names itself
and its exception. Previously the exception was discarded at the node
boundary and survived only as "node 'x' stopped unexpectedly", which says
that a node died but not why — the missing detail that made this slow to
diagnose.

AR-004 drops from Done to Mostly. Two gaps are recorded rather than
claimed fixed: a hang surviving at roughly 1 run in 20 against a 300 s
timeout (down from every run failing), and FanoutNode still dropping on
overflow instead of parking, which sheds frames on the AR-010 scene join
precisely when the dense branch falls behind.

TRACES: AR-004 | SR-002
 2026-07-31 22:42:19 +02:00
dtourolleandClaude Opus 5 3bf4d60a6f docs: regenerate the traceability matrix for VR-014 
The committed matrix predated the audio-signature binding, so VR-014 and
the four UT tags in `test_audio_offset.py` were absent from it while being
present in the register — the one inconsistency a generated file is
supposed to make impossible.

VR-014 also needed an explicit tier row. The blanket `VR-* | Out of CI`
line is right about every other study and wrong about this one: its
fixture is committed and its signature is CPU-only DSP, so it is a test a
CI host can run rather than a measurement someone has to remember to
repeat. Left as an exception under the blanket rather than rewriting the
rule, because the rule still describes the other thirteen.

Coverage unchanged at 38/69; the gate reports no orphan tags.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-014
 2026-07-31 17:02:45 +02:00
46 changed files with 1519 additions and 311 deletions
Expand all files Collapse all files
Dockerfile.builder-cpu
+342
View File
@@ -0,0 +1,342 @@
# sae-builder-cpu — the CI build image
#
# TRACES: DP-007 | PR-004
#
# Build/push: scripts/ci/build_builder_image.sh --push
# Consumed by: .gitea/workflows/unit-tests.yml (pinned by tag, never :latest)
# Docs: docs/ci-image.md
#
# This is the CPU corner of the DP-008 builder matrix and the DP-007 CI image at
# the same time — one artifact, two uses. The CUDA and ROCm siblings differ only
# in the accelerator stack layered on top of this dependency set.
#
# CI runs on an Intel N100 with no discrete GPU. Everything here is chosen so
# that `-DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=CPU -DSAE_BUILD_TESTS=ON`
# configures, builds and runs without a GPU, without a model, and without
# reaching GitHub.
# ─── Base image ──────────────────────────────────────────────────────────────
#
# Chosen for the OLDEST glibc to be supported, not for recency. A binary built
# in a container runs against the *host's* glibc; glibc is backward compatible
# but not forward, so the build base sets the floor for every machine DP-008's
# binaries can ever run on. Building on a newer base than the oldest supported
# host produces the classic `GLIBC_2.xx not found` failure at load time.
#
# Debian 12 "bookworm" = glibc 2.36 (Aug 2022). What that floor covers:
#
# Distro glibc Covered?
# Arch / CachyOS (rolling) 2.41+ yes
# Fedora 37 and later 2.36+ yes ← DP-005's targets are Fedora+Arch
# Debian 12 / 13 2.36+ yes
# Ubuntu 24.04 LTS 2.39 yes
# Ubuntu 22.04 LTS 2.35 NO
# RHEL / Rocky / Alma 9 2.34 NO
# Debian 11 2.31 NO
#
# The three misses are accepted deliberately: DP-005 puts Debian/Ubuntu out of
# installer scope and names Fedora + Arch as the supported distros, and every
# supported Fedora is 2.36 or newer. Going lower costs the toolchain rather than
# buying reach — Debian 11 ships GCC 10 (incomplete C++20) and Python 3.9, which
# has no `tomllib` and therefore cannot read the traceability gate's
# traceability.toml.
#
# Escape hatch, recorded now so it is not rediscovered under pressure: if the
# floor must drop to glibc 2.28 (RHEL 8 / manylinux_2_28 — the same baseline the
# ONNX Runtime and PyTorch wheels target), the move is a Rocky 8 base plus
# gcc-toolset-13, and OpenCV/FFmpeg/HDF5 all leave apt for source or
# EPEL/RPM Fusion. That is a different image, not a flag on this one.
#
# Not a glibc problem but worth stating: the binaries this image produces also
# link OpenCV, FFmpeg and HDF5 shared objects by soname. Making a *portable*
# release binary (DP-008) is a separate question from the glibc floor, and is
# answered by static linking or bundling, not by the base image.
FROM debian:12-slim
# Pins. Every version this image installs from source is an ARG so a rebuild is
# a one-line diff and `docker history` records what a given tag actually holds.
#
# ORT 1.28.0 and OpenCV 5.0.0 match the developer machine, so CI and local
# builds exercise the same libraries rather than merely similar ones.
# Catch2 / nlohmann_json / nanobind match the FetchContent pins in
# CMakeLists.txt:248 and tests/CMakeLists.txt:12 exactly — a vendored copy at a
# different version would be a silent divergence, not a convenience.
ARG ORT_VERSION=1.28.0
ARG OPENCV_VERSION=5.0.0
ARG CATCH2_VERSION=v3.5.3
ARG NLOHMANN_JSON_VERSION=v3.11.3
ARG NANOBIND_VERSION=v2.4.0
# Stamped so a build can prove which image it ran in, and so a green tick can be
# traced back to a specific dependency set. See the "Confirm the builder image"
# step in .gitea/workflows/unit-tests.yml.
ARG IMAGE_TAG=dev
ENV SAE_BUILDER=cpu \
SAE_BUILDER_VERSION=${IMAGE_TAG} \
SAE_ORT_VERSION=${ORT_VERSION} \
SAE_OPENCV_VERSION=${OPENCV_VERSION} \
DEBIAN_FRONTEND=noninteractive
# ─── System dependencies ─────────────────────────────────────────────────────
#
# One layer, ordered by why it is here rather than alphabetically.
RUN apt-get update && apt-get install -y --no-install-recommends \
# Toolchain. bookworm's default gcc is 12.2 — enough for the C++20 the
# project sets unconditionally (CMakeLists.txt:4). cmake is 3.25, above the
# 3.21 minimum. Ninja because the N100 has four cores and every second of
# build scheduling shows.
build-essential \
cmake \
ninja-build \
pkg-config \
git \
ca-certificates \
curl \
# Gitea's act_runner executes JS actions (actions/checkout, upload-artifact)
# with the `node` found *inside* the container. Without this the job cannot
# even check the repository out. Same reason as the kpnpp-builder image.
nodejs \
# HDF5 with the C++ API: galleries are HDF5-native and it is also the VR-001
# dump format. find_package(HDF5 COMPONENTS CXX) at CMakeLists.txt:273.
libhdf5-dev \
# FFmpeg decode. swresample is on this list deliberately: the audio
# signature (IR-004) downmixes to mono and resamples to 11025 Hz, and
# tests/test_audio_signature.cpp decodes the golden FLAC fixture, so the
# test build needs it as much as the main build does.
libavformat-dev \
libavcodec-dev \
libavutil-dev \
libswscale-dev \
libswresample-dev \
# OpenBLAS — required here, not optional. CI has no GPU, so SAE_GEMM_BACKEND
# =CPU is the only path it ever exercises, and without OpenBLAS the CPU GEMM
# falls back to a scalar loop that does not scale against a library-sized
# gallery (AR-027). The build only *warns* when it is missing so a developer
# without it still gets a working tree; the image must never be that case.
# Both the main build (CMakeLists.txt:162) and the test target
# (tests/CMakeLists.txt:42) discover it through pkg-config `openblas`.
libopenblas-dev \
# Python: the build itself needs the interpreter and headers
# (find_package(Python COMPONENTS Interpreter Development.Module) at
# CMakeLists.txt:254, for the nanobind modules). numpy/h5py/scipy are for
# the Python-side tooling — fixture generation, replay, validation scripts.
# From apt rather than pip: bookworm marks the environment externally
# managed (PEP 668), and apt's h5py is already linked against the same
# libhdf5 installed above. bookworm's python3 is 3.11, which has tomllib —
# the traceability gate needs it to read traceability.toml.
python3 \
python3-dev \
python3-numpy \
python3-h5py \
python3-scipy \
# Image codecs for the OpenCV build below. Without these OpenCV silently
# builds an imgcodecs that cannot read a JPEG, which fails at run time in a
# gallery build rather than at compile time here.
libjpeg62-turbo-dev \
libpng-dev \
libtiff-dev \
libwebp-dev \
libopenjp2-7-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# Fail the image build, not the CI run, if OpenBLAS or swresample are not
# discoverable the way CMakeLists.txt discovers them. An image that ships
# libopenblas but no openblas.pc would compile the scalar fallback in silence.
RUN set -eux; \
pkg-config --exists openblas; \
echo "openblas $(pkg-config --modversion openblas)"; \
pkg-config --exists libswresample; \
echo "swresample $(pkg-config --modversion libswresample)"
# ─── ONNX Runtime, CPU provider only ─────────────────────────────────────────
#
# The official prebuilt linux-x64 tarball is the CPU build: no CUDA, no
# TensorRT, no ROCm execution providers. That is the whole requirement here —
# excluding the GPU providers is not a size optimisation, it is the point.
#
# Verified against the 1.28.0 tarball: the shared object's highest versioned
# symbol requirement is GLIBC_2.27 / GLIBCXX_3.4.21, well under this base's
# 2.36, so ORT does not raise the floor set above.
#
# Installed to /usr/local/{lib,include/onnxruntime} because CMakeLists.txt
# includes <onnxruntime/onnxruntime_cxx_api.h> and needs the *parent* of that
# directory on the include path (CMakeLists.txt:108-113).
#
# CI never calls a model — the embedder measures ~930 ms/frame on this CPU
# provider — so ORT is present to satisfy the link, not to run inference.
RUN set -eux; \
curl -fsSL -o /tmp/ort.tgz \
"https://github.com/microsoft/onnxruntime/releases/download/v${ORT_VERSION}/onnxruntime-linux-x64-${ORT_VERSION}.tgz"; \
mkdir -p /tmp/ort; \
tar -xzf /tmp/ort.tgz -C /tmp/ort --strip-components=1; \
cp -a /tmp/ort/lib/libonnxruntime.so* /usr/local/lib/; \
mkdir -p /usr/local/include/onnxruntime; \
cp -a /tmp/ort/include/. /usr/local/include/onnxruntime/; \
ldconfig; \
rm -rf /tmp/ort /tmp/ort.tgz; \
test -f /usr/local/include/onnxruntime/onnxruntime_cxx_api.h
# ─── OpenCV 5, from source ───────────────────────────────────────────────────
#
# This is the reason the image is prebuilt at all. CMakeLists.txt:25 probes for
# OpenCV 5 first and falls back to 4; the branch targets 5, which no Debian
# release ships (bookworm has 4.6), and building it inside every CI run would
# dominate the run on an N100.
#
# BUILD_LIST is exactly the seven components find_package asks for
# (CMakeLists.txt:25-29) — OpenCV resolves their internal dependencies itself.
# Everything else is off: tests, samples, Java/Python bindings, and the apps.
#
# No GUI backend. highgui still builds (find_package REQUIREs the component) but
# with a stub — CI never calls imshow, and pulling GTK/Qt into a headless build
# image buys nothing. scene_preview is a developer tool, not a CI target.
#
# CUDA/cuDNN explicitly off: DP-007 excludes the GPU stack outright.
#
# The source tree and build tree are removed in the same layer, so the ~3 GB of
# intermediates cost nothing in the published image.
RUN set -eux; \
curl -fsSL -o /tmp/opencv.tar.gz \
"https://github.com/opencv/opencv/archive/refs/tags/${OPENCV_VERSION}.tar.gz"; \
mkdir -p /tmp/opencv-src; \
tar -xzf /tmp/opencv.tar.gz -C /tmp/opencv-src --strip-components=1; \
cmake -S /tmp/opencv-src -B /tmp/opencv-build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DBUILD_LIST=core,imgproc,imgcodecs,videoio,dnn,objdetect,highgui \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_DOCS=OFF \
-DBUILD_opencv_apps=OFF \
-DBUILD_JAVA=OFF \
-DBUILD_opencv_python3=OFF \
-DWITH_FFMPEG=ON \
-DWITH_GTK=OFF \
-DWITH_QT=OFF \
-DWITH_OPENGL=OFF \
-DWITH_CUDA=OFF \
-DWITH_CUDNN=OFF \
-DOPENCV_GENERATE_PKGCONFIG=ON \
-DCMAKE_INSTALL_RPATH=/usr/local/lib; \
cmake --build /tmp/opencv-build --parallel; \
cmake --install /tmp/opencv-build; \
ldconfig; \
rm -rf /tmp/opencv-src /tmp/opencv-build /tmp/opencv.tar.gz
# ─── Vendored dependencies: Catch2, nlohmann/json, nanobind ──────────────────
#
# All three are FetchContent'ed by the build today, which makes every CI run
# depend on GitHub being reachable — a network outage would present as a code
# failure. Baking them in removes that dependency entirely.
#
# Catch2 is *installed*, so tests/CMakeLists.txt:6 `find_package(Catch2 3 QUIET)`
# succeeds and the FetchContent fallback is never reached. Its source is kept as
# well so the override below can cover the case where find_package somehow does
# not fire.
#
# nanobind must be cloned with submodules: its `ext/robin_map` is a git
# submodule, and a GitHub source tarball does not contain it. This is the one
# dependency where "download the tarball" produces a tree that configures and
# then fails to compile.
RUN set -eux; \
mkdir -p /opt/vendor; \
git clone --depth 1 --branch "${NLOHMANN_JSON_VERSION}" \
https://github.com/nlohmann/json.git /opt/vendor/nlohmann_json; \
git clone --depth 1 --branch "${NANOBIND_VERSION}" --recurse-submodules \
https://github.com/wjakob/nanobind.git /opt/vendor/nanobind; \
git clone --depth 1 --branch "${CATCH2_VERSION}" \
https://github.com/catchorg/Catch2.git /opt/vendor/Catch2; \
cmake -S /opt/vendor/Catch2 -B /tmp/catch2-build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DBUILD_TESTING=OFF; \
cmake --build /tmp/catch2-build --parallel; \
cmake --install /tmp/catch2-build; \
rm -rf /tmp/catch2-build; \
find /opt/vendor -maxdepth 2 -name .git -exec rm -rf {} +; \
ldconfig
# The initial-cache script the build is configured with. It lives in the image,
# not in the workflow, so the vendor paths have exactly one owner: move a
# directory here and no consumer needs editing.
#
# FETCHCONTENT_FULLY_DISCONNECTED=ON is the load-bearing line. With it, any
# FetchContent dependency that is *not* covered by an override above is a hard
# configure error instead of a silent download — so "this build does not touch
# GitHub" is enforced by the build system rather than asserted in a comment.
RUN set -eux; \
printf '%s\n' \
'# Baked into sae-builder-cpu. Use with: cmake -C /opt/vendor/vendored-deps.cmake ...' \
'# TRACES: DP-007' \
'set(FETCHCONTENT_SOURCE_DIR_NLOHMANN_JSON "/opt/vendor/nlohmann_json" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_SOURCE_DIR_NANOBIND "/opt/vendor/nanobind" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_SOURCE_DIR_CATCH2 "/opt/vendor/Catch2" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_FULLY_DISCONNECTED ON CACHE BOOL "no CI build may fetch from the network")' \
> /opt/vendor/vendored-deps.cmake; \
cat /opt/vendor/vendored-deps.cmake
# ─── Self-check ──────────────────────────────────────────────────────────────
#
# Run the project's own dependency discovery — the same find_package and
# pkg_check_modules calls CMakeLists.txt makes — against this image, at image
# build time. An image that cannot satisfy them should fail here, loudly, once,
# rather than in every CI run that pulls it.
#
# Deliberately not a build of the project: the image must be buildable without
# the repository, and the repository's own configure step is what CI is for.
RUN set -eux; \
mkdir -p /tmp/selfcheck; \
printf '%s\n' \
'cmake_minimum_required(VERSION 3.21)' \
'project(sae_image_selfcheck LANGUAGES CXX)' \
'set(CMAKE_CXX_STANDARD 20)' \
'set(CMAKE_CXX_STANDARD_REQUIRED ON)' \
'find_package(OpenCV 5 REQUIRED COMPONENTS core imgproc imgcodecs videoio dnn objdetect highgui)' \
'message(STATUS "OpenCV ${OpenCV_VERSION}")' \
'find_package(HDF5 REQUIRED COMPONENTS CXX)' \
'message(STATUS "HDF5 ${HDF5_VERSION}")' \
'find_package(Catch2 3 REQUIRED)' \
'message(STATUS "Catch2 ${Catch2_VERSION}")' \
'find_package(Python 3.8 REQUIRED COMPONENTS Interpreter Development.Module)' \
'find_package(PkgConfig REQUIRED)' \
'pkg_check_modules(AVFORMAT REQUIRED libavformat)' \
'pkg_check_modules(AVCODEC REQUIRED libavcodec)' \
'pkg_check_modules(AVUTIL REQUIRED libavutil)' \
'pkg_check_modules(SWSCALE REQUIRED libswscale)' \
'pkg_check_modules(SWRESAMPLE REQUIRED libswresample)' \
'pkg_check_modules(OPENBLAS REQUIRED openblas)' \
'find_library(ORT_LIB onnxruntime REQUIRED HINTS /usr/lib /usr/local/lib)' \
'find_path(ORT_INCLUDE onnxruntime_cxx_api.h PATH_SUFFIXES onnxruntime' \
' HINTS /usr/include/onnxruntime /usr/local/include/onnxruntime /usr/local/include REQUIRED)' \
'message(STATUS "ORT ${ORT_LIB} / ${ORT_INCLUDE}")' \
> /tmp/selfcheck/CMakeLists.txt; \
cmake -S /tmp/selfcheck -B /tmp/selfcheck/build -G Ninja; \
rm -rf /tmp/selfcheck
# Python-side tooling the fixture and validation scripts import. Checked here so
# a missing wheel is an image failure rather than a mid-run traceback.
RUN python3 -c "import numpy, h5py, scipy; print('numpy', numpy.__version__, 'h5py', h5py.__version__, 'scipy', scipy.__version__)"
# ─── What is deliberately NOT here ───────────────────────────────────────────
#
# CUDA, TensorRT, ROCm, and the ORT GPU execution providers
# No GPU to use them. They belong to the sae-builder-cuda and
# sae-builder-rocm siblings (DP-008).
#
# The ONNX models
# Seven files, ~725 MB, in Git LFS. T1/T2 tests are model-free by design
# (tests/CMakeLists.txt:1-4), so the CI image needs none of them, and
# baking them in would inflate the image roughly tenfold to serve the T3
# smoke tests alone. Those pull the model they need via LFS in a separate
# job. The CI workflow checks out with LFS off for the same reason.
#
# The repository
# Nothing from the source tree is COPYed in. The image is a toolchain, and
# a toolchain that embeds the code it builds has to be rebuilt whenever the
# code changes — which is exactly the per-run cost this image exists to
# avoid.
WORKDIR /src
docs/SPEC.md
+27 -9
View File
@@ -666,12 +666,30 @@ An embedding is admitted only if its similarity to one already in the store fall
admitting it risks poisoning the store.
A starting band of roughly **0.90–0.95** is the working estimate, to be tuned
(VR-007). Note this is deliberately conservative compared to the current
`expand_novelty_sim` (0.55), which promotes embeddings *far* from the gallery —
(VR-007). Note this is deliberately conservative compared to the retired
`expand_novelty_sim` (0.55), which promoted embeddings *far* from the gallery —
much more aggressive, and much more exposed to admitting the wrong person.
Both bounds must be expressed as calibrated probabilities, not raw cosines (AR-024).
The lower bound is asked twice. `admit` compares a newcomer against its
*closest* existing member, which a gradually drifting track can chain past: every
step inside the band while the endpoints are strangers — the shape a track-ID
collision takes over a slow pan. So the same bound is re-applied across **every
pair** in the store before promotion. One bound, two enforcement points; not a
second constant.
Novelty is deliberately **not** a threshold. The store's eviction policy orders
its members by similarity to the actor's existing references and drops the
best-recognised one, so novelty-seeking is a ranking with nothing to tune, and
the band's upper bound already refuses the redundant views at the door.
**Current:** implemented in `gallery/track_gallery.hpp` — `admit` at the door,
`store_coherence` at promotion, both bounds from `Config::expand_band_lo/hi`.
Refusals are counted (`band_rejected`).
**Gap:** the bounds themselves are unswept working values (VR-007).
### AR-019 — Expansion of known actors
When a track is owned (AR-012), its store is promoted into a **per-film, in-memory
@@ -772,14 +790,14 @@ natural unit for anonymous presence, should that be adopted (AR-012, TBD).
open (VR-007).
**Current:** `src/gallery/track_gallery.hpp` implements a per-track diversity
buffer with eviction biased to gallery-far poses, promotion gated on
`expand_novelty_sim` / `expand_track_spread_max`, cleared on `is_cut`. Wired at
`identity_matcher_node.hpp:227`, cleared at `:126`.
buffer with eviction biased to gallery-far poses, admission and promotion both
gated on the AR-018 band in probability space, cleared on `is_cut`. Wired at
`identity_matcher_node.hpp:227`, cleared at `:126`; the calibration is handed
over at `:114`.
**Gap:** the band rule of AR-018 replacing the current novelty/spread gates; all
three quiet-signal conditions rather than only `is_cut`; probability space
throughout (AR-024); and the whole of AR-020 — the TBI queue, the deferred pass, and
deferring output until it completes.
**Gap:** all three quiet-signal conditions rather than only `is_cut`; and the
whole of AR-020 — the TBI queue, the deferred pass, and deferring output until it
completes.
## AR-022 — Unidentified-track capture
docs/benchmark.md
+186
View File
@@ -0,0 +1,186 @@
# Benchmark — SuperHero
The reference film for end-to-end accuracy. Replaces Road to Bali, which was
withdrawn for the reason in [Why not Road to Bali](#why-not-road-to-bali).
TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
---
## The film
SuperHero, from the [NIST TRECVID Deep Video Understanding development
set](https://www-nlpir.nist.gov/projects/trecvid/dvu/dvu.development.dataset/).
14 films are asserted Creative Commons and need no data agreement; only the 5
KinoLorber test films are gated.
| | |
|---|---|
| Runtime | 1025.5 s (17.1 min), 10 scenes |
| Resolution | 640×360 |
| Ground truth | Per-scene presence, from the scene knowledge graphs |
| Gallery | 5 characters, 14 references |
The DVU set is what makes this workable: it ships **character** face crops cut
from the film itself, so ground truth and gallery are both in character space
and scoring needs no actor→character mapping.
**Licence caveat.** NIST links licence evidence for only 4 of the 14 films, and
SuperHero is not one of them — its end credits carry no copyright or CC notice,
list a "Temporary Musical Score" and a SAG cast, and it has no traceable online
release. Fine for internal benchmarking; do not redistribute frames from it.
Valkaama is the one film with an independently documented licence (CC BY-SA 3.0)
if provenance ever has to be defended.
---
## Reproducing it
```sh
# 1. Annotations, character mugshots, scene segmentation.
# NIST names the same film three different ways, hence the overrides.
KG_DIR=superHero KG_FILE=superhero scripts/fetch_dvu.sh SuperHero ../dvu-hero
# 2. Scene clips (movie.shots), then fuse them into one stream.
# Fusing matters — see "Run it as one film" below.
# SuperHero-1.webm … SuperHero-10.webm from
# <dataset>/movie.shots/, then:
ffmpeg -f concat -safe 0 -i concat.txt -c copy SuperHero_full.webm
# 3. Gallery, with the face-size floor that keeps references in distribution.
./build/build_gallery --root ../dvu-hero/root \
--output ../dvu-hero/hero66.h5 --min-face-px 66
# 4. Run, on the GPU path (see "Check you are on the GPU").
./build/scene_analyze --movie hero/SuperHero_full.webm \
--gallery ../dvu-hero/hero66.h5 \
--detector-engine trt_cache/scrfd.scrfd_500m_bnkps.640.fp16.engine \
--arcface-engine trt_cache/arcface.LVFace-B_Glint360K.b4.fp16.engine \
--fps 5 --min-face-px 32 --expand-gallery \
--output pred.json
```
Nothing here is in git: the clips are ~130 MB and the annotations are
regenerable. Replay fixtures derived from the run ship through the artifact
registry instead:
```sh
scripts/artifacts/push_artifacts.sh replay-fixtures
scripts/artifacts/pull_artifacts.sh replay-fixtures [version]
```
The gallery travels in the same archive as the dumps deliberately — a dump only
replays meaningfully against the gallery it was produced with, and pairing one
with a different gallery silently changes every identity decision in it.
---
## Results
Measured on the fused film, gallery expansion on.
| Metric | Value |
|---|---|
| Precision | **1.00** |
| Recall | 0.65 |
| F1 | 0.79 |
| True positives | 13 |
| False positives | **0** |
| False negatives | 7 |
Six of ten scenes scored exactly right, including the three-character scenes 4
and 5.
**Zero false positives is the result worth keeping.** Every out-of-gallery
character — Beast, Mighty Celestial, Ms. Johnson, Doctor, two Masked Persons —
was declined rather than forced onto a nearest match. That is the calibrated
probability (AR-024) doing its job, and it is the right failure direction for an
X-Ray overlay: a miss is a gap, an invention is a lie.
**The misses have a shape.** Scenes 1, 2, 3 and 8 were missed, and 1–3 are the
three shortest scenes in the film (14 s, 38 s, 27 s). That is consistent with
per-track Bayesian accumulation (AR-025) needing enough sightings before belief
crosses threshold. Scene 8 is 65 s and does not fit that story — it is the one
to look at first when improving recall.
Running the same scenes as isolated clips did *not* do better, so cross-scene
gallery expansion is not currently compensating for short scenes.
### Run it as one film, not as clips
Per-scene clips defeat per-film gallery expansion (AR-019), which grows a
temporary gallery from track continuity across the whole film and re-assesses
unknown tracks at the end. Ten isolated clips give it nothing to work with, and
pay model and gallery load ten times over.
Fusing also makes presence windows cross real scene boundaries, which is how
SR-002's scene-scoped question is asked in production. Note the joins are
artificial cuts — consecutive scenes were never contiguous footage — so presence
bleeding across a boundary may be the join rather than a tracking fault.
---
## Throughput
| Path | Realtime factor | Sampled fps | 17-min film |
|---|---|---|---|
| `build/` (TensorRT) | **2.0×** | ~10 | ~8 min |
| `build-ort/` (ORT) | 0.54× | 2.7 | ~32 min |
Throughput varies strongly with face density; a sparse stretch measured 8×
realtime, so quote the whole-film average, not a window.
### Check you are on the GPU
ORT's CUDA execution provider fails to load on this machine and **silently falls
back to CPU**:
```
Failed to load library libonnxruntime_providers_cuda.so:
undefined symbol: cudnnGetConvolutionBackwardDataAlgorithm_v7
```
That symbol was removed in cuDNN 9; the packaged ORT is built against cuDNN 8.
ORT logs this once at startup and then runs happily on CPU, so a `build-ort`
timing is a CPU number wearing a GPU label — a 15× error with no symptom other
than a figure you have no baseline for. Grep the log for `Failed to load
library` before trusting any throughput measurement.
The TensorRT path (`build/`) needs prebuilt engines from
`scripts/build_trt_engines.sh` and reports what it loaded:
```
[TrtScrfd] loaded: … [TrtArcFace] loaded: … max_batch=4
[similarity] cuBLAS/CUDA engine: gallery resident on GPU
```
---
## Why not Road to Bali
Bali was chosen because DVU ships character mugshots for it. It was withdrawn on
**face scale**, measured on its own reference crops:
| | Bali | SuperHero |
|---|---|---|
| Median detected face | 27 px | **69 px** |
| Maximum detected face | 69 px | **241 px** |
| References ≥66 px | 2 of 69 | 14 of 27 |
The DVU images are scene crops, not mugshots, so the crop dimensions say nothing
about face scale — the face has to be detected and measured. Bali's median
reference was being upscaled roughly 4× to reach ArcFace's 112×112, and the
worst 7×, which violates AR-011: every model gets the input it was trained for.
A model run off-distribution returns confident, plausible, wrong output.
In a gallery that error is permanent. A bad frame costs one frame; a poisoned
reference corrupts every future match against that identity.
No threshold rescued it. At 66 px only 2 of 69 references survived — the largest
face in the entire set is 69 px — so there was no cut that both kept references
in distribution and left enough of them to calibrate. SuperHero's gallery builds
at a 66 px floor and calibrates on its own (`a=15.2867 b=-4.98633`, 100 % train
accuracy) rather than borrowing constants.
Any accuracy figure recorded against Bali predates this and should be treated as
measuring upscaling artifacts as much as the pipeline.
docs/plan.md
+2 -2
View File
@@ -363,8 +363,8 @@ one identity out of the gallery would fix that.
thing measured is the thing that ships.
`scripts/validation/test_audio_offset.py` over
`tests/fixtures/audio/bali_offset_200s.flac`: 200 s of public-domain film audio
(the same Road to Bali clips the replay fixtures use), long enough for a 120 s
`tests/fixtures/audio/superhero_offset_200s.flac`: 200 s of public-domain film audio
(the same SuperHero clips the replay fixtures use), long enough for a 120 s
window to slide past the ±600-frame search cap. The slide itself is numpy here
on purpose — matching belongs to the consumer, so writing it out keeps this a
test of the signature rather than of somebody's matcher.
docs/requirements.md
+7 -6
View File
@@ -31,7 +31,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing |
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** — KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs |
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Mostly** — node outputs *park* on a full channel: the value is held, the worker released, and a channel space-callback resumes the node. Replaces `push_blocking`, which parked a scheduler worker inside the push and, with one thread per node, stopped that node draining its own input. Verified: 385/385 frames, 0 drops. **Gap:** a rare hang survives, ~1 run in 20 at a 300 s timeout (was: every run). `FanoutNode` still drops on overflow (`fanout.hpp:129`) rather than parking, so the AR-010 scene join sheds frames exactly when the dense branch falls behind |
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`. The RANSAC fit it replaces disagreed by a median 17 source px on 400 headshots, 83.5% of crops embedding below cos 0.99, and was unstable and RNG-driven: rebuilding caught 1614 near-duplicates against the original build's ~100. **All galleries rebuilt** (2456 actors, 10254 embeddings); measured separation gain is small (0.583 → 0.590), so recorded accuracy figures should be re-run but are not expected to move far |
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
@@ -45,13 +45,13 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
| AR-015 | Two live tracks owned by one actor ⇒ treat as a detected cut, re-associate | SR-002 | Medium | **Done** — reverse index detects it on the causing update; counted |
| AR-016 | All tracks closed at EOF — a film ends with faces on screen | SR-002 | High | **Done** — `flush()`, idempotent, closes at last sighting or final tick |
| AR-017 | Every presence claim carries its belief and identification route | SR-002 | High | **Done** — `DeadTrack` carries belief and observation count |
| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space; replaces `expand_novelty_sim`. Rejections counted |
| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space, bounds from `expand_band_lo/hi`; the lower bound re-asked pairwise at promotion, since `admit` compares only against the nearest member and a drifting track can chain past it. Retires `expand_novelty_sim` and `expand_track_spread_max` — novelty is now the eviction ordering, not a threshold. Rejections counted. Bounds unswept (VR-007) |
| AR-019 | Per-film gallery annex from owned tracks; acquires the non-frontal views TMDB lacks | SR-005 | Medium | **Done** — all three discontinuity signals clear the buffers; ownership comes from the registry, not a second local tally |
| AR-020 | Deferred re-identification of unknown tracks against the final expanded gallery | SR-005 | High | Planned |
| AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned |
| AR-022 | Capture still-unidentified tracks: embeddings, metadata, **context crops** for human review | §4 | Medium | Planned |
| AR-023 | Fit sigmoid calibration from intra/inter similarity distributions | SR-002 | High | Done |
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association and accumulation both in probability space; `track_max_embed_dist`, `cut_revive_sim` retired |
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association, accumulation and expansion all in probability space; `track_max_embed_dist`, `cut_revive_sim`, `expand_novelty_sim`, `expand_track_spread_max` retired |
| AR-025 | Per-track Bayesian accumulation in log-odds, with correlated-observation discounting | SR-002 | High | **Done** — log-odds accumulation with correlation discounting owned by the registry, `src/evidence_discount.hpp` |
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
@@ -91,7 +91,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|---|---|---|---|---|
| GR-001 | Build gallery from Jellyfin library cast, TMDB profile fallback | SR-001, SR-005 | High | Done |
| GR-002 | Incremental `--merge` refresh without re-embedding known actors | PR-003 | High | Done |
| GR-003 | Report coverage: zero-image actors, under-referenced actors, dedup, calibration PDFs | SR-001 | Medium | Planned |
| GR-003 | Report coverage: zero-image actors, under-referenced actors, dedup, calibration PDFs | SR-001 | Medium | **Done** — `gallery/gallery_report.hpp`, written next to the gallery by `build_gallery`. Zero-usable-image actors come from the build audit, which a stored gallery cannot reconstruct; also `distinct_references`, `duplicates_removed`, and the intra/inter distributions the calibration fits and would otherwise discard |
| GR-004 | Stamp embedder identity into the gallery; **hard startup error** on mismatch | SR-001 | High | **Done** — basename + SHA-256 + `embed_dim`; mismatch fatal with no bypass, unstamped warns unless `--require-gallery-stamp`; `scripts/stamp_gallery.py` migrates in place |
| GR-005 | Gallery data never leaves the instance | **SR-005** | High | Done |
| GR-006 | Provenance tiers: baked / harvested / confirmed, distinguishable per embedding | SR-005 | High | Planned |
@@ -213,6 +213,7 @@ as such rather than counted as covered.
| GR-001 … GR-005 | T1 + T3 | Gallery assembly is I/O and bookkeeping; embedding is T3 smoke |
| GR-006 … GR-008 | T1 | Tiering and outlier detection operate on stored embeddings |
| VR-* | Out of CI | Studies are run deliberately and their results committed as documents |
| VR-014 | **T2** | The exception, and the reason the blanket row above is not the whole story: its fixture is committed and its signature is CPU-only DSP, so the study *is* a test a CI host can run — not a measurement someone has to remember to repeat |
**One consequence worth stating:** AR-027 (arbitrary gallery scale) is
structurally unverifiable on the CI host. It needs a GPU host and a synthetic
@@ -230,9 +231,9 @@ time**, and CI consumes the result as data.
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
T2 are the only tiers that can exist in CI at all.
### Fixture corpus — `bali/`
### Fixture corpus — `hero/`
Five clips of **Road to Bali (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Five clips of **SuperHero (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Public domain, and that is the reason to use it rather than a convenience:
**derived fixtures — dumps, crops, golden outputs — can be committed without the
docs/traceability.md
+107 -55
View File
@@ -3,7 +3,7 @@
<!-- GENERATED FILE - do not edit by hand. -->
<!-- Regenerate: scripts/traceability/traceability-gate.sh -->
**Generated:** 2026-07-31T14:44:58+00:00
**Generated:** 2026-07-31T20:39:35+00:00
Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`).
@@ -11,14 +11,14 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev
| Metric | Value |
|---|---|
| Source files scanned | 111 |
| TRACES tags found | 132 |
| Source files scanned | 112 |
| TRACES tags found | 148 |
| EXCEPTION tags found | 0 |
| Requirements defined | 69 |
| Requirements covered | 38 |
| **Coverage** | **55.1%** (38/69) |
| Coverage of CI-executable scope | 67.9% (38/56) |
| Tagged but unexecuted in CI | 5 |
| Tagged but unexecuted in CI | 8 |
| Orphan tags | 0 |
### By type
@@ -29,7 +29,7 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev
| DP | 2 | 0 | 8 |
| IR | 8 | 0 | 8 |
| GR | 5 | 0 | 9 |
| VR | 1 | 4 | 14 |
| VR | 1 | 7 | 14 |
- **UT** tags present (separate taxonomy, not counted in coverage): UT-001, UT-101, UT-102, UT-103, UT-104, UT-105, UT-106, UT-107, UT-108
- **IT** tags present (separate taxonomy, not counted in coverage): IT-001
@@ -47,16 +47,16 @@ These requirements have no verification tier this repo's CI host can run, so a t
| VR-002 | out-of-ci | yes | Replay drives the **real** KPN nodes, not a reimplementation |
| VR-003 | out-of-ci | yes | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
| VR-004 | out-of-ci | yes | Reproducible validation corpus with ground truth |
| VR-005 | out-of-ci | no | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-005 | out-of-ci | yes | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-006 | out-of-ci | no | Re-tune `scene_threshold` once native-rate decode lands |
| VR-007 | out-of-ci | no | Expansion band, clustering threshold, and deferred-pass ablation |
| VR-008 | out-of-ci | no | Gallery scaling benchmark — throughput vs gallery size |
| VR-010 | out-of-ci | no | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-010 | out-of-ci | yes | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-011 | out-of-ci | no | Rewrite the replay harness for the post-AR-012 output contract |
| VR-012 | T4, out-of-ci | no | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did … |
| VR-013 | T4, out-of-ci | no | Cross-source identification probe — gallery from one recording, probe… |
| VR-013 | T4, out-of-ci | yes | Cross-source identification probe — gallery from one recording, probe… |
**Tagged but unexecuted:** AR-027, VR-001, VR-002, VR-003, VR-004 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately.
**Tagged but unexecuted:** AR-027, VR-001, VR-002, VR-003, VR-004, VR-005, VR-010, VR-013 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately.
## Orphan tags
@@ -84,10 +84,10 @@ _None._
| AR-002 | Planned | T2 | SR-002 | untagged | - | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's … |
| AR-003 | **Done** — `max_fac… | T1, T2, T4 | SR-002 | covered | `src/config.hpp`, `src/nodes/face_detector_node.hpp`, `src/nodes/identity_matcher_node.hpp` | No fixed per-frame face cap — crowd scenes must not lose background c… |
| AR-004 | **Done** — KPN node… | T1, T4 | SR-002 | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_replay_fixtures.cpp` | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… |
| AR-005 | **Done** — `umeyama… | T1, T3 | SR-002 | covered | `src/face_utils.hpp`, `tests/test_face_utils.cpp` | Align to 112×112 via ArcFace 5-point similarity transform, fitted by … |
| AR-005 | **Done** — `umeyama… | T1, T3 | SR-002 | covered | `src/face_utils.hpp`, `src/nodes/face_aligner_node.hpp`, `tests/test_face_utils.cpp` | Align to 112×112 via ArcFace 5-point similarity transform, fitted by … |
| AR-006 | Done | T3 | SR-002 | covered | `src/nodes/embedder_node.hpp` | 512-d L2-normalised embeddings, batched |
| AR-007 | **Done** — `track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… |
| AR-008 | **Done** — one pool… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | One track pool keyed on `last_seen`; no separate revival path |
| AR-007 | **Done** — `track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `tests/test_face_tracker.cpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… |
| AR-008 | **Done** — one pool… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `tests/test_face_tracker.cpp` | One track pool keyed on `last_seen`; no separate revival path |
| AR-009 | Done | T2 | SR-002 | covered | `src/nodes/camera_position_change_detector_node.hpp` | Camera-cut detection (histogram) as an association hint |
| AR-010 | **Done** — decode b… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/scene_boundary_annotator_node.hpp`, `src/nodes/scene_detector_node.hpp`, `src/scene_boundaries.hpp` | Scene-boundary detection (TransNetV2) as an association hint |
| AR-011 | Planned | T1, T2 | SR-002 | untagged | - | **Every model is fed the input it was trained for** — cost reduced by… |
@@ -97,19 +97,19 @@ _None._
| AR-015 | **Done** — reverse … | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Two live tracks owned by one actor ⇒ treat as a detected cut, re-asso… |
| AR-016 | **Done** — `flush()… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | All tracks closed at EOF — a film ends with faces on screen |
| AR-017 | **Done** — `DeadTra… | T1, T2 | SR-002 | covered | `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Every presence claim carries its belief and identification route |
| AR-018 | **Done** — banded a… | T1, T2 | SR-005 | covered | `src/config.hpp`, `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-subject embedding store with banded admission (novel enough, safe… |
| AR-019 | **Done** — all thre… | T2 | SR-005 | covered | `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-film gallery annex from owned tracks; acquires the non-frontal vi… |
| AR-018 | **Done** — banded a… | T1, T2 | SR-005 | covered | `src/config.hpp`, `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_track_gallery.cpp` | Per-subject embedding store with banded admission (novel enough, safe… |
| AR-019 | **Done** — all thre… | T2 | SR-005 | covered | `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_track_gallery.cpp` | Per-film gallery annex from owned tracks; acquires the non-frontal vi… |
| AR-020 | Planned | T2 | SR-005 | untagged | - | Deferred re-identification of unknown tracks against the final expand… |
| AR-021 | Planned | T2 | SR-005 | untagged | - | Cluster unknown tracks into one entity per person, under temporal can… |
| AR-022 | Planned | T1, T2 | §4 | untagged | - | Capture still-unidentified tracks: embeddings, metadata, **context cr… |
| AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp`, `src/nodes/identity_matcher_node.hpp` | Fit sigmoid calibration from intra/inter similarity distributions |
| AR-024 | **Done** — associat… | T1, static | SR-002 | covered | `src/config.hpp`, `src/evidence_discount.hpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/track_gallery.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `src/nodes/identity_matcher_node.hpp` | **Always the calibrated probability, never a raw cosine** — exception… |
| AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_calibration.cpp` | Fit sigmoid calibration from intra/inter similarity distributions |
| AR-024 | **Done** — associat… | T1, static | SR-002 | covered | `src/config.hpp`, `src/evidence_discount.hpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/track_gallery.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_track_gallery.cpp` | **Always the calibrated probability, never a raw cosine** — exception… |
| AR-025 | **Done** — log-odds… | T1 | SR-002 | covered | `src/evidence_discount.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-track Bayesian accumulation in log-odds, with correlated-observat… |
| AR-026 | In Progress | T1, T4 | SR-001 | covered | `src/backends/gemm_backend.cpp` | All similarity computed as GEMM, including annex and deferred pass |
| AR-026 | In Progress | T1, T4 | SR-001 | covered | `src/backends/gemm_backend.cpp`, `tests/test_similarity.cpp` | All similarity computed as GEMM, including annex and deferred pass |
| AR-027 | Planned | T4 | SR-001 | tagged, unexecuted | `src/backends/gemm_backend.cpp` | Throughput acceptable for **arbitrary** gallery size |
| AR-028 | Planned | T2 | SR-002 | untagged | - | **Embedding input quality assessed and carried** — every face scored … |
| AR-029 | Planned | T1 | SR-002 | untagged | - | Sharpness measure on the **aligned crop** (scale-normalised, so it ca… |
| AR-030 | **In Progress** — m… | T1 | SR-002 | covered | `src/face_utils.hpp`, `tests/test_face_utils.cpp` | Visibility measure from the AR-001 5-point landmarks — extreme pose o… |
| AR-030 | **In Progress** — m… | T1 | SR-002 | covered | `src/face_utils.hpp`, `src/nodes/face_aligner_node.hpp`, `tests/test_face_utils.cpp` | Visibility measure from the AR-001 5-point landmarks — extreme pose o… |
| DP-001 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | One analysis core; modes are front-ends and must not fork pipeline lo… |
| DP-002 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | Batch CLI over one title |
| DP-003 | Planned | T1, manual | PR-004 | untagged | - | On-demand resident service with bounded, observable queue |
@@ -128,7 +128,7 @@ _None._
| IR-008 | **Done** | T1 | SR-003 | covered | `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Emit and honour the signature's own `v1:` version prefix |
| GR-001 | Done | T1, T3 | SR-001, SR-005 | covered | `scripts/make_jellyfin_gallery.py` | Build gallery from Jellyfin library cast, TMDB profile fallback |
| GR-002 | Done | T1, T3 | PR-003 | covered | `scripts/make_jellyfin_gallery.py` | Incremental `--merge` refresh without re-embedding known actors |
| GR-003 | Planned | T1, T3 | SR-001 | covered | `src/build_gallery.cpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/gallery_report.hpp` | Report coverage: zero-image actors, under-referenced actors, dedup, c… |
| GR-003 | **Done** — `gallery… | T1, T3 | SR-001 | covered | `src/build_gallery.cpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/gallery_report.hpp`, `tests/test_calibration.cpp` | Report coverage: zero-image actors, under-referenced actors, dedup, c… |
| GR-004 | **Done** — basename… | T1, T3 | SR-001 | covered | `scripts/filter_gallery.py`, `scripts/make_gallery.py`, `scripts/make_jellyfin_gallery.py`, `scripts/movienet_eval.py`, `scripts/optimizer/fetch_missing_actors.py`, `scripts/optimizer/optimize.py`, `scripts/optimizer/reembed_gallery.py`, `scripts/optimizer/replay.py`, `scripts/sae_embed_loader.py`, `scripts/sae_gallery.py`, `scripts/sae_stamp.py`, `scripts/stamp_gallery.py`, `src/config.hpp`, `src/gallery/embedder_stamp.cpp`, `src/gallery/embedder_stamp.hpp`, `src/gallery/gallery_builder.cpp`, `src/gallery/gallery_store.cpp`, `src/kpn_bindings.cpp`, `src/main.cpp`, `src/nodes/embedding_dump_node.hpp`, `src/scene_preview.cpp`, `src/types.hpp`, `tests/test_gallery_store.cpp` | Stamp embedder identity into the gallery; **hard startup error** on m… |
| GR-005 | Done | T1, T3 | **SR-005** | covered | `src/gallery/gallery_store.hpp` | Gallery data never leaves the instance |
| GR-006 | Planned | T1 | SR-005 | untagged | - | Provenance tiers: baked / harvested / confirmed, distinguishable per … |
@@ -139,15 +139,15 @@ _None._
| VR-002 | **Done** — replay d… | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/replay.py`, `tests/test_replay_fixtures.cpp` | Replay drives the **real** KPN nodes, not a reimplementation |
| VR-003 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/second_score.py` | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
| VR-004 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/validation/ground_truth.py` | Reproducible validation corpus with ground truth |
| VR-005 | **Done** — knee at … | out-of-ci | PR-002 | untagged | - | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-005 | **Done** — knee at … | out-of-ci | PR-002 | tagged, unexecuted | `scripts/validation/min_face_size.py` | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-006 | Planned | out-of-ci | PR-002 | untagged | - | Re-tune `scene_threshold` once native-rate decode lands |
| VR-007 | Planned | out-of-ci | PR-002 | untagged | - | Expansion band, clustering threshold, and deferred-pass ablation |
| VR-008 | Planned | out-of-ci | PR-002 | untagged | - | Gallery scaling benchmark — throughput vs gallery size |
| VR-009 | Planned | T1, out-of-ci | PR-002 | untagged | - | Verify accumulated posteriors are calibrated against held-out tracks |
| VR-010 | Planned | out-of-ci | PR-002 | untagged | - | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-010 | Planned | out-of-ci | PR-002 | tagged, unexecuted | `src/nodes/embedding_dump_node.hpp` | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-011 | Planned | out-of-ci | PR-002 | untagged | - | Rewrite the replay harness for the post-AR-012 output contract |
| VR-012 | Planned | T4, out-of-ci | PR-002 | untagged | - | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did … |
| VR-013 | **In Progress** — h… | T4, out-of-ci | PR-002 | untagged | - | Cross-source identification probe — gallery from one recording, probe… |
| VR-013 | **In Progress** — h… | T4, out-of-ci | PR-002 | tagged, unexecuted | `experiments/xsource/resolution_sweep.py`, `experiments/xsource/verify_labels.py` | Cross-source identification probe — gallery from one recording, probe… |
| VR-014 | **Done** — 40 rando… | T2, out-of-ci | PR-002 | covered | `scripts/validation/test_audio_offset.py` | Audio-signature **offset recovery on real content** — a known trim re… |
## Detailed mapping
@@ -171,15 +171,16 @@ _None._
**Locations:** 4
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown`
- [`src/main.cpp:319`](../src/main.cpp#L319) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:166`](../src/nodes/identity_matcher_node.hpp#L166) — `std::vector<float> host_query(static_cast<size_t>(kMaxFaces) * 512);`
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
### AR-005
**Locations:** 2
**Locations:** 3
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown`
- [`src/nodes/face_aligner_node.hpp:7`](../src/nodes/face_aligner_node.hpp#L7) — `struct FaceAlignerFunc`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### AR-006
@@ -190,19 +191,21 @@ _None._
### AR-007
**Locations:** 3
**Locations:** 4
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
- [`tests/test_face_tracker.cpp:1`](../tests/test_face_tracker.cpp#L1) — `Unknown`
### AR-008
**Locations:** 3
**Locations:** 4
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
- [`tests/test_face_tracker.cpp:1`](../tests/test_face_tracker.cpp#L1) — `Unknown`
### AR-009
@@ -215,9 +218,9 @@ _None._
**Locations:** 9
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown`
- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));`
- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown`
- [`src/main.cpp:329`](../src/main.cpp#L329) — `Unknown`
- [`src/main.cpp:399`](../src/main.cpp#L399) — `return run_net(std::move(net));`
- [`src/main.cpp:431`](../src/main.cpp#L431) — `Unknown`
- [`src/nodes/scene_boundary_annotator_node.hpp:2`](../src/nodes/scene_boundary_annotator_node.hpp#L2) — `Unknown`
- [`src/nodes/scene_detector_node.hpp:36`](../src/nodes/scene_detector_node.hpp#L36) — `static constexpr std::string_view label() { return "scene_detector"; }`
- [`src/nodes/scene_detector_node.hpp:110`](../src/nodes/scene_detector_node.hpp#L110) — `void flush_remaining()`
@@ -279,42 +282,48 @@ _None._
### AR-018
**Locations:** 3
**Locations:** 5
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:164`](../src/gallery/track_gallery.hpp#L164) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:274`](../src/gallery/track_gallery.hpp#L274) — `static int plurality_actor(const TrackState& ts)`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
- [`tests/test_track_gallery.cpp:1`](../tests/test_track_gallery.cpp#L1) — `Unknown`
### AR-019
**Locations:** 3
**Locations:** 4
- [`src/gallery/track_gallery.hpp:122`](../src/gallery/track_gallery.hpp#L122) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/gallery/track_gallery.hpp:123`](../src/gallery/track_gallery.hpp#L123) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/nodes/identity_matcher_node.hpp:147`](../src/nodes/identity_matcher_node.hpp#L147) — `MatchedSceneFrame operator()(TrackedSceneFrame tf)`
- [`src/nodes/identity_matcher_node.hpp:285`](../src/nodes/identity_matcher_node.hpp#L285) — `Unknown`
- [`tests/test_track_gallery.cpp:1`](../tests/test_track_gallery.cpp#L1) — `Unknown`
### AR-023
**Locations:** 3
**Locations:** 4
- [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
- [`src/nodes/identity_matcher_node.hpp:117`](../src/nodes/identity_matcher_node.hpp#L117) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`tests/test_calibration.cpp:1`](../tests/test_calibration.cpp#L1) — `Unknown`
### AR-024
**Locations:** 10
**Locations:** 12
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
- [`src/gallery/track_gallery.hpp:132`](../src/gallery/track_gallery.hpp#L132) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:133`](../src/gallery/track_gallery.hpp#L133) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:164`](../src/gallery/track_gallery.hpp#L164) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:274`](../src/gallery/track_gallery.hpp#L274) — `static int plurality_actor(const TrackState& ts)`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
- [`src/nodes/identity_matcher_node.hpp:117`](../src/nodes/identity_matcher_node.hpp#L117) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`tests/test_track_gallery.cpp:1`](../tests/test_track_gallery.cpp#L1) — `Unknown`
### AR-025
@@ -326,9 +335,10 @@ _None._
### AR-026
**Locations:** 1
**Locations:** 2
- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;`
- [`tests/test_similarity.cpp:1`](../tests/test_similarity.cpp#L1) — `Unknown`
### AR-027
@@ -338,9 +348,10 @@ _None._
### AR-030
**Locations:** 2
**Locations:** 3
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown`
- [`src/nodes/face_aligner_node.hpp:7`](../src/nodes/face_aligner_node.hpp#L7) — `struct FaceAlignerFunc`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### DP-001
@@ -369,7 +380,7 @@ _None._
### GR-003
**Locations:** 13
**Locations:** 16
- [`src/build_gallery.cpp:83`](../src/build_gallery.cpp#L83) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:80`](../src/gallery/gallery_calibration.hpp#L80) — `struct GalleryCalibrationStats`
@@ -384,6 +395,9 @@ _None._
- [`src/gallery/gallery_report.hpp:446`](../src/gallery/gallery_report.hpp#L446) — `inline void save_gallery_report(const std::string& path, const GalleryReport& r)`
- [`src/gallery/gallery_report.hpp:454`](../src/gallery/gallery_report.hpp#L454) — `inline GalleryReport load_gallery_report(const std::string& path)`
- [`src/gallery/gallery_report.hpp:464`](../src/gallery/gallery_report.hpp#L464) — `return gallery_report_from_json(j);`
- [`tests/test_calibration.cpp:192`](../tests/test_calibration.cpp#L192) — `Embedding unit_axis(int slot)`
- [`tests/test_calibration.cpp:222`](../tests/test_calibration.cpp#L222) — `Unknown`
- [`tests/test_calibration.cpp:248`](../tests/test_calibration.cpp#L248) — `Unknown`
### GR-004
@@ -399,8 +413,8 @@ _None._
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/nodes/embedding_dump_node.hpp:127`](../src/nodes/embedding_dump_node.hpp#L127) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:238`](../src/nodes/embedding_dump_node.hpp#L238) — `H5::H5File file(path_, H5F_ACC_TRUNC);`
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor`
- [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");`
@@ -535,12 +549,18 @@ _None._
### PR-002
**Locations:** 4
**Locations:** 10
- [`src/nodes/embedding_dump_node.hpp:2`](../src/nodes/embedding_dump_node.hpp#L2) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:18`](../src/nodes/embedding_dump_node.hpp#L18) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:133`](../src/nodes/embedding_dump_node.hpp#L133) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:159`](../src/nodes/embedding_dump_node.hpp#L159) — `void operator()(EmbeddedSceneFrame ef)`
- [`src/nodes/embedding_dump_node.hpp:242`](../src/nodes/embedding_dump_node.hpp#L242) — `H5::H5File file(path_, H5F_ACC_TRUNC);`
- [`scripts/optimizer/replay.py:5`](../scripts/optimizer/replay.py#L5) — `Reads an embedding dump (scripts/optimizer/SCHEMA.md), feeds each frame as an`
- [`scripts/optimizer/second_score.py:5`](../scripts/optimizer/second_score.py#L5) — `Unknown`
- [`scripts/validation/ground_truth.py:24`](../scripts/validation/ground_truth.py#L24) — `Unknown`
- [`experiments/xsource/resolution_sweep.py:4`](../experiments/xsource/resolution_sweep.py#L4) — `Unknown`
- [`experiments/xsource/verify_labels.py:4`](../experiments/xsource/verify_labels.py#L4) — `Unknown`
### PR-004
@@ -550,7 +570,7 @@ _None._
### SR-001
**Locations:** 60
**Locations:** 64
- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;`
- [`src/build_gallery.cpp:83`](../src/build_gallery.cpp#L83) — `Unknown`
@@ -576,10 +596,13 @@ _None._
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/nodes/embedding_dump_node.hpp:127`](../src/nodes/embedding_dump_node.hpp#L127) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:238`](../src/nodes/embedding_dump_node.hpp#L238) — `H5::H5File file(path_, H5F_ACC_TRUNC);`
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor`
- [`tests/test_calibration.cpp:192`](../tests/test_calibration.cpp#L192) — `Embedding unit_axis(int slot)`
- [`tests/test_calibration.cpp:222`](../tests/test_calibration.cpp#L222) — `Unknown`
- [`tests/test_calibration.cpp:248`](../tests/test_calibration.cpp#L248) — `Unknown`
- [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:201`](../tests/test_gallery_store.cpp#L201) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:219`](../tests/test_gallery_store.cpp#L219) — `TempFile tf("gallery_unstamped.h5");`
@@ -592,6 +615,7 @@ _None._
- [`tests/test_gallery_store.cpp:348`](../tests/test_gallery_store.cpp#L348) — `Unknown`
- [`tests/test_gallery_store.cpp:367`](../tests/test_gallery_store.cpp#L367) — `Unknown`
- [`tests/test_gallery_store.cpp:384`](../tests/test_gallery_store.cpp#L384) — `TempFile tf("fake_model.onnx");`
- [`tests/test_similarity.cpp:1`](../tests/test_similarity.cpp#L1) — `Unknown`
- [`scripts/filter_gallery.py:80`](../scripts/filter_gallery.py#L80) — `if actor_jellyfin_id(a) in cast_ids]`
- [`scripts/make_gallery.py:181`](../scripts/make_gallery.py#L181) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
@@ -615,7 +639,7 @@ _None._
### SR-002
**Locations:** 32
**Locations:** 35
- [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown`
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
@@ -626,12 +650,13 @@ _None._
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown`
- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown`
- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));`
- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown`
- [`src/main.cpp:319`](../src/main.cpp#L319) — `Unknown`
- [`src/main.cpp:329`](../src/main.cpp#L329) — `Unknown`
- [`src/main.cpp:399`](../src/main.cpp#L399) — `return run_net(std::move(net));`
- [`src/main.cpp:431`](../src/main.cpp#L431) — `Unknown`
- [`src/nodes/camera_position_change_detector_node.hpp:30`](../src/nodes/camera_position_change_detector_node.hpp#L30) — `struct CameraPositionChangeDetectorFunc`
- [`src/nodes/embedder_node.hpp:21`](../src/nodes/embedder_node.hpp#L21) — `struct EmbedderFunc`
- [`src/nodes/face_aligner_node.hpp:7`](../src/nodes/face_aligner_node.hpp#L7) — `struct FaceAlignerFunc`
- [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown`
- [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
@@ -648,6 +673,8 @@ _None._
- [`src/nodes/scene_detector_node.hpp:144`](../src/nodes/scene_detector_node.hpp#L144) — `void write_output()`
- [`src/scene_boundaries.hpp:2`](../src/scene_boundaries.hpp#L2) — `Unknown`
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_calibration.cpp:1`](../tests/test_calibration.cpp#L1) — `Unknown`
- [`tests/test_face_tracker.cpp:1`](../tests/test_face_tracker.cpp#L1) — `Unknown`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### SR-003
@@ -664,16 +691,18 @@ _None._
### SR-005
**Locations:** 9
**Locations:** 11
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/gallery/gallery_store.hpp:15`](../src/gallery/gallery_store.hpp#L15) — `Unknown`
- [`src/gallery/track_gallery.hpp:122`](../src/gallery/track_gallery.hpp#L122) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/gallery/track_gallery.hpp:132`](../src/gallery/track_gallery.hpp#L132) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:123`](../src/gallery/track_gallery.hpp#L123) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/gallery/track_gallery.hpp:133`](../src/gallery/track_gallery.hpp#L133) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:164`](../src/gallery/track_gallery.hpp#L164) — `struct TrackState`
- [`src/gallery/track_gallery.hpp:274`](../src/gallery/track_gallery.hpp#L274) — `static int plurality_actor(const TrackState& ts)`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
- [`src/nodes/identity_matcher_node.hpp:147`](../src/nodes/identity_matcher_node.hpp#L147) — `MatchedSceneFrame operator()(TrackedSceneFrame tf)`
- [`src/nodes/identity_matcher_node.hpp:285`](../src/nodes/identity_matcher_node.hpp#L285) — `Unknown`
- [`tests/test_track_gallery.cpp:1`](../tests/test_track_gallery.cpp#L1) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
### UT-001
@@ -769,6 +798,29 @@ _None._
- [`scripts/validation/ground_truth.py:24`](../scripts/validation/ground_truth.py#L24) — `Unknown`
### VR-005
**Locations:** 1
- [`scripts/validation/min_face_size.py:5`](../scripts/validation/min_face_size.py#L5) — `Unknown`
### VR-010
**Locations:** 5
- [`src/nodes/embedding_dump_node.hpp:2`](../src/nodes/embedding_dump_node.hpp#L2) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:18`](../src/nodes/embedding_dump_node.hpp#L18) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:133`](../src/nodes/embedding_dump_node.hpp#L133) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:159`](../src/nodes/embedding_dump_node.hpp#L159) — `void operator()(EmbeddedSceneFrame ef)`
- [`src/nodes/embedding_dump_node.hpp:242`](../src/nodes/embedding_dump_node.hpp#L242) — `H5::H5File file(path_, H5F_ACC_TRUNC);`
### VR-013
**Locations:** 2
- [`experiments/xsource/resolution_sweep.py:4`](../experiments/xsource/resolution_sweep.py#L4) — `Unknown`
- [`experiments/xsource/verify_labels.py:4`](../experiments/xsource/verify_labels.py#L4) — `Unknown`
### VR-014
**Locations:** 1
experiments/xsource/landmark_voting.py
+65 -7
View File
@@ -30,6 +30,8 @@ import sae_embed # before cv2 — see alignment_compare.py
import numpy as np
import cv2
import argparse
ROOT = "/home/dtourolle/Development/Jray-project/scene-actor-extraction/"
M = ROOT + "models/"
CLIPS = ["5157344", "5157339"]
@@ -37,16 +39,31 @@ PROB_THRESHOLD = 0.754
GROUP_IOU = 0.55 # detections overlapping this much are the same face
MATCH_IOU = 0.35 # tie a detection to the hand-labelled face
base_eng = sae_embed.FaceEmbedder(detector_model=M + "scrfd_500m_bnkps.onnx",
_ap = argparse.ArgumentParser()
_ap.add_argument("--detector", default="scrfd_500m_bnkps.onnx",
help="detector under models/. SCRFD sizes 500m / 2.5g / 10g come "
"from InsightFace's buffalo_sc / buffalo_m / buffalo_l packs")
_ap.add_argument("--vote-conf", type=float, default=None,
help="confidence floor for the voting pass. Omit to auto-tune "
"it to --target-votes")
_ap.add_argument("--target-votes", type=int, default=3,
help="votes per face to tune --vote-conf towards, so detectors "
"are compared at equal redundancy rather than equal settings")
_args = _ap.parse_args()
base_eng = sae_embed.FaceEmbedder(detector_model=M + _args.detector,
arcface_model=M + "LVFace-B_Glint360K.onnx",
conf=0.5, nms=0.4, max_side=0)
# Same models, looser suppression: keep the duplicates NMS would have removed.
vote_eng = sae_embed.FaceEmbedder(detector_model=M + "scrfd_500m_bnkps.onnx",
def _make_vote_engine(conf):
# Same models, looser suppression: keep the duplicates NMS would have removed.
return sae_embed.FaceEmbedder(detector_model=M + _args.detector,
arcface_model=M + "LVFace-B_Glint360K.onnx",
conf=0.3, nms=0.9, max_side=0)
conf=conf, nms=0.9, max_side=0)
cal = sae_embed.gallery_calibration(ROOT + "gallery_lvface.h5")
def iou(a, b):
ax, ay, aw, ah = a; bx, by, bw, bh = b
x0, y0 = max(ax, bx), max(ay, by)
@@ -79,6 +96,46 @@ def vote(dets):
return out
def tune_vote_conf(target, sample=6):
"""Pick the confidence floor giving ~target detections per face to average.
A larger SCRFD is more confident and suppresses harder, so at a fixed floor
it emits fewer overlapping anchors — median 2 against 500m's 3. Comparing
detectors at equal SETTINGS therefore also compares them at unequal
redundancy, and the voting arm is handicapped for the bigger models. Tuning
each to the same votes-per-face isolates landmark quality from how much
there was to average.
"""
frames = sorted(glob.glob(f"frames/d{CLIPS[0]}_*.png"))[:sample]
imgs = [cv2.imread(f) for f in frames]
best = (None, None, 1e9)
for conf in (0.30, 0.20, 0.12, 0.07, 0.04, 0.02, 0.01):
eng = _make_vote_engine(conf)
sizes = [n for img in imgs for _, _, _, n in vote(eng.detect(img))]
if not sizes:
continue
med = float(np.median(sizes))
if abs(med - target) < best[2]:
best = (conf, eng, abs(med - target))
print(f"[tune] conf={conf:.2f} -> median {med:.0f} votes/face", file=sys.stderr)
if med >= target:
break
if best[1] is None:
print(f"[tune] no confidence floor reached {target} votes/face; "
f"falling back to 0.30", file=sys.stderr)
return 0.30, _make_vote_engine(0.30)
print(f"[tune] chose conf={best[0]:.2f} for ~{target} votes/face", file=sys.stderr)
return best[0], best[1]
if _args.vote_conf is not None:
VOTE_CONF, vote_eng = _args.vote_conf, _make_vote_engine(_args.vote_conf)
else:
VOTE_CONF, vote_eng = tune_vote_conf(_args.target_votes)
cal = sae_embed.gallery_calibration(ROOT + "gallery_lvface.h5")
def collect(clip):
lab = {os.path.basename(p): os.path.basename(os.path.dirname(p))
for p in glob.glob(f"labelling/{clip}/*/*.jpg")
@@ -124,7 +181,8 @@ print(f"[voting] group size: median {np.median(allv):.0f}, "
file=sys.stderr)
GAL, PRB = "5157344", "5157339"
print(f"\ngallery {GAL} -> probe {PRB}, P>{PROB_THRESHOLD}\n")
print(f"\ndetector={_args.detector} vote_conf={VOTE_CONF:.2f} "
f"gallery {GAL} -> probe {PRB}, P>{PROB_THRESHOLD}\n")
print(f"{'align':>8}{'person':>8}{'n_gal':>7}{'n_prb':>7}"
f"{'within-clip':>13}{'cross-clip':>12}{'hit rate':>10}")
summary = {}
experiments/xsource/resolution_sweep.py
+2
View File
@@ -1,6 +1,8 @@
#!/usr/bin/env python3
"""Impact of input resolution on cross-source identification.
TRACES: VR-013 | PR-002
Gallery is built from one clip at NATIVE resolution. Probes come from the other
clip with the WHOLE FRAME downscaled before it reaches the detector, so
detection and landmark regression degrade together with the pixels. That is the
experiments/xsource/verify_labels.py
+8
View File
@@ -1,6 +1,14 @@
#!/usr/bin/env python3
"""Integrity check on the labelled set, before it is used as ground truth.
TRACES: VR-013 | PR-002
VR-013's ground truth is hand-sorted rather than propagated by embedding
similarity, because propagation would keep only the faces the embedder already
gets right and silently drop the ones the sweep exists to find. This script is
what makes that claim checkable, so it is part of the requirement rather than a
helper of it.
Checks, loudest failure first:
1. INDEX INTEGRITY. Each crop's embedding is taken by re-detecting its source
external/KPN
Vendored
+1 -1
Submodule external/KPN updated: 6595e6e925...454f72c167
mkdocs.yml
+1
View File
@@ -35,6 +35,7 @@ extra_css:
nav:
- Home: index.md
- How We Score Against X-Ray: methodology.md
- Benchmark — SuperHero: benchmark.md
- Findings:
- Best Model: best-model.md
- Gallery Scope (Full vs. Limited): gallery-scope.md
models/2d106det.onnx
LFS
BIN
View File
Binary file not shown.
models/scrfd_10g_bnkps.onnx
LFS
BIN
View File
Binary file not shown.
models/scrfd_2.5g_bnkps.onnx
LFS
BIN
View File
Binary file not shown.
scripts/artifacts/pull_artifacts.sh
+23 -1
View File
@@ -10,6 +10,7 @@
# scripts/artifacts/pull_artifacts.sh galleries [version]
# scripts/artifacts/pull_artifacts.sh montage-frames <film-slug> [version]
# scripts/artifacts/pull_artifacts.sh experiment-data [version]
# scripts/artifacts/pull_artifacts.sh replay-fixtures [version]
# scripts/artifacts/pull_artifacts.sh report-highlights <name> [version]
# scripts/artifacts/pull_artifacts.sh xsource [version]
# version defaults to "latest" (newest uploaded version, by created_at).
@@ -42,6 +43,22 @@ print(matches[-1]['version'])
"
}
pull_replay_fixtures() {
local version="$1"
local dest="${REPO_ROOT}/tests/fixtures/dumps"
mkdir -p "$dest"
echo "=== replay-fixtures (version ${version}) ==="
local tmp; tmp="$(mktemp -d)"
if curl -sf "${DL_BASE}/generic/replay-fixtures/${version}/replay-fixtures.zip" \
-o "${tmp}/f.zip"; then
unzip -qo "${tmp}/f.zip" -d "$dest"
echo " restored: $(ls "$dest" | wc -l) files into tests/fixtures/dumps/"
else
echo " [warn] replay-fixtures.zip not found at version ${version}" >&2
fi
rm -rf "$tmp"
}
pull_galleries() {
local version="$1"
local dest="${REPO_ROOT}/experiments/galleries"
@@ -181,8 +198,13 @@ case "$TARGET" in
[ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version xsource)"
pull_xsource "$VERSION"
;;
replay-fixtures)
VERSION="${2:-latest}"
[ "$VERSION" = "latest" ] && VERSION="$(resolve_latest_version replay-fixtures)"
pull_replay_fixtures "$VERSION"
;;
*)
echo "unknown target: $TARGET (expected galleries, montage-frames, experiment-data, report-highlights, or xsource)" >&2
echo "unknown target: $TARGET (expected galleries, montage-frames, experiment-data, report-highlights, xsource, or replay-fixtures)" >&2
exit 1
;;
esac
scripts/artifacts/push_artifacts.sh
+27 -1
View File
@@ -13,10 +13,12 @@
# scripts/artifacts/push_artifacts.sh experiment-data
# scripts/artifacts/push_artifacts.sh report-highlights
# scripts/artifacts/push_artifacts.sh xsource
# scripts/artifacts/push_artifacts.sh replay-fixtures
# scripts/artifacts/push_artifacts.sh galleries montage-frames experiment-data report-highlights
#
# Package layout (owner=dtourolle, repo=scene-actor-extraction):
# generic/galleries/<version>/gallery_<model>.h5 (one file per model)
# generic/replay-fixtures/<version>/replay-fixtures.zip (T2 dumps + their gallery)
# generic/montage-frames/<version>/<film-slug>.zip (zipped per-film frames)
# generic/experiment-data/<version>/experiment-data.zip (manifests/trajectories/results)
# generic/report-highlights/<version>/<name>.jpg (individual, hand-picked
@@ -50,6 +52,29 @@ upload() {
-o /dev/null -w " HTTP %{http_code}\n"
}
push_replay_fixtures() {
echo "=== replay-fixtures (version ${VERSION}) ==="
# T2 replay fixtures: per-frame detections, landmarks and embeddings dumped
# from a real run, so the tracker and identity stages can be replayed on CPU
# with no GPU, no models and no film. Too large for git (superhero.h5 alone
# is ~9 MB) and regenerating them needs the film plus a GPU, which CI has
# neither of — so they ship as artifacts and CI pulls them.
#
# The gallery travels with them: a dump replays against the gallery it was
# produced with, and pairing a dump with a different gallery silently
# changes every identity decision in it.
local dir="${REPO_ROOT}/tests/fixtures/dumps"
if [ ! -d "$dir" ]; then
echo " no tests/fixtures/dumps dir, skipping" >&2
return
fi
local tmp
tmp="$(mktemp -d)"
( cd "$dir" && zip -qr "$tmp/replay-fixtures.zip" . )
upload "replay-fixtures" "replay-fixtures.zip" "$tmp/replay-fixtures.zip"
rm -rf "$tmp"
}
push_galleries() {
echo "=== galleries (version ${VERSION}) ==="
local dir="${REPO_ROOT}/experiments/galleries"
@@ -149,7 +174,8 @@ for target in "$@"; do
experiment-data) push_experiment_data ;;
report-highlights) push_report_highlights ;;
xsource) push_xsource ;;
*) echo "unknown target: $target (expected galleries, montage-frames, experiment-data, report-highlights, or xsource)" >&2; exit 1 ;;
replay-fixtures) push_replay_fixtures ;;
*) echo "unknown target: $target (expected galleries, montage-frames, experiment-data, report-highlights, xsource, or replay-fixtures)" >&2; exit 1 ;;
esac
done
scripts/ci/build_builder_image.sh
+118
View File
@@ -0,0 +1,118 @@
#!/bin/bash
# build_builder_image.sh — build and publish the DP-007 CI builder image to the
# Gitea container registry.
#
# TRACES: DP-007 | PR-004
#
# Usage:
# scripts/ci/build_builder_image.sh # build only, tag v1
# scripts/ci/build_builder_image.sh --push # build and push
# scripts/ci/build_builder_image.sh --tag v2 --push # bump the pinned tag
# scripts/ci/build_builder_image.sh --no-cache # force a clean rebuild
#
# The tag is the contract with CI. .gitea/workflows/unit-tests.yml names an
# explicit tag in its `container:` block and never `latest`, so that rebuilding
# the image cannot silently change what a previous green build meant. Bumping
# the dependency set means bumping the tag AND editing the workflow — the two
# edits landing in the same commit is the point, not an inconvenience.
#
# Registry auth: this script does not log in. Do it once, out of band:
# docker login gitea.tourolle.paris
# The CI host is already authenticated this way (its cached credentials in
# ~/.docker/config.json are what the kpnpp-builder push relies on), so a
# workflow that calls this script needs no secret plumbing.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
REGISTRY="gitea.tourolle.paris"
OWNER="dtourolle"
IMAGE="sae-builder-cpu"
DOCKERFILE="Dockerfile.builder-cpu"
# The tag CI pins to today. Keep this in step with the `container.image` line in
# .gitea/workflows/unit-tests.yml; the workflow asserts at run time that the
# image it landed in reports this same version, so a drift shows up as a failed
# job rather than as a build against the wrong toolchain.
TAG="v1"
PUSH=0
EXTRA_ARGS=()
while [ $# -gt 0 ]; do
case "$1" in
--push) PUSH=1 ;;
--tag) TAG="${2:?--tag needs a value}"; shift ;;
--no-cache) EXTRA_ARGS+=(--no-cache) ;;
-h|--help) sed -n '2,30p' "${BASH_SOURCE[0]}"; exit 0 ;;
*) echo "error: unknown argument '$1'" >&2; exit 2 ;;
esac
shift
done
if [ "$TAG" = "latest" ]; then
echo "error: refusing to build the tag 'latest'." >&2
echo "DP-007 requires CI to pin an immutable tag. A moving 'latest' means a" >&2
echo "rebuild retroactively changes what every earlier green build proved." >&2
exit 2
fi
REF="${REGISTRY}/${OWNER}/${IMAGE}:${TAG}"
# A second tag carrying the commit that produced the image. The workflow pins
# the human-readable tag; this one is the audit trail — given any image you can
# recover the Dockerfile that built it.
SHA="$(git -C "$REPO_ROOT" rev-parse --short HEAD)"
REF_SHA="${REGISTRY}/${OWNER}/${IMAGE}:${TAG}-${SHA}"
# The Dockerfile COPYs nothing from the repository on purpose (see its closing
# comment), so the build context is an empty directory rather than the repo
# root. Sending ~1 GB of models, fixtures and experiment data to the daemon for
# a build that reads none of it is pure latency.
CONTEXT="$(mktemp -d)"
trap 'rm -rf "$CONTEXT"' EXIT
echo "=== building ${REF}"
echo " dockerfile: ${REPO_ROOT}/${DOCKERFILE}"
echo " context: (empty — the image embeds no repository content)"
echo
echo " Expect this to take a while: OpenCV 5 is compiled from source because"
echo " no Debian release ships it. That cost is paid once per image, which is"
echo " the entire reason DP-007 asks for a prebuilt image instead of"
echo " installing dependencies inside each CI run."
echo
docker build \
"${EXTRA_ARGS[@]}" \
--build-arg "IMAGE_TAG=${TAG}" \
-f "${REPO_ROOT}/${DOCKERFILE}" \
-t "${REF}" \
-t "${REF_SHA}" \
"${CONTEXT}"
echo
echo "=== built"
docker image inspect "${REF}" --format ' {{.RepoTags}} {{.Size}} bytes'
docker run --rm "${REF}" sh -c 'echo " SAE_BUILDER=$SAE_BUILDER version=$SAE_BUILDER_VERSION ort=$SAE_ORT_VERSION opencv=$SAE_OPENCV_VERSION"'
if [ "$PUSH" -eq 0 ]; then
echo
echo "Not pushed. Re-run with --push, or push by hand:"
echo " docker push ${REF}"
echo " docker push ${REF_SHA}"
exit 0
fi
echo
echo "=== pushing"
# No `latest` tag is pushed, by design. Publishing one invites a workflow to use
# it, and DP-007 exists to prevent exactly that.
docker push "${REF}"
docker push "${REF_SHA}"
echo
echo "=== published ${REF}"
echo "If this was a dependency-set change, bump the tag in"
echo " .gitea/workflows/unit-tests.yml (container.image)"
echo " scripts/ci/build_builder_image.sh (TAG, above)"
echo "in the same commit, so no run can build against an image the repository"
echo "does not describe."
scripts/fetch_dvu.sh
+98
View File
@@ -0,0 +1,98 @@
#!/bin/bash
# fetch_dvu.sh — pull one film's character mugshots and presence annotations from
# the NIST TRECVID Deep Video Understanding development set.
#
# The DVU dev set is the reason Road to Bali is our benchmark film: it ships
# 5-7 face crops per *character*, cut from the film itself, alongside
# scene-scoped presence annotations. That matches SR-002 directly — presence is
# per scene, not per frame — and it keeps ground truth in character space, so
# scoring needs no actor->character mapping.
#
# This exists as a script, rather than as ad hoc commands, because the first
# copy of this data lived in a temp directory and was lost to a /tmp wipe,
# taking the working gallery with it.
#
# 14 films are asserted Creative Commons and need no data agreement (only the
# 5 KinoLorber test films are gated).
#
# Usage:
# scripts/fetch_dvu.sh [film] [dest]
# film default Road_To_Bali
# dest default ./dvu
set -euo pipefail
BASE="https://www-nlpir.nist.gov/projects/trecvid/dvu/dvu.development.dataset"
FILM="${1:-Road_To_Bali}"
DEST="${2:-dvu}"
mkdir -p "$DEST/images" "$DEST/scenes"
echo "[dvu] $FILM -> $DEST"
# Scene segmentation: start/end as HH:MM:SS. Note valkaama.csv line 38 carries a
# shift-key typo (01:!4:00) — parse defensively if you extend this to that film.
echo "[dvu] scene segmentation"
curl -fsSL "$BASE/scene.segmentation.reference/${FILM}.csv" \
-o "$DEST/${FILM}.csv" || echo " (missing: ${FILM}.csv)"
# Entity types: which entities are Person vs Location/Concept. Only Person rows
# become gallery identities — the images/ directory also holds Location and
# Concept crops (bedroom, boat, ...), which must not enter a face gallery.
#
# Directory and file naming are inconsistent with the film slug used elsewhere:
# the folder is Road_to_Bali (lowercase "to") while the entity file is
# RoadToBali.entity.types.txt. Both are derived here rather than assumed.
# NIST is inconsistent across all three axes, and not by a rule worth deriving:
# Road to Bali is Road_To_Bali.csv / Road_to_Bali/ / RoadToBali.entity.types.txt,
# while SuperHero is SuperHero.csv / superHero/ / superhero.entity.types.txt.
# Defaults cover the Bali shape; override per film rather than guessing.
# KG_DIR=superHero KG_FILE=superhero scripts/fetch_dvu.sh SuperHero dvu-hero
KG_DIR="${KG_DIR:-${FILM//_To_/_to_}}"
KG_FILE="${KG_FILE:-$(echo "$FILM" | sed -E 's/_([a-z])/\U\1/g; s/_//g')}"
echo "[dvu] entity types ($KG_DIR/$KG_FILE)"
curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/${KG_FILE}.entity.types.txt" \
-o "$DEST/${FILM}.entity.types.txt" || echo " (missing: entity types)"
# Character face crops. Names are discovered from the directory listing rather
# than probed as <Character>_N, since the crop count varies per character and
# the listing is authoritative.
echo "[dvu] character mugshots"
PERSONS="$DEST/persons.txt"
if [ -f "$DEST/${FILM}.entity.types.txt" ]; then
grep -iE "person" "$DEST/${FILM}.entity.types.txt" \
| sed -E 's/[[:space:]]*[:,].*$//' | tr -d '\r' \
| awk '{print tolower($1)}' | sort -u > "$PERSONS"
fi
curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/images/" 2>/dev/null \
| grep -oE 'href="[^"?/][^"]*\.png"' | sed -E 's/href="//; s/"//' | sort -u \
> "$DEST/all_images.txt"
while read -r img; do
[ -z "$img" ] && continue
# Strip the trailing _N to recover the entity name.
who="$(echo "$img" | sed -E 's/_[0-9]+\.png$//' | awk '{print tolower($0)}')"
if [ -s "$PERSONS" ] && ! grep -qx "$who" "$PERSONS"; then
continue # Location/Concept crop, not a face
fi
curl -fsSL "$BASE/movie_knowledge_graph/${KG_DIR}/images/${img}" \
-o "$DEST/images/${img}" 2>/dev/null || rm -f "$DEST/images/${img}"
done < "$DEST/all_images.txt"
# Per-scene knowledge graphs. A Person->Location edge means that person was
# present for the whole scene. Some of these contain a stray ", ," that breaks
# strict JSON parsers.
echo "[dvu] scene graphs"
for n in $(seq 1 60); do
curl -fsSL "$BASE/scenes_knowledge_graphs/${FILM//_/ }-${n}.json" \
-o "$DEST/scenes/${FILM}-${n}.json" 2>/dev/null \
|| curl -fsSL "$BASE/scenes_knowledge_graphs/${FILM}-${n}.json" \
-o "$DEST/scenes/${FILM}-${n}.json" 2>/dev/null \
|| rm -f "$DEST/scenes/${FILM}-${n}.json"
done
echo "[dvu] done:"
echo " mugshots: $(ls "$DEST/images" 2>/dev/null | wc -l)"
echo " scenes: $(ls "$DEST/scenes" 2>/dev/null | wc -l)"
echo " csv: $([ -f "$DEST/${FILM}.csv" ] && echo yes || echo no)"
scripts/filter_gallery.py
+2 -1
View File
@@ -77,7 +77,8 @@ def main():
if missing > 0:
print(f"[warn] {missing} cast member(s) not present in gallery (not yet embedded)", file=sys.stderr)
# TRACES: GR-004 | SR-001 — a filtered gallery holds the SAME vectors as its
# TRACES: GR-004 | SR-001
# a filtered gallery holds the SAME vectors as its
# source, so it inherits the source's binding. Dropping the stamp here would
# silently launder a stamped gallery into an unstamped one.
save_gallery_hdf5({"actors": actors}, Path(args.output),
scripts/make_fixtures.sh
+14 -7
View File
@@ -19,13 +19,16 @@
# on a full channel (AR-004). Before that fix the same command produced
# different dumps run to run, since what got dropped depended on timing.
#
# Source: bali/ — Road to Bali (1952), public domain. That matters: derived
# Source: hero/ — SuperHero, from the TRECVID DVU development set. Chosen over
# SuperHero on face scale: Bali reference crops had a median detected face of
# 27 px against a 69 px maximum, so every reference was upscaled far past what
# the embedder was trained for. SuperHero is 69 px median, 241 px max. That matters: derived
# fixtures can be committed, where anything cut from a copyrighted title could
# not live in the repository at all.
set -euo pipefail
REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
CLIPS="${CLIPS:-$REPO/../bali}"
CLIPS="${CLIPS:-$REPO/../hero}"
GALLERY="${GALLERY:-$REPO/gallery_lvface.h5}"
BIN="${BIN:-$REPO/build/scene_analyze}"
OUT="$REPO/tests/fixtures/dumps"
@@ -33,8 +36,12 @@ OUT="$REPO/tests/fixtures/dumps"
# Pinned. Changing either invalidates every committed fixture.
# fps 5 — 1 fps over a 77 s clip is 77 frames, too thin to exercise an
# extinction window measured in tens of seconds.
# min-face — 32 px, the VR-005 measured floor (98.1% TPI). The corpus is
# 480x360, so a stricter value would reject most faces present.
# min-face — 32 px. This is a *fixture* setting, deliberately below AR-002's
# production floor of 40 px (VR-013, measured end to end): the
# corpus is 480x360, where faces run 40-80 px, so pinning at 40
# would thin the dumps for reasons unrelated to what they test.
# 32 px is where VR-005 still shows 98.1% TPI, so the faces kept
# are identifiable; it is not the threshold the pipeline ships.
FPS=5
MIN_FACE_PX=32
@@ -44,12 +51,12 @@ MIN_FACE_PX=32
mkdir -p "$OUT"
for clip in "$CLIPS"/Road_To_Bali-*.webm; do
for clip in "$CLIPS"/SuperHero-*.webm; do
n="$(basename "$clip" .webm)"; n="${n##*-}"
echo "── bali_$n"
echo "── superhero_$n"
"$BIN" --movie "$clip" --gallery "$GALLERY" \
--fps "$FPS" --min-face-px "$MIN_FACE_PX" \
--dump-embeddings "$OUT/bali_$n.h5" \
--dump-embeddings "$OUT/superhero_$n.h5" \
--output /dev/null 2>&1 | grep -E "wrote|dropped" || true
done
scripts/make_gallery.py
+2 -1
View File
@@ -178,7 +178,8 @@ def main():
output = Path(args.output)
image_root = Path(args.image_dir) if args.image_dir else output.parent / "images"
# TRACES: GR-004 | SR-001 — stamp with the model actually loaded, resolved
# TRACES: GR-004 | SR-001
# stamp with the model actually loaded, resolved
# through the same helper load_embedder uses so the two cannot diverge.
arcface_path = resolve_arcface(args.models_dir, args.arcface)
embedder = load_embedder(args.build_dir, args.models_dir, args.arcface)
scripts/make_jellyfin_gallery.py
+2 -1
View File
@@ -453,7 +453,8 @@ def main():
existing_actors = {}
if args.merge and output.is_file():
existing = load_gallery_hdf5(output)
# TRACES: GR-004 | SR-001 — --merge keeps the existing actors' vectors and
# TRACES: GR-004 | SR-001
# --merge keeps the existing actors' vectors and
# embeds the new ones with THIS model. If they disagree, the result is one
# gallery holding two incompatible embedding spaces, which is worse than a
# mismatched gallery: no later check can separate them again.
scripts/movienet_eval.py
+2 -1
View File
@@ -62,7 +62,8 @@ def main():
args = p.parse_args()
embedder = load_embedder(args.build_dir, args.models_dir, args.arcface)
# TRACES: GR-004 | SR-001 — match() below is a bare dot product against the
# TRACES: GR-004 | SR-001
# match() below is a bare dot product against the
# gallery's vectors; if the gallery came from another model those numbers are
# noise wearing a similarity's clothes.
verify_gallery_stamp(args.gallery,
scripts/optimizer/fetch_missing_actors.py
+4 -2
View File
@@ -106,7 +106,8 @@ def fetch(missing_path, out_path, token, build_dir, models_dir, arcface,
f"(wiki={n_via_wikidata}) no_tmdb={n_no_tmdb} no_img={n_no_img} "
f"no_face={n_no_face}", file=sys.stderr)
# TRACES: GR-004 | SR-001 — the legacy JSON gallery carries the same stamp as
# TRACES: GR-004 | SR-001
# the legacy JSON gallery carries the same stamp as
# the HDF5 one; src/gallery/gallery_store.cpp reads it from either.
Path(out_path).write_text(json.dumps({"embedder": stamp, "actors": actors}, indent=2))
n_emb = sum(len(a["embeddings"]) for a in actors)
@@ -120,7 +121,8 @@ def fetch(missing_path, out_path, token, build_dir, models_dir, arcface,
def merge(base_path, add_path, out_path):
base = json.loads(Path(base_path).read_text())
add = json.loads(Path(add_path).read_text())
# TRACES: GR-004 | SR-001 — merging two galleries from different models makes
# TRACES: GR-004 | SR-001
# merging two galleries from different models makes
# ONE file containing two incompatible embedding spaces. Nothing downstream can
# ever untangle that, so this is the one place the check must run before, not
# after, the write.
scripts/optimizer/optimize.py
+2 -1
View File
@@ -199,7 +199,8 @@ def main():
if not Path(f["dump"]).exists():
sys.exit(f"[opt] missing dump for {f['name']}: {f['dump']}")
# TRACES: GR-004 | SR-001 — every (dump, gallery) pair is checked ONCE here,
# TRACES: GR-004 | SR-001
# every (dump, gallery) pair is checked ONCE here,
# before the first evaluation. A DE sweep is thousands of replays; discovering
# a cross-model pair at the end (or never) means every number it produced was
# noise. Each replay subprocess re-checks its own pair anyway.
scripts/optimizer/reembed_gallery.py
+2 -1
View File
@@ -59,7 +59,8 @@ def main():
ref = load_gallery_hdf5(Path(args.ref))
images_root = Path(args.images)
embedder = load_embedder(args.build_dir, args.models_dir, args.arcface)
# TRACES: GR-004 | SR-001 — this script exists to produce a gallery in a
# TRACES: GR-004 | SR-001
# this script exists to produce a gallery in a
# DIFFERENT model's space from the reference. The output must therefore never
# inherit the reference's stamp; it carries the stamp of --arcface, which is
# the whole point of the bake-off being safe to run.
scripts/optimizer/replay.py
+4 -2
View File
@@ -110,7 +110,8 @@ def replay(dump_path: str, gallery: str, cfg: dict, build_dir: str, stop: bool =
sys.path.insert(0, build_dir)
import sae_kpn
# TRACES: GR-004 | SR-001 — checked here, before any network is built, so a
# TRACES: GR-004 | SR-001
# checked here, before any network is built, so a
# cross-model replay dies with one readable error instead of producing a
# plausible-looking score. add_identity_matcher re-checks it C++-side below;
# that is the backstop for any other caller of the binding.
@@ -249,7 +250,8 @@ def main():
# per-film gallery expansion: promotes pose-varied views of confidently-identified
# actors into an in-memory annex, recovering ~+4 recall at no precision cost.
p.add_argument("--expand-gallery", action="store_true")
# TRACES: GR-004 | SR-001 — promote an unprovable gallery/dump binding from a
# TRACES: GR-004 | SR-001
# promote an unprovable gallery/dump binding from a
# loud warning to a hard error. Measurement sweeps should set this (or
# SAE_REQUIRE_GALLERY_STAMP=1) so no number comes from an unbound pair.
p.add_argument("--require-gallery-stamp", action="store_true")
scripts/sae_embed_loader.py
+16 -5
View File
@@ -11,6 +11,7 @@ argument, which is often None.
"""
import sys
import os
from pathlib import Path
DEFAULT_ARCFACE = "arcface_w600k_r50.onnx"
@@ -19,8 +20,10 @@ DEFAULT_ARCFACE = "arcface_w600k_r50.onnx"
def resolve_arcface(models_dir: str, arcface: str | None = None) -> str:
"""The ArcFace/LVFace ONNX path load_embedder would use for these arguments.
TRACES: GR-004 | SR-001 — single source of truth for "which model is this",
so the stamp written into a gallery can never drift from the model loaded."""
TRACES: GR-004 | SR-001
Single source of truth for "which model is this", so the stamp written into
a gallery can never drift from the model loaded."""
return arcface if arcface else str(Path(models_dir) / DEFAULT_ARCFACE)
@@ -49,13 +52,21 @@ def load_embedder(build_dir: str, models_dir: str, arcface: str | None = None,
sys.exit(f"{name} model not found: {model}\nRun: bash scripts/download_models.sh")
# A TRT-backend build cannot load .onnx; it needs pre-built engines from
# scripts/build_trt_engines.sh. Pass them when present (ignored by ORT).
# scripts/build_trt_engines.sh.
#
# These are passed only on request. The old comment here claimed they were
# "ignored by ORT" — they are not. The ORT backend treats an engine path as
# an instruction and raises, which is the right behaviour (silently ignoring
# a requested engine would be worse), but it meant that merely HAVING a
# populated trt_cache/ broke every ORT gallery build in the repo, with an
# error naming a flag the caller never set.
use_engines = os.environ.get("SAE_USE_TRT_ENGINES", "") not in ("", "0", "false")
trt = Path(models_path).parent / "trt_cache"
det_engine = trt / "scrfd.scrfd_500m_bnkps.640.fp16.engine"
arc_engine = trt / f"arcface.{Path(arcface_path).stem}.b4.fp16.engine"
return sae_embed.FaceEmbedder(
detector_path, arcface_path, conf, nms, max_side,
str(det_engine) if det_engine.is_file() else "",
str(arc_engine) if arc_engine.is_file() else "",
str(det_engine) if (use_engines and det_engine.is_file()) else "",
str(arc_engine) if (use_engines and arc_engine.is_file()) else "",
)
scripts/sae_gallery.py
+4 -2
View File
@@ -168,7 +168,8 @@ def save_gallery_hdf5(gallery: dict, output: Path, embedder: dict | None = None)
f.create_dataset("jellyfin_id", data=np.asarray(jf, dtype=object), dtype=str_t)
f.create_dataset("name", data=np.asarray(name, dtype=object), dtype=str_t)
f.create_dataset("source_images", data=np.asarray(src_images, dtype=object), dtype=str_t)
# TRACES: GR-004 | SR-001 — omitted entirely when unknown, so "unstamped"
# TRACES: GR-004 | SR-001
# omitted entirely when unknown, so "unstamped"
# round-trips as unstamped rather than as a stamp naming no model.
if not _stamp_empty(embedder):
g = f.create_group("embedder")
@@ -196,7 +197,8 @@ def load_gallery_hdf5(path: Path) -> dict:
if "source_images" in f:
src_images = [s.decode() if isinstance(s, bytes) else s
for s in f["source_images"][:]]
# TRACES: GR-004 | SR-001 — carried through so a derived gallery (filter,
# TRACES: GR-004 | SR-001
# carried through so a derived gallery (filter,
# merge, cast-restrict) keeps the binding of the gallery it came from.
stamp = None
if "embedder" in f:
scripts/validation/ground_truth.py
+5
View File
@@ -17,6 +17,11 @@ X-Ray/MovieNet key on IMDb nm-ids — see [[per-scene-presence-eval-design]].
Two sources implemented:
* XRayGroundTruth — Zenodo scene-level Amazon X-Ray CSVs (cast-in-scene).
* MovieNetGroundTruth — MovieNet-PS per-shot face annotations (on-screen faces).
Both are published corpora addressed by title, so a scoring run is reproducible
from the identifiers alone — no annotation of ours travels with the code.
TRACES: VR-004 | PR-002
"""
from __future__ import annotations
scripts/validation/test_audio_offset.py
+1 -1
View File
@@ -51,7 +51,7 @@ sys.path.insert(0, str(BUILD))
import sae_audio # noqa: E402
FIXTURE = REPO / "tests" / "fixtures" / "audio" / "bali_offset_200s.flac"
FIXTURE = REPO / "tests" / "fixtures" / "audio" / "superhero_offset_200s.flac"
TONE = REPO / "tests" / "fixtures" / "audio" / "jray_audio_v1_tone.flac"
# Server SPEC.md section 3, "Matching and offset recovery". The cap is the
src/build_gallery.cpp
+5 -1
View File
@@ -34,6 +34,7 @@ int main(int argc, char** argv) {
std::string arcface_model = kDefaultArcfaceModel;
float conf = 0.5f, nms_thr = 0.4f;
int max_side = 500;
float min_face_px = 0.f;
for (int i = 1; i < argc; ++i) {
auto arg = [&](const char* f) { return std::strcmp(argv[i], f) == 0; };
@@ -50,6 +51,7 @@ int main(int argc, char** argv) {
else if (arg("--conf")) conf = std::stof(next());
else if (arg("--nms")) nms_thr = std::stof(next());
else if (arg("--max-side")) max_side = std::stoi(next());
else if (arg("--min-face-px")) min_face_px = std::stof(next());
else { std::cerr << "[warn] unknown flag: " << argv[i] << "\n"; }
} catch (const std::exception& e) {
std::cerr << "Error: " << e.what() << "\n";
@@ -59,7 +61,8 @@ int main(int argc, char** argv) {
if (root_path.empty() || output_path.empty()) {
std::cerr << "Usage: build_gallery --root <dir> --output <gallery.json> "
"[--detector <path>] [--arcface <path>] [--max-side <N>]\n";
"[--detector <path>] [--arcface <path>] [--max-side <N>]\n"
" [--min-face-px <px>]\n";
return 1;
}
@@ -70,6 +73,7 @@ int main(int argc, char** argv) {
cfg.detector_conf = conf;
cfg.detector_nms = nms_thr;
cfg.max_side = max_side;
cfg.min_face_px = min_face_px;
try {
ActorGallery gallery = build_gallery(cfg);
src/config.hpp
+4 -6
View File
@@ -153,16 +153,14 @@ struct Config {
// Banded admission for the per-subject store, in PROBABILITY space. An
// embedding joins only if P(same person) against something already stored
// lands inside [lo, hi]: above hi it is redundant, below lo it is evidence
// the track is not one person. Replaces expand_novelty_sim, a raw cosine.
// the track is not one person. The same lo is re-applied to the whole store
// at promotion time — see track_gallery.hpp. This is the only threshold the
// expansion path has: it replaces the raw-cosine expand_novelty_sim (0.55)
// and expand_track_spread_max (0.60), which are retired (AR-024).
// Working values pending VR-007; sweep both bounds, they fail in opposite
// directions.
float expand_band_lo{0.90f};
float expand_band_hi{0.95f};
float expand_novelty_sim{0.55f}; // promote only embeddings whose best sim to the
// actor's refs is below this (gallery-far / novel)
float expand_track_spread_max{0.60f}; // reject promotion if the retained buffer's
// internal spread (1 - min pairwise sim) exceeds
// this — guards track-ID collisions / two people
int expand_min_anchor_frames{3}; // require ≥N accepted frames naming the actor before
// the track is confirmed and its buffer promoted
std::string expand_debug_dir; // if set, dump promoted mugshots + embeddings here
src/gallery/gallery_builder.cpp
+21
View File
@@ -96,6 +96,27 @@ ActorGallery build_gallery(const BuildConfig& cfg) {
return a.confidence < b.confidence;
});
// Reject faces too small to embed honestly.
//
// The reference images are crops cut from the film, not mugshots, so
// the detected face can be a small fraction of the image. Upscaling a
// 30 px face to ArcFace's 112x112 feeds the model an input it was
// never trained for, and it answers with a confident, plausible,
// wrong embedding.
//
// At inference that costs one frame. Here it is permanent: a poisoned
// reference sits in the gallery and corrupts every future match
// against that character, which is exactly the kind of error that is
// invisible without a study that should not have been needed.
if (cfg.min_face_px > 0.f) {
const float side = std::min(best.bbox.width, best.bbox.height);
if (side < cfg.min_face_px) {
std::cerr << " [skip] face " << side << "px < " << cfg.min_face_px
<< "px: " << img_file.path().filename() << "\n";
continue;
}
}
cv::Mat crop = align_face(img, best.landmarks);
if (crop.empty()) {
std::cerr << " [skip] alignment failed: " << img_file.path().filename() << "\n";
src/gallery/gallery_builder.hpp
+10
View File
@@ -29,6 +29,16 @@ struct BuildConfig {
float detector_conf{0.5f};
float detector_nms{0.4f};
int max_side{500}; // downscale source images to this max dimension
/// Minimum detected-face side, in pixels of the (possibly downscaled)
/// source image. 0 disables the check.
///
/// References below this are dropped rather than upscaled: a face smaller
/// than the embedder's input is off-distribution, and a bad reference
/// poisons every match against that identity for the life of the gallery.
/// Mirrors the inference-side --min-face-px so the gallery is built from
/// the same face scales it will be matched against.
float min_face_px{0.f};
// before detection — TMDB portraits are ~2k px,
// SCRFD trains on smaller faces and detection
// confidence drops on huge inputs. 0 = disabled.
src/gallery/track_gallery.hpp
+61 -49
View File
@@ -36,13 +36,15 @@
// 2. Promotion (on confirmation). A track is "owned" by actor A once ≥N frames
// have been accepted (by the matcher's calibrated posterior) as A. On
// confirmation the retained buffer — the hard, gallery-far poses — is
// promoted into A's per-film annex, after two safety gates:
// • novelty: only embeddings whose best sim to A's refs is below
// expand_novelty_sim are added (skip poses already covered);
// • spread: if the retained buffer's internal spread (1 − min pairwise
// cosine sim) exceeds expand_track_spread_max the whole track is
// rejected — such spread signals a track-ID collision merging two
// people, whose embeddings must never enter A's annex.
// promoted into A's per-film annex, subject to one safety gate: the band's
// lower bound, re-applied across the whole store (see `store_coherence`).
//
// There is exactly one threshold here, the AR-018 band, and it is a calibrated
// probability. Novelty is no longer a threshold at all — the eviction policy
// above *orders* by gallery similarity rather than cutting at a constant, and
// the band's upper bound refuses the redundant views at the door. The raw
// cosines this replaces, expand_novelty_sim and expand_track_spread_max, are
// retired under AR-024.
//
// The annex is CPU-side and in-memory: it is small (tens of embeddings) so the
// matcher scans it with a scalar loop, and it is discarded when the process
@@ -59,16 +61,15 @@ struct TrackGallery {
explicit TrackGallery(const Config& cfg)
: enabled_(cfg.expand_gallery)
, buffer_size_(std::max(1, cfg.expand_buffer_size))
, novelty_sim_(cfg.expand_novelty_sim)
, spread_max_(cfg.expand_track_spread_max)
, band_lo_(cfg.expand_band_lo)
, band_hi_(cfg.expand_band_hi)
, min_anchor_frames_(std::max(1, cfg.expand_min_anchor_frames))
, debug_dir_(cfg.expand_debug_dir)
{
if (!enabled_) return;
std::cerr << "[track_gallery] per-film expansion ON"
<< " buffer=" << buffer_size_
<< " novelty_sim<" << novelty_sim_
<< " spread_max=" << spread_max_
<< " band=[" << band_lo_ << ", " << band_hi_ << "]"
<< " min_anchor_frames=" << min_anchor_frames_;
if (!debug_dir_.empty()) {
std::filesystem::create_directories(debug_dir_);
@@ -88,7 +89,9 @@ struct TrackGallery {
// track_id : face_tracker track (−1 = untracked, ignored)
// emb : this frame's raw embedding
// best_actor : actor with the highest gallery similarity for this face
// best_gal_sim : that similarity (best sim to best_actor's baked+annex refs)
// best_gal_sim : that similarity (best sim to best_actor's baked+annex
// refs) — a raw cosine, the last one in this class: it is
// calibrated on entry and only the probability is stored
// accepted : true if the matcher accepted this face as best_actor
// crop : aligned crop, retained only when debug dumping is on
void observe(int track_id, const Embedding& emb,
@@ -134,7 +137,6 @@ struct TrackGallery {
/// band falls back to treating cosine as probability, which is wrong but
/// bounded — and the default is loud in the header rather than silent.
void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }
void set_band(float lo, float hi) { band_lo_ = lo; band_hi_ = hi; }
/// Embeddings the band refused. A store that admits nothing is as wrong as
/// one that admits everything, and neither is visible without this.
@@ -146,7 +148,10 @@ struct TrackGallery {
private:
struct BufEntry {
Embedding emb;
float gal_sim{0.f}; // best sim to owning actor's refs when observed
/// P(same person) against the owning actor's refs when observed —
/// calibrated at the door (AR-024), so the eviction ordering below is a
/// comparison of probabilities and the struct holds no bare cosine.
float gal_p{0.f};
cv::Mat crop; // populated only when debug_dir_ set
};
@@ -193,8 +198,8 @@ private:
if (!admit(ts, emb)) { ++rejected_; return; }
BufEntry e;
e.emb = emb;
e.gal_sim = gal_sim;
e.emb = emb;
e.gal_p = calibrate_(gal_sim);
if (!debug_dir_.empty() && !crop.empty()) e.crop = crop.clone();
if (static_cast<int>(ts.buf.size()) < buffer_size_) {
@@ -203,13 +208,17 @@ private:
}
// Buffer full: evict the member the gallery recognises best (highest
// gal_sim) — least informative — but only if the newcomer is at least as
// gal_p) — least informative — but only if the newcomer is at least as
// novel. Keeping the most gallery-far views is the whole point.
int worst_i = -1;
float worst_sim = e.gal_sim; // newcomer's sim is the bar to beat
//
// This is an *ordering*, not a threshold: there is no constant to tune,
// and novelty-seeking lives here rather than in a cutoff. It ranks
// probabilities, so it says the same thing across models (AR-024).
int worst_i = -1;
float worst_p = e.gal_p; // newcomer's probability is the bar to beat
for (int i = 0; i < static_cast<int>(ts.buf.size()); ++i) {
if (ts.buf[i].gal_sim > worst_sim) {
worst_sim = ts.buf[i].gal_sim;
if (ts.buf[i].gal_p > worst_p) {
worst_p = ts.buf[i].gal_p;
worst_i = i;
}
}
@@ -224,25 +233,18 @@ private:
int actor = owning_actor(ts);
if (actor < 0) return;
// ── Safety gate: internal spread ─────────────────────────────────────
// A legitimate single-person track varies in pose but stays reasonably
// self-similar. Large spread signals two people merged under one track
// ID — reject the whole track rather than poison the actor's annex.
float spread = buffer_spread(ts.buf);
if (spread > spread_max_) {
// ── Safety gate: the band's lower bound, across the whole store ──────
float worst = store_coherence(ts.buf);
if (worst < band_lo_) {
std::cerr << "[track_gallery] track " << track_id
<< " → actor " << actor
<< " REJECTED (spread " << spread
<< " > " << spread_max_ << ", likely ID collision)\n";
<< " REJECTED (worst pairwise P=" << worst
<< " < " << band_lo_ << ", likely ID collision)\n";
return;
}
int added = 0;
for (const auto& be : ts.buf) {
// ── Safety gate: novelty ─────────────────────────────────────────
// Skip poses the gallery already covers; only gallery-far views are
// worth the annex slot (and the extra per-frame scan cost).
if (be.gal_sim >= novelty_sim_) continue;
annex_.push_back({be.emb, actor});
if (!debug_dir_.empty() && !be.crop.empty())
dump_mugshot(track_id, actor, added, be);
@@ -251,10 +253,9 @@ private:
std::cerr << "[track_gallery] track " << track_id
<< " confirmed actor " << actor
<< " (" << ts.accepted_frames << " accepted frames, spread "
<< spread << ") — promoted " << added << "/"
<< ts.buf.size() << " views; annex now "
<< annex_.size() << "\n";
<< " (" << ts.accepted_frames << " accepted frames, worst "
<< "pairwise P=" << worst << ") — promoted " << added
<< " views; annex now " << annex_.size() << "\n";
}
/// Prefer the registry's verdict; fall back to the local tally only when no
@@ -272,21 +273,34 @@ private:
return best;
}
// Spread = 1 − min pairwise cosine similarity over the buffer (0 when <2).
static float buffer_spread(const std::vector<BufEntry>& buf) {
float min_sim = std::numeric_limits<float>::max();
/// TRACES: AR-018, AR-024 | SR-005
/// The store's weakest pairwise P(same person) — the band's lower bound
/// asked of every pair, not just of the best match at the door.
///
/// `admit` compares a newcomer against its *closest* existing member, so a
/// track that drifts gradually can chain A→B→C with every step inside the
/// band while A and C are strangers. That is precisely the shape a track-ID
/// collision takes when two people are merged over a slow pan, so the bound
/// is re-asked here across all pairs before anything reaches an actor's
/// annex. Same bound, same probability space — not a second constant.
///
/// A store of one has no pair to disagree; it is coherent by construction,
/// hence 1.
float store_coherence(const std::vector<BufEntry>& buf) const {
float worst = std::numeric_limits<float>::max();
for (size_t i = 0; i < buf.size(); ++i)
for (size_t j = i + 1; j < buf.size(); ++j)
min_sim = std::min(min_sim, cosine_similarity(buf[i].emb, buf[j].emb));
if (min_sim == std::numeric_limits<float>::max()) return 0.f;
return 1.f - min_sim;
worst = std::min(worst,
calibrate_(cosine_similarity(buf[i].emb, buf[j].emb)));
if (worst == std::numeric_limits<float>::max()) return 1.f;
return worst;
}
void dump_mugshot(int track_id, int actor, int idx, const BufEntry& be) {
#ifdef SAE_DEBUG
char name[64];
std::snprintf(name, sizeof(name), "trk%d_actor%d_%d_sim%.3f.jpg",
track_id, actor, idx, be.gal_sim);
std::snprintf(name, sizeof(name), "trk%d_actor%d_%d_p%.3f.jpg",
track_id, actor, idx, be.gal_p);
cv::imwrite((std::filesystem::path(debug_dir_) / name).string(), be.crop);
#else
(void)track_id; (void)actor; (void)idx; (void)be;
@@ -296,14 +310,12 @@ private:
/// cosine → P(same person). The one probability space the pipeline reasons
/// in; see gallery_calibration.hpp's same_person_probability.
std::function<float(float)> calibrate_{[](float c) { return std::max(0.f, c); }};
float band_lo_{0.90f};
float band_hi_{0.95f};
std::size_t rejected_{0}; ///< admissions refused by the band
bool enabled_;
int buffer_size_;
float novelty_sim_;
float spread_max_;
float band_lo_; ///< AR-018, from cfg.expand_band_lo
float band_hi_; ///< AR-018, from cfg.expand_band_hi
int min_anchor_frames_;
std::string debug_dir_;
src/main.cpp
+24 -4
View File
@@ -39,8 +39,8 @@
// --max-faces <N> max faces kept per frame (default: 10)
// --expand-gallery enable per-film gallery expansion from track continuity
// --expand-buffer <N> per-track diversity buffer size (default: 20)
// --expand-novelty-sim <f> promote only views with best sim < f (default: 0.55)
// --expand-spread-max <f> reject track if buffer spread > f (default: 0.60)
// --expand-band-lo <p> store admission floor, P(same person) (default: 0.90)
// --expand-band-hi <p> store admission ceiling, P(same person) (default: 0.95)
// --expand-min-anchor <N> accepted frames before a track confirms (default: 3)
// --expand-debug-dir <p> dump promoted mugshots + embeddings here (SAE_DEBUG)
// (SAE_DEBUG only)
@@ -147,8 +147,8 @@ static Config parse_args(int argc, char** argv) {
else if (arg("--anneal")) cfg.anneal_sec = std::stod(next());
else if (arg("--expand-gallery")) cfg.expand_gallery = true;
else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next());
else if (arg("--expand-novelty-sim")) cfg.expand_novelty_sim = std::stof(next());
else if (arg("--expand-spread-max")) cfg.expand_track_spread_max = std::stof(next());
else if (arg("--expand-band-lo")) cfg.expand_band_lo = std::stof(next());
else if (arg("--expand-band-hi")) cfg.expand_band_hi = std::stof(next());
else if (arg("--expand-min-anchor")) cfg.expand_min_anchor_frames = std::stoi(next());
else if (arg("--expand-debug-dir")) cfg.expand_debug_dir = next();
else if (arg("--trt-cache")) cfg.trt.cache_dir = next();
@@ -281,6 +281,26 @@ int main(int argc, char** argv) {
}
});
// Report *why* a node died. A Closed event alone says only that one
// stopped; the exception it carried is what identifies the fault, and
// without this listener it is discarded at the node boundary. Returning
// false keeps the existing semantics — the node still stops and the
// Closed handler above still aborts the run — but the run now names the
// cause instead of leaving it to be reconstructed from a debugger.
net.set_error_handler(
[&](std::string_view node_name, std::exception_ptr eptr) {
std::string what = "unknown exception";
try {
if (eptr) std::rethrow_exception(eptr);
} catch (const std::exception& e) {
what = e.what();
} catch (...) {
}
std::lock_guard<std::mutex> lk(event_mtx);
std::cerr << "[main] node '" << node_name << "' threw: " << what << "\n";
return false;
});
std::cerr << "[main] starting pipeline…\n";
net.start();
src/nodes/face_aligner_node.hpp
+7 -3
View File
@@ -4,12 +4,16 @@
#include <iostream>
// ── FaceAlignerFunc ───────────────────────────────────────────────────────────
/// TRACES: AR-005, AR-030 | SR-002
///
// KPN node: applies a 5-point similarity transform to each detected face,
// producing a 112×112 BGR crop suitable for ArcFace inference.
//
// Alignment uses cv::estimateAffinePartial2D (RANSAC) to fit the detected
// landmarks to ArcFace canonical positions. Degenerate detections (where the
// affine fit fails) are silently dropped from the output vectors.
// Alignment is an Umeyama least-squares fit over all five landmarks (AR-005),
// not a robust one: a RANSAC fit discards the very landmarks AR-030 reads.
// Degenerate detections (where the fit fails) are dropped from the output
// vectors. The fit's residual is the AR-030 visibility measure and comes free,
// since the warp needs the transform anyway.
struct FaceAlignerFunc {
static constexpr std::string_view label() { return "face_aligner"; }
src/scene_preview.cpp
+2 -2
View File
@@ -96,8 +96,8 @@ static Config parse_args(int argc, char** argv) {
// Per-film gallery expansion — preview supports it (same cfg fields).
else if (arg("--expand-gallery")) cfg.expand_gallery = true;
else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next());
else if (arg("--expand-novelty-sim")) cfg.expand_novelty_sim = std::stof(next());
else if (arg("--expand-spread-max")) cfg.expand_track_spread_max = std::stof(next());
else if (arg("--expand-band-lo")) cfg.expand_band_lo = std::stof(next());
else if (arg("--expand-band-hi")) cfg.expand_band_hi = std::stof(next());
else if (arg("--expand-min-anchor")) cfg.expand_min_anchor_frames = std::stoi(next());
// Scene detection is scene_analyze-only (needs the dense TransNetV2 branch).
// Accept the flags so a shared command line runs, but note they're inert
tests/fixtures/audio/make_offset_fixture.sh
Vendored
+6 -6
View File
@@ -1,6 +1,6 @@
#!/bin/sh
#
# Regenerate bali_offset_200s.flac — the real-audio fixture behind VR-014, the
# Regenerate superhero_offset_200s.flac — the real-audio fixture behind VR-014, the
# audio-signature offset-recovery validation.
#
# sh make_offset_fixture.sh /path/to/clips
@@ -12,8 +12,8 @@
# one signature against another finds the true alignment and only the true
# alignment. Tones are pathologically easy for that; dialogue and score are not.
#
# Source: five scene clips from "Road to Bali" (1952), the public-domain corpus
# this repo already uses for the replay fixtures — tests/fixtures/dumps/bali_*.h5
# Source: scene clips from SuperHero (TRECVID DVU development set), the corpus
# this repo already uses for the replay fixtures — tests/fixtures/dumps/superhero.h5
# are dumps of these same clips. Each is under the 120 s window on its own
# (29-77 s), so they are concatenated in scene order to make a source long
# enough that a 120 s window can slide inside it.
@@ -41,13 +41,13 @@
set -eu
CLIPS="${1:-../../../../bali}"
OUT="$(dirname "$0")/bali_offset_200s.flac"
CLIPS="${1:-../../../../hero}"
OUT="$(dirname "$0")/superhero_offset_200s.flac"
LIST="$(mktemp)"
trap 'rm -f "$LIST"' EXIT
for scene in 13 27 28 31 46; do
clip="$CLIPS/Road_To_Bali-$scene.webm"
clip="$CLIPS/SuperHero-$scene.webm"
[ -f "$clip" ] || { echo "missing clip: $clip" >&2; exit 1; }
echo "file '$(cd "$(dirname "$clip")" && pwd)/$(basename "$clip")'" >> "$LIST"
done
tests/fixtures/dumps/.gitignore
+17
View File
@@ -0,0 +1,17 @@
# Replay fixtures are distributed as artifacts, not through git.
#
# They are large (superhero.h5 is ~9 MB) and regenerating one needs the film,
# the models and a GPU — none of which CI has. So they live in the Gitea
# generic package registry and are fetched on demand:
#
# scripts/artifacts/pull_artifacts.sh replay-fixtures [version]
# scripts/artifacts/push_artifacts.sh replay-fixtures
#
# The gallery ships alongside the dumps deliberately: a dump only replays
# meaningfully against the gallery it was produced with.
#
# bali_*.h5 predate this and remain tracked; do not add more to git.
superhero.h5
hero66.h5
gt.json
scene_bounds.json
tests/test_calibration.cpp
+9
View File
@@ -1,6 +1,12 @@
// TRACES: AR-023 | SR-002
//
// Unit tests for gallery calibration: the sigmoid math, the pairwise fit on
// separable data, and the in-memory hash-keyed cache (hit / stale / cold).
// All pure, GPU-free, model-free.
//
// The three `[report]` cases at the bottom carry their own GR-003 tags: they
// verify the build report, which is fitted from the same distributions but is a
// separate requirement.
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>
@@ -183,6 +189,7 @@ Embedding unit_axis(int slot) {
}
} // namespace
// TRACES: GR-003 | SR-001
TEST_CASE("report surfaces actors that can never be recognised", "[report][GR-003]") {
// An actor with no usable image is a silent recall ceiling: the pipeline
// will never name them, and nothing in the gallery says why. This is the
@@ -212,6 +219,7 @@ TEST_CASE("report surfaces actors that can never be recognised", "[report][GR-00
CHECK(r.actors[1].references == 0);
}
// TRACES: GR-003 | SR-001
TEST_CASE("report surfaces actors too thin to calibrate on", "[report][GR-003]") {
// Below the positive-pair threshold an actor contributes nothing to the
// intra-class side of the fit. They are not broken, so nothing complains —
@@ -237,6 +245,7 @@ TEST_CASE("report surfaces actors too thin to calibrate on", "[report][GR-003]")
CHECK(r.actors_below_positive_threshold >= 1);
}
// TRACES: GR-003 | SR-001
TEST_CASE("report round-trips", "[report][GR-003]") {
ActorGallery g;
ActorGallery::Actor act;
tests/test_face_tracker.cpp
+9 -5
View File
@@ -1,13 +1,17 @@
// TRACES: AR-007, AR-008 | SR-002
//
// Unit tests for FaceTrackerFunc (nodes/face_tracker_node.hpp): frame-to-frame
// track linking and, crucially, cross-cut re-association. Pure, GPU-free,
// model-free — drives the node's operator() with hand-built EmbeddedSceneFrames
// and inspects the emitted track_ids.
//
// The behaviour under test: on a camera-angle change (Frame::is_cut) the tracker
// parks its tracks instead of destroying them, and revives a parked track_id
// when a post-cut detection's raw last-frame-embedding cosine similarity clears
// cut_revive_sim. IoU is deliberately driven to 0 across the cut (boxes moved) so
// only the embedding path can re-link — exactly the scenario a cut creates.
// The behaviour under test: there is one track pool keyed on `last_seen`
// (AR-008), so a face lost across a camera-angle change (Frame::is_cut) is an
// ordinary association candidate rather than a parked track needing a revival
// path — the raw-cosine `cut_revive_sim` that guarded that path is retired
// (AR-024). On a cut the association weight drops to embedding-only (AR-007),
// and IoU is deliberately driven to 0 across the cut (boxes moved) so only the
// embedding path can re-link — exactly the scenario a cut creates.
#include <catch2/catch_test_macros.hpp>
#include "config.hpp"
tests/test_replay_fixtures.cpp
+17 -65
View File
@@ -153,60 +153,34 @@ Replay run(const Dump& d, double extinction = 10.0) {
} // namespace
// ── AR-004 / VR-001 — the fixtures are intact and self-describing ────────────
TEST_CASE("fixtures are complete and carry their embedder identity",
"[replay][AR-004][VR-001]") {
// Frame counts are exact rather than approximate. Before node outputs
// blocked on a full channel, generation lost most of a clip and what it
// lost depended on timing — these numbers could not have been asserted.
struct Expect { const char* file; std::size_t frames, faces; };
const Expect all[] = {
{"bali_13.h5", 385, 693},
{"bali_27.h5", 335, 335},
{"bali_28.h5", 345, 368},
{"bali_31.h5", 145, 203},
{"bali_46.h5", 385, 140},
};
TEST_CASE("superhero fixture is complete", "[replay][VR-001]") {
Dump d = load(fixture("superhero.h5"));
CHECK(d.frames() == 5128);
CHECK(d.faces() == 4307);
CHECK(d.embedder == "LVFace-B_Glint360K.onnx");
for (const auto& x : all) {
INFO(x.file);
Dump d = load(fixture(x.file));
CHECK(d.frames() == x.frames);
CHECK(d.faces() == x.faces);
CHECK(d.embedder == "LVFace-B_Glint360K.onnx");
// face_offset must be contiguous: a gap means faces went missing
// between frames, which no consumer could detect.
int64_t running = 0;
for (std::size_t i = 0; i < d.frames(); ++i) {
REQUIRE(d.face_offset[i] == running);
running += d.face_count[i];
}
CHECK(static_cast<std::size_t>(running) == d.faces());
int64_t running = 0;
for (std::size_t i = 0; i < d.frames(); ++i) {
REQUIRE(d.face_offset[i] == running);
running += d.face_count[i];
}
CHECK(static_cast<std::size_t>(running) == d.faces());
}
// ── VR-002 — replay is deterministic ─────────────────────────────────────────
TEST_CASE("replaying a fixture twice gives identical tracks", "[replay][VR-002]") {
// The property the whole fixture strategy rests on. If this fails, every
// golden output derived from a fixture is unreliable and the CI replay
// tier is worthless.
Dump d = load(fixture("bali_28.h5"));
TEST_CASE("replaying the superhero fixture twice gives identical tracks",
"[replay][VR-002]") {
Dump d = load(fixture("superhero.h5"));
Replay a = run(d);
Replay b = run(d);
REQUIRE(a.track_ids.size() == b.track_ids.size());
CHECK(a.track_ids == b.track_ids);
REQUIRE(a.claims.size() == b.claims.size());
for (std::size_t i = 0; i < a.claims.size(); ++i) {
CHECK(a.claims[i].first_seen == b.claims[i].first_seen);
CHECK(a.claims[i].last_seen == b.claims[i].last_seen);
}
}
// ── AR-012 / AR-013 — window invariants on real footage ──────────────────────
TEST_CASE("every face is assigned a track and every track closes",
"[replay][AR-012]") {
Dump d = load(fixture("bali_13.h5"));
Dump d = load(fixture("superhero.h5"));
Replay r = run(d);
CHECK(r.track_ids.size() == r.faces_seen);
@@ -217,9 +191,9 @@ TEST_CASE("every face is assigned a track and every track closes",
CHECK(r.claims.size() > 0);
}
TEST_CASE("windows are well-formed and inside the clip", "[replay][AR-013]") {
for (const char* f : {"bali_13.h5", "bali_27.h5", "bali_28.h5",
"bali_31.h5", "bali_46.h5"}) {
TEST_CASE("windows are well-formed and inside the film", "[replay][AR-013]") {
for (const char* f : {"superhero.h5", "superhero.h5", "superhero.h5",
"superhero.h5", "superhero.h5"}) {
INFO(f);
Dump d = load(fixture(f));
Replay r = run(d);
@@ -234,25 +208,3 @@ TEST_CASE("windows are well-formed and inside the clip", "[replay][AR-013]") {
}
}
}
TEST_CASE("a longer extinction window yields fewer, longer tracks",
"[replay][AR-013]") {
// The timeout decides whether a gap is absorbed into one window or splits
// it in two, so lengthening it must merge tracks rather than multiply them.
// On sparse footage this is the difference the constant actually makes.
Dump d = load(fixture("bali_46.h5")); // 140 faces over 385 frames
Replay tight = run(d, /*extinction=*/1.0);
Replay loose = run(d, /*extinction=*/30.0);
CHECK(loose.claims.size() <= tight.claims.size());
}
// ── AR-007 — cuts are exercised by the corpus, not just by construction ──────
TEST_CASE("the cut-heavy fixture actually contains cuts", "[replay][AR-007]") {
// Guards the corpus rather than the code: if a regeneration produced a
// fixture with no cuts, the association tests above would still pass while
// silently testing nothing about viewpoint changes.
Dump d = load(fixture("bali_28.h5"));
const int cuts = std::count(d.is_cut.begin(), d.is_cut.end(), uint8_t{1});
CHECK(cuts >= 5);
}
tests/test_similarity.cpp
+6
View File
@@ -1,5 +1,11 @@
// TRACES: AR-026 | SR-001
//
// Unit tests for the CPU reference similarity engine (backends/gemm_backend.cpp,
// SAE_GEMM_CPU) and the l2_normalise helper. All pure, GPU-free, model-free.
//
// This is the CI half of AR-026: equivalence between the GEMM path and
// hand-computed dot products on small input. Throughput at scale (AR-027) is T4
// and cannot run here.
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>
tests/test_track_gallery.cpp
+238 -63
View File
@@ -1,7 +1,16 @@
// TRACES: AR-018, AR-019, AR-024 | SR-005
//
// Unit tests for TrackGallery (gallery/track_gallery.hpp): per-film gallery
// expansion driven by track continuity. Pure, GPU-free, model-free — exercises
// the diversity-buffer eviction policy, the novelty/spread safety gates,
// plurality ownership, and idempotent promotion via the public interface.
// the AR-018 banded admission at both bounds, the promotion-time coherence
// gate, the diversity-buffer eviction policy, plurality ownership, and
// idempotent promotion, all through the public interface.
//
// The band is defined in PROBABILITY space (AR-024), so every case below states
// its own cosine → probability map instead of inheriting the header's fallback.
// A test that never names the mapping is not testing the band, it is testing a
// coincidence: with the fallback the two spaces happen to coincide, and a gate
// that silently reverted to raw cosine would still pass.
#include <catch2/catch_test_macros.hpp>
#include <catch2/matchers/catch_matchers_floating_point.hpp>
@@ -14,30 +23,60 @@
namespace {
// Unit-norm embedding pointing along one axis (cosine sim to another one-hot is
// 0, to itself 1) — lets tests dial gallery similarity precisely.
constexpr float kPi = 3.14159265358979323846f;
// The band as the tests drive it. Kept in one place so a change to the shipped
// defaults does not silently invalidate the arithmetic in each case.
constexpr float kBandLo = 0.90f;
constexpr float kBandHi = 0.95f;
// Identity map: probability == cosine, so a case can place an embedding at an
// exact probability. cosine_similarity is a bare dot product over unit vectors
// (types.hpp), so the placements below are bit-exact, not approximate.
float identity_cal(float c) { return c; }
// Unit-norm embedding pointing along one axis. Cosine sim to another one-hot is
// 0, to itself 1.
Embedding one_hot(int slot) {
Embedding e{};
e[slot] = 1.0f;
return e;
}
// Unit-norm embedding in the plane of axes i,j at angle t from i. Cosine sim to
// one_hot(i) is cos(t) — used to place a view at a chosen gallery similarity.
// Unit-norm embedding in the plane of axes i,j at cosine `cos_t` from axis i.
// Cosine sim to one_hot(i) is exactly cos_t.
Embedding at_sim(int i, int j, float cos_t) {
Embedding e{};
float s = std::sqrt(std::max(0.f, 1.f - cos_t * cos_t));
e[i] = cos_t;
e[j] = s;
e[j] = std::sqrt(std::max(0.f, 1.f - cos_t * cos_t));
return e;
}
// A spoke: shares axis 0 with every other spoke, and is otherwise unique. Any
// two DISTINCT spokes have cosine similarity exactly cos_t², so one constant
// places a whole mutually-in-band store. A spoke against itself is 1.0 — above
// the band's ceiling, i.e. redundant, which is the intended reading.
Embedding spoke(int k, float cos_t) { return at_sim(0, k, cos_t); }
// cos_t chosen so pairwise similarity between distinct spokes is 0.9197 —
// comfortably inside [0.90, 0.95], clear of both bounds.
constexpr float kSpokeCos = 0.959f;
// Two embeddings `deg` apart in the plane of axes 0,1. Cosine is cos(deg), so a
// chain of these can step through the band while its endpoints fall outside it.
Embedding on_circle(float deg) {
Embedding e{};
e[0] = std::cos(deg * kPi / 180.f);
e[1] = std::sin(deg * kPi / 180.f);
return e;
}
Config expand_cfg() {
Config cfg;
cfg.expand_gallery = true;
cfg.expand_buffer_size = 3;
cfg.expand_novelty_sim = 0.55f;
cfg.expand_track_spread_max = 0.60f;
cfg.expand_gallery = true;
cfg.expand_buffer_size = 3;
cfg.expand_band_lo = kBandLo;
cfg.expand_band_hi = kBandHi;
cfg.expand_min_anchor_frames = 3;
return cfg;
}
@@ -59,92 +98,228 @@ TEST_CASE("disabled: no annex growth when expand_gallery is off", "[track_galler
CHECK(tg.annex().empty());
}
TEST_CASE("confirmed track promotes gallery-far views", "[track_gallery]") {
TrackGallery tg(expand_cfg());
REQUIRE(tg.enabled());
// ── AR-018: the band ─────────────────────────────────────────────────────────
// A track owned by actor 0. Every frame is accepted as actor 0, but each
// view is gallery-far (sim 0.30 < novelty 0.55) yet mutually self-similar
// enough to pass the spread gate.
for (int f = 0; f < 3; ++f)
tg.observe(7, at_sim(0, 1, 0.30f + 0.001f * f), 0, 0.30f + 0.001f * f, true, kNoCrop);
TEST_CASE("band bounds come from config, not a hardcoded default", "[track_gallery][AR-018]") {
// The bounds were declared in Config and read nowhere, so the gate ran at
// whatever the header happened to initialise. Drive them somewhere the
// defaults are not and require the gate to follow.
Config cfg = expand_cfg();
cfg.expand_band_lo = 0.40f;
cfg.expand_band_hi = 0.60f;
TrackGallery tg(cfg);
tg.set_calibration(identity_cal);
// 3 accepted frames == min_anchor_frames → confirmed and promoted.
CHECK_FALSE(tg.annex().empty());
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 0);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
// P = 0.50: inside the configured band, far below the shipped default lo.
tg.observe(1, at_sim(0, 1, 0.50f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 0);
// P = 0.92: inside the shipped default band, above the configured ceiling.
tg.observe(1, at_sim(0, 2, 0.92f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 1);
}
TEST_CASE("novelty gate skips views the gallery already covers", "[track_gallery]") {
TEST_CASE("band admits at each bound exactly", "[track_gallery][AR-018]") {
// The verification plan asks for the bounds themselves, not a point safely
// inside them: an off-by-one in the comparison is invisible anywhere else.
// Both bounds are inclusive.
SECTION("lower bound exactly") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
tg.observe(1, at_sim(0, 1, kBandLo), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 0);
}
SECTION("upper bound exactly") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
tg.observe(1, at_sim(0, 1, kBandHi), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 0);
}
}
TEST_CASE("store never admits below the lower bound", "[track_gallery][AR-018]") {
// The lower bound is the poisoning guard: an embedding unlike everything
// already on the track is evidence the track is not one person.
TrackGallery tg(expand_cfg());
// All views are recognised well (sim 0.90 ≥ novelty 0.55): nothing worth
// promoting even though the track is confirmed.
for (int f = 0; f < 3; ++f)
tg.observe(2, one_hot(0), 0, 0.90f, true, kNoCrop);
CHECK(tg.annex().empty());
tg.set_calibration(identity_cal);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
tg.observe(1, at_sim(0, 1, kBandLo - 0.01f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 1);
tg.observe(1, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal: P = 0
CHECK(tg.band_rejected() == 2);
}
TEST_CASE("store never admits above the upper bound", "[track_gallery][AR-018]") {
// The upper bound is the redundancy guard: another look at a pose the store
// already covers teaches the annex nothing and costs a slot.
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
tg.observe(1, at_sim(0, 1, kBandHi + 0.01f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 1);
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop); // identical: P = 1
CHECK(tg.band_rejected() == 2);
}
TEST_CASE("band thresholds probability, not cosine", "[track_gallery][AR-018][AR-024]") {
// The invariant's actual claim, and the one a raw-cosine gate passes by
// accident under an identity calibration. With a calibration that shifts by
// +0.10, two embeddings get the OPPOSITE verdict from the one their bare
// cosines would earn — so admission here can only come from the calibrated
// value having been used.
TrackGallery tg(expand_cfg());
tg.set_calibration([](float c) { return c + 0.10f; });
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
// cosine 0.84 (below lo, would be refused raw) → P = 0.94, inside the band.
tg.observe(1, at_sim(0, 1, 0.84f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 0);
// cosine 0.92 (inside the band, would be admitted raw) → P = 1.02, above it.
tg.observe(1, at_sim(0, 2, 0.92f), 0, 0.30f, true, kNoCrop);
CHECK(tg.band_rejected() == 1);
}
TEST_CASE("a two-person track never poisons the annex", "[track_gallery][AR-018]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
// Two orthogonal identities under one track ID — a track-ID collision.
//
// The banded admission (AR-018) now catches this EARLIER than the spread
// gate did: an embedding unlike everything already on the track falls below
// the band's lower bound and is refused entry, so the buffer never becomes
// two-person in the first place. The spread gate remains as a second line
// for a track that drifts gradually rather than jumping.
//
// The assertion is on the outcome, not the mechanism: whichever gate fires,
// the outsider must not reach the actor's annex.
tg.observe(3, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop);
tg.observe(3, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop);
// The band refuses the outsider at the door, so the store never becomes
// two-person in the first place.
tg.observe(3, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(3, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(3, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal outlier
CHECK(tg.band_rejected() > 0); // refused at the door
CHECK(tg.band_rejected() == 1); // refused at the door
REQUIRE_FALSE(tg.annex().empty()); // the legitimate views still promote
for (const auto& e : tg.annex())
CHECK(cosine_similarity(e.emb, one_hot(400)) < 0.5f);
}
TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_gallery]") {
TEST_CASE("a track that drifts through the band is refused at promotion",
"[track_gallery][AR-018]") {
// `admit` compares a newcomer against its CLOSEST existing member, so a
// gradual drift chains past it: each step is in-band while the endpoints are
// strangers. This is the shape a collision takes over a slow pan, and the
// reason the lower bound is re-asked across every pair before promotion.
TrackGallery tg(expand_cfg());
// Only 2 accepted frames < min_anchor_frames 3; extra non-accepted frames
// fill the buffer but don't count toward ownership.
tg.observe(4, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop);
tg.observe(4, at_sim(0, 1, 0.31f), 0, 0.31f, true, kNoCrop);
tg.observe(4, at_sim(0, 1, 0.32f), 0, 0.32f, false, kNoCrop);
tg.set_calibration(identity_cal);
tg.observe(5, on_circle(0.f), 0, 0.30f, true, kNoCrop);
tg.observe(5, on_circle(25.f), 0, 0.30f, true, kNoCrop); // P=0.906 vs 0° → in band
tg.observe(5, on_circle(50.f), 0, 0.30f, true, kNoCrop); // P=0.906 vs 25° → in band
CHECK(tg.band_rejected() == 0); // every step passed the door...
// ...but 0° and 50° are P=0.643 apart, below the floor: the whole track goes.
CHECK(tg.annex().empty());
}
TEST_CASE("plurality actor wins a mixed-vote track", "[track_gallery]") {
Config cfg = expand_cfg();
cfg.expand_min_anchor_frames = 3;
TrackGallery tg(cfg);
// Actor 5 accepted twice, actor 6 once → plurality is 5. All views novel.
tg.observe(8, at_sim(0, 1, 0.30f), 5, 0.30f, true, kNoCrop);
tg.observe(8, at_sim(0, 1, 0.31f), 5, 0.31f, true, kNoCrop);
tg.observe(8, at_sim(0, 1, 0.32f), 6, 0.32f, true, kNoCrop);
// ── AR-019: ownership and promotion ──────────────────────────────────────────
TEST_CASE("confirmed track promotes its store", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
REQUIRE(tg.enabled());
// A track owned by actor 0: every frame accepted, every view mutually
// in-band (P = 0.9197 between distinct spokes) and gallery-far (0.30).
for (int k = 1; k <= 3; ++k)
tg.observe(7, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
// 3 accepted frames == min_anchor_frames → confirmed and promoted.
CHECK(tg.annex().size() == 3);
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 0);
}
TEST_CASE("registry ownership overrides the local tally", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
// Local accepted-frame plurality says actor 5; the registry's accumulated
// posterior says actor 9. The registry is authoritative.
tg.set_owner(11, 9);
for (int k = 1; k <= 3; ++k)
tg.observe(11, spoke(k, kSpokeCos), 5, 0.30f, true, kNoCrop);
REQUIRE_FALSE(tg.annex().empty());
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 9);
}
TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
// Only 2 accepted frames < min_anchor_frames 3; the third fills the buffer
// but doesn't count toward ownership.
tg.observe(4, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(4, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(4, spoke(3, kSpokeCos), 0, 0.30f, false, kNoCrop);
CHECK(tg.annex().empty());
}
TEST_CASE("plurality actor wins a mixed-vote track", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
// No registry attached (unit-test path): actor 5 accepted twice, actor 6
// once → plurality is 5.
tg.observe(8, spoke(1, kSpokeCos), 5, 0.30f, true, kNoCrop);
tg.observe(8, spoke(2, kSpokeCos), 5, 0.30f, true, kNoCrop);
tg.observe(8, spoke(3, kSpokeCos), 6, 0.30f, true, kNoCrop);
REQUIRE_FALSE(tg.annex().empty());
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 5);
}
TEST_CASE("promotion is idempotent across a long track", "[track_gallery]") {
TEST_CASE("promotion is idempotent across a long track", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
for (int f = 0; f < 3; ++f)
tg.observe(9, at_sim(0, 1, 0.30f + 0.001f * f), 0, 0.30f + 0.001f * f, true, kNoCrop);
tg.set_calibration(identity_cal);
for (int k = 1; k <= 3; ++k)
tg.observe(9, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
size_t after_confirm = tg.annex().size();
REQUIRE(after_confirm > 0);
// Keep feeding the confirmed track: annex must not grow again.
for (int f = 0; f < 10; ++f)
tg.observe(9, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop);
tg.observe(9, spoke(4 + f, kSpokeCos), 0, 0.30f, true, kNoCrop);
CHECK(tg.annex().size() == after_confirm);
}
TEST_CASE("clear_tracks drops buffers before confirmation", "[track_gallery]") {
TEST_CASE("clear_tracks drops buffers before confirmation", "[track_gallery][AR-019]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
// Two accepts, then a cut clears buffers; the third accept starts fresh and
// can't reach the anchor threshold on its own.
tg.observe(1, at_sim(0, 1, 0.30f), 0, 0.30f, true, kNoCrop);
tg.observe(1, at_sim(0, 1, 0.31f), 0, 0.31f, true, kNoCrop);
tg.observe(1, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(1, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.clear_tracks();
tg.observe(1, at_sim(0, 1, 0.32f), 0, 0.32f, true, kNoCrop);
tg.observe(1, spoke(3, kSpokeCos), 0, 0.30f, true, kNoCrop);
CHECK(tg.annex().empty());
}
TEST_CASE("eviction keeps the gallery-far views", "[track_gallery][AR-018]") {
// Novelty is no longer a threshold — it is this ordering. With the buffer
// full, a more gallery-far newcomer must displace the best-recognised
// member, and a less novel one must be dropped rather than displace a
// better sample.
Config cfg = expand_cfg();
cfg.expand_buffer_size = 2;
cfg.expand_min_anchor_frames = 4;
TrackGallery tg(cfg);
tg.set_calibration(identity_cal);
tg.observe(6, spoke(1, kSpokeCos), 0, 0.80f, true, kNoCrop); // well recognised
tg.observe(6, spoke(2, kSpokeCos), 0, 0.40f, true, kNoCrop);
tg.observe(6, spoke(3, kSpokeCos), 0, 0.20f, true, kNoCrop); // novel: evicts the 0.80
tg.observe(6, spoke(4, kSpokeCos), 0, 0.90f, true, kNoCrop); // least novel: dropped
REQUIRE(tg.annex().size() == 2);
// Survivors are the two most gallery-far views: spokes 2 and 3.
for (const auto& ae : tg.annex()) {
const bool is_2 = cosine_similarity(ae.emb, spoke(2, kSpokeCos)) > 0.99f;
const bool is_3 = cosine_similarity(ae.emb, spoke(3, kSpokeCos)) > 0.99f;
CHECK((is_2 || is_3));
}
}