feat(scene): feed TransNetV2 at native rate, derive the dedup window from it

Closes both violations SPEC.md named under "Every model gets the input it
was trained for". They are one bug, not two.

The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window
spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over
twice its temporal context. Boundary timestamps stayed correct throughout,
which is exactly why the degradation was invisible and why the compressed
separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a
property of the ONNX export rather than of the input.

Dedup then merged boundaries closer than a literal 0.04 s — one frame at
25 fps, and wider than a frame at 30, so two cuts on consecutive frames
became one. Nothing in scenes.json showed it; the file simply had fewer
boundaries. Native rate is where that constant did the most damage, which
is why fixing the decode rate without fixing the dedup would have made
things worse.

dedup_window_sec() now takes the median interval the detector was actually
fed and halves it. Half a frame rather than a whole one: the only thing
being merged is one frame scored by two overlapping windows, and two
distinct frames are a full interval apart.

Cost is real — dense decode is the pipeline's cost driver. It is accepted;
dense_scale and scene_stride remain the reductions that do not run the
model off-distribution. scene_threshold 0.60 was fitted against the 12 fps
input and is now stale, so VR-006 goes from Low to Medium: it is no longer
a refinement, it is a constant that no longer describes the input.

AR-002 rides along because it was already implemented, just untagged and
unverified — the register said Planned while the code was correct. The size
filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold
and at dense_scale 0.5, and checked end to end against the superhero dump,
whose smallest face is exactly its recorded 32 px minimum, so the fixture
check cannot pass vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001
This commit is contained in:
2026-08-04 21:17:57 +02:00
co-authored by Claude Opus 5
parent 079b490ede
commit f33403fff8
10 changed files with 378 additions and 44 deletions
+2
View File
@@ -22,6 +22,8 @@ add_executable(sae_tests
test_track_gallery.cpp
test_face_tracker.cpp
test_track_registry.cpp
test_face_detector_node.cpp
test_scene_detector_node.cpp
test_replay_fixtures.cpp
test_audio_signature.cpp
${CMAKE_SOURCE_DIR}/src/backends/gemm_backend.cpp
+114
View File
@@ -0,0 +1,114 @@
// AR-002 — minimum face size, in original video resolution.
//
// TRACES: AR-002 | SR-002 | UT-002
//
// Tier T1 here, T2 in test_replay_fixtures.cpp. The requirement is arithmetic on
// bounding boxes, so the two edge cases that matter — a face sitting exactly on
// the threshold, and the same face seen through a downscaled decode — are
// reachable without a detector, a model or a GPU. What the fixture check adds is
// that the rule was actually applied on the way to a dump; what this adds is that
// it is applied *correctly*, which no real dump can demonstrate because real
// footage does not contain a 39.999 px face on demand.
//
// FaceDetectorFunc is never constructed: its constructor loads SCRFD. Only the
// static rule is called, so make_face_detector() is never odr-used and nothing
// here needs a backend.
#include <catch2/catch_test_macros.hpp>
#include "nodes/face_detector_node.hpp"
#include "types.hpp"
#include <vector>
namespace {
DetectedFace box(float w, float h) {
DetectedFace f;
f.bbox = cv::Rect2f(10.f, 10.f, w, h);
f.confidence = 0.9f;
return f;
}
// Sizes the rule kept, in the order given.
std::vector<float> surviving_widths(std::vector<DetectedFace> faces,
float min_face_px, float bbox_upscale) {
FaceDetectorFunc::drop_undersized(faces, min_face_px, bbox_upscale);
std::vector<float> out;
for (const auto& f : faces) out.push_back(f.bbox.width);
return out;
}
constexpr float kMin = 40.f; // Config::min_face_px default, and AR-002's number
} // namespace
// ── Exactly at the threshold ─────────────────────────────────────────────────
// The boundary case is the whole content of a minimum: "40x40" has to mean 40 is
// admissible, or the requirement says 41.
TEST_CASE("a face exactly at the minimum is kept", "[detector][AR-002]") {
CHECK(surviving_widths({box(kMin, kMin)}, kMin, 1.f).size() == 1);
}
TEST_CASE("a face one tenth of a pixel under the minimum is dropped",
"[detector][AR-002]") {
CHECK(surviving_widths({box(39.9f, 100.f)}, kMin, 1.f).empty());
CHECK(surviving_widths({box(100.f, 39.9f)}, kMin, 1.f).empty());
}
TEST_CASE("both sides must clear the minimum, not the larger one",
"[detector][AR-002]") {
// A wide, short box has enough pixels and is still unusable: ArcFace
// alignment needs both dimensions. Area would admit this; the rule must not.
CHECK(surviving_widths({box(400.f, 20.f)}, kMin, 1.f).empty());
}
TEST_CASE("the filter is a filter, not a reordering", "[detector][AR-002]") {
auto kept = surviving_widths(
{box(80.f, 80.f), box(10.f, 10.f), box(60.f, 60.f), box(39.f, 39.f)},
kMin, 1.f);
REQUIRE(kept.size() == 2);
// Order is load-bearing downstream (AR-003's largest-first sort tie-breaks on
// it, and the Hungarian solver tie-breaks on index) — erase-remove must not
// shuffle the survivors.
CHECK(kept[0] == 80.f);
CHECK(kept[1] == 60.f);
}
// ── The dense_scale interaction the requirement exists for ───────────────────
// dense_scale 0.5 halves the decoded frame, so the detector reports a 40 px face
// as 20 px. If the threshold were applied to those numbers, turning on a
// throughput knob would silently double the minimum face size the pipeline
// accepts — a recall change with no line in the config to explain it. AR-002
// pins the minimum to the ORIGINAL resolution instead.
TEST_CASE("at dense_scale 0.5 the cutoff stays 40 px of original footage",
"[detector][AR-002]") {
constexpr float kUpscale = 2.f; // frame_source_node: 1 / dense_scale
// 20 px in downscaled space is exactly 40 px of original footage: kept.
CHECK(surviving_widths({box(20.f, 20.f)}, kMin, kUpscale).size() == 1);
// 19.9 px downscaled is 39.8 px original: dropped.
CHECK(surviving_widths({box(19.9f, 19.9f)}, kMin, kUpscale).empty());
// And the interaction stated as one claim: a face of a given original size is
// admitted or refused identically whether or not the decode was downscaled.
for (float original : {30.f, 39.f, 40.f, 41.f, 80.f}) {
INFO("original size " << original);
const bool full = !surviving_widths({box(original, original)},
kMin, 1.f).empty();
const bool dense = !surviving_widths({box(original / kUpscale,
original / kUpscale)},
kMin, kUpscale).empty();
CHECK(full == dense);
CHECK(full == (original >= kMin));
}
}
TEST_CASE("an absent or degenerate upscale falls back to the raw threshold",
"[detector][AR-002]") {
// bbox_upscale is 1 on every non-dense frame; 0 would mean the frame source
// never set it. Dividing by that would reject every face in the film, which
// is a failure worth not having.
CHECK(surviving_widths({box(kMin, kMin)}, kMin, 0.f).size() == 1);
CHECK(surviving_widths({box(39.f, 39.f)}, kMin, 0.f).empty());
}
+42 -1
View File
@@ -1,6 +1,6 @@
// Replay tests — the real tracker and registry driven from committed fixtures.
//
// TRACES: AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001
// TRACES: AR-002, AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001
//
// Tier T2: composition, not units. The registry tests construct awkward states
// directly; these check that the pieces behave when wired together and fed real
@@ -27,6 +27,7 @@
#include <algorithm>
#include <cmath>
#include <limits>
#include <memory>
#include <string>
#include <vector>
@@ -44,6 +45,8 @@ struct Dump {
std::vector<Embedding> emb;
std::vector<float> bbox; // 4 per face
std::string embedder;
float min_face_px{0.f}; // AR-002, as the run was configured
float bbox_upscale{1.f}; // bbox × this = original resolution
std::size_t frames() const { return ts.size(); }
std::size_t faces() const { return emb.size(); }
@@ -97,6 +100,17 @@ Dump load(const std::string& path) {
H5::StrType vlen(H5::PredType::C_S1, H5T_VARIABLE);
f.openAttribute("embedder_model").read(vlen, d.embedder);
}
// AR-002: the threshold the run was configured with, and the scale its boxes
// are in. Read from the dump rather than assumed, so the check is against
// what this fixture was actually generated with — the hero clips predate the
// move to 40 px and were dumped at 32.
if (f.attrExists("min_face_px"))
f.openAttribute("min_face_px").read(H5::PredType::NATIVE_FLOAT,
&d.min_face_px);
if (f.attrExists("bbox_upscale"))
f.openAttribute("bbox_upscale").read(H5::PredType::NATIVE_FLOAT,
&d.bbox_upscale);
return d;
}
@@ -167,6 +181,33 @@ TEST_CASE("superhero fixture is complete", "[replay][VR-001]") {
CHECK(static_cast<std::size_t>(running) == d.faces());
}
// ── AR-002 — the size filter held, all the way to the dump ───────────────────
// The T1 arithmetic is in test_face_detector_node.cpp. This is the other half:
// that the rule was applied on real footage and nothing downstream of it let an
// undersized face back in.
TEST_CASE("no dumped face is below the configured minimum size",
"[replay][AR-002]") {
Dump d = load(fixture("superhero.h5"));
// A dump that did not record its threshold cannot be checked against one.
REQUIRE(d.min_face_px > 0.f);
REQUIRE(d.bbox_upscale > 0.f);
float smallest_side = std::numeric_limits<float>::max();
for (std::size_t i = 0; i < d.faces(); ++i) {
const float w = d.bbox[i * 4 + 2] * d.bbox_upscale; // original resolution
const float h = d.bbox[i * 4 + 3] * d.bbox_upscale;
REQUIRE(w >= d.min_face_px);
REQUIRE(h >= d.min_face_px);
smallest_side = std::min({smallest_side, w, h});
}
// And the filter was binding, not vacuously satisfied. 480x360 footage puts
// faces right on the cutoff, which is what makes this fixture worth checking:
// if the threshold stopped being applied the assertions above would still
// pass on a corpus of close-ups.
CHECK(smallest_side < d.min_face_px * 1.05f);
}
TEST_CASE("replaying the superhero fixture twice gives identical tracks",
"[replay][VR-002]") {
Dump d = load(fixture("superhero.h5"));
+86
View File
@@ -0,0 +1,86 @@
// AR-011 — the boundary dedup window is derived from the stream's cadence, not
// assumed.
//
// TRACES: AR-011 | SR-002 | UT-003
//
// Tier T1: the derivation is arithmetic on frame timestamps, so it is checked
// against synthetic cadences at 24, 25 and 30 fps rather than against a decode.
// The number this replaced was 0.04 s — one frame at 25 fps, correct for exactly
// one of those three and quietly wrong for the other two.
//
// SceneDetectorFunc is never constructed: its constructor loads TransNetV2. Only
// the static rule is called, so make_scene_detector() is never odr-used.
#include <catch2/catch_test_macros.hpp>
#include "nodes/scene_detector_node.hpp"
#include <vector>
namespace {
// The intervals the node accumulates from a steady stream at `fps`.
std::vector<double> cadence(double fps, int n = 200) {
return std::vector<double>(static_cast<std::size_t>(n), 1.0 / fps);
}
double window(double fps) {
return SceneDetectorFunc::dedup_window_sec(cadence(fps));
}
} // namespace
// ── The property that has to hold at every rate ──────────────────────────────
// The window has exactly one job: tell "one frame scored twice by two
// overlapping windows" (a gap of zero) from "two adjacent frames, both of them
// real cuts" (a gap of one frame interval). It has to sit strictly between.
TEST_CASE("the dedup window separates a duplicate from an adjacent frame",
"[scene][AR-011]") {
for (double fps : {24.0, 25.0, 30.0, 23.976, 29.97, 50.0, 60.0}) {
INFO("source at " << fps << " fps");
const double frame = 1.0 / fps;
const double w = window(fps);
CHECK(w > 0.0); // a duplicate (gap 0) is still merged
CHECK(w < frame); // two consecutive frames both survive
}
}
// The concrete failure the hardcoded constant caused: at 30 fps a frame is
// 0.0333 s, so a 0.04 s window swallowed a cut on the very next frame. Nothing in
// the output showed it — the file just had fewer boundaries.
TEST_CASE("cuts on consecutive frames survive at 30 fps", "[scene][AR-011]") {
const double frame = 1.0 / 30.0;
CHECK(window(30.0) < frame);
CHECK(0.04 > frame); // the constant that was there, for the record
}
TEST_CASE("the window tracks the rate rather than a constant",
"[scene][AR-011]") {
// If it were still assumed, these would be equal.
CHECK(window(24.0) > window(30.0));
CHECK(window(30.0) > window(60.0));
CHECK(window(25.0) == 0.5 / 25.0);
}
// ── Robustness of the estimate ───────────────────────────────────────────────
TEST_CASE("a seek or a dropped frame does not move the derived cadence",
"[scene][AR-011]") {
auto intervals = cadence(25.0);
intervals[0] = 3.5; // a seek at the start
intervals[97] = 0.4; // a gap where the decoder lost frames
// Median, not mean: two long intervals out of 200 cannot shift it at all.
CHECK(SceneDetectorFunc::dedup_window_sec(intervals) == 0.5 / 25.0);
}
TEST_CASE("too few frames to have a cadence yields an inert window",
"[scene][AR-011]") {
// Under two frames there is no interval to measure — and also no second
// boundary to merge with, so a window of 0 changes nothing. Guessing a rate
// here would be the mistake this requirement is about.
CHECK(SceneDetectorFunc::dedup_window_sec({}) == 0.0);
}
TEST_CASE("a single observed interval is enough", "[scene][AR-011]") {
CHECK(SceneDetectorFunc::dedup_window_sec({1.0 / 24.0}) == 0.5 / 24.0);
}