From f33403fff841b2e12eb80ef3f6a0a4efdc975484 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Tue, 4 Aug 2026 21:17:57 +0200 Subject: [PATCH] feat(scene): feed TransNetV2 at native rate, derive the dedup window from it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes both violations SPEC.md named under "Every model gets the input it was trained for". They are one bug, not two. The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over twice its temporal context. Boundary timestamps stayed correct throughout, which is exactly why the degradation was invisible and why the compressed separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a property of the ONNX export rather than of the input. Dedup then merged boundaries closer than a literal 0.04 s — one frame at 25 fps, and wider than a frame at 30, so two cuts on consecutive frames became one. Nothing in scenes.json showed it; the file simply had fewer boundaries. Native rate is where that constant did the most damage, which is why fixing the decode rate without fixing the dedup would have made things worse. dedup_window_sec() now takes the median interval the detector was actually fed and halves it. Half a frame rather than a whole one: the only thing being merged is one frame scored by two overlapping windows, and two distinct frames are a full interval apart. Cost is real — dense decode is the pipeline's cost driver. It is accepted; dense_scale and scene_stride remain the reductions that do not run the model off-distribution. scene_threshold 0.60 was fitted against the 12 fps input and is now stale, so VR-006 goes from Low to Medium: it is no longer a refinement, it is a constant that no longer describes the input. AR-002 rides along because it was already implemented, just untagged and unverified — the register said Planned while the code was correct. The size filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold and at dense_scale 0.5, and checked end to end against the superhero dump, whose smallest face is exactly its recorded 32 px minimum, so the fixture check cannot pass vacuously. Co-Authored-By: Claude Opus 5 TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001 --- docs/SPEC.md | 45 ++++++++---- docs/requirements.md | 6 +- src/config.hpp | 29 +++++--- src/main.cpp | 5 +- src/nodes/face_detector_node.hpp | 39 +++++++--- src/nodes/scene_detector_node.hpp | 53 +++++++++++++- tests/CMakeLists.txt | 2 + tests/test_face_detector_node.cpp | 114 +++++++++++++++++++++++++++++ tests/test_replay_fixtures.cpp | 43 ++++++++++- tests/test_scene_detector_node.cpp | 86 ++++++++++++++++++++++ 10 files changed, 378 insertions(+), 44 deletions(-) create mode 100644 tests/test_face_detector_node.cpp create mode 100644 tests/test_scene_detector_node.cpp diff --git a/docs/SPEC.md b/docs/SPEC.md index 88f963d..3a6d27a 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -438,7 +438,7 @@ Both feed AR-007 as **association hints**: they tell the tracker that spatial continuity is broken and that association should weight embedding over IoU. Neither ends a presence window (AR-012). -In dense mode the source decodes at `scene_decode_fps` (default 12) and a +In dense mode the source decodes at `scene_decode_fps` (default 0 = native) and a decimator splits the stream: full-resolution sampled frames to the face pipeline, downscaled dense frames to the scene detector (`frame_source_node.hpp:63`). `sample_fps` is independent of this — the face @@ -459,31 +459,44 @@ degrading what a single inference sees. A model run off-distribution produces confident, plausible, wrong output, and the error is invisible without a study that should not have been necessary. -Two places this is currently violated: +Two places this was violated, both now closed: -1. **`scene_decode_fps = 12` starves TransNetV2.** `kWindow` is 100 frames. At - native 25 fps that window spans ~4 s; at 12 fps it spans ~8.3 s, so the model - sees roughly half-speed motion over twice the temporal context it was trained - on. **Requirement: feed TransNetV2 at the source's native frame rate**, so a - 100-frame window covers the duration the model expects. The - "tolerates ~12fps" note in `config.hpp` describes a compromise, and the - recorded margin is consistent with it — a non-boundary baseline at ~0.50 with +1. **`scene_decode_fps = 12` starved TransNetV2.** `kWindow` is 100 frames. At + native 25 fps that window spans ~4 s; at 12 fps it spanned ~8.3 s, so the + model saw roughly half-speed motion over twice the temporal context it was + trained on. **Requirement: feed TransNetV2 at the source's native frame + rate**, so a 100-frame window covers the duration the model expects. The + "tolerates ~12fps" note in `config.hpp` described a compromise, and the + recorded margin was consistent with it — a non-boundary baseline at ~0.50 with real boundaries reaching only ~0.7+ is a compressed separation, not a healthy - one. + one. **Done:** `scene_decode_fps` defaults to 0. -2. **Hardcoded 25 fps in boundary dedup.** `scene_detector_node.hpp:138` merges - boundaries closer than `0.04 s` — "~1 frame @25fps". **Requirement: derive - this from the source's actual frame rate.** +2. **Hardcoded 25 fps in boundary dedup.** The node merged boundaries closer than + `0.04 s` — "~1 frame @25fps". **Requirement: derive this from the source's + actual frame rate. Done:** `SceneDetectorFunc::dedup_window_sec()` takes the + median of the frame intervals the detector was actually fed and halves it. + Half a frame rather than a whole one, because the only thing being merged is + one frame scored by two overlapping windows; two distinct frames are a full + interval apart and both have to survive. + +The two are one change, not two. A native-rate stream is where the old constant +did the most damage — at 30 fps, 0.04 s is wider than a frame, so two cuts on +consecutive frames merged into one and the loss showed up nowhere: the file +simply had fewer boundaries. Dense decode is the pipeline's cost driver, so (1) is not free. The cost is accepted: the alternative is a boundary signal that steers association (AR-007) while being quietly unreliable. `dense_scale` remains available as a spatial reduction, since downscaling is a documented, understood degradation rather than a temporal -one the model has no defence against. +one the model has no defence against — and TransNetV2 downsamples to 48×27 +regardless. **Current:** histogram cut in the decoder; `scene_detector_node.hpp` for -TransNetV2. **Gap:** native-rate dense decode; framerate-derived dedup; -`--scene-detect` is default-off despite now feeding association. +TransNetV2, fed at native rate with a framerate-derived dedup window. +**Gap:** `scene_threshold` (0.60) is still the value picked against 12 fps input +and is now certainly wrong — VR-006 re-fits it, and until it does, boundary +recall at native rate is untuned rather than better. `--scene-detect` is +default-off despite now feeding association. ## AR-012 … AR-017 — Track-level identity propagation — **CHANGED BEHAVIOUR** diff --git a/docs/requirements.md b/docs/requirements.md index 5c3a6cd..5d49c83 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -29,7 +29,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | ID | Requirement | Traces to | Priority | Status | |---|---|---|---|---| | AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done | -| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned | +| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | **Done** — `FaceDetectorFunc::drop_undersized()`. The threshold is divided by `bbox_upscale` rather than every box multiplied, which keeps the comparison on the detector's own numbers and means turning `dense_scale` on cannot silently raise the minimum face the pipeline accepts. Verified at the threshold and at `dense_scale` 0.5 (UT-002), and end to end on the fixture (IT-001) — the superhero dump's smallest side is *exactly* its recorded 32 px, so the filter is binding there rather than vacuously satisfied | | AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing | | AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Mostly** — node outputs *park* on a full channel: the value is held, the worker released, and a channel space-callback resumes the node. Replaces `push_blocking`, which parked a scheduler worker inside the push and, with one thread per node, stopped that node draining its own input. Verified: 385/385 frames, 0 drops. **Gap:** a rare hang survives, ~1 run in 20 at a 300 s timeout (was: every run). `FanoutNode` still drops on overflow (`fanout.hpp:129`) rather than parking, so the AR-010 scene join sheds frames exactly when the dense branch falls behind | | AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`. The RANSAC fit it replaces disagreed by a median 17 source px on 400 headshots, 83.5% of crops embedding below cos 0.99, and was unstable and RNG-driven: rebuilding caught 1614 near-duplicates against the original build's ~100. **All galleries rebuilt** (2456 actors, 10254 embeddings); measured separation gain is small (0.583 → 0.590), so recorded accuracy figures should be re-run but are not expected to move far | @@ -38,7 +38,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted | | AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done | | AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Done** — decode butterfly joined via `SceneBoundaries`; the sampled branch waits for the detector's watermark. Frames past its last scored window are counted as unverified, never assumed boundary-free | -| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned | +| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | **Done** — both violations SPEC.md named are closed. (1) `scene_decode_fps` defaults to 0 (native): at 12 fps a 100-frame `kWindow` spanned ~8.3 s instead of the ~4 s TransNetV2 was trained on, half-speed motion over twice its temporal context. (2) The boundary dedup window is derived from the cadence the detector was actually fed (`SceneDetectorFunc::dedup_window_sec()`, median observed interval, halved) rather than the literal 0.04 s — one frame at 25 fps, and at 30 fps wider than a frame, so two cuts on consecutive frames merged into one and the loss was invisible: the file simply had fewer boundaries. Derivation checked at 24/25/30 fps and under a seek (UT-003). **Consequence, not a gap:** `scene_threshold` 0.60 was fitted against the 12 fps input and is now certainly wrong — VR-006 re-fits it, and until then boundary recall at native rate is untuned rather than better. Dense decode is the cost driver, so this is not free; `dense_scale` and `scene_stride` remain the reductions that do not run the model off-distribution | | AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done** — `src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track | | AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done** — `last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed | | AR-014 | Belief swap A→B terminates the track and starts a new one | SR-002 | Medium | **Done** — swap closes at `last_seen` and opens a successor at the swap frame; counted | @@ -108,7 +108,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done | | VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done | | VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 24–32 px; 32 px gives 98.1% TPI, 0.0 FPI at every size. Degrades an already-aligned 112×112 crop, so it isolates the embedder and is an **upper bound**; VR-013 measures the same question end to end and AR-002 takes its number, not this one | -| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned | +| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | **Medium** | **Planned, now unblocked** — native-rate decode landed with AR-011, so the prerequisite is met and the current 0.60 is a value fitted against input the pipeline no longer produces. Raised from Low for that reason: it is no longer a refinement, it is a stale constant | | VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned | | VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned | | VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned | diff --git a/src/config.hpp b/src/config.hpp index c56472c..5ae8061 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -91,17 +91,28 @@ struct Config { // at ~0.50; real boundaries spike to ~0.7+) int scene_stride{50}; // frames advanced between windows (≤ kWindow) - // Dense-decode throughput knobs (only active with scene_detect). Dense decode - // of every native-rate frame is the pipeline's cost driver; these trade a - // little boundary precision for a large speedup. - // scene_decode_fps: rate the source decodes at in dense mode. Lower = - // fewer frames decoded. TransNetV2 tolerates ~12fps; boundary timestamps - // stay correct (keyed off each frame's real timestamp). 0 = native fps. + // Dense-decode knobs (only active with scene_detect). Dense decode of every + // native-rate frame is the pipeline's cost driver, which is what made the + // temporal shortcut below tempting. + /// TRACES: AR-011 | SR-002 + // scene_decode_fps: rate the source decodes at in dense mode. + // **0 = native, and native is the only correct setting.** kWindow is 100 + // frames: at native 25 fps that window spans ~4 s, which is what + // TransNetV2 was trained on; at the 12 fps this used to default to it + // spans ~8.3 s, so the model saw half-speed motion over twice its + // temporal context. Boundary *timestamps* stay right either way — which + // is exactly why the degradation was invisible, and why the compressed + // separation it produced (~0.50 baseline against ~0.7+ peaks) was read + // as a property of the export rather than of the input. Lowering this + // buys decode time by running the model off-distribution; reach for + // dense_scale or scene_stride instead, which do not. // dense_scale: downscale factor applied to decoded frames in dense mode // (0 pre-built TransNetV2 TRT engine (TRT backend) // --scene-threshold boundary sigmoid prob above this → cut (default: 0.60) // --scene-stride frames between TransNetV2 windows (default: 50, ≤100) -// --scene-decode-fps dense decode rate in scene-detect mode (default: 12; -// 0 = native fps). Lower = faster, coarser boundaries. +// --scene-decode-fps dense decode rate in scene-detect mode (default: 0 = +// native, the only rate TransNetV2 is calibrated for; +// AR-011). Lowering it runs the model off-distribution. // --dense-scale downscale decoded frames in scene-detect mode (0 max faces kept per frame (default: 10) diff --git a/src/nodes/face_detector_node.hpp b/src/nodes/face_detector_node.hpp index 6689291..6b409de 100644 --- a/src/nodes/face_detector_node.hpp +++ b/src/nodes/face_detector_node.hpp @@ -6,6 +6,7 @@ #include #include #include +#include // ── FaceDetectorFunc ────────────────────────────────────────────────────────── // KPN node: runs SCRFD-500MF to detect ALL faces in a frame. @@ -22,22 +23,38 @@ struct FaceDetectorFunc { , min_face_px_(cfg.min_face_px) {} - SceneFrame operator()(Frame f) { - if (f.eof) return {std::move(f), {}}; - - auto faces = detector_->detect(f.image); - - // Drop faces below minimum pixel size (too small for reliable ArcFace - // alignment). Note: when dense_scale downscaled the frame, both the - // detection coords and min_face_px are in downscaled space — so scale - // the threshold down to match, keeping the physical size cutoff constant. - const float min_px = (f.bbox_upscale != 1.f) - ? min_face_px_ / f.bbox_upscale : min_face_px_; + /// TRACES: AR-002 | SR-002 + // Drop faces below the minimum size — too small for reliable ArcFace + // alignment, and below the resolution where identification still holds + // (VR-013 measured the knee end to end). + // + // The minimum is expressed in ORIGINAL video resolution, which is what makes + // it a property of the footage rather than of a throughput knob. When + // dense_scale downscaled the frame the detector's boxes are in downscaled + // space, and `bbox_upscale` is what maps them back; dividing the threshold by + // it rather than multiplying every box keeps the comparison on the detector's + // own numbers and the physical cutoff constant across scales. + // + // Strictly less-than: a face exactly at the minimum is admissible, which is + // what a "minimum of 40x40" means. + static void drop_undersized(std::vector& faces, + float min_face_px, + float bbox_upscale) { + const float min_px = (bbox_upscale > 0.f) ? min_face_px / bbox_upscale + : min_face_px; faces.erase( std::remove_if(faces.begin(), faces.end(), [&](const DetectedFace& d) { return d.bbox.width < min_px || d.bbox.height < min_px; }), faces.end()); + } + + SceneFrame operator()(Frame f) { + if (f.eof) return {std::move(f), {}}; + + auto faces = detector_->detect(f.image); + + drop_undersized(faces, min_face_px_, f.bbox_upscale); // Sort largest-first so max_faces_ keeps the most informative detections std::sort(faces.begin(), faces.end(), diff --git a/src/nodes/scene_detector_node.hpp b/src/nodes/scene_detector_node.hpp index d694e67..bb9305c 100644 --- a/src/nodes/scene_detector_node.hpp +++ b/src/nodes/scene_detector_node.hpp @@ -67,6 +67,15 @@ struct SceneDetectorFunc { return; } + /// TRACES: AR-011 | SR-002 + // Learn the cadence of the stream from the stream itself, rather than + // assuming one. See dedup_window_sec(). + if (prev_ts_ >= 0.0 && intervals_.size() < kCadenceSamples) { + const double dt = f.timestamp_sec - prev_ts_; + if (dt > 0.0) intervals_.push_back(dt); + } + prev_ts_ = f.timestamp_sec; + images_.push_back(f.image); times_.push_back(f.timestamp_sec); @@ -81,6 +90,34 @@ struct SceneDetectorFunc { } } + /// TRACES: AR-011 | SR-002 + // How close two boundaries have to be before they are the same boundary, + // derived from the cadence the detector was actually fed. + // + // What this replaces is a literal 0.04 s — one frame at 25 fps, and silently + // wrong at any other rate. On a 30 fps source it spans more than a frame, so + // two cuts on consecutive frames merge into one and a real boundary is lost; + // the output does not show this, it simply contains fewer cuts. Assuming a + // frame rate is the same class of mistake as feeding a model the wrong rate, + // which is why this belongs to AR-011 and not to a tidy-up. + // + // Half a frame, not a whole one, because the only thing being deduplicated is + // one frame scored by two overlapping windows — a gap of zero. Two distinct + // frames are a full interval apart and must both survive. Half an interval + // separates those two cases without putting the decision on the knife-edge + // where floating-point error settles it. + // + // Median, not mean: a seek, or a gap where the decoder dropped a frame, + // contributes one long interval that would drag a mean and cannot move a + // median. + static double dedup_window_sec(std::vector intervals) { + if (intervals.empty()) return 0.0; // <2 frames: nothing to deduplicate + const std::size_t mid = intervals.size() / 2; + std::nth_element(intervals.begin(), intervals.begin() + mid, + intervals.end()); + return intervals[mid] * 0.5; + } + private: // Run TransNetV2 on the leading kWindow frames of the buffer and record any // boundaries found within the trusted centre region. @@ -154,6 +191,8 @@ private: written_ = true; // Merge boundaries closer than one frame apart (dedup across window seams). + const double dedup_sec = dedup_window_sec(intervals_); + std::sort(boundaries_.begin(), boundaries_.end(), [](const Boundary& a, const Boundary& b) { return a.t < b.t; @@ -167,7 +206,7 @@ private: nlohmann::json cuts = nlohmann::json::array(); double last_t = -1e9; for (const auto& b : boundaries_) { - if (b.t - last_t < 0.04) continue; // ~1 frame @25fps dedup + if (b.t - last_t < dedup_sec) continue; cuts.push_back({{"t", b.t}, {"probability", b.prob}}); last_t = b.t; } @@ -180,8 +219,12 @@ private: return; } f << root.dump(2) << "\n"; + // Report the derived cadence: VR-006 re-tunes scene_threshold against it, + // and a rate that is not the source's is the first thing to suspect. std::cerr << "\n[scene_detector] wrote " << root["cuts"].size() - << " boundaries → " << output_path_ << "\n"; + << " boundaries → " << output_path_ + << " (dedup=" << dedup_sec << "s from " + << (dedup_sec > 0.0 ? 0.5 / dedup_sec : 0.0) << " fps)\n"; } static int kLast_() { return ISceneDetector::kWindow - 1; } @@ -195,6 +238,10 @@ private: struct Boundary { double t; float prob; }; + // Enough to establish a rate; bounded so a feature-length film does not + // accumulate one double per frame for a number that stops moving early. + static constexpr std::size_t kCadenceSamples = 512; + std::unique_ptr detector_; float threshold_; int stride_; @@ -207,6 +254,8 @@ private: std::deque times_; int64_t window_base_{0}; // frame index of images_.front() std::vector boundaries_; + double prev_ts_{-1.0}; // AR-011: cadence, learned not assumed + std::vector intervals_; bool written_{false}; std::shared_ptr shared_; ///< AR-010 join point }; diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 8615f1c..dfb2587 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -22,6 +22,8 @@ add_executable(sae_tests test_track_gallery.cpp test_face_tracker.cpp test_track_registry.cpp + test_face_detector_node.cpp + test_scene_detector_node.cpp test_replay_fixtures.cpp test_audio_signature.cpp ${CMAKE_SOURCE_DIR}/src/backends/gemm_backend.cpp diff --git a/tests/test_face_detector_node.cpp b/tests/test_face_detector_node.cpp new file mode 100644 index 0000000..5af0f24 --- /dev/null +++ b/tests/test_face_detector_node.cpp @@ -0,0 +1,114 @@ +// AR-002 — minimum face size, in original video resolution. +// +// TRACES: AR-002 | SR-002 | UT-002 +// +// Tier T1 here, T2 in test_replay_fixtures.cpp. The requirement is arithmetic on +// bounding boxes, so the two edge cases that matter — a face sitting exactly on +// the threshold, and the same face seen through a downscaled decode — are +// reachable without a detector, a model or a GPU. What the fixture check adds is +// that the rule was actually applied on the way to a dump; what this adds is that +// it is applied *correctly*, which no real dump can demonstrate because real +// footage does not contain a 39.999 px face on demand. +// +// FaceDetectorFunc is never constructed: its constructor loads SCRFD. Only the +// static rule is called, so make_face_detector() is never odr-used and nothing +// here needs a backend. +#include + +#include "nodes/face_detector_node.hpp" +#include "types.hpp" + +#include + +namespace { + +DetectedFace box(float w, float h) { + DetectedFace f; + f.bbox = cv::Rect2f(10.f, 10.f, w, h); + f.confidence = 0.9f; + return f; +} + +// Sizes the rule kept, in the order given. +std::vector surviving_widths(std::vector faces, + float min_face_px, float bbox_upscale) { + FaceDetectorFunc::drop_undersized(faces, min_face_px, bbox_upscale); + std::vector out; + for (const auto& f : faces) out.push_back(f.bbox.width); + return out; +} + +constexpr float kMin = 40.f; // Config::min_face_px default, and AR-002's number + +} // namespace + +// ── Exactly at the threshold ───────────────────────────────────────────────── +// The boundary case is the whole content of a minimum: "40x40" has to mean 40 is +// admissible, or the requirement says 41. +TEST_CASE("a face exactly at the minimum is kept", "[detector][AR-002]") { + CHECK(surviving_widths({box(kMin, kMin)}, kMin, 1.f).size() == 1); +} + +TEST_CASE("a face one tenth of a pixel under the minimum is dropped", + "[detector][AR-002]") { + CHECK(surviving_widths({box(39.9f, 100.f)}, kMin, 1.f).empty()); + CHECK(surviving_widths({box(100.f, 39.9f)}, kMin, 1.f).empty()); +} + +TEST_CASE("both sides must clear the minimum, not the larger one", + "[detector][AR-002]") { + // A wide, short box has enough pixels and is still unusable: ArcFace + // alignment needs both dimensions. Area would admit this; the rule must not. + CHECK(surviving_widths({box(400.f, 20.f)}, kMin, 1.f).empty()); +} + +TEST_CASE("the filter is a filter, not a reordering", "[detector][AR-002]") { + auto kept = surviving_widths( + {box(80.f, 80.f), box(10.f, 10.f), box(60.f, 60.f), box(39.f, 39.f)}, + kMin, 1.f); + REQUIRE(kept.size() == 2); + // Order is load-bearing downstream (AR-003's largest-first sort tie-breaks on + // it, and the Hungarian solver tie-breaks on index) — erase-remove must not + // shuffle the survivors. + CHECK(kept[0] == 80.f); + CHECK(kept[1] == 60.f); +} + +// ── The dense_scale interaction the requirement exists for ─────────────────── +// dense_scale 0.5 halves the decoded frame, so the detector reports a 40 px face +// as 20 px. If the threshold were applied to those numbers, turning on a +// throughput knob would silently double the minimum face size the pipeline +// accepts — a recall change with no line in the config to explain it. AR-002 +// pins the minimum to the ORIGINAL resolution instead. +TEST_CASE("at dense_scale 0.5 the cutoff stays 40 px of original footage", + "[detector][AR-002]") { + constexpr float kUpscale = 2.f; // frame_source_node: 1 / dense_scale + + // 20 px in downscaled space is exactly 40 px of original footage: kept. + CHECK(surviving_widths({box(20.f, 20.f)}, kMin, kUpscale).size() == 1); + + // 19.9 px downscaled is 39.8 px original: dropped. + CHECK(surviving_widths({box(19.9f, 19.9f)}, kMin, kUpscale).empty()); + + // And the interaction stated as one claim: a face of a given original size is + // admitted or refused identically whether or not the decode was downscaled. + for (float original : {30.f, 39.f, 40.f, 41.f, 80.f}) { + INFO("original size " << original); + const bool full = !surviving_widths({box(original, original)}, + kMin, 1.f).empty(); + const bool dense = !surviving_widths({box(original / kUpscale, + original / kUpscale)}, + kMin, kUpscale).empty(); + CHECK(full == dense); + CHECK(full == (original >= kMin)); + } +} + +TEST_CASE("an absent or degenerate upscale falls back to the raw threshold", + "[detector][AR-002]") { + // bbox_upscale is 1 on every non-dense frame; 0 would mean the frame source + // never set it. Dividing by that would reject every face in the film, which + // is a failure worth not having. + CHECK(surviving_widths({box(kMin, kMin)}, kMin, 0.f).size() == 1); + CHECK(surviving_widths({box(39.f, 39.f)}, kMin, 0.f).empty()); +} diff --git a/tests/test_replay_fixtures.cpp b/tests/test_replay_fixtures.cpp index 8fb894d..518214f 100644 --- a/tests/test_replay_fixtures.cpp +++ b/tests/test_replay_fixtures.cpp @@ -1,6 +1,6 @@ // Replay tests — the real tracker and registry driven from committed fixtures. // -// TRACES: AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001 +// TRACES: AR-002, AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001 // // Tier T2: composition, not units. The registry tests construct awkward states // directly; these check that the pieces behave when wired together and fed real @@ -27,6 +27,7 @@ #include #include +#include #include #include #include @@ -44,6 +45,8 @@ struct Dump { std::vector emb; std::vector bbox; // 4 per face std::string embedder; + float min_face_px{0.f}; // AR-002, as the run was configured + float bbox_upscale{1.f}; // bbox × this = original resolution std::size_t frames() const { return ts.size(); } std::size_t faces() const { return emb.size(); } @@ -97,6 +100,17 @@ Dump load(const std::string& path) { H5::StrType vlen(H5::PredType::C_S1, H5T_VARIABLE); f.openAttribute("embedder_model").read(vlen, d.embedder); } + + // AR-002: the threshold the run was configured with, and the scale its boxes + // are in. Read from the dump rather than assumed, so the check is against + // what this fixture was actually generated with — the hero clips predate the + // move to 40 px and were dumped at 32. + if (f.attrExists("min_face_px")) + f.openAttribute("min_face_px").read(H5::PredType::NATIVE_FLOAT, + &d.min_face_px); + if (f.attrExists("bbox_upscale")) + f.openAttribute("bbox_upscale").read(H5::PredType::NATIVE_FLOAT, + &d.bbox_upscale); return d; } @@ -167,6 +181,33 @@ TEST_CASE("superhero fixture is complete", "[replay][VR-001]") { CHECK(static_cast(running) == d.faces()); } +// ── AR-002 — the size filter held, all the way to the dump ─────────────────── +// The T1 arithmetic is in test_face_detector_node.cpp. This is the other half: +// that the rule was applied on real footage and nothing downstream of it let an +// undersized face back in. +TEST_CASE("no dumped face is below the configured minimum size", + "[replay][AR-002]") { + Dump d = load(fixture("superhero.h5")); + // A dump that did not record its threshold cannot be checked against one. + REQUIRE(d.min_face_px > 0.f); + REQUIRE(d.bbox_upscale > 0.f); + + float smallest_side = std::numeric_limits::max(); + for (std::size_t i = 0; i < d.faces(); ++i) { + const float w = d.bbox[i * 4 + 2] * d.bbox_upscale; // original resolution + const float h = d.bbox[i * 4 + 3] * d.bbox_upscale; + REQUIRE(w >= d.min_face_px); + REQUIRE(h >= d.min_face_px); + smallest_side = std::min({smallest_side, w, h}); + } + + // And the filter was binding, not vacuously satisfied. 480x360 footage puts + // faces right on the cutoff, which is what makes this fixture worth checking: + // if the threshold stopped being applied the assertions above would still + // pass on a corpus of close-ups. + CHECK(smallest_side < d.min_face_px * 1.05f); +} + TEST_CASE("replaying the superhero fixture twice gives identical tracks", "[replay][VR-002]") { Dump d = load(fixture("superhero.h5")); diff --git a/tests/test_scene_detector_node.cpp b/tests/test_scene_detector_node.cpp new file mode 100644 index 0000000..0ed590d --- /dev/null +++ b/tests/test_scene_detector_node.cpp @@ -0,0 +1,86 @@ +// AR-011 — the boundary dedup window is derived from the stream's cadence, not +// assumed. +// +// TRACES: AR-011 | SR-002 | UT-003 +// +// Tier T1: the derivation is arithmetic on frame timestamps, so it is checked +// against synthetic cadences at 24, 25 and 30 fps rather than against a decode. +// The number this replaced was 0.04 s — one frame at 25 fps, correct for exactly +// one of those three and quietly wrong for the other two. +// +// SceneDetectorFunc is never constructed: its constructor loads TransNetV2. Only +// the static rule is called, so make_scene_detector() is never odr-used. +#include + +#include "nodes/scene_detector_node.hpp" + +#include + +namespace { + +// The intervals the node accumulates from a steady stream at `fps`. +std::vector cadence(double fps, int n = 200) { + return std::vector(static_cast(n), 1.0 / fps); +} + +double window(double fps) { + return SceneDetectorFunc::dedup_window_sec(cadence(fps)); +} + +} // namespace + +// ── The property that has to hold at every rate ────────────────────────────── +// The window has exactly one job: tell "one frame scored twice by two +// overlapping windows" (a gap of zero) from "two adjacent frames, both of them +// real cuts" (a gap of one frame interval). It has to sit strictly between. +TEST_CASE("the dedup window separates a duplicate from an adjacent frame", + "[scene][AR-011]") { + for (double fps : {24.0, 25.0, 30.0, 23.976, 29.97, 50.0, 60.0}) { + INFO("source at " << fps << " fps"); + const double frame = 1.0 / fps; + const double w = window(fps); + + CHECK(w > 0.0); // a duplicate (gap 0) is still merged + CHECK(w < frame); // two consecutive frames both survive + } +} + +// The concrete failure the hardcoded constant caused: at 30 fps a frame is +// 0.0333 s, so a 0.04 s window swallowed a cut on the very next frame. Nothing in +// the output showed it — the file just had fewer boundaries. +TEST_CASE("cuts on consecutive frames survive at 30 fps", "[scene][AR-011]") { + const double frame = 1.0 / 30.0; + CHECK(window(30.0) < frame); + CHECK(0.04 > frame); // the constant that was there, for the record +} + +TEST_CASE("the window tracks the rate rather than a constant", + "[scene][AR-011]") { + // If it were still assumed, these would be equal. + CHECK(window(24.0) > window(30.0)); + CHECK(window(30.0) > window(60.0)); + CHECK(window(25.0) == 0.5 / 25.0); +} + +// ── Robustness of the estimate ─────────────────────────────────────────────── +TEST_CASE("a seek or a dropped frame does not move the derived cadence", + "[scene][AR-011]") { + auto intervals = cadence(25.0); + intervals[0] = 3.5; // a seek at the start + intervals[97] = 0.4; // a gap where the decoder lost frames + + // Median, not mean: two long intervals out of 200 cannot shift it at all. + CHECK(SceneDetectorFunc::dedup_window_sec(intervals) == 0.5 / 25.0); +} + +TEST_CASE("too few frames to have a cadence yields an inert window", + "[scene][AR-011]") { + // Under two frames there is no interval to measure — and also no second + // boundary to merge with, so a window of 0 changes nothing. Guessing a rate + // here would be the mistake this requirement is about. + CHECK(SceneDetectorFunc::dedup_window_sec({}) == 0.0); +} + +TEST_CASE("a single observed interval is enough", "[scene][AR-011]") { + CHECK(SceneDetectorFunc::dedup_window_sec({1.0 / 24.0}) == 0.5 / 24.0); +}