diff --git a/docs/SPEC.md b/docs/SPEC.md index 88f963d..3a6d27a 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -438,7 +438,7 @@ Both feed AR-007 as **association hints**: they tell the tracker that spatial continuity is broken and that association should weight embedding over IoU. Neither ends a presence window (AR-012). -In dense mode the source decodes at `scene_decode_fps` (default 12) and a +In dense mode the source decodes at `scene_decode_fps` (default 0 = native) and a decimator splits the stream: full-resolution sampled frames to the face pipeline, downscaled dense frames to the scene detector (`frame_source_node.hpp:63`). `sample_fps` is independent of this — the face @@ -459,31 +459,44 @@ degrading what a single inference sees. A model run off-distribution produces confident, plausible, wrong output, and the error is invisible without a study that should not have been necessary. -Two places this is currently violated: +Two places this was violated, both now closed: -1. **`scene_decode_fps = 12` starves TransNetV2.** `kWindow` is 100 frames. At - native 25 fps that window spans ~4 s; at 12 fps it spans ~8.3 s, so the model - sees roughly half-speed motion over twice the temporal context it was trained - on. **Requirement: feed TransNetV2 at the source's native frame rate**, so a - 100-frame window covers the duration the model expects. The - "tolerates ~12fps" note in `config.hpp` describes a compromise, and the - recorded margin is consistent with it — a non-boundary baseline at ~0.50 with +1. **`scene_decode_fps = 12` starved TransNetV2.** `kWindow` is 100 frames. At + native 25 fps that window spans ~4 s; at 12 fps it spanned ~8.3 s, so the + model saw roughly half-speed motion over twice the temporal context it was + trained on. **Requirement: feed TransNetV2 at the source's native frame + rate**, so a 100-frame window covers the duration the model expects. The + "tolerates ~12fps" note in `config.hpp` described a compromise, and the + recorded margin was consistent with it — a non-boundary baseline at ~0.50 with real boundaries reaching only ~0.7+ is a compressed separation, not a healthy - one. + one. **Done:** `scene_decode_fps` defaults to 0. -2. **Hardcoded 25 fps in boundary dedup.** `scene_detector_node.hpp:138` merges - boundaries closer than `0.04 s` — "~1 frame @25fps". **Requirement: derive - this from the source's actual frame rate.** +2. **Hardcoded 25 fps in boundary dedup.** The node merged boundaries closer than + `0.04 s` — "~1 frame @25fps". **Requirement: derive this from the source's + actual frame rate. Done:** `SceneDetectorFunc::dedup_window_sec()` takes the + median of the frame intervals the detector was actually fed and halves it. + Half a frame rather than a whole one, because the only thing being merged is + one frame scored by two overlapping windows; two distinct frames are a full + interval apart and both have to survive. + +The two are one change, not two. A native-rate stream is where the old constant +did the most damage — at 30 fps, 0.04 s is wider than a frame, so two cuts on +consecutive frames merged into one and the loss showed up nowhere: the file +simply had fewer boundaries. Dense decode is the pipeline's cost driver, so (1) is not free. The cost is accepted: the alternative is a boundary signal that steers association (AR-007) while being quietly unreliable. `dense_scale` remains available as a spatial reduction, since downscaling is a documented, understood degradation rather than a temporal -one the model has no defence against. +one the model has no defence against — and TransNetV2 downsamples to 48×27 +regardless. **Current:** histogram cut in the decoder; `scene_detector_node.hpp` for -TransNetV2. **Gap:** native-rate dense decode; framerate-derived dedup; -`--scene-detect` is default-off despite now feeding association. +TransNetV2, fed at native rate with a framerate-derived dedup window. +**Gap:** `scene_threshold` (0.60) is still the value picked against 12 fps input +and is now certainly wrong — VR-006 re-fits it, and until it does, boundary +recall at native rate is untuned rather than better. `--scene-detect` is +default-off despite now feeding association. ## AR-012 … AR-017 — Track-level identity propagation — **CHANGED BEHAVIOUR** diff --git a/docs/requirements.md b/docs/requirements.md index 5c3a6cd..5d49c83 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -29,7 +29,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | ID | Requirement | Traces to | Priority | Status | |---|---|---|---|---| | AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done | -| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned | +| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | **Done** — `FaceDetectorFunc::drop_undersized()`. The threshold is divided by `bbox_upscale` rather than every box multiplied, which keeps the comparison on the detector's own numbers and means turning `dense_scale` on cannot silently raise the minimum face the pipeline accepts. Verified at the threshold and at `dense_scale` 0.5 (UT-002), and end to end on the fixture (IT-001) — the superhero dump's smallest side is *exactly* its recorded 32 px, so the filter is binding there rather than vacuously satisfied | | AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing | | AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Mostly** — node outputs *park* on a full channel: the value is held, the worker released, and a channel space-callback resumes the node. Replaces `push_blocking`, which parked a scheduler worker inside the push and, with one thread per node, stopped that node draining its own input. Verified: 385/385 frames, 0 drops. **Gap:** a rare hang survives, ~1 run in 20 at a 300 s timeout (was: every run). `FanoutNode` still drops on overflow (`fanout.hpp:129`) rather than parking, so the AR-010 scene join sheds frames exactly when the dense branch falls behind | | AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`. The RANSAC fit it replaces disagreed by a median 17 source px on 400 headshots, 83.5% of crops embedding below cos 0.99, and was unstable and RNG-driven: rebuilding caught 1614 near-duplicates against the original build's ~100. **All galleries rebuilt** (2456 actors, 10254 embeddings); measured separation gain is small (0.583 → 0.590), so recorded accuracy figures should be re-run but are not expected to move far | @@ -38,7 +38,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted | | AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done | | AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Done** — decode butterfly joined via `SceneBoundaries`; the sampled branch waits for the detector's watermark. Frames past its last scored window are counted as unverified, never assumed boundary-free | -| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned | +| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | **Done** — both violations SPEC.md named are closed. (1) `scene_decode_fps` defaults to 0 (native): at 12 fps a 100-frame `kWindow` spanned ~8.3 s instead of the ~4 s TransNetV2 was trained on, half-speed motion over twice its temporal context. (2) The boundary dedup window is derived from the cadence the detector was actually fed (`SceneDetectorFunc::dedup_window_sec()`, median observed interval, halved) rather than the literal 0.04 s — one frame at 25 fps, and at 30 fps wider than a frame, so two cuts on consecutive frames merged into one and the loss was invisible: the file simply had fewer boundaries. Derivation checked at 24/25/30 fps and under a seek (UT-003). **Consequence, not a gap:** `scene_threshold` 0.60 was fitted against the 12 fps input and is now certainly wrong — VR-006 re-fits it, and until then boundary recall at native rate is untuned rather than better. Dense decode is the cost driver, so this is not free; `dense_scale` and `scene_stride` remain the reductions that do not run the model off-distribution | | AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done** — `src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track | | AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done** — `last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed | | AR-014 | Belief swap A→B terminates the track and starts a new one | SR-002 | Medium | **Done** — swap closes at `last_seen` and opens a successor at the swap frame; counted | @@ -108,7 +108,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done | | VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done | | VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 24–32 px; 32 px gives 98.1% TPI, 0.0 FPI at every size. Degrades an already-aligned 112×112 crop, so it isolates the embedder and is an **upper bound**; VR-013 measures the same question end to end and AR-002 takes its number, not this one | -| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned | +| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | **Medium** | **Planned, now unblocked** — native-rate decode landed with AR-011, so the prerequisite is met and the current 0.60 is a value fitted against input the pipeline no longer produces. Raised from Low for that reason: it is no longer a refinement, it is a stale constant | | VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned | | VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned | | VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned | diff --git a/src/config.hpp b/src/config.hpp index c56472c..5ae8061 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -91,17 +91,28 @@ struct Config { // at ~0.50; real boundaries spike to ~0.7+) int scene_stride{50}; // frames advanced between windows (≤ kWindow) - // Dense-decode throughput knobs (only active with scene_detect). Dense decode - // of every native-rate frame is the pipeline's cost driver; these trade a - // little boundary precision for a large speedup. - // scene_decode_fps: rate the source decodes at in dense mode. Lower = - // fewer frames decoded. TransNetV2 tolerates ~12fps; boundary timestamps - // stay correct (keyed off each frame's real timestamp). 0 = native fps. + // Dense-decode knobs (only active with scene_detect). Dense decode of every + // native-rate frame is the pipeline's cost driver, which is what made the + // temporal shortcut below tempting. + /// TRACES: AR-011 | SR-002 + // scene_decode_fps: rate the source decodes at in dense mode. + // **0 = native, and native is the only correct setting.** kWindow is 100 + // frames: at native 25 fps that window spans ~4 s, which is what + // TransNetV2 was trained on; at the 12 fps this used to default to it + // spans ~8.3 s, so the model saw half-speed motion over twice its + // temporal context. Boundary *timestamps* stay right either way — which + // is exactly why the degradation was invisible, and why the compressed + // separation it produced (~0.50 baseline against ~0.7+ peaks) was read + // as a property of the export rather than of the input. Lowering this + // buys decode time by running the model off-distribution; reach for + // dense_scale or scene_stride instead, which do not. // dense_scale: downscale factor applied to decoded frames in dense mode // (0 pre-built TransNetV2 TRT engine (TRT backend) // --scene-threshold boundary sigmoid prob above this → cut (default: 0.60) // --scene-stride frames between TransNetV2 windows (default: 50, ≤100) -// --scene-decode-fps dense decode rate in scene-detect mode (default: 12; -// 0 = native fps). Lower = faster, coarser boundaries. +// --scene-decode-fps dense decode rate in scene-detect mode (default: 0 = +// native, the only rate TransNetV2 is calibrated for; +// AR-011). Lowering it runs the model off-distribution. // --dense-scale downscale decoded frames in scene-detect mode (0 max faces kept per frame (default: 10) diff --git a/src/nodes/face_detector_node.hpp b/src/nodes/face_detector_node.hpp index 6689291..6b409de 100644 --- a/src/nodes/face_detector_node.hpp +++ b/src/nodes/face_detector_node.hpp @@ -6,6 +6,7 @@ #include #include #include +#include // ── FaceDetectorFunc ────────────────────────────────────────────────────────── // KPN node: runs SCRFD-500MF to detect ALL faces in a frame. @@ -22,22 +23,38 @@ struct FaceDetectorFunc { , min_face_px_(cfg.min_face_px) {} - SceneFrame operator()(Frame f) { - if (f.eof) return {std::move(f), {}}; - - auto faces = detector_->detect(f.image); - - // Drop faces below minimum pixel size (too small for reliable ArcFace - // alignment). Note: when dense_scale downscaled the frame, both the - // detection coords and min_face_px are in downscaled space — so scale - // the threshold down to match, keeping the physical size cutoff constant. - const float min_px = (f.bbox_upscale != 1.f) - ? min_face_px_ / f.bbox_upscale : min_face_px_; + /// TRACES: AR-002 | SR-002 + // Drop faces below the minimum size — too small for reliable ArcFace + // alignment, and below the resolution where identification still holds + // (VR-013 measured the knee end to end). + // + // The minimum is expressed in ORIGINAL video resolution, which is what makes + // it a property of the footage rather than of a throughput knob. When + // dense_scale downscaled the frame the detector's boxes are in downscaled + // space, and `bbox_upscale` is what maps them back; dividing the threshold by + // it rather than multiplying every box keeps the comparison on the detector's + // own numbers and the physical cutoff constant across scales. + // + // Strictly less-than: a face exactly at the minimum is admissible, which is + // what a "minimum of 40x40" means. + static void drop_undersized(std::vector& faces, + float min_face_px, + float bbox_upscale) { + const float min_px = (bbox_upscale > 0.f) ? min_face_px / bbox_upscale + : min_face_px; faces.erase( std::remove_if(faces.begin(), faces.end(), [&](const DetectedFace& d) { return d.bbox.width < min_px || d.bbox.height < min_px; }), faces.end()); + } + + SceneFrame operator()(Frame f) { + if (f.eof) return {std::move(f), {}}; + + auto faces = detector_->detect(f.image); + + drop_undersized(faces, min_face_px_, f.bbox_upscale); // Sort largest-first so max_faces_ keeps the most informative detections std::sort(faces.begin(), faces.end(), diff --git a/src/nodes/scene_detector_node.hpp b/src/nodes/scene_detector_node.hpp index d694e67..bb9305c 100644 --- a/src/nodes/scene_detector_node.hpp +++ b/src/nodes/scene_detector_node.hpp @@ -67,6 +67,15 @@ struct SceneDetectorFunc { return; } + /// TRACES: AR-011 | SR-002 + // Learn the cadence of the stream from the stream itself, rather than + // assuming one. See dedup_window_sec(). + if (prev_ts_ >= 0.0 && intervals_.size() < kCadenceSamples) { + const double dt = f.timestamp_sec - prev_ts_; + if (dt > 0.0) intervals_.push_back(dt); + } + prev_ts_ = f.timestamp_sec; + images_.push_back(f.image); times_.push_back(f.timestamp_sec); @@ -81,6 +90,34 @@ struct SceneDetectorFunc { } } + /// TRACES: AR-011 | SR-002 + // How close two boundaries have to be before they are the same boundary, + // derived from the cadence the detector was actually fed. + // + // What this replaces is a literal 0.04 s — one frame at 25 fps, and silently + // wrong at any other rate. On a 30 fps source it spans more than a frame, so + // two cuts on consecutive frames merge into one and a real boundary is lost; + // the output does not show this, it simply contains fewer cuts. Assuming a + // frame rate is the same class of mistake as feeding a model the wrong rate, + // which is why this belongs to AR-011 and not to a tidy-up. + // + // Half a frame, not a whole one, because the only thing being deduplicated is + // one frame scored by two overlapping windows — a gap of zero. Two distinct + // frames are a full interval apart and must both survive. Half an interval + // separates those two cases without putting the decision on the knife-edge + // where floating-point error settles it. + // + // Median, not mean: a seek, or a gap where the decoder dropped a frame, + // contributes one long interval that would drag a mean and cannot move a + // median. + static double dedup_window_sec(std::vector intervals) { + if (intervals.empty()) return 0.0; // <2 frames: nothing to deduplicate + const std::size_t mid = intervals.size() / 2; + std::nth_element(intervals.begin(), intervals.begin() + mid, + intervals.end()); + return intervals[mid] * 0.5; + } + private: // Run TransNetV2 on the leading kWindow frames of the buffer and record any // boundaries found within the trusted centre region. @@ -154,6 +191,8 @@ private: written_ = true; // Merge boundaries closer than one frame apart (dedup across window seams). + const double dedup_sec = dedup_window_sec(intervals_); + std::sort(boundaries_.begin(), boundaries_.end(), [](const Boundary& a, const Boundary& b) { return a.t < b.t; @@ -167,7 +206,7 @@ private: nlohmann::json cuts = nlohmann::json::array(); double last_t = -1e9; for (const auto& b : boundaries_) { - if (b.t - last_t < 0.04) continue; // ~1 frame @25fps dedup + if (b.t - last_t < dedup_sec) continue; cuts.push_back({{"t", b.t}, {"probability", b.prob}}); last_t = b.t; } @@ -180,8 +219,12 @@ private: return; } f << root.dump(2) << "\n"; + // Report the derived cadence: VR-006 re-tunes scene_threshold against it, + // and a rate that is not the source's is the first thing to suspect. std::cerr << "\n[scene_detector] wrote " << root["cuts"].size() - << " boundaries → " << output_path_ << "\n"; + << " boundaries → " << output_path_ + << " (dedup=" << dedup_sec << "s from " + << (dedup_sec > 0.0 ? 0.5 / dedup_sec : 0.0) << " fps)\n"; } static int kLast_() { return ISceneDetector::kWindow - 1; } @@ -195,6 +238,10 @@ private: struct Boundary { double t; float prob; }; + // Enough to establish a rate; bounded so a feature-length film does not + // accumulate one double per frame for a number that stops moving early. + static constexpr std::size_t kCadenceSamples = 512; + std::unique_ptr detector_; float threshold_; int stride_; @@ -207,6 +254,8 @@ private: std::deque times_; int64_t window_base_{0}; // frame index of images_.front() std::vector boundaries_; + double prev_ts_{-1.0}; // AR-011: cadence, learned not assumed + std::vector intervals_; bool written_{false}; std::shared_ptr shared_; ///< AR-010 join point }; diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 8615f1c..dfb2587 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -22,6 +22,8 @@ add_executable(sae_tests test_track_gallery.cpp test_face_tracker.cpp test_track_registry.cpp + test_face_detector_node.cpp + test_scene_detector_node.cpp test_replay_fixtures.cpp test_audio_signature.cpp ${CMAKE_SOURCE_DIR}/src/backends/gemm_backend.cpp diff --git a/tests/test_face_detector_node.cpp b/tests/test_face_detector_node.cpp new file mode 100644 index 0000000..5af0f24 --- /dev/null +++ b/tests/test_face_detector_node.cpp @@ -0,0 +1,114 @@ +// AR-002 — minimum face size, in original video resolution. +// +// TRACES: AR-002 | SR-002 | UT-002 +// +// Tier T1 here, T2 in test_replay_fixtures.cpp. The requirement is arithmetic on +// bounding boxes, so the two edge cases that matter — a face sitting exactly on +// the threshold, and the same face seen through a downscaled decode — are +// reachable without a detector, a model or a GPU. What the fixture check adds is +// that the rule was actually applied on the way to a dump; what this adds is that +// it is applied *correctly*, which no real dump can demonstrate because real +// footage does not contain a 39.999 px face on demand. +// +// FaceDetectorFunc is never constructed: its constructor loads SCRFD. Only the +// static rule is called, so make_face_detector() is never odr-used and nothing +// here needs a backend. +#include + +#include "nodes/face_detector_node.hpp" +#include "types.hpp" + +#include + +namespace { + +DetectedFace box(float w, float h) { + DetectedFace f; + f.bbox = cv::Rect2f(10.f, 10.f, w, h); + f.confidence = 0.9f; + return f; +} + +// Sizes the rule kept, in the order given. +std::vector surviving_widths(std::vector faces, + float min_face_px, float bbox_upscale) { + FaceDetectorFunc::drop_undersized(faces, min_face_px, bbox_upscale); + std::vector out; + for (const auto& f : faces) out.push_back(f.bbox.width); + return out; +} + +constexpr float kMin = 40.f; // Config::min_face_px default, and AR-002's number + +} // namespace + +// ── Exactly at the threshold ───────────────────────────────────────────────── +// The boundary case is the whole content of a minimum: "40x40" has to mean 40 is +// admissible, or the requirement says 41. +TEST_CASE("a face exactly at the minimum is kept", "[detector][AR-002]") { + CHECK(surviving_widths({box(kMin, kMin)}, kMin, 1.f).size() == 1); +} + +TEST_CASE("a face one tenth of a pixel under the minimum is dropped", + "[detector][AR-002]") { + CHECK(surviving_widths({box(39.9f, 100.f)}, kMin, 1.f).empty()); + CHECK(surviving_widths({box(100.f, 39.9f)}, kMin, 1.f).empty()); +} + +TEST_CASE("both sides must clear the minimum, not the larger one", + "[detector][AR-002]") { + // A wide, short box has enough pixels and is still unusable: ArcFace + // alignment needs both dimensions. Area would admit this; the rule must not. + CHECK(surviving_widths({box(400.f, 20.f)}, kMin, 1.f).empty()); +} + +TEST_CASE("the filter is a filter, not a reordering", "[detector][AR-002]") { + auto kept = surviving_widths( + {box(80.f, 80.f), box(10.f, 10.f), box(60.f, 60.f), box(39.f, 39.f)}, + kMin, 1.f); + REQUIRE(kept.size() == 2); + // Order is load-bearing downstream (AR-003's largest-first sort tie-breaks on + // it, and the Hungarian solver tie-breaks on index) — erase-remove must not + // shuffle the survivors. + CHECK(kept[0] == 80.f); + CHECK(kept[1] == 60.f); +} + +// ── The dense_scale interaction the requirement exists for ─────────────────── +// dense_scale 0.5 halves the decoded frame, so the detector reports a 40 px face +// as 20 px. If the threshold were applied to those numbers, turning on a +// throughput knob would silently double the minimum face size the pipeline +// accepts — a recall change with no line in the config to explain it. AR-002 +// pins the minimum to the ORIGINAL resolution instead. +TEST_CASE("at dense_scale 0.5 the cutoff stays 40 px of original footage", + "[detector][AR-002]") { + constexpr float kUpscale = 2.f; // frame_source_node: 1 / dense_scale + + // 20 px in downscaled space is exactly 40 px of original footage: kept. + CHECK(surviving_widths({box(20.f, 20.f)}, kMin, kUpscale).size() == 1); + + // 19.9 px downscaled is 39.8 px original: dropped. + CHECK(surviving_widths({box(19.9f, 19.9f)}, kMin, kUpscale).empty()); + + // And the interaction stated as one claim: a face of a given original size is + // admitted or refused identically whether or not the decode was downscaled. + for (float original : {30.f, 39.f, 40.f, 41.f, 80.f}) { + INFO("original size " << original); + const bool full = !surviving_widths({box(original, original)}, + kMin, 1.f).empty(); + const bool dense = !surviving_widths({box(original / kUpscale, + original / kUpscale)}, + kMin, kUpscale).empty(); + CHECK(full == dense); + CHECK(full == (original >= kMin)); + } +} + +TEST_CASE("an absent or degenerate upscale falls back to the raw threshold", + "[detector][AR-002]") { + // bbox_upscale is 1 on every non-dense frame; 0 would mean the frame source + // never set it. Dividing by that would reject every face in the film, which + // is a failure worth not having. + CHECK(surviving_widths({box(kMin, kMin)}, kMin, 0.f).size() == 1); + CHECK(surviving_widths({box(39.f, 39.f)}, kMin, 0.f).empty()); +} diff --git a/tests/test_replay_fixtures.cpp b/tests/test_replay_fixtures.cpp index 8fb894d..518214f 100644 --- a/tests/test_replay_fixtures.cpp +++ b/tests/test_replay_fixtures.cpp @@ -1,6 +1,6 @@ // Replay tests — the real tracker and registry driven from committed fixtures. // -// TRACES: AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001 +// TRACES: AR-002, AR-004, AR-012, AR-013 | VR-001, VR-002 | IT-001 // // Tier T2: composition, not units. The registry tests construct awkward states // directly; these check that the pieces behave when wired together and fed real @@ -27,6 +27,7 @@ #include #include +#include #include #include #include @@ -44,6 +45,8 @@ struct Dump { std::vector emb; std::vector bbox; // 4 per face std::string embedder; + float min_face_px{0.f}; // AR-002, as the run was configured + float bbox_upscale{1.f}; // bbox × this = original resolution std::size_t frames() const { return ts.size(); } std::size_t faces() const { return emb.size(); } @@ -97,6 +100,17 @@ Dump load(const std::string& path) { H5::StrType vlen(H5::PredType::C_S1, H5T_VARIABLE); f.openAttribute("embedder_model").read(vlen, d.embedder); } + + // AR-002: the threshold the run was configured with, and the scale its boxes + // are in. Read from the dump rather than assumed, so the check is against + // what this fixture was actually generated with — the hero clips predate the + // move to 40 px and were dumped at 32. + if (f.attrExists("min_face_px")) + f.openAttribute("min_face_px").read(H5::PredType::NATIVE_FLOAT, + &d.min_face_px); + if (f.attrExists("bbox_upscale")) + f.openAttribute("bbox_upscale").read(H5::PredType::NATIVE_FLOAT, + &d.bbox_upscale); return d; } @@ -167,6 +181,33 @@ TEST_CASE("superhero fixture is complete", "[replay][VR-001]") { CHECK(static_cast(running) == d.faces()); } +// ── AR-002 — the size filter held, all the way to the dump ─────────────────── +// The T1 arithmetic is in test_face_detector_node.cpp. This is the other half: +// that the rule was applied on real footage and nothing downstream of it let an +// undersized face back in. +TEST_CASE("no dumped face is below the configured minimum size", + "[replay][AR-002]") { + Dump d = load(fixture("superhero.h5")); + // A dump that did not record its threshold cannot be checked against one. + REQUIRE(d.min_face_px > 0.f); + REQUIRE(d.bbox_upscale > 0.f); + + float smallest_side = std::numeric_limits::max(); + for (std::size_t i = 0; i < d.faces(); ++i) { + const float w = d.bbox[i * 4 + 2] * d.bbox_upscale; // original resolution + const float h = d.bbox[i * 4 + 3] * d.bbox_upscale; + REQUIRE(w >= d.min_face_px); + REQUIRE(h >= d.min_face_px); + smallest_side = std::min({smallest_side, w, h}); + } + + // And the filter was binding, not vacuously satisfied. 480x360 footage puts + // faces right on the cutoff, which is what makes this fixture worth checking: + // if the threshold stopped being applied the assertions above would still + // pass on a corpus of close-ups. + CHECK(smallest_side < d.min_face_px * 1.05f); +} + TEST_CASE("replaying the superhero fixture twice gives identical tracks", "[replay][VR-002]") { Dump d = load(fixture("superhero.h5")); diff --git a/tests/test_scene_detector_node.cpp b/tests/test_scene_detector_node.cpp new file mode 100644 index 0000000..0ed590d --- /dev/null +++ b/tests/test_scene_detector_node.cpp @@ -0,0 +1,86 @@ +// AR-011 — the boundary dedup window is derived from the stream's cadence, not +// assumed. +// +// TRACES: AR-011 | SR-002 | UT-003 +// +// Tier T1: the derivation is arithmetic on frame timestamps, so it is checked +// against synthetic cadences at 24, 25 and 30 fps rather than against a decode. +// The number this replaced was 0.04 s — one frame at 25 fps, correct for exactly +// one of those three and quietly wrong for the other two. +// +// SceneDetectorFunc is never constructed: its constructor loads TransNetV2. Only +// the static rule is called, so make_scene_detector() is never odr-used. +#include + +#include "nodes/scene_detector_node.hpp" + +#include + +namespace { + +// The intervals the node accumulates from a steady stream at `fps`. +std::vector cadence(double fps, int n = 200) { + return std::vector(static_cast(n), 1.0 / fps); +} + +double window(double fps) { + return SceneDetectorFunc::dedup_window_sec(cadence(fps)); +} + +} // namespace + +// ── The property that has to hold at every rate ────────────────────────────── +// The window has exactly one job: tell "one frame scored twice by two +// overlapping windows" (a gap of zero) from "two adjacent frames, both of them +// real cuts" (a gap of one frame interval). It has to sit strictly between. +TEST_CASE("the dedup window separates a duplicate from an adjacent frame", + "[scene][AR-011]") { + for (double fps : {24.0, 25.0, 30.0, 23.976, 29.97, 50.0, 60.0}) { + INFO("source at " << fps << " fps"); + const double frame = 1.0 / fps; + const double w = window(fps); + + CHECK(w > 0.0); // a duplicate (gap 0) is still merged + CHECK(w < frame); // two consecutive frames both survive + } +} + +// The concrete failure the hardcoded constant caused: at 30 fps a frame is +// 0.0333 s, so a 0.04 s window swallowed a cut on the very next frame. Nothing in +// the output showed it — the file just had fewer boundaries. +TEST_CASE("cuts on consecutive frames survive at 30 fps", "[scene][AR-011]") { + const double frame = 1.0 / 30.0; + CHECK(window(30.0) < frame); + CHECK(0.04 > frame); // the constant that was there, for the record +} + +TEST_CASE("the window tracks the rate rather than a constant", + "[scene][AR-011]") { + // If it were still assumed, these would be equal. + CHECK(window(24.0) > window(30.0)); + CHECK(window(30.0) > window(60.0)); + CHECK(window(25.0) == 0.5 / 25.0); +} + +// ── Robustness of the estimate ─────────────────────────────────────────────── +TEST_CASE("a seek or a dropped frame does not move the derived cadence", + "[scene][AR-011]") { + auto intervals = cadence(25.0); + intervals[0] = 3.5; // a seek at the start + intervals[97] = 0.4; // a gap where the decoder lost frames + + // Median, not mean: two long intervals out of 200 cannot shift it at all. + CHECK(SceneDetectorFunc::dedup_window_sec(intervals) == 0.5 / 25.0); +} + +TEST_CASE("too few frames to have a cadence yields an inert window", + "[scene][AR-011]") { + // Under two frames there is no interval to measure — and also no second + // boundary to merge with, so a window of 0 changes nothing. Guessing a rate + // here would be the mistake this requirement is about. + CHECK(SceneDetectorFunc::dedup_window_sec({}) == 0.0); +} + +TEST_CASE("a single observed interval is enough", "[scene][AR-011]") { + CHECK(SceneDetectorFunc::dedup_window_sec({1.0 / 24.0}) == 0.5 / 24.0); +}