From fa1c494825ae00efd494d4a2fed78e2ada70c275 Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Fri, 31 Jul 2026 14:27:05 +0200 Subject: [PATCH] docs: AR-010 is blocked on a design decision, not an implementation gap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Making SceneDetectorFunc a pass-through does not work. TransNetV2 buffers 100 dense frames before it can score any of them and trusts only each window's centre, so a boundary at time T is not known until roughly 3.3s after T at 30 fps. The face pipeline runs on a parallel branch and has long since passed T. An association hint that arrives after the association is worthless. Three options recorded with their costs: two-pass (correct, doubles the decode that already dominates runtime), delaying the face branch (couples the two branches' timing, which invites heisenbugs under backpressure), or leaving it unwired. Leaving it unwired costs less than it looks, which is what makes this a decision rather than a defect. The redesign made cuts and boundaries do the same thing — both say "spatial continuity is broken, associate on embedding" — so TransNetV2 adds nothing over the histogram except on transitions the histogram cannot see: slow dissolves and fades. That gap is real but narrow. Where TransNetV2 still earns its cost is AR-019, whose promotion gate wants a span free of cuts and boundaries. A late answer is fine there, because promotion happens on track confirmation rather than per frame — so it can be wired offline against the collected boundary list, off the hot path entirely. Recommendation: leave the association path on is_cut alone, wire boundaries into AR-019, and revisit if dissolve-heavy material shows association failures. Co-Authored-By: Claude Opus 5 TRACES: AR-010, AR-019 | SR-002 --- docs/SPEC.md | 42 +++++++++++++++++++++++++++-------- docs/requirements.md | 2 +- docs/traceability.md | 52 +++++++++++++++++++++++++++----------------- 3 files changed, 66 insertions(+), 30 deletions(-) diff --git a/docs/SPEC.md b/docs/SPEC.md index 31b7efd..f890f41 100644 --- a/docs/SPEC.md +++ b/docs/SPEC.md @@ -204,16 +204,40 @@ Two distinct signals, deliberately kept separate: > `--dump-embeddings` branch *before* the `scene_detect` branch at `:296`, so no > dump-producing path even instantiates the detector. > -> This makes AR-010 **not implemented**, not "in progress" — and it means a T2 -> test of the frame-dependent `track_alpha` (AR-007) would **pass vacuously**, -> which is the worst possible failure for a verification gate. The fix is in the -> producer, not the schema: make `SceneDetectorFunc` a pass-through (or add a -> boundary annotator before the decimator) and add the scene branch to -> `dump_embeddings.cpp`. **No `schema_version` bump** — the column exists and -> merely stops being constant. > -> Fixtures generated before the fix must be marked in provenance, since `0` is -> presently indistinguishable from "no boundary here". +> **It cannot be fixed by making the node a pass-through.** TransNetV2 buffers +> `kWindow` = 100 dense frames before it can score any of them, runs inference +> every `scene_stride` (50) frames, and trusts only each window's centre. So a +> boundary at time *T* is not known until roughly 100 dense frames after *T* — +> about **3.3 s at 30 fps**. The face pipeline runs on a parallel branch and has +> long since passed *T* by then. An association hint that arrives after the +> association is worthless. +> +> Three ways out, none free: +> +> 1. **Two-pass.** Run scene detection to completion, then analyse faces with +> boundaries already known. Simple and correct; costs a second decode of the +> whole file, and dense decode is already the pipeline's dominant cost. +> 2. **Delay the face branch** by the detector's window latency. Keeps one pass; +> adds a buffering stage and couples the two branches' timing, which is the +> kind of coupling that produces heisenbugs under backpressure. +> 3. **Leave it unwired.** Accept that `is_cut` is the only association hint. +> +> **Option 3 costs less than it appears**, which is why this is a decision rather +> than a bug. Since the redesign made cuts and boundaries do the *same thing* — +> both say "spatial continuity is broken, associate on embedding" — TransNetV2 +> adds nothing over the histogram except on transitions the histogram misses: +> slow dissolves and fades, where there is no frame-to-frame discontinuity to +> detect. That is a real but narrow gap. +> +> The value TransNetV2 retains is in **AR-019**, whose promotion gate requires a +> span with no cut *and* no boundary. There a late answer is still usable, +> because promotion happens when a track is confirmed rather than per frame. +> Wiring it there — offline, against the collected boundary list — is cheaper +> than any of the three options above and does not touch the hot path. +> +> **Recommendation: option 3 plus the AR-019 wiring**, and revisit if dissolve- +> heavy material shows association failures the histogram misses. Both feed AR-007 as **association hints**: they tell the tracker that spatial continuity is broken and that association should weight embedding over IoU. diff --git a/docs/requirements.md b/docs/requirements.md index 6dcf028..fcf0060 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -37,7 +37,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn` | AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks | | AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted | | AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done | -| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Not started** — `is_scene_boundary` has no producer; `SceneDetectorFunc` is a terminal sink and never annotates the frame | +| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Blocked on a design decision** — the detector needs ~100 dense frames (~3.3 s) before it can score, so a boundary arrives long after the face branch has passed it. An association hint that late is useless. See SPEC AR-009…AR-011 | | AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned | | AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done** — `src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track | | AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done** — `last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed | diff --git a/docs/traceability.md b/docs/traceability.md index 5f876d6..c38d44c 100644 --- a/docs/traceability.md +++ b/docs/traceability.md @@ -3,7 +3,7 @@ -**Generated:** 2026-07-31T09:23:18+00:00 +**Generated:** 2026-07-31T12:22:20+00:00 Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`). @@ -11,13 +11,13 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev | Metric | Value | |---|---| -| Source files scanned | 96 | -| TRACES tags found | 92 | +| Source files scanned | 102 | +| TRACES tags found | 95 | | EXCEPTION tags found | 0 | | Requirements defined | 63 | -| Requirements covered | 27 | -| **Coverage** | **42.9%** (27/63) | -| Coverage of CI-executable scope | 51.9% (27/52) | +| Requirements covered | 28 | +| **Coverage** | **44.4%** (28/63) | +| Coverage of CI-executable scope | 53.8% (28/52) | | Tagged but unexecuted in CI | 3 | | Orphan tags | 0 | @@ -25,7 +25,7 @@ Denominators are read from [`requirements.md`](requirements.md) at run time, nev | Type | Covered | Tagged but unexecuted | Defined | |---|---|---|---| -| AR | 14 | 0 | 27 | +| AR | 15 | 0 | 27 | | DP | 2 | 0 | 8 | | IR | 8 | 0 | 8 | | GR | 3 | 0 | 9 | @@ -80,8 +80,8 @@ _None._ |---|---|---|---|---|---|---| | AR-001 | Done | T3 | SR-002 | covered | `src/nodes/face_detector_node.hpp` | Detect faces in sampled frames; emit bbox, confidence, 5-point landma… | | AR-002 | Planned | unset | SR-002 | untagged | - | Minimum face size **32×32 px** (VR-005 measured), expressed in **orig… | -| AR-003 | Planned | T1, T2, T4 | SR-002 | untagged | - | No fixed per-frame face cap — crowd scenes must not lose background c… | -| AR-004 | **Done** — KPN node… | T1, T4 | SR-002 | covered | `src/main.cpp`, `tests/test_replay_fixtures.cpp` | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… | +| AR-003 | **Done** — `max_fac… | T1, T2, T4 | SR-002 | covered | `src/config.hpp`, `src/nodes/face_detector_node.hpp`, `src/nodes/identity_matcher_node.hpp` | No fixed per-frame face cap — crowd scenes must not lose background c… | +| AR-004 | **Done** — KPN node… | T1, T4 | SR-002 | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_replay_fixtures.cpp` | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… | | AR-005 | Done | T1, T3 | SR-002 | covered | `src/face_utils.hpp` | Align to 112×112 via ArcFace 5-point similarity transform | | AR-006 | Done | T3 | SR-002 | untagged | - | 512-d L2-normalised embeddings, batched | | AR-007 | **Done** — `track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… | @@ -150,11 +150,20 @@ _None._ - [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown` +### AR-003 + +**Locations:** 3 + +- [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown` +- [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:` +- [`src/nodes/identity_matcher_node.hpp:149`](../src/nodes/identity_matcher_node.hpp#L149) — `std::vector host_query(static_cast(kMaxFaces) * 512);` + ### AR-004 -**Locations:** 2 +**Locations:** 3 - [`src/main.cpp:285`](../src/main.cpp#L285) — `std::lock_guard lk(event_mtx);` +- [`src/nodes/identity_matcher_node.hpp:149`](../src/nodes/identity_matcher_node.hpp#L149) — `std::vector host_query(static_cast(kMaxFaces) * 512);` - [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown` ### AR-005 @@ -167,7 +176,7 @@ _None._ **Locations:** 3 -- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown` +- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` @@ -175,7 +184,7 @@ _None._ **Locations:** 3 -- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown` +- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` @@ -186,7 +195,7 @@ _None._ - [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }` -- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown` +- [`src/nodes/identity_matcher_node.hpp:255`](../src/nodes/identity_matcher_node.hpp#L255) — `Unknown` - [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }` - [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };` - [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown` @@ -244,7 +253,7 @@ _None._ **Locations:** 6 -- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown` +- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown` - [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const` - [`src/main.cpp:199`](../src/main.cpp#L199) — `reg_cfg, EvidenceDiscounter(same_person));` @@ -257,7 +266,7 @@ _None._ - [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown` - [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }` -- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown` +- [`src/nodes/identity_matcher_node.hpp:255`](../src/nodes/identity_matcher_node.hpp#L255) — `Unknown` ### DP-001 @@ -287,7 +296,7 @@ _None._ **Locations:** 44 -- [`src/config.hpp:49`](../src/config.hpp#L49) — `Unknown` +- [`src/config.hpp:54`](../src/config.hpp#L54) — `Unknown` - [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown` - [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown` - [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)` @@ -440,7 +449,7 @@ _None._ **Locations:** 46 -- [`src/config.hpp:49`](../src/config.hpp#L49) — `Unknown` +- [`src/config.hpp:54`](../src/config.hpp#L54) — `Unknown` - [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown` - [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown` - [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)` @@ -489,9 +498,10 @@ _None._ ### SR-002 -**Locations:** 17 +**Locations:** 20 -- [`src/config.hpp:103`](../src/config.hpp#L103) — `Unknown` +- [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown` +- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown` - [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown` - [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `inline cv::Mat align_face(const cv::Mat& img,` - [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown` @@ -500,10 +510,12 @@ _None._ - [`src/main.cpp:216`](../src/main.cpp#L216) — `reg_cfg, EvidenceDiscounter(same_person));` - [`src/main.cpp:285`](../src/main.cpp#L285) — `std::lock_guard lk(event_mtx);` - [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown` +- [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:` - [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown` - [`src/nodes/identity_matcher_node.hpp:111`](../src/nodes/identity_matcher_node.hpp#L111) — `const GalleryCalibration& calibration() const { return cal_; }` - [`src/nodes/identity_matcher_node.hpp:119`](../src/nodes/identity_matcher_node.hpp#L119) — `const GalleryCalibration& calibration() const { return cal_; }` -- [`src/nodes/identity_matcher_node.hpp:242`](../src/nodes/identity_matcher_node.hpp#L242) — `Unknown` +- [`src/nodes/identity_matcher_node.hpp:149`](../src/nodes/identity_matcher_node.hpp#L149) — `std::vector host_query(static_cast(kMaxFaces) * 512);` +- [`src/nodes/identity_matcher_node.hpp:255`](../src/nodes/identity_matcher_node.hpp#L255) — `Unknown` - [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }` - [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function fn) { pre_write_ = std::move(fn); }` - [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`