feat(scene): feed TransNetV2 at native rate, derive the dedup window from it

Closes both violations SPEC.md named under "Every model gets the input it
was trained for". They are one bug, not two.

The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window
spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over
twice its temporal context. Boundary timestamps stayed correct throughout,
which is exactly why the degradation was invisible and why the compressed
separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a
property of the ONNX export rather than of the input.

Dedup then merged boundaries closer than a literal 0.04 s — one frame at
25 fps, and wider than a frame at 30, so two cuts on consecutive frames
became one. Nothing in scenes.json showed it; the file simply had fewer
boundaries. Native rate is where that constant did the most damage, which
is why fixing the decode rate without fixing the dedup would have made
things worse.

dedup_window_sec() now takes the median interval the detector was actually
fed and halves it. Half a frame rather than a whole one: the only thing
being merged is one frame scored by two overlapping windows, and two
distinct frames are a full interval apart.

Cost is real — dense decode is the pipeline's cost driver. It is accepted;
dense_scale and scene_stride remain the reductions that do not run the
model off-distribution. scene_threshold 0.60 was fitted against the 12 fps
input and is now stale, so VR-006 goes from Low to Medium: it is no longer
a refinement, it is a constant that no longer describes the input.

AR-002 rides along because it was already implemented, just untagged and
unverified — the register said Planned while the code was correct. The size
filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold
and at dense_scale 0.5, and checked end to end against the superhero dump,
whose smallest face is exactly its recorded 32 px minimum, so the fixture
check cannot pass vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001
This commit is contained in:
2026-08-04 21:17:57 +02:00
co-authored by Claude Opus 5
parent 079b490ede
commit f33403fff8
10 changed files with 378 additions and 44 deletions
+29 -16
View File
@@ -438,7 +438,7 @@ Both feed AR-007 as **association hints**: they tell the tracker that spatial
continuity is broken and that association should weight embedding over IoU.
Neither ends a presence window (AR-012).
In dense mode the source decodes at `scene_decode_fps` (default 12) and a
In dense mode the source decodes at `scene_decode_fps` (default 0 = native) and a
decimator splits the stream: full-resolution sampled frames to the face pipeline,
downscaled dense frames to the scene detector
(`frame_source_node.hpp:63`). `sample_fps` is independent of this — the face
@@ -459,31 +459,44 @@ degrading what a single inference sees. A model run off-distribution produces
confident, plausible, wrong output, and the error is invisible without a study
that should not have been necessary.
Two places this is currently violated:
Two places this was violated, both now closed:
1. **`scene_decode_fps = 12` starves TransNetV2.** `kWindow` is 100 frames. At
native 25 fps that window spans ~4 s; at 12 fps it spans ~8.3 s, so the model
sees roughly half-speed motion over twice the temporal context it was trained
on. **Requirement: feed TransNetV2 at the source's native frame rate**, so a
100-frame window covers the duration the model expects. The
"tolerates ~12fps" note in `config.hpp` describes a compromise, and the
recorded margin is consistent with it — a non-boundary baseline at ~0.50 with
1. **`scene_decode_fps = 12` starved TransNetV2.** `kWindow` is 100 frames. At
native 25 fps that window spans ~4 s; at 12 fps it spanned ~8.3 s, so the
model saw roughly half-speed motion over twice the temporal context it was
trained on. **Requirement: feed TransNetV2 at the source's native frame
rate**, so a 100-frame window covers the duration the model expects. The
"tolerates ~12fps" note in `config.hpp` described a compromise, and the
recorded margin was consistent with it — a non-boundary baseline at ~0.50 with
real boundaries reaching only ~0.7+ is a compressed separation, not a healthy
one.
one. **Done:** `scene_decode_fps` defaults to 0.
2. **Hardcoded 25 fps in boundary dedup.** `scene_detector_node.hpp:138` merges
boundaries closer than `0.04 s` — "~1 frame @25fps". **Requirement: derive
this from the source's actual frame rate.**
2. **Hardcoded 25 fps in boundary dedup.** The node merged boundaries closer than
`0.04 s` — "~1 frame @25fps". **Requirement: derive this from the source's
actual frame rate. Done:** `SceneDetectorFunc::dedup_window_sec()` takes the
median of the frame intervals the detector was actually fed and halves it.
Half a frame rather than a whole one, because the only thing being merged is
one frame scored by two overlapping windows; two distinct frames are a full
interval apart and both have to survive.
The two are one change, not two. A native-rate stream is where the old constant
did the most damage — at 30 fps, 0.04 s is wider than a frame, so two cuts on
consecutive frames merged into one and the loss showed up nowhere: the file
simply had fewer boundaries.
Dense decode is the pipeline's cost driver, so (1) is not free. The cost is
accepted: the alternative is a boundary signal that steers association (AR-007) while
being quietly unreliable. `dense_scale` remains available as a spatial reduction,
since downscaling is a documented, understood degradation rather than a temporal
one the model has no defence against.
one the model has no defence against — and TransNetV2 downsamples to 48×27
regardless.
**Current:** histogram cut in the decoder; `scene_detector_node.hpp` for
TransNetV2. **Gap:** native-rate dense decode; framerate-derived dedup;
`--scene-detect` is default-off despite now feeding association.
TransNetV2, fed at native rate with a framerate-derived dedup window.
**Gap:** `scene_threshold` (0.60) is still the value picked against 12 fps input
and is now certainly wrong — VR-006 re-fits it, and until it does, boundary
recall at native rate is untuned rather than better. `--scene-detect` is
default-off despite now feeding association.
## AR-012 … AR-017 — Track-level identity propagation — **CHANGED BEHAVIOUR**