feat(scene): feed TransNetV2 at native rate, derive the dedup window from it
Closes both violations SPEC.md named under "Every model gets the input it was trained for". They are one bug, not two. The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over twice its temporal context. Boundary timestamps stayed correct throughout, which is exactly why the degradation was invisible and why the compressed separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a property of the ONNX export rather than of the input. Dedup then merged boundaries closer than a literal 0.04 s — one frame at 25 fps, and wider than a frame at 30, so two cuts on consecutive frames became one. Nothing in scenes.json showed it; the file simply had fewer boundaries. Native rate is where that constant did the most damage, which is why fixing the decode rate without fixing the dedup would have made things worse. dedup_window_sec() now takes the median interval the detector was actually fed and halves it. Half a frame rather than a whole one: the only thing being merged is one frame scored by two overlapping windows, and two distinct frames are a full interval apart. Cost is real — dense decode is the pipeline's cost driver. It is accepted; dense_scale and scene_stride remain the reductions that do not run the model off-distribution. scene_threshold 0.60 was fitted against the 12 fps input and is now stale, so VR-006 goes from Low to Medium: it is no longer a refinement, it is a constant that no longer describes the input. AR-002 rides along because it was already implemented, just untagged and unverified — the register said Planned while the code was correct. The size filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold and at dense_scale 0.5, and checked end to end against the superhero dump, whose smallest face is exactly its recorded 32 px minimum, so the fixture check cannot pass vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001
This commit is contained in:
+29
-16
@@ -438,7 +438,7 @@ Both feed AR-007 as **association hints**: they tell the tracker that spatial
|
||||
continuity is broken and that association should weight embedding over IoU.
|
||||
Neither ends a presence window (AR-012).
|
||||
|
||||
In dense mode the source decodes at `scene_decode_fps` (default 12) and a
|
||||
In dense mode the source decodes at `scene_decode_fps` (default 0 = native) and a
|
||||
decimator splits the stream: full-resolution sampled frames to the face pipeline,
|
||||
downscaled dense frames to the scene detector
|
||||
(`frame_source_node.hpp:63`). `sample_fps` is independent of this — the face
|
||||
@@ -459,31 +459,44 @@ degrading what a single inference sees. A model run off-distribution produces
|
||||
confident, plausible, wrong output, and the error is invisible without a study
|
||||
that should not have been necessary.
|
||||
|
||||
Two places this is currently violated:
|
||||
Two places this was violated, both now closed:
|
||||
|
||||
1. **`scene_decode_fps = 12` starves TransNetV2.** `kWindow` is 100 frames. At
|
||||
native 25 fps that window spans ~4 s; at 12 fps it spans ~8.3 s, so the model
|
||||
sees roughly half-speed motion over twice the temporal context it was trained
|
||||
on. **Requirement: feed TransNetV2 at the source's native frame rate**, so a
|
||||
100-frame window covers the duration the model expects. The
|
||||
"tolerates ~12fps" note in `config.hpp` describes a compromise, and the
|
||||
recorded margin is consistent with it — a non-boundary baseline at ~0.50 with
|
||||
1. **`scene_decode_fps = 12` starved TransNetV2.** `kWindow` is 100 frames. At
|
||||
native 25 fps that window spans ~4 s; at 12 fps it spanned ~8.3 s, so the
|
||||
model saw roughly half-speed motion over twice the temporal context it was
|
||||
trained on. **Requirement: feed TransNetV2 at the source's native frame
|
||||
rate**, so a 100-frame window covers the duration the model expects. The
|
||||
"tolerates ~12fps" note in `config.hpp` described a compromise, and the
|
||||
recorded margin was consistent with it — a non-boundary baseline at ~0.50 with
|
||||
real boundaries reaching only ~0.7+ is a compressed separation, not a healthy
|
||||
one.
|
||||
one. **Done:** `scene_decode_fps` defaults to 0.
|
||||
|
||||
2. **Hardcoded 25 fps in boundary dedup.** `scene_detector_node.hpp:138` merges
|
||||
boundaries closer than `0.04 s` — "~1 frame @25fps". **Requirement: derive
|
||||
this from the source's actual frame rate.**
|
||||
2. **Hardcoded 25 fps in boundary dedup.** The node merged boundaries closer than
|
||||
`0.04 s` — "~1 frame @25fps". **Requirement: derive this from the source's
|
||||
actual frame rate. Done:** `SceneDetectorFunc::dedup_window_sec()` takes the
|
||||
median of the frame intervals the detector was actually fed and halves it.
|
||||
Half a frame rather than a whole one, because the only thing being merged is
|
||||
one frame scored by two overlapping windows; two distinct frames are a full
|
||||
interval apart and both have to survive.
|
||||
|
||||
The two are one change, not two. A native-rate stream is where the old constant
|
||||
did the most damage — at 30 fps, 0.04 s is wider than a frame, so two cuts on
|
||||
consecutive frames merged into one and the loss showed up nowhere: the file
|
||||
simply had fewer boundaries.
|
||||
|
||||
Dense decode is the pipeline's cost driver, so (1) is not free. The cost is
|
||||
accepted: the alternative is a boundary signal that steers association (AR-007) while
|
||||
being quietly unreliable. `dense_scale` remains available as a spatial reduction,
|
||||
since downscaling is a documented, understood degradation rather than a temporal
|
||||
one the model has no defence against.
|
||||
one the model has no defence against — and TransNetV2 downsamples to 48×27
|
||||
regardless.
|
||||
|
||||
**Current:** histogram cut in the decoder; `scene_detector_node.hpp` for
|
||||
TransNetV2. **Gap:** native-rate dense decode; framerate-derived dedup;
|
||||
`--scene-detect` is default-off despite now feeding association.
|
||||
TransNetV2, fed at native rate with a framerate-derived dedup window.
|
||||
**Gap:** `scene_threshold` (0.60) is still the value picked against 12 fps input
|
||||
and is now certainly wrong — VR-006 re-fits it, and until it does, boundary
|
||||
recall at native rate is untuned rather than better. `--scene-detect` is
|
||||
default-off despite now feeding association.
|
||||
|
||||
## AR-012 … AR-017 — Track-level identity propagation — **CHANGED BEHAVIOUR**
|
||||
|
||||
|
||||
Reference in New Issue
Block a user