feat: join the decode butterfly so scene boundaries reach the face branch

AR-010 — is_scene_boundary had no producer: SceneDetectorFunc was a terminal
sink writing scenes.json and never annotating the frames flowing to face
detection. The flag was permanently false, so the boundary half of AR-007's
frame-dependent association was dead code that a test could still exercise
synthetically and appear to verify.

The topology already forks after decode — dense frames to TransNetV2, sampled
frames to face detection — so this is a fork-join. SceneBoundaries is the join:
the detector publishes each window's verdict with a watermark, and an annotator
on the sampled branch stamps the flag.

The watermark is the part that matters. TransNetV2 buffers 100 frames before it
can score any of them, so at any instant it has an opinion up to some time T and
none after. Without recording T a consumer cannot tell "no boundary" from "not
scored yet", and those demand opposite behaviour — treating unscored frames as
boundary-free is exactly what makes a downstream check pass while verifying
nothing.

Buffering alone does not work, which was my first attempt. Channel depth creates
lag only when the consumer is slower, and the face branch runs four orders of
magnitude faster per frame than TransNetV2 (0.01ms vs 400ms), so its channels
drain instantly and no lag accumulates. Measured: 106 of 364 frames outran the
detector. The annotator therefore waits on the watermark explicitly. The
detector signals completion so the tail cannot deadlock, and publishes from
flush_remaining too — without that the final frames arrive with no verdict.

Boundaries are deduped on publish, matching what scenes.json does at write time.
A run of adjacent high-scoring frames is one boundary, not several; leaving them
raw made this view report 357 where the file said 13. Now the two agree exactly.

Frames past the detector's last scored window remain unverified and are counted
as such rather than silently marked boundary-free.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-010 | SR-002
This commit is contained in:
2026-07-31 14:56:38 +02:00
co-authored by Claude Opus 5
parent a2c699a844
commit bbab5aed23
7 changed files with 393 additions and 15 deletions
+67 -2
View File
@@ -60,6 +60,8 @@
#include "nodes/identity_matcher_node.hpp"
#include "nodes/scene_tracker_node.hpp"
#include "nodes/scene_detector_node.hpp"
#include "scene_boundaries.hpp"
#include "nodes/scene_boundary_annotator_node.hpp"
#include "nodes/result_sink_node.hpp"
#include "nodes/embedding_dump_node.hpp"
#ifdef SAE_DEBUG
@@ -81,6 +83,18 @@
// ── CLI parsing ───────────────────────────────────────────────────────────────
/// TRACES: AR-010, AR-004 | SR-002
/// How deeply the sampled branch is buffered behind the dense one. TransNetV2
/// needs kWindow (100) dense frames before it can score any of them, so the face
/// branch must lag by at least that much or it asks about frames nobody has
/// looked at yet. Backpressure turns depth into lag: the fanout blocks on the
/// slower branch rather than dropping, so the detector simply runs ahead.
static constexpr std::size_t kSceneJoinDepth = 256;
/// Set when the scene branch is built, so shutdown can report whether the join
/// actually worked.
static std::shared_ptr<SceneBoundaries> scene_stats;
static Config parse_args(int argc, char** argv) {
Config cfg;
cfg.detector_model = kDefaultDetectorModel;
@@ -292,6 +306,21 @@ int main(int argc, char** argv) {
//
// Reporting it in a footer and exiting 0 made both invisible: the run
// "succeeded" and the truth file looked complete. Fail instead.
/// TRACES: AR-010 | SR-002
if (scene_stats) {
std::cerr << "[scene_annotate] boundaries=" << scene_stats->count()
<< " scored_through=" << scene_stats->scored_through() << "s";
// The tail is expected: frames after the detector's last full
// window are never covered, and no amount of buffering changes
// that. They are counted rather than silently treated as
// boundary-free, which is the distinction that matters.
if (scene_stats->outran() > 0)
std::cerr << " unscored=" << scene_stats->outran()
<< " frame(s) past the detector's last window — treated as"
" boundary-free, which is unverified rather than known";
std::cerr << "\n";
}
bool dropped = false;
{
std::lock_guard<std::mutex> lk(event_mtx);
@@ -346,6 +375,21 @@ int main(int argc, char** argv) {
if (cfg.scene_detect) {
scene_done.store(false, std::memory_order_release); // now a real terminal branch
SceneDetectorFunc scene_fn{cfg, scene_done};
/// TRACES: AR-010 | SR-002
// The join of the decode butterfly. source fans out to the dense
// TransNetV2 branch and the sampled face branch; boundaries found on the
// first have to reach the second, and cannot ride the frames because the
// branches run in parallel.
//
// TransNetV2 buffers kWindow frames before it can score any of them, so
// the face branch must lag by at least that much or it will ask about
// frames nobody has looked at yet. Channel depth is what creates the lag:
// with backpressure (AR-004) the fanout blocks on the slower branch, so
// a deep face-branch channel lets the detector run ahead by its window
// rather than dropping anything.
auto boundaries = std::make_shared<SceneBoundaries>();
scene_fn.set_boundaries(boundaries);
kpn::ObjectNode<SceneDetectorFunc, kpn::in<"dense">, kpn::out<>, "scene_detector", 0>
scene_node(scene_fn, 128);
@@ -362,13 +406,34 @@ int main(int argc, char** argv) {
return true;
}
return false;
}, 32);
}, kSceneJoinDepth);
/// TRACES: AR-010 | SR-002
// Stamp is_scene_boundary from the detector's published verdict. tol is
// half a sample interval: the two branches sample at different rates, so
// a boundary found on a dense frame rarely lands exactly on a sampled
// one, and half an interval attributes it to the nearest sampled frame
// and no further.
//
// outran() counts frames that arrived before the detector had scored
// them. Nonzero means the join depth is too shallow for the window, and
// those frames were annotated from an incomplete verdict — which would
// otherwise look exactly like "no boundary here".
SceneBoundaryAnnotatorFunc annotate_fn{boundaries, 0.5 / cfg.sample_fps};
kpn::ObjectNode<SceneBoundaryAnnotatorFunc, kpn::in<"frame">, kpn::out<"frame">,
"scene_annotate", 0> annotate(annotate_fn, kSceneJoinDepth);
// Reported at shutdown: without this the join is unverifiable, and an
// annotator that never fired looks identical to footage with no
// boundaries.
scene_stats = boundaries;
auto net = kpn::make_network(
kpn::edge(source.output<"raw">(), campos.input<"raw">()),
kpn::edge(source.output<"raw">(), scene_node.input<"dense">()),
kpn::edge(campos.output<"frame">(), decimate.input<0>()),
kpn::edge(decimate.output<0>(), detector.input<"frame">()),
kpn::edge(decimate.output<0>(), annotate.input<"frame">()),
kpn::edge(annotate.output<"frame">(), detector.input<"frame">()),
kpn::edge(detector.output<"scene">(), aligner.input<"scene">()),
kpn::edge(aligner.output<"aligned">(), embedder.input<"aligned">()),
kpn::edge(embedder.output<"embedded">(), ftracker.input<"embedded">()),