dtourolle
718dad688d
Status for the two changes just landed, plus the consequence AR-004's fix has for the scene join. The annotator's old comment said blocking there was safe because the branches are independent. That stopped being true when the fanout became lossless: it now stops popping once one branch stops taking, so a starved detector and a waiting annotator would wedge. What actually makes it safe is join depth -- the fanout can run the dense branch ahead by the whole of the sampled branch's buffering, which at kSceneJoinDepth 256 and sample_fps 5 against a 25 fps source is ~1200 dense frames against TransNetV2's 100-frame window. Cutting kSceneJoinDepth below the window would reintroduce the wedge, so it is now a correctness precondition rather than a tuning knob. TRACES: AR-004, AR-010, AR-028, AR-029 | VR-015 | SR-002
76 lines
3.5 KiB
C++
76 lines
3.5 KiB
C++
#pragma once
|
|
/// TRACES: AR-010 | SR-002
|
|
///
|
|
/// SceneBoundaryAnnotatorFunc — the join of the decode butterfly.
|
|
///
|
|
/// `source` fans out to two branches: dense frames to TransNetV2, sampled frames
|
|
/// to face detection. A boundary found on the first has to reach the second, and
|
|
/// cannot ride along in the frame because the branches run in parallel.
|
|
///
|
|
/// This node sits on the sampled branch and stamps `Frame::is_scene_boundary`
|
|
/// from the detector's published verdict.
|
|
///
|
|
/// **It only works because the sampled branch lags.** TransNetV2 buffers
|
|
/// `kWindow` frames before it can score any of them, so this node must not reach
|
|
/// a frame before the detector has an opinion about it. Channel depth creates
|
|
/// that lag: with backpressure (AR-004) the fanout blocks on the slower branch,
|
|
/// so a deep channel here lets the detector run ahead by its window instead of
|
|
/// anything being dropped.
|
|
///
|
|
/// When the lag is insufficient the node **counts it** rather than guessing.
|
|
/// Annotating an unscored frame as boundary-free is indistinguishable from a
|
|
/// genuine "no boundary here", and that is the failure that makes a downstream
|
|
/// test pass while verifying nothing.
|
|
|
|
#include "scene_boundaries.hpp"
|
|
#include "types.hpp"
|
|
|
|
#include <memory>
|
|
#include <string_view>
|
|
#include <utility>
|
|
|
|
struct SceneBoundaryAnnotatorFunc {
|
|
static constexpr std::string_view label() { return "scene_annotate"; }
|
|
|
|
/// `tol` is half a sample interval. The branches sample at different rates,
|
|
/// so a boundary found on a dense frame rarely lands exactly on a sampled
|
|
/// one; half an interval attributes it to the nearest sampled frame and no
|
|
/// further.
|
|
SceneBoundaryAnnotatorFunc(std::shared_ptr<SceneBoundaries> b, double tol)
|
|
: bounds_(std::move(b)), tol_(tol) {}
|
|
|
|
Frame operator()(Frame f) {
|
|
if (f.eof || !bounds_) return f;
|
|
|
|
// Wait for the detector's verdict to cover this frame. Channel depth
|
|
// alone cannot provide the lag: it holds frames back only when the
|
|
// consumer is slower, and this branch is orders of magnitude faster per
|
|
// frame than TransNetV2. Blocking here is what makes the join real.
|
|
//
|
|
// What makes that safe is **join depth**, not branch independence. Now
|
|
// that the fanout is lossless (AR-004) it stops popping once this branch
|
|
// stops taking, so stalling here does eventually starve the detector —
|
|
// the two would wedge if this node could ask about a frame the detector
|
|
// has not been given the frames to score. It cannot, by a wide margin:
|
|
// the fanout can run the dense branch ahead by the whole of this
|
|
// branch's buffering, which is kSceneJoinDepth (256) *sampled* frames,
|
|
// and at sample_fps 5 against a ~25 fps source that is on the order of
|
|
// 1200 dense frames against TransNetV2's 100-frame window.
|
|
//
|
|
// Cutting kSceneJoinDepth below the window would reintroduce the wedge.
|
|
if (!bounds_->wait_until_scored(f.timestamp_sec)) {
|
|
// The detector finished without covering this frame — the tail after
|
|
// its last full window. Unknown, not negative; counted so it cannot
|
|
// pass for "no boundary here".
|
|
bounds_->note_outran();
|
|
return f;
|
|
}
|
|
f.is_scene_boundary = bounds_->is_boundary(f.timestamp_sec, tol_);
|
|
return f;
|
|
}
|
|
|
|
private:
|
|
std::shared_ptr<SceneBoundaries> bounds_;
|
|
double tol_{0.0};
|
|
};
|