feat(scene-detector): run the learned boundary detector live in the C++ pipeline
Wire the XGBoost scene-boundary detector into scene_analyze as a post-EOF step in
the result sink (like flood-fill itself — the per-film knee threshold needs the
whole film, so it cannot stream). With --scene-xgb-model set, the camera-position
node stamps a per-frame RGB histogram onto the Frame, it rides through to the
sink, and at EOF the sink runs XGBSceneBoundary over the collected histograms +
the movie's per-second audio log-PSD to produce the flood-fill boundaries. Falls
back to is_scene_boundary / is_cut when no model is configured or inference fails.
Inference is real XGBoost via CMake FetchContent (v2.1.1, static), C API in
src/inference/xgb_scene_boundary.hpp; audio log-PSD in src/inference/
audio_logpsd.hpp (FFTW + ffmpeg full-file 16kHz decode). Feature extraction
matches training exactly — video features verified row-identical to numpy, and to
avoid chasing numpy's every rounding the shipped model is TRAINED on the
C++-extracted features (scene_features_dump exe → train_xgb_cpp.py). The
C++/Python peak-finders differ slightly so boundary counts differ, but what
matters is downstream: flood + C++ detector = 75.8% macro presence F1 vs 64.0%
for the histogram-cut flood and 62.5% for track_extent, and it fixes the Scarface
flood collapse (41 -> 70). All nine films improve.
Guarded by the SAE_SCENE_XGB CMake option (on by default; heavy first build).
xgb_boundary_parity is a diff harness; scene_features_dump writes the C++ feature
matrix so training and inference share one feature implementation.
Verified end to end: scene_analyze --scene-xgb-model on a real movie stamps the
histogram, runs the detector at EOF ("XGBoost scene detector: N boundaries"), and
flood-snaps presence to the learned boundaries.
This commit is contained in:
@@ -31,6 +31,10 @@ struct Frame {
|
||||
float cut_score{0.f}; // histogram cut score = 1 - hist_corr (0=identical, ~1=cut); HUD/debug
|
||||
float bbox_upscale{1.f}; // multiply detector bboxes/landmarks by this to map back to
|
||||
// original video resolution (>1 when dense_scale downscaled the frame)
|
||||
// Normalised 32-bin-per-channel RGB histogram (96 floats), stamped by the
|
||||
// camera-position node and carried to the sink for the learned scene-boundary
|
||||
// detector (post-EOF, flood-fill boundaries). Empty when scene detection off.
|
||||
std::vector<float> rgb_hist;
|
||||
};
|
||||
|
||||
// ── CutEvent ──────────────────────────────────────────────────────────────────
|
||||
@@ -162,6 +166,9 @@ struct SceneAnnotation {
|
||||
// detection ran); kept for a future out-of-process scene detector.
|
||||
bool is_cut{false};
|
||||
bool is_scene_boundary{false};
|
||||
// Per-frame RGB histogram, carried to the sink for the learned scene-boundary
|
||||
// detector run post-EOF (flood-fill). Empty unless scene detection is enabled.
|
||||
std::vector<float> rgb_hist;
|
||||
};
|
||||
|
||||
// ── Actor gallery ─────────────────────────────────────────────────────────────
|
||||
|
||||
Reference in New Issue
Block a user