dtourolle
e5204a831a
Wire the XGBoost scene-boundary detector into scene_analyze as a post-EOF step in
the result sink (like flood-fill itself — the per-film knee threshold needs the
whole film, so it cannot stream). With --scene-xgb-model set, the camera-position
node stamps a per-frame RGB histogram onto the Frame, it rides through to the
sink, and at EOF the sink runs XGBSceneBoundary over the collected histograms +
the movie's per-second audio log-PSD to produce the flood-fill boundaries. Falls
back to is_scene_boundary / is_cut when no model is configured or inference fails.
Inference is real XGBoost via CMake FetchContent (v2.1.1, static), C API in
src/inference/xgb_scene_boundary.hpp; audio log-PSD in src/inference/
audio_logpsd.hpp (FFTW + ffmpeg full-file 16kHz decode). Feature extraction
matches training exactly — video features verified row-identical to numpy, and to
avoid chasing numpy's every rounding the shipped model is TRAINED on the
C++-extracted features (scene_features_dump exe → train_xgb_cpp.py). The
C++/Python peak-finders differ slightly so boundary counts differ, but what
matters is downstream: flood + C++ detector = 75.8% macro presence F1 vs 64.0%
for the histogram-cut flood and 62.5% for track_extent, and it fixes the Scarface
flood collapse (41 -> 70). All nine films improve.
Guarded by the SAE_SCENE_XGB CMake option (on by default; heavy first build).
xgb_boundary_parity is a diff harness; scene_features_dump writes the C++ feature
matrix so training and inference share one feature implementation.
Verified end to end: scene_analyze --scene-xgb-model on a real movie stamps the
histogram, runs the detector at EOF ("XGBoost scene detector: N boundaries"), and
flood-snaps presence to the learned boundaries.
50 lines
2.3 KiB
C++
50 lines
2.3 KiB
C++
#pragma once
|
|
/// TRACES: AR-012, AR-013 | SR-002
|
|
///
|
|
/// FrameAnnotationFunc — project a matched frame into a per-frame annotation.
|
|
///
|
|
/// Stateless, and that is the entire point of it.
|
|
///
|
|
/// It replaces `SceneTrackerFunc`, which kept an extinction timer per actor and
|
|
/// reported an actor as visible for `extinction_sec` (57.4 s) after their last
|
|
/// detection. docs/SPEC.md specified that node's deletion -- "anneal_sec and
|
|
/// extinction_sec are deleted, not re-tuned ... SceneTrackerFunc goes with
|
|
/// them", with a removal list ending "grep for both names and expect no
|
|
/// survivors" -- and docs/requirements.md recorded both constants as Withdrawn,
|
|
/// deleted "rather than retained at zero", on the grounds that a field naming a
|
|
/// mechanism the pipeline no longer has is actively misleading. None of that
|
|
/// removal had happened. The node was still wired into both shipped pipelines
|
|
/// and still printed its timeout at every startup.
|
|
///
|
|
/// **Presence is not this node's business.** AR-012 moved it to TrackRegistry,
|
|
/// where a window is `[first_seen, last_seen]` of a track an actor owns, and
|
|
/// AR-013 ends that window at the last sighting rather than after it. A
|
|
/// keep-alive here answered the same question a second time and answered it
|
|
/// worse: it re-opened the trailing cool-down the registry exists to refuse.
|
|
///
|
|
/// What a consumer sees change: `--verbosity standard`'s `frames[].identified`
|
|
/// used to list every actor still inside the keep-alive, including ones absent
|
|
/// from the frame. It now lists what was actually matched in that frame. The
|
|
/// minimal and xray outputs are unaffected -- they were already built from
|
|
/// registry claims and never consulted this node.
|
|
|
|
#include "types.hpp"
|
|
|
|
#include <string_view>
|
|
#include <utility>
|
|
|
|
struct FrameAnnotationFunc {
|
|
static constexpr std::string_view label() { return "frame_annotation"; }
|
|
|
|
SceneAnnotation operator()(MatchedSceneFrame mf) {
|
|
if (mf.source.eof) return {0.0, {}, /*eof=*/true};
|
|
SceneAnnotation sa;
|
|
sa.timestamp_sec = mf.source.timestamp_sec;
|
|
sa.visible_actors = std::move(mf.actors);
|
|
sa.is_cut = mf.source.is_cut;
|
|
sa.is_scene_boundary = mf.source.is_scene_boundary;
|
|
sa.rgb_hist = std::move(mf.source.rgb_hist);
|
|
return sa;
|
|
}
|
|
};
|