The boundary-detection paragraph understated the detector. Replace the stale
"~34% F1 vs ~27%" (a pre-C++-retrain figure with no backing artifact) with the
measured numbers at the shipped ±20s tolerance:
- leave-one-out macro boundary F1 = 44.1% (honest generalisation)
- grayscale baseline = 29.8%
- train-all (shipped model) = 72.9% (per-film 51-86%)
computed from experiments/results/scene_boundary/xgb_report.json and per-film
leave-one-out runs of train_xgb_cpp.py. Also correct the false claim that the
low-contrast grades "cannot generalise held out" — Scarface held out scores 32%,
Café Society 51%, both above grayscale (0% and 31%).
Split the evolution figure into two panels so the strict-±2s feature-development
curve is no longer mistaken for the shipped result: left = feature progress at
±2s, right = shipped detector at the ±20s tolerance the pipeline uses.
make_figures.py gains the DE-landscape, calibration, and per-film leave-one-out
holdout figures used by the experiment log. regen_frame_examples.sh replays all
nine films with the shipped learned-boundary flood config and draws GT-aware
TP/FP/FN frames, so every annotated image in the docs is reproducible.
Auto-match each named July frame by film+actor+class and re-extract it from
the current learned-boundary flood replay, drawing GT-aware boxes: green TP,
red FP, orange unknown, plus the blue off-screen/missed (X-Ray cast, no face)
FN panel. Adds rematch_frames.py, the tool that does the matching.
Zooey Deschanel is dropped: the current pipeline no longer makes that
false identification, so the frame is removed and the July deep-dive notes
the fix rather than showing a stale error.
New docs/scene-boundary-detector.md: why the grayscale cut detector wasn't
enough (Scarface: 1 cut in 10k frames → flood-fill P=26%), what X-Ray boundaries
are and why they're hard, the feature/model design (delta histograms, multi-scale
ramp bank, scene-length debounce, soft-target XGBoost regressor, per-film knee),
and the measured dead ends (audio-only, raw features, LSTM, TransNetV2).
Headline result, honest leave-one-out (each film scored by a detector trained on
the other eight): flood + learned detector = 74.9% macro presence F1, vs 64.0%
for grayscale-cut flood and 62.6% for track-extent — +12.3pp, improving all nine
films. Fixes the Scarface flood collapse (grayscale 40.9 → learned 74.9, on a
film the detector never trained on) and swings Downton +37pp.
Figures are generated by scripts/scene_detector/make_figures.py from the saved
results (experiments/results/scene_boundary/downstream_loo.json); the PNGs
themselves follow the repo convention of not committing regenerable chart assets.
Added to the mkdocs nav.
Wire the XGBoost scene-boundary detector into scene_analyze as a post-EOF step in
the result sink (like flood-fill itself — the per-film knee threshold needs the
whole film, so it cannot stream). With --scene-xgb-model set, the camera-position
node stamps a per-frame RGB histogram onto the Frame, it rides through to the
sink, and at EOF the sink runs XGBSceneBoundary over the collected histograms +
the movie's per-second audio log-PSD to produce the flood-fill boundaries. Falls
back to is_scene_boundary / is_cut when no model is configured or inference fails.
Inference is real XGBoost via CMake FetchContent (v2.1.1, static), C API in
src/inference/xgb_scene_boundary.hpp; audio log-PSD in src/inference/
audio_logpsd.hpp (FFTW + ffmpeg full-file 16kHz decode). Feature extraction
matches training exactly — video features verified row-identical to numpy, and to
avoid chasing numpy's every rounding the shipped model is TRAINED on the
C++-extracted features (scene_features_dump exe → train_xgb_cpp.py). The
C++/Python peak-finders differ slightly so boundary counts differ, but what
matters is downstream: flood + C++ detector = 75.8% macro presence F1 vs 64.0%
for the histogram-cut flood and 62.5% for track_extent, and it fixes the Scarface
flood collapse (41 -> 70). All nine films improve.
Guarded by the SAE_SCENE_XGB CMake option (on by default; heavy first build).
xgb_boundary_parity is a diff harness; scene_features_dump writes the C++ feature
matrix so training and inference share one feature implementation.
Verified end to end: scene_analyze --scene-xgb-model on a real movie stamps the
histogram, runs the detector at EOF ("XGBoost scene detector: N boundaries"), and
flood-snaps presence to the learned boundaries.
A boosted-tree scene-boundary detector that replaces the grayscale
histogram-correlation cut detector as the flood-fill boundary source, and
substantially improves actor-presence accuracy.
Downstream result (per-second X-Ray presence F1, macro over 9 films):
track_extent 62.3% | flood + histogram cuts 64.0% | flood + this 76.9%
+12.9pp, and it wins on every film — notably fixing the histogram flood's
Scarface collapse (61 -> 41 -> 71) and lifting Downton 41 -> 84.
Design (each choice measured — see the memory / report):
- XGBoost REGRESSOR on a ±3s window of DELTA features (symmetric RGB-hist
and audio-PSD deltas at k=1,2,4,8s + ramp bank + time-since-last-peak
debounce). Raw histograms dilute; deltas separate boundaries ~4-5x.
- SOFT Gaussian proximity target (sigma=10s) so near-misses train as
near-correct, not hard negatives; regression -> smooth score -> NMS peaks.
- KNEE per-film threshold: self-calibrates the boundary count to ~the true
scene count, no global rate. Evaluated at ±20s (X-Ray scenes ~170s).
- Trained on all 9 films (Cafe/Scarface low-contrast grades must be seen).
Honest held-out ~41% boundary-F1 @±20s vs ~27% grayscale.
Scripts: train_xgb_boundary.py (shipped detector), extract_audio_features.py
(per-second log-PSD), downstream_presence.py (the A/B above), density_floor.py
(fallback for detection-starved films), plus the LSTM/DE explorations kept
for provenance. Model: models/scene_boundary_xgb.json.
Not yet wired into the live C++ pipeline — boundaries are a post-EOF step in
the sink (like flood-fill itself); libxgboost C++ integration is the next step.
dtourolle