feat(scene-detector): learned scene-boundary detector for flood-fill presence

A boosted-tree scene-boundary detector that replaces the grayscale
histogram-correlation cut detector as the flood-fill boundary source, and
substantially improves actor-presence accuracy.

Downstream result (per-second X-Ray presence F1, macro over 9 films):
  track_extent 62.3%  |  flood + histogram cuts 64.0%  |  flood + this 76.9%
+12.9pp, and it wins on every film — notably fixing the histogram flood's
Scarface collapse (61 -> 41 -> 71) and lifting Downton 41 -> 84.

Design (each choice measured — see the memory / report):
- XGBoost REGRESSOR on a ±3s window of DELTA features (symmetric RGB-hist
  and audio-PSD deltas at k=1,2,4,8s + ramp bank + time-since-last-peak
  debounce). Raw histograms dilute; deltas separate boundaries ~4-5x.
- SOFT Gaussian proximity target (sigma=10s) so near-misses train as
  near-correct, not hard negatives; regression -> smooth score -> NMS peaks.
- KNEE per-film threshold: self-calibrates the boundary count to ~the true
  scene count, no global rate. Evaluated at ±20s (X-Ray scenes ~170s).
- Trained on all 9 films (Cafe/Scarface low-contrast grades must be seen).
  Honest held-out ~41% boundary-F1 @±20s vs ~27% grayscale.

Scripts: train_xgb_boundary.py (shipped detector), extract_audio_features.py
(per-second log-PSD), downstream_presence.py (the A/B above), density_floor.py
(fallback for detection-starved films), plus the LSTM/DE explorations kept
for provenance. Model: models/scene_boundary_xgb.json.

Not yet wired into the live C++ pipeline — boundaries are a post-EOF step in
the sink (like flood-fill itself); libxgboost C++ integration is the next step.
This commit is contained in:
2026-08-09 19:21:04 +02:00
parent 8dd2255125
commit 0e35dac951
9 changed files with 1003 additions and 0 deletions
+65
View File
@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""
density_floor.py — synthesise scene boundaries when detection is starved.
Flood-fill presence snaps each actor claim to the shot it sits in, so a film
whose boundary detector fires almost nothing (Scarface: 1 cut in 171 min) floods
every actor across the whole film. This is a safety floor: when a film's DETECTED
boundary density is far below what a working detector should produce, fill the
long gaps between real detections with uniformly-spaced synthetic boundaries so no
flood-fill span can exceed ~1/target-density.
Design points (measured on the X-Ray corpus):
- The target density is a PRIOR from the central 60 min of films (avoids credits/
intro/outro skew): median ~0.35 scenes/min.
- The trigger is detected-vs-prior, not prior-vs-anything: only fire when detected
density < TRIGGER_FRAC × prior. Legitimately sparse films (long-scene ensembles
like Downton/Many Saints) detect fine and are left alone.
- Real detections are never moved or dropped; synthetic boundaries only subdivide
gaps that are longer than the target scene length.
"""
from __future__ import annotations
PRIOR_SCENES_PER_MIN = 0.35 # central-60min X-Ray median
TRIGGER_FRAC = 0.30 # fire only when detected < 30% of prior
def apply_density_floor(boundaries: list[float], duration_sec: float,
prior_per_min: float = PRIOR_SCENES_PER_MIN,
trigger_frac: float = TRIGGER_FRAC) -> list[float]:
"""Return boundaries augmented with synthetic ones iff detection is starved.
boundaries: detected boundary timestamps (s), any order.
duration_sec: film length.
Returns a sorted list; unchanged (just sorted) when the film is not starved.
"""
b = sorted(t for t in boundaries if 0.0 < t < duration_sec)
minutes = duration_sec / 60.0
if minutes <= 0:
return b
detected_density = len(b) / minutes
if detected_density >= trigger_frac * prior_per_min:
return b # detector produced a reasonable amount — leave it alone
target_gap = 60.0 / prior_per_min # seconds per expected scene
edges = [0.0] + b + [duration_sec]
out = list(b)
for lo, hi in zip(edges[:-1], edges[1:]):
gap = hi - lo
if gap <= target_gap:
continue
n_insert = int(gap // target_gap) # how many synthetic cuts fit
step = gap / (n_insert + 1)
for k in range(1, n_insert + 1):
out.append(lo + k * step)
return sorted(out)
if __name__ == "__main__":
# self-check on the Scarface failure and a healthy film
scar = apply_density_floor([88.0], 171*60) # 1 detected cut, 171 min
print(f"Scarface: 1 detected → {len(scar)} after floor "
f"({len(scar)/171:.2f}/min, prior {PRIOR_SCENES_PER_MIN})")
healthy = apply_density_floor([i*130.0 for i in range(1, 47)], 122*60)
print(f"healthy (46 detected/122min={46/122:.2f}/min): "
f"{len(healthy)} after floor (unchanged = not triggered)")