dtourolle
0e35dac951
A boosted-tree scene-boundary detector that replaces the grayscale histogram-correlation cut detector as the flood-fill boundary source, and substantially improves actor-presence accuracy. Downstream result (per-second X-Ray presence F1, macro over 9 films): track_extent 62.3% | flood + histogram cuts 64.0% | flood + this 76.9% +12.9pp, and it wins on every film — notably fixing the histogram flood's Scarface collapse (61 -> 41 -> 71) and lifting Downton 41 -> 84. Design (each choice measured — see the memory / report): - XGBoost REGRESSOR on a ±3s window of DELTA features (symmetric RGB-hist and audio-PSD deltas at k=1,2,4,8s + ramp bank + time-since-last-peak debounce). Raw histograms dilute; deltas separate boundaries ~4-5x. - SOFT Gaussian proximity target (sigma=10s) so near-misses train as near-correct, not hard negatives; regression -> smooth score -> NMS peaks. - KNEE per-film threshold: self-calibrates the boundary count to ~the true scene count, no global rate. Evaluated at ±20s (X-Ray scenes ~170s). - Trained on all 9 films (Cafe/Scarface low-contrast grades must be seen). Honest held-out ~41% boundary-F1 @±20s vs ~27% grayscale. Scripts: train_xgb_boundary.py (shipped detector), extract_audio_features.py (per-second log-PSD), downstream_presence.py (the A/B above), density_floor.py (fallback for detection-starved films), plus the LSTM/DE explorations kept for provenance. Model: models/scene_boundary_xgb.json. Not yet wired into the live C++ pipeline — boundaries are a post-EOF step in the sink (like flood-fill itself); libxgboost C++ integration is the next step.
66 lines
3.0 KiB
Python
66 lines
3.0 KiB
Python
#!/usr/bin/env python3
|
||
"""
|
||
density_floor.py — synthesise scene boundaries when detection is starved.
|
||
|
||
Flood-fill presence snaps each actor claim to the shot it sits in, so a film
|
||
whose boundary detector fires almost nothing (Scarface: 1 cut in 171 min) floods
|
||
every actor across the whole film. This is a safety floor: when a film's DETECTED
|
||
boundary density is far below what a working detector should produce, fill the
|
||
long gaps between real detections with uniformly-spaced synthetic boundaries so no
|
||
flood-fill span can exceed ~1/target-density.
|
||
|
||
Design points (measured on the X-Ray corpus):
|
||
- The target density is a PRIOR from the central 60 min of films (avoids credits/
|
||
intro/outro skew): median ~0.35 scenes/min.
|
||
- The trigger is detected-vs-prior, not prior-vs-anything: only fire when detected
|
||
density < TRIGGER_FRAC × prior. Legitimately sparse films (long-scene ensembles
|
||
like Downton/Many Saints) detect fine and are left alone.
|
||
- Real detections are never moved or dropped; synthetic boundaries only subdivide
|
||
gaps that are longer than the target scene length.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
PRIOR_SCENES_PER_MIN = 0.35 # central-60min X-Ray median
|
||
TRIGGER_FRAC = 0.30 # fire only when detected < 30% of prior
|
||
|
||
|
||
def apply_density_floor(boundaries: list[float], duration_sec: float,
|
||
prior_per_min: float = PRIOR_SCENES_PER_MIN,
|
||
trigger_frac: float = TRIGGER_FRAC) -> list[float]:
|
||
"""Return boundaries augmented with synthetic ones iff detection is starved.
|
||
|
||
boundaries: detected boundary timestamps (s), any order.
|
||
duration_sec: film length.
|
||
Returns a sorted list; unchanged (just sorted) when the film is not starved.
|
||
"""
|
||
b = sorted(t for t in boundaries if 0.0 < t < duration_sec)
|
||
minutes = duration_sec / 60.0
|
||
if minutes <= 0:
|
||
return b
|
||
detected_density = len(b) / minutes
|
||
if detected_density >= trigger_frac * prior_per_min:
|
||
return b # detector produced a reasonable amount — leave it alone
|
||
|
||
target_gap = 60.0 / prior_per_min # seconds per expected scene
|
||
edges = [0.0] + b + [duration_sec]
|
||
out = list(b)
|
||
for lo, hi in zip(edges[:-1], edges[1:]):
|
||
gap = hi - lo
|
||
if gap <= target_gap:
|
||
continue
|
||
n_insert = int(gap // target_gap) # how many synthetic cuts fit
|
||
step = gap / (n_insert + 1)
|
||
for k in range(1, n_insert + 1):
|
||
out.append(lo + k * step)
|
||
return sorted(out)
|
||
|
||
|
||
if __name__ == "__main__":
|
||
# self-check on the Scarface failure and a healthy film
|
||
scar = apply_density_floor([88.0], 171*60) # 1 detected cut, 171 min
|
||
print(f"Scarface: 1 detected → {len(scar)} after floor "
|
||
f"({len(scar)/171:.2f}/min, prior {PRIOR_SCENES_PER_MIN})")
|
||
healthy = apply_density_floor([i*130.0 for i in range(1, 47)], 122*60)
|
||
print(f"healthy (46 detected/122min={46/122:.2f}/min): "
|
||
f"{len(healthy)} after floor (unchanged = not triggered)")
|