docs: correct scene-boundary detector F1 numbers to measured values

The boundary-detection paragraph understated the detector. Replace the stale
"~34% F1 vs ~27%" (a pre-C++-retrain figure with no backing artifact) with the
measured numbers at the shipped ±20s tolerance:
  - leave-one-out macro boundary F1 = 44.1% (honest generalisation)
  - grayscale baseline               = 29.8%
  - train-all (shipped model)        = 72.9% (per-film 51-86%)
computed from experiments/results/scene_boundary/xgb_report.json and per-film
leave-one-out runs of train_xgb_cpp.py. Also correct the false claim that the
low-contrast grades "cannot generalise held out" — Scarface held out scores 32%,
Café Society 51%, both above grayscale (0% and 31%).

Split the evolution figure into two panels so the strict-±2s feature-development
curve is no longer mistaken for the shipped result: left = feature progress at
±2s, right = shipped detector at the ±20s tolerance the pipeline uses.
This commit is contained in:
2026-08-11 20:54:15 +02:00
parent b26c66dcce
commit 13437e0d8b
3 changed files with 47 additions and 14 deletions
+21 -7
View File
@@ -62,7 +62,13 @@ Everything is per second, aligned to the 1-fps presence grid.
the films where video is weak (Downton, Sound of Metal), so it is included and
the model uses it where it helps.
![Detector development: features and model](assets/images/scene_detector_evolution.png)
![Detector development at strict ±2 s tolerance, and where the shipped detector landed at the ±20 s tolerance the pipeline uses](assets/images/scene_detector_evolution.png)
The left panel is the *feature* development, scored at a strict ±2 s tolerance so
each change is visible — this is where "delta beats raw histogram" was measured, not
the shipped tolerance. The right panel is the shipped detector at the ±20 s
tolerance the pipeline actually uses (see below). The two panels are on different
tolerances by design and must not be read as one curve.
Dead ends, all measured and discarded: audio-only detection; raw
histograms/PSDs as input; a two-tower BiLSTM (no better than the tree, far slower);
@@ -81,14 +87,22 @@ and TransNetV2 (a Conv3D net that will not co-reside with the ROCm/VAAPI stack).
real boundaries give way to noise. Selecting at the knee **self-calibrates the
boundary count** to roughly the true scene count, per film, with no global
threshold that would be wrong for every grade.
- **Trained on all nine films** for the shipped model. Café Society and Scarface
(the low-contrast grades) *must* be in training — held out, the model cannot
generalise to them; in training they reach 7086% boundary-F1.
- **Trained on all nine films** for the shipped model. Keeping the low-contrast
grades (Café Society, Scarface) in training matters most: on its own training
films the shipped model reaches **72.9% macro boundary-F1** (per-film 5186%),
versus **29.8%** for the grayscale baseline on the same films.
Boundary detection, held out (leave-one-out, ±20 s tolerance — appropriate given
~170 s scenes): **~34% F1, versus ~27% for the grayscale baseline.** The absolute
number is capped by the narrative-vs-audiovisual mismatch above; the point is the
downstream effect.
~170 s scenes): **44.1% macro F1, versus 29.8% for the grayscale baseline** — the
honest generalisation number, each film scored by a detector trained on the other
eight. Even the low-contrast grades generalise (Scarface held out 32%, Café Society
51%), where the grayscale detector scores 0% and 31%. The absolute number is capped
by the narrative-vs-audiovisual mismatch above — many boundaries have no
audio-visual signature at all — so the point is the downstream effect, below.
| boundary-F1 @±20 s | grayscale | learned (LOO) | learned (train-all) |
| ------------------ | --------: | ------------: | ------------------: |
| macro over 9 films | 29.8% | **44.1%** | 72.9% |
## The result that matters: actor presence