docs: correct scene-boundary detector F1 numbers to measured values
The boundary-detection paragraph understated the detector. Replace the stale "~34% F1 vs ~27%" (a pre-C++-retrain figure with no backing artifact) with the measured numbers at the shipped ±20s tolerance: - leave-one-out macro boundary F1 = 44.1% (honest generalisation) - grayscale baseline = 29.8% - train-all (shipped model) = 72.9% (per-film 51-86%) computed from experiments/results/scene_boundary/xgb_report.json and per-film leave-one-out runs of train_xgb_cpp.py. Also correct the false claim that the low-contrast grades "cannot generalise held out" — Scarface held out scores 32%, Café Society 51%, both above grayscale (0% and 31%). Split the evolution figure into two panels so the strict-±2s feature-development curve is no longer mistaken for the shipped result: left = feature progress at ±2s, right = shipped detector at the ±20s tolerance the pipeline uses.
This commit is contained in:
@@ -57,15 +57,34 @@ def fig_macro():
|
||||
fig.tight_layout(); fig.savefig(OUT/"scene_presence_macro.png"); plt.close(fig)
|
||||
|
||||
# ── Figure 3: feature/model evolution (boundary-F1 development) ──────────────
|
||||
# Two panels, because the development curve and the shipped result are measured
|
||||
# at DIFFERENT tolerances and must not be plotted on one axis:
|
||||
# left — relative feature progress at the strict ±2 s tolerance (how the LSTM
|
||||
# experiments were scored; establishes which features helped)
|
||||
# right — the shipped XGBoost detector at the ±20 s tolerance the pipeline
|
||||
# actually uses and scores at (grayscale vs learned-LOO vs train-all)
|
||||
def fig_evolution():
|
||||
fig,(axl,axr)=plt.subplots(1,2,figsize=(11,4.5),gridspec_kw={"width_ratios":[1.15,1]})
|
||||
|
||||
steps=["grayscale\nbaseline","raw-hist\nLSTM","delta\nLSTM","XGBoost\n(delta+debounce)"]
|
||||
f1=[7.2,7.5,10.8,15.2] # boundary-F1 @±2s during development
|
||||
fig,ax=plt.subplots(figsize=(6.5,4.5))
|
||||
ax.plot(steps,f1,marker="o",color="#3d7ea6",lw=2,ms=8)
|
||||
for i,v in enumerate(f1): ax.text(i,v+0.4,f"{v:.1f}%",ha="center",fontsize=10)
|
||||
ax.set_ylabel("held-out boundary F1 @±2s (%)")
|
||||
ax.set_title("Detector development: features + model")
|
||||
ax.set_ylim(0,18)
|
||||
dev=[7.2,7.5,10.8,15.2] # boundary-F1 @±2s during LSTM-era development
|
||||
axl.plot(steps,dev,marker="o",color="#9aa7b4",lw=2,ms=8)
|
||||
for i,v in enumerate(dev): axl.text(i,v+0.4,f"{v:.1f}%",ha="center",fontsize=9)
|
||||
axl.set_ylabel("boundary F1 @±2 s (%)")
|
||||
axl.set_title("Feature progress (strict ±2 s)")
|
||||
axl.set_ylim(0,18)
|
||||
|
||||
# shipped detector at the ±20s tolerance the pipeline uses — real measured
|
||||
# macro numbers: grayscale (xgb_report gray_F1), learned LOO, learned train-all
|
||||
names=["grayscale","learned\n(LOO)","learned\n(train-all)"]
|
||||
f20=[29.8,44.1,72.9]; cols=["#e07a5f","#3d7ea6","#8fb8cf"]
|
||||
bars=axr.bar(names,f20,color=cols)
|
||||
for b,v in zip(bars,f20): axr.text(b.get_x()+b.get_width()/2,v+1.2,f"{v:.1f}%",
|
||||
ha="center",fontsize=10,fontweight="bold")
|
||||
axr.set_ylabel("boundary F1 @±20 s (%)")
|
||||
axr.set_title("Shipped detector (±20 s, macro/9 films)")
|
||||
axr.set_ylim(0,80)
|
||||
fig.suptitle("Detector development, and where it landed",fontsize=13)
|
||||
fig.tight_layout(); fig.savefig(OUT/"scene_detector_evolution.png"); plt.close(fig)
|
||||
|
||||
import csv as _csv
|
||||
|
||||
Reference in New Issue
Block a user