docs: opencv5 experiment log + rewritten Home
Add model-bakeoff.md for the opencv5 build: the ten-knob DE tuning and where each shipped config default comes from, the replay architecture, and the flood-fill-on-learned-boundaries step change (62.6% -> 74.9% presence F1, LOO). Rewrite index.md to lead with the learned scene-boundary result and point at the current pages, with the July four-model bake-off moved to an Archive section. Both pages build with no broken links.
This commit is contained in:
+54
-46
@@ -17,62 +17,70 @@ two credited cast members without a visible face are correctly reported
|
||||
present but not visible. This matches Amazon X-Ray's own record for this
|
||||
second exactly.
|
||||
|
||||
Results are not uniform across films. The hardest held-out film scores 46%
|
||||
F1. This report documents why: one tunable trade (extinction bridging at
|
||||
hard cuts), one structural limit (X-Ray credits people whose faces never
|
||||
appear on screen), and a small number of cases where the pipeline is
|
||||
correct and X-Ray's ground truth is not. Read
|
||||
[how we score against X-Ray](methodology.md) first. X-Ray's ground truth is
|
||||
scene-level; the pipeline's output is per-second. That difference shapes
|
||||
every finding below.
|
||||
## The headline: learned scene boundaries
|
||||
|
||||
## Findings
|
||||
The current opencv5 build's biggest gain is **flood-fill presence on a
|
||||
learned scene-boundary detector**. An actor seen once inside a shot is
|
||||
reported for the whole shot — but only if the shot boundaries are good. A
|
||||
learned XGBoost boundary detector, scored **leave-one-out** so no film is
|
||||
ever measured by a detector that trained on it, lifts per-second X-Ray
|
||||
presence F1 across nine films and improves every one of them:
|
||||
|
||||
<div class="grid cards" markdown>
|
||||
| boundary source for flood-fill | presence F1 |
|
||||
| ------------------------------ | ----------: |
|
||||
| track-extent (flood off) | 62.6% |
|
||||
| flood + grayscale cuts | 64.0% |
|
||||
| **flood + learned detector (LOO)** | **74.9%** |
|
||||
|
||||
- :material-trophy:{ .lg .middle } **[Which model is best?](best-model.md)**
|
||||

|
||||
|
||||
---
|
||||
The full story — why the old grayscale cut detector broke Scarface, what
|
||||
features work, and the per-film breakdown — is on the
|
||||
[learned scene-boundary detector](scene-boundary-detector.md) page.
|
||||
|
||||
Calibration curves first, independent of any threshold, then held-out
|
||||
F1 across three models. LVFace-B Glint360K wins both, and wins on every
|
||||
held-out film.
|
||||
## What the numbers mean, and their limits
|
||||
|
||||
- :material-filter:{ .lg .middle } **[Whole vs. cast-restricted gallery](gallery-scope.md)**
|
||||
Results are not uniform across films, and they should not be. X-Ray's ground
|
||||
truth is scene-level and credits people whose faces never appear on screen;
|
||||
the pipeline's output is per-second and can only name a face it can see.
|
||||
That difference is a structural recall ceiling, not a bug. Read
|
||||
[how we score against X-Ray](methodology.md) first — it defines F1,
|
||||
precision, recall, and misID, and explains the two limits (off-screen cast
|
||||
and gallery coverage) that shape every finding.
|
||||
|
||||
---
|
||||
|
||||
Restricting the matcher to a film's credited cast improves F1,
|
||||
recall, and misID rate at once, but is not a shipped runtime feature
|
||||
yet.
|
||||
|
||||
- :material-account-convert:{ .lg .middle } **[Does pose expansion help?](pose-expansion.md)**
|
||||
|
||||
---
|
||||
|
||||
A training-set effect that did not reproduce on 5 held-out films once
|
||||
two methodology bugs in the comparison harness were found and fixed.
|
||||
|
||||
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
|
||||
|
||||
---
|
||||
|
||||
The held-out generalization gap, the two mechanisms behind its errors,
|
||||
and every distinct case where it names someone outside the film's
|
||||
credited cast.
|
||||
|
||||
</div>
|
||||
Precision on identified faces is near-perfect: where the pipeline names a
|
||||
face, it is almost always a name X-Ray also credits to that scene. The
|
||||
frames throughout this documentation make the tension visual — **green** =
|
||||
true positive, **red** = false positive, **orange** = unknown, and a
|
||||
**blue** panel lists credited cast present with no visible face.
|
||||
|
||||
## Full experiment log
|
||||
|
||||
- **[Full experiment log](model-bakeoff.md)**: the complete log behind the
|
||||
four pages above, including how replaying against cached embeddings
|
||||
inside the same KPN network makes a full model and configuration
|
||||
comparison practical, the full results table, and every caveat. This is
|
||||
where the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
|
||||
defaults come from.
|
||||
- **[Service conversion (proposal)](service-conversion.md)**: design
|
||||
sketch for a native idle-GPU worker gated on screen lock, not yet built.
|
||||
- **[Full experiment log (opencv5)](model-bakeoff.md)**: the complete log
|
||||
behind the current build — the ten-knob differential-evolution tuning, the
|
||||
shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults and
|
||||
where each comes from, the replay architecture that makes a nine-film
|
||||
search tractable, and the flood-fill step change.
|
||||
- **[Learned scene-boundary detector](scene-boundary-detector.md)**: the
|
||||
features, the model, leave-one-out results, and the two headline films.
|
||||
- **[Benchmark — SuperHero](benchmark.md)**: the benchmark harness.
|
||||
- **[Service conversion (proposal)](service-conversion.md)**: design sketch
|
||||
for a native idle-GPU worker gated on screen lock, not yet built.
|
||||
|
||||
## Archive (July 2026)
|
||||
|
||||
The pre-opencv5 four-model ArcFace/LVFace bake-off is kept for provenance.
|
||||
Its numbers are historical; the current build supersedes them.
|
||||
|
||||
- [Best model (July)](best-model-2026-07.md) — LVFace-B Glint360K wins on
|
||||
calibration and on every held-out film.
|
||||
- [Gallery scope (July)](gallery-scope-2026-07.md) — cast-restricted
|
||||
gallery improves F1, recall, and misID at once.
|
||||
- [Pose expansion (July)](pose-expansion-2026-07.md) — a training-set
|
||||
effect that did not reproduce held-out.
|
||||
- [LVFace deep dive (July)](lvface-deep-dive-2026-07.md) — the
|
||||
generalization gap and every out-of-cast identification.
|
||||
- [Full experiment log (July)](model-bakeoff-2026-07.md).
|
||||
|
||||
## Reproducing the benchmarks
|
||||
|
||||
|
||||
Reference in New Issue
Block a user