docs: richer report — data figures, success/failure frames, commit-pinned repo links

- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
This commit is contained in:
2026-07-19 22:06:56 +02:00
parent 4925443e56
commit b1efefac6f
17 changed files with 702 additions and 146 deletions
+16 -7
View File
@@ -9,7 +9,7 @@ open question: is LVFace (455MB) actually better, or just the biggest?
Each gallery carries a fitted Platt sigmoid `P(match | cosine similarity) =
σ(a·sim + b)`, embedded directly in the gallery's HDF5 file
(`src/gallery/gallery_calibration.hpp`). This is a property of the embedding
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)). This is a property of the embedding
space alone — computed from intra/inter-actor reference-image pairs, no
tracking or scene logic involved — so it's a clean first read on discriminative
power before running a single benchmark.
@@ -31,7 +31,7 @@ head start before the tracking/scoring pipeline is even involved.
## Second signal: F1 on the actual benchmark
Best full-gallery (no cast-restriction) result per model, from the 16-combo
rep4 matrix (`rep4-optimizer-results.md`):
bake-off matrix ([full experiment log](model-bakeoff.md)):
| model | F1 | P | R | misID |
|---|---|---|---|---|
@@ -40,16 +40,24 @@ rep4 matrix (`rep4-optimizer-results.md`):
| ArcFace R18 | 69.1% | 87.6% | 57.7% | 242 |
| ArcFace w600k-R50 | 68.5% | 94.0% | 54.1% | 150 |
The full 16-combo picture makes the model ordering visible at a glance — LVFace
(yellow) tops both the restricted and full columns, and R18 (green) props up
the bottom of the full-gallery ranking:
![All 16 bake-off combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
LVFace wins outright, with the highest recall of any full-mode combo. This
reverses an earlier conclusion from a prior (superseded) benchmarking pass
using a scene-union metric, which found the three models statistically
indistinguishable (~85% each) and concluded LVFace wasn't worth its size — that
metric hid out-of-cast false positives behind a gallery∩cast recall mask (see
`optimizer-experiments.md`); the per-second metric used here does not.
[the prior optimizer round](optimizer-experiments.md)); the per-second metric
used here does not.
Held-out validation (5 films never seen by the optimizer) confirms LVFace's
lead holds up out of sample — see the deep-dive page for the full breakdown,
including where it fails.
lead holds up out of sample — see the
[LVFace deep dive](lvface-deep-dive.md) for the full breakdown, including
where it fails.
## Caveat: model choice is an operational change
@@ -57,5 +65,6 @@ Switching the default embedder isn't just flipping a config value — the
gallery itself is model-specific (embeddings from different models aren't
comparable), so any existing gallery built against ArcFace w600k-R50 needs to
be rebuilt from source images against LVFace before the new default takes
effect. `scripts/optimizer/reembed_gallery.py` does this from a reference
gallery's cached source images without re-downloading anything.
effect. [`scripts/optimizer/reembed_gallery.py`](https://REPOLINK/scripts/optimizer/reembed_gallery.py)
does this from a reference gallery's cached source images without
re-downloading anything.