docs: archive the July 2026 report; new methodology for the opencv5 run

The July report (4-model ArcFace/LVFace bake-off, pre-opencv5 framework,
3-film training + held-out validation) is superseded by the opencv5 build:
single-model LVFace-B, a 6-knob DE sweep over all 9 films, flood-fill
presence, and the registry/decode fixes. Rather than overwrite it, archive
it date-suffixed and start the current report fresh.

- Rename the six July result pages to *-2026-07.md, rewrite their
  intra-archive cross-links, and add an "Archived (July 2026)" banner to each.
- mkdocs nav: current report at top, the July set under an Archive section.
- New docs/methodology.md for the opencv5 run: corrects the withdrawn
  anneal_sec/extinction_sec presence bridging (windows are now
  [first_seen, last_seen], AR-012/013), documents the two presence modes
  (track_extent / flood), and records that every eval scores all 9 films.

The current experiment log (model-bakeoff.md) and Home rewrite land once
the DE sweep converges and the final optimum is known.
This commit is contained in:
2026-08-09 10:48:45 +02:00
parent e1423062e2
commit ea922356f1
8 changed files with 243 additions and 103 deletions
+71
View File
@@ -0,0 +1,71 @@
> **Archived (July 2026).** This report covers the pre-opencv5 framework and the 4-model ArcFace/LVFace bake-off. It is superseded by the current [experiment log](model-bakeoff.md) for the opencv5 build. Kept for provenance; the numbers here are historical.
# Whole gallery vs. cast-restricted gallery
Two ways to run the matcher. Full mode scores every detected face against
the entire 2418-actor gallery. Restricted mode pre-filters each film's
gallery down to just its Jellyfin-credited cast (typically around 15
top-billed actors) before the matcher runs.
## Result
Averaged across the 3 compared models (r50 excluded, see
[the full experiment log](model-bakeoff-2026-07.md)) and both expansion settings, on
the 4 training films:
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once, not a precision/recall trade:
+4.8pp F1, +6.0pp recall, roughly a quarter the total misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections.
Every model's best-scoring combo in the training matrix uses the
restricted gallery:
![All combos ranked by training-set F1, filled dots are restricted](assets/images/rep4_matrix_f1.png)
See [the full experiment log](model-bakeoff-2026-07.md) for the complete table. One
combo reaches zero true out-of-cast misidentifications,
`arcface_w600k_mbf_restricted_exp` (F1 76.2%), and it is a restricted one,
consistent with restriction, not expansion, being what suppresses cross-film
confusions.
The restriction effect (+4.8pp averaged across models) is larger than the
model-choice effect: LVFace beats r18 by 6.2pp in full mode but beats mbf by
3.3pp. Restriction is the single strongest lever in the matrix.
## Why this is not the shipped default
Cast restriction is implemented today only as an offline optimizer
technique
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
it pre-builds a filtered gallery file per film using Jellyfin's cast list
before the benchmark calls the matcher. There is no runtime "restrict to
this title's credited cast" switch in the shipped application;
`scene_analyze` always matches against whatever single gallery file it is
given.
Building this as a real feature requires:
- A live Jellyfin cast lookup at analysis time. The title is already known,
and [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
already performs this lookup for its own `filter_gallery`-based
restriction path; it is not wired into `scene_analyze` as a first-class
option.
- A decision on the fallback case: what happens to a real, uncredited
cameo (see the Germar Terrell Gardner and Talia Balsam cases in the
[LVFace deep dive](lvface-deep-dive-2026-07.md#where-lvface-beat-x-ray)) if the
restricted gallery never includes them at all.
- Regenerating the restricted-gallery cache whenever a title's Jellyfin
cast list changes.
The shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults use
the full-mode winner (`LVFace-B_Glint360K_full_exp`, F1 75.3% training,
67.4% held-out macro) rather than the higher-scoring `restricted_exp`
(78.3%), because 78.3% describes a capability the application does not
have yet.