docs: richer report — data figures, success/failure frames, commit-pinned repo links

- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
This commit is contained in:
2026-07-19 22:06:56 +02:00
parent 4925443e56
commit b1efefac6f
17 changed files with 702 additions and 146 deletions
+14 -5
View File
@@ -7,7 +7,7 @@ pre-filters each film's gallery down to just its Jellyfin-credited cast (typical
## The result
Averaged across all 4 models and both expansion settings, on the 4 rep4 training
Averaged across all 4 models and both expansion settings, on the 4 bake-off training
films:
| scope | F1 | P | R | total misID (8 evals) |
@@ -23,7 +23,13 @@ with someone in the film, but isn't actually in it), and the recall gain shows
it isn't costing real detections to get there.
Per-model, every single model's best-scoring combo in the full 16-way matrix is
a `restricted` variant — see the full table in `rep4-optimizer-results.md`. Two
a `restricted` variant — visible directly in the ranking below (filled dots =
restricted, open = full; the filled dots cluster at the top for every color):
![All 16 bake-off combos — filled dots (restricted) dominate the top](assets/images/rep4_matrix_f1.png)
See the full table in the
[bake-off experiment log](model-bakeoff.md). Two
combos hit **zero** true out-of-cast misidentifications:
`arcface_w600k_mbf_restricted_exp` (F1 76.5%) and, in full mode,
`LVFace-B_Glint360K_full_noexp` (F1 72.4%) — restriction isn't the only way to
@@ -32,7 +38,8 @@ reach misid=0, but it's the more reliable one.
## Why this isn't the shipped default
Cast-restriction is implemented today only as an **offline optimizer technique**
(`scripts/optimizer/cast_restrict.py`): it pre-builds a filtered gallery file
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
it pre-builds a filtered gallery file
per film, using Jellyfin's own cast list, before the benchmark ever calls the
matcher. There's no runtime "restrict matching to this title's credited cast"
switch in the shipped application — `scene_analyze` always matches against
@@ -41,7 +48,8 @@ whatever single gallery file it's given.
Building that as a real feature would need, at minimum:
- A live Jellyfin cast lookup at analysis time (the title is already known —
`run_from_jellyfin.py` already does this same lookup for its own
[`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
already does this same lookup for its own
`filter_gallery`-based restriction path, just not wired into `scene_analyze`
itself as a first-class option).
- A decision on the *fallback*: what happens to a real, uncredited cameo
@@ -50,7 +58,8 @@ Building that as a real feature would need, at minimum:
- Regenerating the restricted-gallery cache whenever the title's Jellyfin cast
list changes.
This is why the shipped `src/config.hpp` defaults use the `full`-mode winner
This is why the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
defaults use the `full`-mode winner
(`LVFace-B_Glint360K_full_exp`, F1 75.3% training / 67.4% held-out macro) rather
than the higher-scoring `restricted_exp` (78.3%) — the 78.3% number describes a
capability the app doesn't have yet, not what actually ships.