docs: richer report — data figures, success/failure frames, commit-pinned repo links
- experiment_charts.py generates 4 figures from experiments/ artifacts: held-out per-film F1, 16-combo ranking, DE search landscape, and the Downton detector-vs-tracker ghost timeline (replaces the blank title-card screenshot) - new frames: 19-correct wedding shot (success case), Many Saints ghost-vs-unknown frame (three error classes in one image) - rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only as the on-disk artifact prefix, explained once - repo file references are now links via https://REPOLINK/<path> placeholders; build_site.sh pins them to the HEAD commit's raw URLs and fails the build if a linked path doesn't exist at HEAD - drop references to removed scripts (scene_score.py, score_config.py) and to session-memory names; mark artifact-registry paths with their pull commands - commit readme_example.jpg + pipeline_topology.svg so README renders on the plain Gitea repo view - deploy_pages.sh: push built site/ to the gitea-pages branch
This commit is contained in:
+14
-5
@@ -7,7 +7,7 @@ pre-filters each film's gallery down to just its Jellyfin-credited cast (typical
|
||||
|
||||
## The result
|
||||
|
||||
Averaged across all 4 models and both expansion settings, on the 4 rep4 training
|
||||
Averaged across all 4 models and both expansion settings, on the 4 bake-off training
|
||||
films:
|
||||
|
||||
| scope | F1 | P | R | total misID (8 evals) |
|
||||
@@ -23,7 +23,13 @@ with someone in the film, but isn't actually in it), and the recall gain shows
|
||||
it isn't costing real detections to get there.
|
||||
|
||||
Per-model, every single model's best-scoring combo in the full 16-way matrix is
|
||||
a `restricted` variant — see the full table in `rep4-optimizer-results.md`. Two
|
||||
a `restricted` variant — visible directly in the ranking below (filled dots =
|
||||
restricted, open = full; the filled dots cluster at the top for every color):
|
||||
|
||||

|
||||
|
||||
See the full table in the
|
||||
[bake-off experiment log](model-bakeoff.md). Two
|
||||
combos hit **zero** true out-of-cast misidentifications:
|
||||
`arcface_w600k_mbf_restricted_exp` (F1 76.5%) and, in full mode,
|
||||
`LVFace-B_Glint360K_full_noexp` (F1 72.4%) — restriction isn't the only way to
|
||||
@@ -32,7 +38,8 @@ reach misid=0, but it's the more reliable one.
|
||||
## Why this isn't the shipped default
|
||||
|
||||
Cast-restriction is implemented today only as an **offline optimizer technique**
|
||||
(`scripts/optimizer/cast_restrict.py`): it pre-builds a filtered gallery file
|
||||
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
|
||||
it pre-builds a filtered gallery file
|
||||
per film, using Jellyfin's own cast list, before the benchmark ever calls the
|
||||
matcher. There's no runtime "restrict matching to this title's credited cast"
|
||||
switch in the shipped application — `scene_analyze` always matches against
|
||||
@@ -41,7 +48,8 @@ whatever single gallery file it's given.
|
||||
Building that as a real feature would need, at minimum:
|
||||
|
||||
- A live Jellyfin cast lookup at analysis time (the title is already known —
|
||||
`run_from_jellyfin.py` already does this same lookup for its own
|
||||
[`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
|
||||
already does this same lookup for its own
|
||||
`filter_gallery`-based restriction path, just not wired into `scene_analyze`
|
||||
itself as a first-class option).
|
||||
- A decision on the *fallback*: what happens to a real, uncredited cameo
|
||||
@@ -50,7 +58,8 @@ Building that as a real feature would need, at minimum:
|
||||
- Regenerating the restricted-gallery cache whenever the title's Jellyfin cast
|
||||
list changes.
|
||||
|
||||
This is why the shipped `src/config.hpp` defaults use the `full`-mode winner
|
||||
This is why the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
|
||||
defaults use the `full`-mode winner
|
||||
(`LVFace-B_Glint360K_full_exp`, F1 75.3% training / 67.4% held-out macro) rather
|
||||
than the higher-scoring `restricted_exp` (78.3%) — the 78.3% number describes a
|
||||
capability the app doesn't have yet, not what actually ships.
|
||||
|
||||
Reference in New Issue
Block a user