docs: richer report — data figures, success/failure frames, commit-pinned repo links
- experiment_charts.py generates 4 figures from experiments/ artifacts: held-out per-film F1, 16-combo ranking, DE search landscape, and the Downton detector-vs-tracker ghost timeline (replaces the blank title-card screenshot) - new frames: 19-correct wedding shot (success case), Many Saints ghost-vs-unknown frame (three error classes in one image) - rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only as the on-disk artifact prefix, explained once - repo file references are now links via https://REPOLINK/<path> placeholders; build_site.sh pins them to the HEAD commit's raw URLs and fails the build if a linked path doesn't exist at HEAD - drop references to removed scripts (scene_score.py, score_config.py) and to session-memory names; mark artifact-registry paths with their pull commands - commit readme_example.jpg + pipeline_topology.svg so README renders on the plain Gitea repo view - deploy_pages.sh: push built site/ to the gitea-pages branch
This commit is contained in:
@@ -45,13 +45,17 @@ GT set = actors X-Ray lists for that scene. Per scene TP/FP/FN, then:
|
||||
scenes count more) → **equal-weight mean across movies** (macro; each film counts
|
||||
the same regardless of length). This is the DE objective.
|
||||
|
||||
Implemented in `scripts/optimizer/scene_score.py`.
|
||||
Implemented in `scripts/optimizer/scene_score.py` — since **removed** along
|
||||
with this metric; its per-second successor is
|
||||
[`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)
|
||||
(see the [bake-off round](model-bakeoff.md)).
|
||||
|
||||
## The gallery coverage gap
|
||||
|
||||
Diagnosing low recall: only **131 of 392** X-Ray cast were in the gallery (33%). Every
|
||||
in-gallery actor HAD embeddings (gallery well-formed) — the gap was pure coverage.
|
||||
`scripts/optimizer/fetch_missing_actors.py` recovers missing actors:
|
||||
[`scripts/optimizer/fetch_missing_actors.py`](https://REPOLINK/scripts/optimizer/fetch_missing_actors.py)
|
||||
recovers missing actors:
|
||||
`nm-id → TMDB /find external_ids → /person/{id}/images → download → embed (sae_embed)`,
|
||||
with a `--wikidata` fallback (P345→P18 Commons photo).
|
||||
|
||||
@@ -71,7 +75,8 @@ face-recognition pipeline vs X-Ray's presence semantics, not a fixable gap.
|
||||
|
||||
## Optimizer
|
||||
|
||||
`scripts/optimizer/optimize.py` — scipy `differential_evolution` over the knob space,
|
||||
[`scripts/optimizer/optimize.py`](https://REPOLINK/scripts/optimizer/optimize.py)
|
||||
— scipy `differential_evolution` over the knob space,
|
||||
each candidate = full replay of all films through the **real** C++ nodes (see the
|
||||
KPN replay architecture below) scored by the metric above. Global objective (one
|
||||
config for all films, not per-film).
|
||||
@@ -91,7 +96,8 @@ the tightly-converged knobs were adopted as defaults.
|
||||
|
||||
The optimizer never re-decodes video. `scene_analyze --dump-embeddings out.h5` runs the
|
||||
expensive half once (decode→detect→align→embed) and dumps per-frame face embeddings
|
||||
+ metadata to HDF5 (`scripts/optimizer/SCHEMA.md`). `scripts/optimizer/replay.py` then
|
||||
+ metadata to HDF5 ([`scripts/optimizer/SCHEMA.md`](https://REPOLINK/scripts/optimizer/SCHEMA.md)).
|
||||
[`scripts/optimizer/replay.py`](https://REPOLINK/scripts/optimizer/replay.py) then
|
||||
replays that dump through the **real** C++ `face_tracker → identity_matcher →
|
||||
scene_tracker` assembled in a Python KPN network (`sae_kpn` nanobind module), varying
|
||||
Config knobs freely — no GPU embedding, no decode. Verified BYTE-EXACT against
|
||||
@@ -110,8 +116,10 @@ python scripts/optimizer/optimize.py --manifest films.json --gallery gallery.jso
|
||||
--params prob_threshold:0.5:0.999 anneal_sec:1:30 extinction_sec:1:15 \
|
||||
--popsize 8 --maxiter 20 --trajectory traj.jsonl --out opt.json
|
||||
# 4. score a fixed config / validate on a held-out set
|
||||
python scripts/optimizer/score_config.py --manifest heldout.json --gallery gallery.json \
|
||||
--config '{"prob_threshold":0.76,"extinction_sec":1.5,"anneal_sec":10}'
|
||||
# (historical: score_config.py and scene_score.py were removed with the
|
||||
# scene-union metric — use scripts/optimizer/second_score.py, per-second)
|
||||
python scripts/optimizer/second_score.py --help
|
||||
```
|
||||
|
||||
See also memory: kpn-python-replay-optimizer, gallery-coverage-gap, xray-validation-*.
|
||||
Superseded by the [model bake-off + re-tune](model-bakeoff.md), which
|
||||
replaced this round's scene-union metric with per-second scoring.
|
||||
|
||||
Reference in New Issue
Block a user