dtourolle
b1efefac6f
- experiment_charts.py generates 4 figures from experiments/ artifacts: held-out per-film F1, 16-combo ranking, DE search landscape, and the Downton detector-vs-tracker ghost timeline (replaces the blank title-card screenshot) - new frames: 19-correct wedding shot (success case), Many Saints ghost-vs-unknown frame (three error classes in one image) - rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only as the on-disk artifact prefix, explained once - repo file references are now links via https://REPOLINK/<path> placeholders; build_site.sh pins them to the HEAD commit's raw URLs and fails the build if a linked path doesn't exist at HEAD - drop references to removed scripts (scene_score.py, score_config.py) and to session-memory names; mark artifact-registry paths with their pull commands - commit readme_example.jpg + pipeline_topology.svg so README renders on the plain Gitea repo view - deploy_pages.sh: push built site/ to the gitea-pages branch
62 lines
3.2 KiB
Markdown
62 lines
3.2 KiB
Markdown
# scene-actor-extraction
|
|
|
|
A face-recognition pipeline that finds when each actor appears on screen in a
|
|
film or TV episode — built on [KPN++](https://gitea.tourolle.paris/dtourolle/KPN)
|
|
(a C++20 Kahn Process Network library) for the detect → track → match → scene
|
|
pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated optimizer.
|
|
|
|
This is what a good second looks like — one sampled frame from a held-out film,
|
|
19 faces named, all 19 correct, the rest honestly declared unknown:
|
|
|
|
|
|
|
|
And this is why the work isn't done: on this same film the same config misses
|
|
6 in 10 of the actor-seconds X-Ray says are present, and on the worst held-out
|
|
film it reports ghost actors over empty walls — at 100% confidence. Both
|
|
stories, with the evidence, are in the pages below.
|
|
|
|
## Start here — four questions this bake-off answers
|
|
|
|
- **[Which model is best?](best-model.md)** — calibration curves first
|
|
(discriminative power, independent of any threshold), then F1 on the actual
|
|
benchmark. LVFace-B Glint360K wins both.
|
|
- **[Whole gallery vs. limited (cast-restricted) gallery](gallery-scope.md)** —
|
|
restricting the matcher to a film's credited cast is a clean win on every
|
|
axis (+3.3pp F1, less than a third the misIDs), but isn't a shipped runtime
|
|
feature yet.
|
|
- **[Does pose expansion help?](pose-expansion.md)** — a real training-set
|
|
effect that didn't reproduce on 5 held-out films once two methodology bugs
|
|
were caught and fixed. An honest null result, not a forced narrative.
|
|
- **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)** — the winning
|
|
model's held-out generalization gap, its two real failure modes (frozen-bbox
|
|
"ghost tracks"), and one case where it correctly identified an actor that
|
|
the X-Ray ground truth itself failed to credit.
|
|
|
|
## The full technical log
|
|
|
|
- **[Model bake-off + threshold re-tune](model-bakeoff.md)** —
|
|
the complete experiment log behind the four pages above: the ROCm teardown
|
|
deadlock root cause and fix, DE concurrency tuning, the full 16-combo
|
|
results table, and every caveat. This is where the shipped
|
|
[`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults come from.
|
|
- **[Optimizer experiments (prior round)](optimizer-experiments.md)** — the
|
|
earlier scene-union-metric tuning pass, superseded by the per-second metric
|
|
used in the bake-off but kept for the ground-truth/architecture background.
|
|
- **[Service conversion (proposal)](service-conversion.md)** — design sketch
|
|
for an idle-GPU Docker worker, not yet built.
|
|
|
|
## Reproducing the benchmarks
|
|
|
|
Gallery `.h5` files, embedding dumps, the X-Ray corpus, montage frame images,
|
|
and DE trajectories are not committed to this repository — they're pushed to
|
|
the Gitea package registry and pulled on demand:
|
|
|
|
```bash
|
|
scripts/artifacts/pull_artifacts.sh galleries
|
|
scripts/artifacts/pull_artifacts.sh experiment-data
|
|
scripts/artifacts/pull_artifacts.sh montage-frames <film-slug>
|
|
```
|
|
|
|
See [`scripts/artifacts/push_artifacts.sh`](https://REPOLINK/scripts/artifacts/push_artifacts.sh)
|
|
for the upload side (requires a `GITEA_TOKEN` with package write scope).
|