From d340da755a08ae9c885545b9d01491d801a916ff Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Sun, 19 Jul 2026 19:12:22 +0200 Subject: [PATCH] docs: rep4 bake-off write-up, MkDocs site, artifact-registry-backed experiments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/rep4-optimizer-results.md is the main deliverable: the model bake-off + threshold re-tune experiment log, including the ROCm teardown deadlock root cause and fix, DE concurrency tuning, the 16-combo results table, held-out validation against 5 films never seen by the optimizer (macro F1 67.4% vs. 75.3% training — a real generalization gap), the frozen-bbox "ghost track" failure mode found via annotated frame evidence, calibration curves per model, and an isolated-effects breakdown of gallery scope vs. pose expansion. MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/ pulls referenced images from the artifact registry and generates the calibration chart at build time (see the tooling commit) rather than committing images to the repo. experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories, manifests, results) moved to the Gitea package registry. film-lut.template.json is the committed placeholder for the gitignored file-lut.json (real local movie paths, never shared — some source filenames carry scene-release tags). Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for the new scene-detection path. --- .gitignore | 21 +- README.md | 8 +- docs/index.md | 33 +++ docs/optimizer-experiments.md | 117 +++++++++ docs/rep4-optimizer-results.md | 408 +++++++++++++++++++++++++++++ docs/service-conversion.md | 215 +++++++++++++++ experiments/.gitignore | 21 ++ experiments/README.md | 59 +++++ experiments/SESSION_STATE.md | 105 ++++++++ experiments/build_manifests.py | 66 +++++ experiments/file-lut.template.json | 11 + experiments/run_montage_all.py | 93 +++++++ experiments/run_rep4_subprocess.sh | 26 ++ mkdocs.yml | 42 +++ models/transnetv2.onnx | 3 + 15 files changed, 1223 insertions(+), 5 deletions(-) create mode 100644 docs/index.md create mode 100644 docs/optimizer-experiments.md create mode 100644 docs/rep4-optimizer-results.md create mode 100644 docs/service-conversion.md create mode 100644 experiments/.gitignore create mode 100644 experiments/README.md create mode 100644 experiments/SESSION_STATE.md create mode 100644 experiments/build_manifests.py create mode 100644 experiments/file-lut.template.json create mode 100644 experiments/run_montage_all.py create mode 100644 experiments/run_rep4_subprocess.sh create mode 100644 mkdocs.yml create mode 100644 models/transnetv2.onnx diff --git a/.gitignore b/.gitignore index 8f8e316..71be62c 100644 --- a/.gitignore +++ b/.gitignore @@ -30,9 +30,12 @@ trt_cache/ # ORT pre-optimized model cache (generated on first run, provider-specific) ort_cache/ -# Gallery JSON files (generated) +# Gallery files (generated — HDF5 only, see src/gallery/gallery_store.cpp). +# Legacy JSON galleries from before that switch are also excluded. gallery.json gallery_*.json +gallery.h5 +gallery_*.h5 # Calibration cache (generated alongside a gallery, per-embedder) *.calib_cache.csv @@ -61,6 +64,22 @@ eval/probe/ # Local reference repos kept for inspiration (each has its own .git) inspiration/ +# experiments/ has its own nested .gitignore for HDF5 galleries/dumps/X-Ray +# corpus (all pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to +# the Gitea generic package registry instead of committed). +# Exception to the blanket *.json rule above: the committed placeholder for +# experiments/file-lut.json (see experiments/.gitignore). +!experiments/file-lut.template.json + +# Site build output (mkdocs build). Rendered site is deployed to a +# gitea-pages branch, never committed to a working branch. +site/ +docs_site/ + +# Images staged into docs/ from the artifact registry at build time +# (scripts/docs/build_site.sh) — not committed, pulled fresh on each build. +docs/assets/ + # Python __pycache__/ *.pyc diff --git a/README.md b/README.md index b9e50c5..bed079f 100644 --- a/README.md +++ b/README.md @@ -46,10 +46,10 @@ Optional flags: ## Models The ONNX model weights live in `models/` (tracked via Git LFS): -- `arcface_w600k_r50.onnx` — primary ArcFace embedder -- `arcface_w600k_mbf.onnx`, `arcface_r18.onnx` — lighter alternatives -- `LVFace-B_Glint360K.onnx` — LVFace embedder (ViT backbone, ICCV 2025), drop-in - ArcFace-compatible: same `[N,3,112,112]` input and 512-d output (see below) +- `LVFace-B_Glint360K.onnx` — LVFace embedder (ViT backbone, ICCV 2025), the default + (best F1 in the rep4 model bake-off, see `docs/rep4-optimizer-results.md`) +- `arcface_w600k_r50.onnx` — ArcFace embedder, previous default +- `arcface_w600k_mbf.onnx`, `arcface_r18.onnx` — lighter ArcFace alternatives - `face_detection_yunet_2023mar.onnx` — YuNet face detector - `scrfd_500m_bnkps.onnx` — SCRFD face detector diff --git a/docs/index.md b/docs/index.md new file mode 100644 index 0000000..9ccdb17 --- /dev/null +++ b/docs/index.md @@ -0,0 +1,33 @@ +# scene-actor-extraction + +A face-recognition pipeline that finds when each actor appears on screen in a +film or TV episode — built on [KPN++](https://gitea.tourolle.paris/dtourolle/KPN) +(a C++20 Kahn Process Network library) for the detect → track → match → scene +pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated optimizer. + +## Start here + +- **[Rep4 model bake-off + threshold re-tune](rep4-optimizer-results.md)** — the + current experiment log: model comparison, DE threshold tuning, held-out + validation, and the visual failure-mode evidence (frozen-bbox ghost tracks). + This is where the shipped `src/config.hpp` defaults come from. +- **[Optimizer experiments (prior round)](optimizer-experiments.md)** — the + earlier scene-union-metric tuning pass, superseded by the per-second metric + used in rep4 but kept for the ground-truth/architecture background. +- **[Service conversion (proposal)](service-conversion.md)** — design sketch + for an idle-GPU Docker worker, not yet built. + +## Reproducing the benchmarks + +Gallery `.h5` files, embedding dumps, the X-Ray corpus, montage frame images, +and DE trajectories are not committed to this repository — they're pushed to +the Gitea package registry and pulled on demand: + +```bash +scripts/artifacts/pull_artifacts.sh galleries +scripts/artifacts/pull_artifacts.sh experiment-data +scripts/artifacts/pull_artifacts.sh montage-frames +``` + +See `scripts/artifacts/push_artifacts.sh` for the upload side (requires a +`GITEA_TOKEN` with package write scope). diff --git a/docs/optimizer-experiments.md b/docs/optimizer-experiments.md new file mode 100644 index 0000000..327e318 --- /dev/null +++ b/docs/optimizer-experiments.md @@ -0,0 +1,117 @@ +# Threshold optimization against Amazon X-Ray — experiment log + +Record of the July 2026 work that tuned the pipeline's recognition/tracking defaults +against ground-truth per-scene actor presence, and the tooling built to do it. + +## TL;DR — what changed + +| knob | old default | new default | why | +| ---- | ----------- | ----------- | --- | +| `prob_threshold` | 0.99 | **0.76** | 0.99 was far too strict — halved recall for a fraction of a precision point. DE optimum, tightly converged. | +| `extinction_sec` | 5.0 | **1.5** | Long extinction smears presence into later scenes → FPs. DE converged tightly low. | +| `anneal_sec` | 10.0 | 10.0 (unchanged) | DE found it **insensitive** (F1 flat ±0.3pp across 3–26s) — kept the round default. | +| `detector_conf` | 0.5 | 0.5 (unchanged) | Sweep showed raising it only trades recall for precision at a net F1 loss — near-threshold detections are real faces, not phantoms. | + +Net effect on the 9-film benchmark (strict per-scene, augmented gallery): +recall **58% → ~72%**, F1 **70% → ~76%**, precision ~85%, at no meaningful precision cost. + +## Ground truth + +Public scene-level **Amazon X-Ray** dataset (Zenodo DOI 10.5281/zenodo.17659734, +CC-BY-4.0): per movie, `people.csv` (name_id/person/character), `scenes.csv` +(scene/start/end ms), `people_in_scenes.csv`. Films matched to the library by an +**authoritative Jellyfin ID join** (query `/Items?IncludeItemTypes=Movie&Fields= +ProviderIds,Path`, join Imdb/Tmdb against X-Ray metadata) — NOT fuzzy title matching, +which collides badly (TV episodes vs same-named films). 9 genuine films with source +video on disk: Benny & Joon, Café Society, Downton Abbey: A New Era, Lord of War, +Lovelace, The Many Saints of Newark, Scarface, Sound of Metal, Valerian. + +## The scoring metric (evolved through review) + +Comparison unit is the **X-Ray scene**, not sampled timepoints. For each scene +`[start,end]`: predicted set = **union** of actors detected anywhere in the span; +GT set = actors X-Ray lists for that scene. Per scene TP/FP/FN, then: + +- **Precision: STRICT.** Any predicted actor not in the scene's X-Ray set is an FP, + *including out-of-cast confusions* (no gallery∩cast masking). An earlier + timepoint-sampled, cast-masked metric HID ~570 such FPs across 9 films and let the + optimizer drive `prob_threshold` to the 0.50 floor — a metric artifact. Counting + them is essential. +- **Recall: FAIR.** FN counts only X-Ray cast members **who are in the gallery**. 67% + of X-Ray cast (261/392) have no gallery reference embedding and can never be + recognised — counting them as misses penalises coverage, not the threshold. Both + `recall` (fair) and `recall_strict` (all) are reported. +- **Aggregation:** per-scene F1 → **duration-weighted average within a movie** (long + scenes count more) → **equal-weight mean across movies** (macro; each film counts + the same regardless of length). This is the DE objective. + +Implemented in `scripts/optimizer/scene_score.py`. + +## The gallery coverage gap + +Diagnosing low recall: only **131 of 392** X-Ray cast were in the gallery (33%). Every +in-gallery actor HAD embeddings (gallery well-formed) — the gap was pure coverage. +`scripts/optimizer/fetch_missing_actors.py` recovers missing actors: +`nm-id → TMDB /find external_ids → /person/{id}/images → download → embed (sae_embed)`, +with a `--wikidata` fallback (P345→P18 Commons photo). + +- **TMDB recovered 143/261** (55%). 0 face-detection failures; the rest had no TMDB + person (60) or no profile photo (58). Coverage 33% → **70%**. +- **Wikidata fallback: 0/118** of the TMDB failures — only 4 even had a Commons photo, + none yielded a detectable face. → **TheTVDB not worth pursuing**: these remaining + actors are obscure enough that no image source covers them, AND (see below) most are + off-camera anyway. + +**Coverage vs detectability.** Adding references lifted recall (58→68% at fixed config) +but modestly. Per-film drill-down (Lord of War: 12 actors recovered, only 1 had a +detectable on-camera face) showed most missing cast are a **detectability gap** — X-Ray +credits them as cast-in-scene (incl. off-camera/background), but their face never +appears clearly for the pipeline to detect. This is a fundamental ceiling of a +face-recognition pipeline vs X-Ray's presence semantics, not a fixable gap. + +## Optimizer + +`scripts/optimizer/optimize.py` — scipy `differential_evolution` over the knob space, +each candidate = full replay of all films through the **real** C++ nodes (see the +KPN replay architecture below) scored by the metric above. Global objective (one +config for all films, not per-film). + +**Convergence stability (augmented gallery, 233 evals):** + +| knob | top-20 range | verdict | +| ---- | ------------ | ------- | +| `prob_threshold` | 0.69–0.83 (σ 0.05) | TIGHT — trust 0.76 | +| `extinction_sec` | 1.0–2.2 (σ 0.33) | TIGHT — trust 1.5 | +| `anneal_sec` | 3.1–26.3 (σ 6.4) | LOOSE — insensitive, not hard-coded | + +F1 varied only 0.3pp across the top-20 → objective is flat near the optimum, so only +the tightly-converged knobs were adopted as defaults. + +## Replay architecture (how the sweep is cheap) + +The optimizer never re-decodes video. `scene_analyze --dump-embeddings out.h5` runs the +expensive half once (decode→detect→align→embed) and dumps per-frame face embeddings ++ metadata to HDF5 (`scripts/optimizer/SCHEMA.md`). `scripts/optimizer/replay.py` then +replays that dump through the **real** C++ `face_tracker → identity_matcher → +scene_tracker` assembled in a Python KPN network (`sae_kpn` nanobind module), varying +Config knobs freely — no GPU embedding, no decode. Verified BYTE-EXACT against +`scene_analyze`'s own output. The dumps are gallery-independent, so testing the +augmented gallery needed no re-dump. `detector_conf` is replayable UPWARD only (the +dump floor is 0.5). + +## Reproduce + +```bash +# 1. dump (once per film, needs video) +scene_analyze --movie --gallery gallery.json --dump-embeddings dump.h5 --fps 1 +# 2. build films manifest by Jellyfin ID join (see scripts/optimizer notes) +# 3. optimize +python scripts/optimizer/optimize.py --manifest films.json --gallery gallery.json \ + --params prob_threshold:0.5:0.999 anneal_sec:1:30 extinction_sec:1:15 \ + --popsize 8 --maxiter 20 --trajectory traj.jsonl --out opt.json +# 4. score a fixed config / validate on a held-out set +python scripts/optimizer/score_config.py --manifest heldout.json --gallery gallery.json \ + --config '{"prob_threshold":0.76,"extinction_sec":1.5,"anneal_sec":10}' +``` + +See also memory: kpn-python-replay-optimizer, gallery-coverage-gap, xray-validation-*. diff --git a/docs/rep4-optimizer-results.md b/docs/rep4-optimizer-results.md new file mode 100644 index 0000000..3ed2e15 --- /dev/null +++ b/docs/rep4-optimizer-results.md @@ -0,0 +1,408 @@ +# Rep4 model bake-off + threshold re-tune — experiment log (2026-07-18/19) + +Follow-on to `docs/optimizer-experiments.md`, which used an older, since-superseded +scene-union metric. This round uses the **per-second** metric +(`scripts/optimizer/second_score.py`) and answers three questions in one 16-run +matrix: which embedding model is best, does cast-restriction cut misIDs, and does +per-film gallery expansion help. + +## Why this experiment, and what it actually delivered + +Four goals going in, and an honest read on each after held-out validation (see +below): + +1. **Find the best default parameters to ship.** Partially delivered. The DE optimum + generalizes *unevenly* — strong on 3 of 5 held-out films, badly broken on 2 (one + with a 974-count misID blowup). The tuned values are shipped anyway (see + Caveats) because they still beat the old defaults on average, but this is not a + settled, film-agnostic optimum. +2. **Find the best default model.** Delivered with more confidence. LVFace beat + r50/r18/mbf across all 4 training combos, and nothing in held-out validation + contradicts the model choice specifically — the held-out failures trace to + `extinction_sec`/threshold interactions and gallery coverage, not the embedder. +3. **Provide insight into how the application works.** The strongest, most durable + output. Found and fixed a real teardown deadlock bug (100% reproducible, not the + assumed rare GPU flake), established a real concurrency ceiling (8 parallel + replays, not more), and found a real parameter interaction (a strict + `prob_threshold` "earns" a longer extinction window before it starts hurting). +4. **Demonstrate limitations.** Delivered, and reinforced hard by held-out + validation — see the "Held-out validation" section below for concrete examples, + including a screenshot of the matcher naming 15 actors, none correctly, on a + completely blank title card. + +## TL;DR — what changed in `src/config.hpp` + +| knob | old default | new default | why | +| ---- | ----------- | ----------- | --- | +| `arcface_model` | `arcface_w600k_r50.onnx` | **`LVFace-B_Glint360K.onnx`** | Best F1 in the full-gallery bake-off (75.3% vs r50's 68.5%). LVFace was worth its size. | +| `prob_threshold` | 0.76 | **0.754** | Re-tuned for LVFace + per-second metric. | +| `extinction_sec` | 1.5 | **57.4** | Reverses the earlier "short is better" finding — see below. | +| `anneal_sec` | 10.0 | **35.5** | Same reversal; previously thought insensitive. | +| `expand_gallery` | false | **true** | Helped recall on the full (unrestricted) gallery for the winning model — opposite of the earlier assumption. | + +These are the **`LVFace-B_Glint360K_full_exp`** winning values — the best result that +uses only features already live in the running app (full gallery, no cast +restriction; see below for why restricted mode isn't applied even though it scored +higher). + +## Why re-run at all + +`docs/optimizer-experiments.md`'s scene-union metric hid out-of-cast false positives +behind a gallery∩cast recall mask, and the earlier 9-film benchmark was scene-level +(union over a whole X-Ray scene), not a fair per-timepoint comparison. SESSION_STATE +flagged the old scene-metric bake-off numbers (R50≈LVFace≈MBF ~85%) as superseded. +This round uses `second_score.py`: uniform per-second sampling, GT = X-Ray scene's +cast at time *t*, pred = actors whose presence window covers *t*, FPI weighted 10× +when the named actor isn't in the film's cast at all (true misID) vs. an in-cast +timing slip. FN only counts gallery-known cast (fair recall — 67% of X-Ray cast have +no reference embedding, see `gallery-coverage-gap` memory). + +## The deadlock that was blocking all of this + +Every replay in this line of work goes through `scripts/optimizer/replay.py`, which +runs the real C++ tracker/matcher/scene_tracker nodes inside a Python-assembled KPN +network. Before this session, every subprocess replay **timed out at 45s, 100% of +the time** — not the documented ~20-30% ROCm rocBLAS-GEMM driver flake, but a plain +logic bug: `replay.py`'s CLI called `replay(net, ..., stop=False)` to *skip* +`net.stop()` (trying to dodge the GEMM deadlock), planning to `os._exit(0)` +immediately after. But: + +- `PyNode::stop()` (`external/KPN/include/kpn/python/bindings.hpp`) is the *only* + code that sets `stop_flag_ = true` before joining the node's worker thread. +- The source node's `run_loop()` has `while (!stop_flag_)` as its only exit + condition (it has no input channels, so it never sees a channel-closed signal + either). +- Skipping `stop()` meant `stop_flag_` never became true. When `replay()` returned, + its local `net` went out of scope immediately, running `~PyNetwork` → `~PyNode` → + `thread_.join()` **synchronously inside `replay()`'s own call frame** — before + `main()` ever got control back to run `os._exit(0)`. + +Root-caused via `gdb -p -batch -ex "thread apply all bt"` on a hung process: +the main thread was stuck in `~PyNode`'s `jthread::join()`; the worker thread was in +an ordinary `time.sleep()` inside the Python source callback, waiting for a stop +signal that was never sent. The two HSA `kfd_wait_on_events` threads visible in the +same trace are normal ROCm runtime housekeeping, not evidence of a wedged GPU kernel. + +**Fix:** `replay.py` now calls `replay(..., stop=True)` (the removed `stop=False` + +`os._exit` workaround was actively harmful). Verified 3/3 clean runs at ~8s each +(down from a guaranteed 45s timeout), and a full DE sweep producing real, sensible +F1/precision/recall instead of flat 0.0%. + +## Concurrency tuning + +With the deadlock fixed, `optimize.py` was extended with DE-level parallelism — +`differential_evolution(..., workers=ThreadPoolExecutor.map)` — so multiple +population candidates evaluate concurrently, each spawning its own per-film replay +subprocesses (`REPLAY_WORKERS`). Total concurrent GPU replay processes ≈ +`DE_WORKERS × REPLAY_WORKERS`. + +| concurrent replays | result | +| --- | --- | +| 3 (`REPLAY_WORKERS=3`, no DE parallelism) | baseline, GPU underutilised | +| 6 (`DE_WORKERS=2 × REPLAY_WORKERS=3`) | clean, real scores, ~1 isolated timeout per run | +| 8 (`DE_WORKERS=2 × REPLAY_WORKERS=4`, 4-film manifest) | clean, real scores | +| 9 (`DE_WORKERS=3 × REPLAY_WORKERS=3`) | **broken** — every replay blew past the 45s timeout, all scores silently degraded to 0.0% | + +9 concurrent replays looks like valid output (well-formed JSON, a real number) while +actually being garbage — a dangerous failure mode, not a crash. **8 concurrent is the +practical ceiling** on this GPU (gfx1100) for this workload. The matrix ran at +`REPLAY_WORKERS=4 DE_WORKERS=2`. + +## Training films (rep4) and validation set + +9 films total have dumped embeddings across all 4 models. 4 were used for +optimization (rep4), leaving 5 held out for validation: + +- **Lord of War** (64-cast, "clean") +- **Scarface** (67-cast, "ensemble/lookalike") +- **Sound of Metal** (14-cast, "high gallery-coverage") +- **Café Society** (62-cast, added this round — similar ensemble size to Scarface but + different genre/lighting; picked to add diversity, not genre-overlap, over + Downton Abbey or The Many Saints of Newark) + +Held out: Benny & Joon, Downton Abbey: A New Era, Lovelace, The Many Saints of +Newark, Valerian and the City of a Thousand Planets. The rep4 numbers below are +training-set fit — see "Held-out validation" further down for the real +generalization test. + +## Search space and DE settings + +`popsize=10, maxiter=15` (3 params → ≤480 evals/combo ceiling; DE's `tol` convergence +usually stops earlier). `anneal_sec`/`extinction_sec` bounds were **widened from +1–30/1–15 to 1–60/1–60 mid-run** (see below) — the 4 `arcface_w600k_r50` combos +finished before the widening and still use the old, narrower bounds, so they are +**not directly comparable** to the other 12 on those two params. Re-running r50 with +the wider bounds was deferred (diminishing-returns judgment call, not yet done). + +## Results — all 16 combos (4 models × {full, restricted} × {expand, noexp}) + +Ranked by F1. `misid` = FPI_misid, count of true wrong-actor identifications (an +actor named who isn't in the film's cast at all) — distinct from `FPI`, which +includes in-cast timing slips. + +| combo | F1 | P | R | TPI | FPI | misid | FN | +|---|---|---|---|---|---|---|---| +| LVFace-B_Glint360K_restricted_exp | **78.3%** | 91.0% | 68.9% | 42830 | 3782 | 60 | 19492 | +| LVFace-B_Glint360K_restricted_noexp | 76.7% | 91.5% | 66.2% | 41149 | 3400 | 59 | 21173 | +| arcface_w600k_mbf_restricted_exp | 76.5% | 90.7% | 66.3% | 41270 | 4431 | **0** | 21052 | +| arcface_r18_restricted_exp | 75.5% | 87.6% | 66.5% | 41399 | 5666 | 60 | 20923 | +| **LVFace-B_Glint360K_full_exp** | **75.3%** | 89.7% | 65.4% | 47757 | 3407 | 232 | 26966 | +| arcface_w600k_mbf_restricted_noexp | 75.0% | 91.1% | 63.9% | 39752 | 3465 | 60 | 22570 | +| arcface_w600k_mbf_full_noexp | 74.2% | 87.4% | 64.4% | 12645 | 1312 | 57 | 6985 | +| arcface_r18_restricted_noexp | 73.5% | 91.3% | 61.7% | 38299 | 3220 | 60 | 24023 | +| LVFace-B_Glint360K_full_noexp | 72.4% | 94.2% | 58.9% | 27077 | 1725 | **0** | 19506 | +| arcface_w600k_mbf_full_exp | 72.0% | 87.7% | 61.4% | 39875 | 3729 | 240 | 26338 | +| arcface_w600k_r50_full_noexp † | 71.6% | 96.7% | 56.9% | 22471 | 361 | 45 | 17012 | +| arcface_w600k_r50_restricted_exp † | 71.1% | 96.5% | 56.4% | 34954 | 1119 | 15 | 27368 | +| arcface_w600k_r50_restricted_noexp † | 69.2% | 97.9% | 53.6% | 21146 | 327 | 15 | 18337 | +| arcface_r18_full_exp | 69.1% | 87.6% | 57.7% | 37342 | 3119 | 242 | 28871 | +| arcface_w600k_r50_full_exp † | 68.5% | 94.0% | 54.1% | 34982 | 903 | 150 | 31231 | +| arcface_r18_full_noexp | 66.6% | 91.3% | 53.1% | 34314 | 2362 | 107 | 31899 | + +† old, narrower anneal/extinction bounds (see above) — not directly comparable to +the other 12 on those two params. + +## Calibration curves — discriminative power, independent of the threshold + +Each model's gallery carries a fitted Platt sigmoid `P(match | sim) = σ(a·sim + b)` +(embedded directly in the gallery HDF5, see `src/gallery/gallery_calibration.hpp`). +Plotting all four side by side shows discriminative power directly, independent of +whatever `prob_threshold` a particular run happened to use: + +![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png) + +LVFace-B has both the steepest curve (`a=17.7`, vs. 15.3–16.2 for the ArcFace +variants) and the lowest P=0.5 decision boundary (similarity 0.23 vs. 0.27–0.31) — +it separates same-actor from different-actor pairs more confidently at a lower +similarity, consistent with it winning the full-gallery F1 comparison below. +Generated by `scripts/docs/calibration_chart.py` (requires each gallery to have +been calibrated at least once — run any replay against it first). + +## Two effects in isolation: gallery scope, and pose expansion + +The matrix crosses two independent variables — averaging across all 4 models +isolates each one from model choice: + +**Gallery scope (whole 2418-actor gallery vs. restricted to the film's credited +cast)** — averaged over both expansion settings and all 4 models: + +| scope | F1 | P | R | total misID (16 evals→8 each) | +|---|---|---|---|---| +| full | 71.2% | 91.1% | 59.0% | 1073 | +| **restricted** | **74.5%** | 92.2% | **62.9%** | **329** | + +Restriction wins outright on every axis — not a precision/recall trade, a clean +win: **+3.3pp F1, +3.9pp recall, and less than a third the total misIDs.** Fewer +candidates in the matcher's search space means fewer opportunities for a +look-alike false match, and (per the recall gain) doesn't cost real detections. +This is the single cleanest signal in the whole matrix — stronger than the model +choice itself — which is exactly why cast-restriction becoming a real runtime +feature (not just an optimizer trick) is the top item in Caveats below. + +**Pose expansion (promoting a confidently-identified track's novel-pose views into +a per-film gallery annex — `track_gallery.hpp`)** is smaller and interacts with +scope rather than acting independently: + +| scope | expansion | F1 | R | misID | +|---|---|---|---|---| +| full | off | 71.2% | 58.3% | 209 | +| full | **on** | 71.2% | 59.7% | **864** | +| restricted | off | 73.6% | 61.3% | 194 | +| restricted | **on** | **75.4%** | **64.5%** | 135 | + +In **restricted** mode, expansion is a clean win (+1.8pp F1, +3.2pp recall, misID +actually *drops*) — the annex only ever competes against the film's own ~15-actor +cast, so a "confidently identified, new pose" view is unlikely to be mistaken for +someone else. In **full** mode, expansion buys essentially nothing on F1 (71.2% → +71.2%, recall +1.4pp) while **quadrupling misIDs** (209 → 864): a novel-pose view +promoted into the annex now competes against the whole 2418-actor gallery, so a +"confident" identity is confident against the wrong universe of candidates — the +expansion mechanism is "learning" a pose correctly, but the enlarged evidence pool +makes it easier for that learned pose to look like a plausible match for a +different actor. **Practical takeaway: gallery expansion should be paired with +cast restriction, not used on the full gallery** — the version currently shipped +as default (`full_exp`, see TL;DR) sits in the worse of these four cells for this +specific knob, even though it's the best available combo without cast-restriction +support in the app yet (see Caveats). + +## What the data says + +- **LVFace was worth its size.** It wins full-gallery mode outright (75.3% vs r50's + 68.5%, r18's 69.1%, mbf's 72.0%) with the highest recall of any full-mode combo — + the earlier scene-union-metric conclusion ("not worth it") doesn't survive the + better metric. +- **Cast-restriction is a consistent, broad win.** Every model's best combo is + `restricted`. It isn't just precision-safe: `arcface_w600k_mbf_restricted_exp` and + `LVFace-B_Glint360K_full_noexp` both hit **misid=0** — zero true wrong-actor + identifications. But restriction is an **offline optimizer technique, not a live + app feature** — it pre-filters each film's gallery to its Jellyfin-credited cast + before the matcher ever runs; there's no runtime "restrict to this film's cast" + switch in the app today. Implementing it for real is future work, tracked + separately from this defaults update. +- **Gallery expansion (`expand_gallery`) is mode-dependent.** It helps on + `restricted` galleries (smaller, so novel-pose promotion adds real signal) and on + LVFace's full gallery, but **hurts** r50 and mbf in full mode (compare + `arcface_w600k_r50_full_exp` 68.5% vs `full_noexp` 71.6%). Don't assume it's a free + win — model- and mode-dependent. +- **arcface_r18 (smallest/cheapest) is last across all 4 modes** — model capacity + matters here, this isn't just parameter-count padding. +- **`anneal_sec`/`extinction_sec` kept pinning at the search ceiling.** With the + original 1–30/1–15 bounds, 3 of 4 r50 combos landed at ~93-98% of the upper bound. + Widened to 1–60/1–60 mid-run (after the r50 combos had already finished) — every + subsequent combo's best config landed at ~90%+ of the *new* ceiling too (e.g. the + LVFace winner: `ann=59.2, ext=59.2`, both ~99% of 60). The likely mechanism: a + strict `prob_threshold` "earns" a long extinction/anneal window — once false + matches are rare, a long window just bridges real presence gaps (occlusion, turned + face) instead of smearing false positives into later scenes, which is what made + short windows look better under the old, laxer thresholds. **Open question, not + resolved**: does this keep climbing past 60s, or does it actually plateau there? + Decided not to chase further this round (diminishing-returns judgment call) — flag + for a future sweep if it matters. + +## Held-out validation — the number that actually matters + +The 16-combo matrix above is training-set fit. This is the real test: the shipped +config (`LVFace-B_Glint360K_full_exp` — `prob_threshold=0.754, anneal_sec=35.5, +extinction_sec=57.4, expand_gallery=true`) replayed against the **5 films never seen +by the optimizer** (Benny & Joon, Downton Abbey: A New Era, Lovelace, The Many +Saints of Newark, Valerian and the City of a Thousand Planets), scored the same way. + +| film | F1 | P | R | agree | TPI | FPI | misid | FN | +|---|---|---|---|---|---|---|---|---| +| Benny & Joon | 83.0% | 89.1% | 77.7% | 72.4% | 15125 | 1846 | 0 | 4337 | +| Lovelace | 77.5% | 90.3% | 67.9% | 72.1% | 14990 | 1085 | 58 | 7085 | +| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 58.8% | 18663 | 548 | 0 | 12467 | +| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 40.6% | 52027 | 1173 | 0 | 80084 | +| **The Many Saints of Newark** | **46.3%** | **54.7%** | 40.1% | 37.0% | 15922 | 4394 | **974** | 23791 | +| **macro average (5 films)** | **67.4%** | 85.8% | 57.0% | 56.2% | 116727 | 9046 | 1032 | 127764 | + +**67.4% held out vs. 75.3% on training** — an ~8pp drop, and a much more informative +number than the training-set F1 alone: a **37pp spread between best and worst film** +(83.0% vs 46.3%). The config does not generalize uniformly. + +Two films are outright failure cases, and rendering bounding boxes + names on the +extracted frames (`replay.py --raw-out` + `dump_error_frames.py --raw`, see +Reproduce) turned what looked like a same-scene misidentification into something +more precise and more damning: + +- **The Many Saints of Newark** (mob-family drama, picked as a training-adjacent + genre test) has **974 true misIDs** — far more than any training combo saw at any + setting. The annotated frame below shows the same mechanism as Downton Abbey, + at smaller scale: **"Jon Bernthal 100%", "Joey Diaz 100%", and "Billy Magnussen + 100%" are all frozen boxes over empty background — a blurred wall, hanging plates — + with no face in them at all.** Only one real face in frame has a box, and it + carries a *second*, colliding label ("Leslie Odom Jr." and "Michael Gandolfini" + both at high confidence on the same box) — likely two tracks whose frozen bboxes + happen to overlap. + + ![Frozen ghost boxes over background, The Many Saints of Newark](assets/images/many_saints_ghost_fpi.jpg) + *`experiments/results/holdout/frames/many_saints/fpi/fpi_t03543.jpg`* + +- **Downton Abbey: A New Era** (large ensemble, 36-cast) has high precision (97.8%) + but recall collapses to 39.4% (FN=80084, by far the largest of the 5). The frame + below is the starkest evidence in this whole experiment: **the matcher named 15 + actors — all of them wrong — over a completely blank closing title card with no + faces on screen at all.** + + ![15 ghost boxes over a blank title card, Downton Abbey: A New Era](assets/images/downton_abbey_ghost_fpi.jpg) + *`experiments/results/holdout/frames/downton_abbey/fpi/fpi_t07242.jpg`* + +Both are the same mechanism, verified directly against the HDF5 dump and the raw +per-frame stream (not just inferred from the screenshot): at the Downton Abbey +title card (t=7242), the dump's own `face_count` is **0 from t≈7240 onward** — no +detector output at all, confirmed independently of the pipeline. Yet all 15 "wrong" +actors are still marked visible, each with the *exact same bbox, unchanged to the +pixel*, repeated every single frame back to t=7222 (verified for Hugh Bonneville: +`(1743.2, 0.0, 171.3, 317.8)` at every sampled second from 7222 through 7279+). That +is `SceneTrackerFunc`'s `active_[actor_idx].last_bbox` (`scene_tracker_node.hpp`) +being re-emitted unchanged — **this is the extinction state machine working exactly +as coded**, not a bug in the logic. The film cuts from a packed group shot straight +into ~40+ seconds of blank titles/credits with zero faces, and `extinction_sec=57.4` +is comfortably long enough to bridge that entire gap without expiring, so the +tracker faithfully keeps reporting "last known position" for a cast that is no +longer on screen at all. + +This reframes the "long extinction window wins" DE-search pattern (see above): it +isn't unambiguously good. It buys recall by bridging real gaps (occlusion, turned +face) in some films, but on others — specifically, hard cuts into long faceless +footage — it manufactures a frozen-bbox ghost the tracker has no way to verify, +precisely the failure mode the *original* short-extinction-window default (`1.5s`) +was chosen to avoid. The training-set films apparently didn't have a long enough +faceless stretch after a confirmed identity to expose this; the held-out set did. + +Frames for all three films (`benny_joon`, `many_saints`, `downton_abbey` — one strong +performer, two failure cases) are under `experiments/results/holdout/frames/`, each +with a `manifest.json` listing the bucket (`best`/`fpi`/`fn`), timestamp, and +predicted vs. ground-truth actors for every dumped frame. Frames are annotated with +bounding boxes + name/confidence (green = identified, orange = unknown), matching +`debug_renderer_node.hpp`'s colour convention. Generated by +`scripts/optimizer/dump_error_frames.py --raw ` (see +Reproduce). + +`dump_error_frames.py --interval-sec 600` also supports a per-N-second sweep +instead of the fixed best/fpi/fn buckets: one best (highest Jaccard) and one worst +(lowest Jaccard) frame per 10-minute window across the whole film, e.g. +`experiments/results/holdout/frames/many_saints_intervals/` (13 windows × 2 = 26 +frames for the ~2h Many Saints runtime) — a way to sample "how are we doing" evenly +across a film's runtime rather than only at its most extreme seconds. + +## Caveats / what this is not + +- **r50's 4 combos used the old, narrower search bounds** and aren't fully + comparable to the other 12 on `anneal_sec`/`extinction_sec`. +- **The applied defaults use `full_exp`, not the higher-scoring `restricted_exp`**, + because cast-restriction isn't a real runtime feature yet (see above). The + 78.3% F1 number is not what the shipped defaults will produce — 75.3% is. +- **`full_exp` is the best full-gallery combo, but not the safest.** Per the + isolated-effects analysis above, `expand_gallery=true` only cleanly pays off + when paired with cast-restriction; on the full gallery it's flat on F1 while + ~4x-ing misIDs (209→864, averaged across models). `full_noexp` scores lower + (72.4% vs 75.3% for LVFace) but with **zero** true misIDs and higher precision + (94.2% vs 89.7%). Kept `full_exp` as shipped since it's the highest-F1 option + available without cast-restriction, but this is a real F1-vs-safety trade, not + a strictly-better choice — worth revisiting if misID rate matters more than + the last few points of F1 for a given deployment. +- **Switching the default model is an operational change, not just a config tweak**: + any existing gallery built from r50 embeddings is incompatible with LVFace + embeddings and needs rebuilding. + +## Reproduce + +```bash +# 4-film matrix, all 4 models × 2 modes × 2 expansion settings +bash experiments/run_rep4_subprocess.sh + +# single combo +SAE_EXPAND=1 REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \ + --manifest experiments/manifests/rep4_LVFace-B_Glint360K_full.json \ + --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ + --params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \ + --popsize 10 --maxiter 15 --trajectory traj.jsonl --out best.json + +# replay the shipped config against a held-out film — --raw-out is needed to draw +# bboxes later (the merged pred.json has no per-frame bbox, only actor windows) +python3 scripts/optimizer/replay.py \ + --dump experiments/dumps/LVFace-B_Glint360K/dump_.h5 \ + --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ + --out pred.json --raw-out raw.jsonl --prob-threshold 0.754 --anneal-sec 35.54 \ + --extinction-sec 57.43 --expand-gallery + +# dump example frames (best-agreement / FPI / FN) for visual inspection, annotated +# with bounding boxes + names (--raw is optional; omit for unannotated frames) +python3 scripts/optimizer/dump_error_frames.py \ + --pred pred.json --raw raw.jsonl --xray experiments/xray/.../ \ + --movie "" \ + --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ + --out-dir experiments/results/holdout/frames/ --n-per-bucket 4 + +# or: one best + one worst frame per 10-minute window across the whole film +python3 scripts/optimizer/dump_error_frames.py \ + --pred pred.json --raw raw.jsonl --xray experiments/xray/.../ \ + --movie "" \ + --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ + --out-dir experiments/results/holdout/frames/_intervals --interval-sec 600 +``` + +See also: `docs/optimizer-experiments.md` (prior round, superseded metric), +`experiments/SESSION_STATE.md`, and memory: kpn-python-replay-optimizer, +gallery-coverage-gap, xray-validation-results, per-scene-presence-eval-design. diff --git a/docs/service-conversion.md b/docs/service-conversion.md new file mode 100644 index 0000000..1979109 --- /dev/null +++ b/docs/service-conversion.md @@ -0,0 +1,215 @@ +# Conversion to service — a native idle-GPU worker + +Status: **design / proposal**. Nothing here is built yet. + +## The idea + +Turn the CLI tools into a **turnkey batch worker that uses the machine's idle +GPU**: it analyses newly-added Jellyfin media when you're not using the computer +(screen locked), and stops the instant you come back. It's an overnight job on +your own Linux box. + +**No Docker.** This runs on your own machine with your own drivers, so a container +buys little and costs a lot: GPU passthrough (nvidia-container-toolkit, or +`/dev/kfd`+`/dev/dri`+`video` group for ROCm) is the single most fragile part of a +containerised setup, and it exists *only* because of the container. Natively, the +GPU just works with the drivers you already have, and the media paths Jellyfin +reports are just real paths — no re-mounting. So we ship a **native installer** +instead of an image builder. + +Two deliverables: + +1. **An installer** — `scripts/build_install.py`. Detects your distro, ensures the + GPU/build dependencies are present (via `dnf`/`pacman`), compiles `scene_analyze` + for your GPU, and installs the binary + Python glue + two systemd **user** + units under `~/.local`. +2. **A screen-lock gate** — one of those systemd units watches logind lock/unlock + and starts/stops the worker. Lock → analyse. Unlock → stop. + +## What already exists (reuse, don't rebuild) + +The processing loop is already implemented — this is packaging, building, and +lock-gating, not new pipeline logic. + +| Piece | Where | What it does | +|---|---|---| +| Analysis engine | `build/scene_analyze` | Video → face detect/align/embed → gallery match → result JSON | +| Backend selection | `CMakeLists.txt` (`SAE_INFERENCE_BACKEND`, `SAE_GEMM_BACKEND`) | ORT/TRT + ROCm/CUDA, chosen **at build time** | +| New-media queue | JRay plugin → `GET /Plugins/JRay/Tasks/Pending` | Backlog of items with no results yet | +| Worker loop | `run_from_jellyfin.py --worker` | Poll Pending → run `scene_analyze` → push results | +| Result push | `PUT /Plugins/JRay/Items/{id}/Truth` | Stores per-actor scene windows back in Jellyfin | +| Incremental gallery | `make_jellyfin_gallery.py --merge` | Embeds only cast not already in the gallery | +| Secrets loader | `.env` via `sae_env.py` | `JELLYFIN_URL`, `JELLYFIN_API_KEY`, `TMDB_API_KEY` | + +## Installer config + +One file. Build-time settings (fixed when we compile) vs. run-time settings (in the +worker's `.env`, editable without recompiling). + +```yaml +# install.yaml — consumed by scripts/build_install.py + +platform: nvidia # nvidia | amd | cpu → picks the cmake backend +model: + arcface: LVFace-B_Glint360K.onnx # embedder compiled against; gallery MUST match +schedule: + gallery_scan_interval: 24h # incremental --merge cadence; 0 disables the scanner +prefix: ~/.local # install root (bin, share, systemd user units) + +# runtime (written to the worker .env, not compiled in): +runtime: + jellyfin_url: http://localhost:8096 + # JELLYFIN_API_KEY / TMDB_API_KEY are filled into .env by hand after install +``` + +**Secrets never go in the repo or a build artifact** — the installer writes a +`.env` under the install prefix with blanks for the keys, and you fill them in +once. `sae_env.py` already loads it. + +**Model ⇄ gallery coupling (guard, don't just document):** embeddings from +different recognition models aren't interchangeable. We compile against one +embedder; the gallery must be built with the same one. Stamp the embedder name +into `gallery.json`, and have the worker **refuse to start** if the gallery's +embedder ≠ the configured `model.arcface`, rather than silently mismatching. + +## Dependencies via the system package manager + +The heavy build/runtime deps (OpenCV, ffmpeg, the GPU stack) are best provided by +the distro, not vendored. The installer ships a per-distro dependency list and +either installs them or prints the exact command. Targets: **Fedora (dnf)** and +**Arch (pacman)** first. + +| Dependency | Fedora (dnf) | Arch (pacman) | +|---|---|---| +| OpenCV | `opencv-devel` | `opencv` | +| ffmpeg | `ffmpeg-free`/`ffmpeg` (RPM Fusion) | `ffmpeg` | +| CMake / toolchain | `cmake gcc-c++` | `cmake gcc` | +| CUDA + TensorRT (nvidia) | NVIDIA CUDA repo + `libnvinfer-*` | `cuda`, `tensorrt` | +| ROCm (amd) | `rocm-hip-sdk` / `rocblas-devel` | `rocm-hip-sdk`, `rocblas` | +| ONNX Runtime | **not packaged** — installer fetches a pinned release tarball into the prefix | AUR `onnxruntime` (or same pinned-tarball fallback) | + +So the flow is: **detect distro → check each package → install via the native +manager (or print `sudo dnf install …` / `sudo pacman -S …`)**, with ONNX Runtime +as the one known gap the installer fills itself (a pinned upstream release +extracted under the install prefix, so it doesn't depend on a system package that +may not exist). CUDA/ROCm being present is *assumed* — you already run a GPU +desktop; the installer verifies and points you at the vendor repo if not. + +## What `build_install.py` does + +``` +build_install.py install.yaml + │ + ├─ detect distro (dnf vs pacman) and platform from config + ├─ ensure deps: install via manager, or print the exact command; fetch ONNX Runtime if needed + ├─ cmake + build scene_analyze with the platform's backend flags: + │ nvidia → -DSAE_INFERENCE_BACKEND=TRT -DSAE_GEMM_BACKEND=CUDA + │ amd → -DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=ROCM + │ cpu → -DSAE_INFERENCE_BACKEND=ORT (CPU EP; slow, for smoke tests) + ├─ install into : + │ bin/sae-scene-analyze the compiled binary + │ share/sae-worker/ Python glue + a venv (requests, etc.), models/ + │ share/sae-worker/.env runtime config (keys blank, url from config) + ├─ install systemd --user units: + │ sae-worker.service runs the worker + gallery-scan supervisor + │ sae-lock-gate.service watches logind lock/unlock, start/stops the worker + └─ print next steps (edit .env, `systemctl --user enable --now sae-lock-gate`) +``` + +## The worker service (supervisor) + +`sae-worker.service` runs a small Python supervisor as its main process: + +- starts the **worker loop** (`run_from_jellyfin.py --worker`) — the hot path, +- starts a **gallery-scan timer** — sleeps `gallery_scan_interval`, runs + `make_jellyfin_gallery.py --merge`, repeats, +- exits cleanly on SIGTERM (see re-queue below). + +## The lock gate + +`sae-lock-gate.service` runs a tiny watcher that subscribes to logind +lock/unlock signals and drives the worker service: + +``` +screen locks → systemctl --user start sae-worker.service +screen unlocks → systemctl --user stop sae-worker.service (SIGTERM) +``` + +**Screen-lock is the only signal — deliberately.** We don't also gate on GPU/CPU +load, because our own worker *is* the load: a load threshold would form a feedback +loop (worker starts → GPU spikes → threshold trips → worker stops → load drops → +restart → …). Lock state is external to what the worker does, so it can't +oscillate. + +Signal source is desktop-dependent: logind `Lock`/`Unlock` (GNOME/KDE via +`loginctl`/D-Bus) covers most setups; a `swayidle`/`xss-lock` hook is the fallback +for wlroots/X-only compositors. The installer picks based on what's present. + +## On resume: hard stop + re-queue (it's free) + +Stopping the worker mid-analysis costs nothing to reschedule, because of how the +JRay queue works: **an item only leaves `/Tasks/Pending` once its results are +pushed** (`push_truth`). A worker stopped mid-`scene_analyze` simply leaves that +item Pending — next lock picks it up again. No re-queue bookkeeping. + +Two small correctness requirements (the only worker changes needed): + +1. **Never push a partial result.** Already true — `push_truth` runs only after + `scene_analyze` returns; a killed run pushes nothing. ✓ (keep it that way). +2. **Clean up on signal.** `process_item` writes a temp filtered-gallery file and + unlinks it in a `finally`; a SIGKILL skips `finally`. Fix: write temps under a + dir the worker wipes on start, and/or a SIGTERM handler that unlinks before + exit. Minor. + +Accepted trade-off: a partially-analysed title restarts from scratch next lock. +Fine for an overnight/idle workload; no mid-video checkpointing. + +## The end-to-end UX + +```bash +# once: build + install for your GPU + model +./scripts/build_install.py install.yaml + # detects Fedora/Arch, ensures deps, compiles, installs units under ~/.local + +# once: set your keys, enable the gate +$EDITOR ~/.local/share/sae-worker/.env # JELLYFIN_API_KEY, TMDB_API_KEY +systemctl --user enable --now sae-lock-gate.service + +# from then on: nothing. Lock your screen → it analyses. Unlock → it stops. +``` + +No Docker, no GPU passthrough config, no media re-mounting — the worker sees the +same filesystem and GPU as everything else on the box. + +## Implementation plan (follow-up commits) + +Ordered so each step stands alone: + +1. **installer skeleton** — `scripts/build_install.py`: parse `install.yaml`, + distro detect, dependency check/print (start with cpu platform so it builds + without a GPU), cmake+build, copy into prefix. +2. **supervisor + cleanup** — `scripts/service.py` (worker loop + gallery-scan + timer + SIGTERM); temp-file cleanup fix in `run_from_jellyfin.py`. +3. **systemd units + lock gate** — generate/install `sae-worker.service`, + `sae-lock-gate.service`, and the logind lock watcher. +4. **gallery/model guard** — stamp embedder into `gallery.json`; startup mismatch + check. +5. **platform + distro matrix** — nvidia/amd backends; dnf/pacman dep lists; ONNX + Runtime fetch fallback. +6. **docs** — README "Run on your idle GPU" section. + +## Settled decisions + +- **ONNX Runtime build** — the installer fetches the **ROCm ORT** release. It + serves the `amd` platform, and its CPU execution provider covers the `cpu` + smoke-test fallback too, so one download handles both. (nvidia uses raw TRT and + doesn't need ORT.) +- **`dnf`/`pacman` invocation** — **auto-install.** The installer runs `sudo dnf + install …` / `sudo pacman -S …` itself (prompting for sudo), rather than only + printing the command. It still prints what it's about to install first. +- **Distro coverage** — **Fedora + Arch only** for now. Debian/Ubuntu (`apt`) is + out of scope. + +## Open questions + +*(none blocking — the spec above is buildable as-is.)* diff --git a/experiments/.gitignore b/experiments/.gitignore new file mode 100644 index 0000000..1ec81d2 --- /dev/null +++ b/experiments/.gitignore @@ -0,0 +1,21 @@ +# experiments/ in git keeps only scripts, README.md, SESSION_STATE.md, and this +# file. Every data artifact — galleries, embedding dumps, the X-Ray corpus, +# montage/frame images, DE trajectories, film manifests, and result summaries — +# is pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to the Gitea +# generic package registry instead (see docs/rep4-optimizer-results.md). +xray/ +dumps/ +galleries/ +*.h5 +manifests/ +trajectories/ +results/ + +# Raw run logs and scratch scripts (regenerated by every run). +_scratch/ + +# Real local media paths (film slug -> path on this machine). Never committed — +# these paths embed the specific source file names, which can include +# scene-release tags. Only file-lut.template.json (placeholders) is tracked; +# copy it to file-lut.json and fill in your own paths. +file-lut.json diff --git a/experiments/README.md b/experiments/README.md new file mode 100644 index 0000000..be5a21e --- /dev/null +++ b/experiments/README.md @@ -0,0 +1,59 @@ +# experiments/ — X-Ray validation & optimizer artifacts + +Durable home (in the repo tree, NOT `/tmp` scratch — a scratch wipe once cost an hour) +for the data behind the X-Ray threshold-optimization and embedding-model bake-off. + +## Layout +- `xray/` — Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734). +- `dumps/` — per-model embedding dumps, one HDF5 per (model, film). Gitignored (large). + Naming: `/dump_.h5`. Regenerate with `scene_analyze --dump-embeddings`. +- `galleries/` — per-model galleries (gitignored JSON). `gallery_.json` + + augmented variants. Regenerate with build_gallery / fetch_missing_actors. +- `manifests/` — film manifests (committed — small, and the Jellyfin ID join is the + authoritative record of which films/paths/X-Ray-dirs were used). +- `trajectories/` — DE trajectories, one JSONL per run (committed — the evidence). +- `results/` — final per-run metrics + the model comparison table (committed). + +## Embedding-model bake-off (July 2026) + +Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest? +Method: **optimize per model** — each model gets its own dumps + gallery + full DE run, +then compare each model at ITS OWN optimum (fairest — no model penalised by another's +threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see +docs/optimizer-experiments.md). + +Models: +| model | file | size | MovieNet rank-1 (prior) | +| ----- | ---- | ---- | ----------------------- | +| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — | +| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% | +| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% | +| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% | + +9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society, +Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface, +Sound of Metal, Valerian. + +## Gallery-mode bake-off (full vs cast-restricted) + +Second axis alongside the model comparison: does restricting the matcher's candidate +set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a +film he's not in) vs. matching against the whole 2418-actor gallery? + +- **full** — match against the entire model gallery (2418 actors). +- **restricted** — per film, match only against its Jellyfin credited cast, filtered + from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production. + +**LIMITATION — Jellyfin stores only ~15 actors per title.** Jellyfin's People list is +capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface: +Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a +query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item +/Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15 +top-billed leads, which caps its achievable recall at whatever fraction of on-screen +actors are top-billed, but should drive out-of-cast misIDs toward zero. A production +deployment wanting fuller cast restriction would need a richer cast source than Jellyfin +(TMDB/IMDb full credits). + +Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline +galleries (no augmentation — avoids test-set leakage on either arm). Scored by the +duration-weighted per-scene metric with the misID split (report_rates.py). diff --git a/experiments/SESSION_STATE.md b/experiments/SESSION_STATE.md new file mode 100644 index 0000000..ff8aeea --- /dev/null +++ b/experiments/SESSION_STATE.md @@ -0,0 +1,105 @@ +# Session state — X-Ray optimizer + model bake-off (as of 2026-07-18) + +Handoff for a fresh session. Everything below is UNCOMMITTED — commit early next session. + +## What we're doing +Optimizing the scene-actor pipeline's thresholds against Amazon X-Ray ground truth, and +running a **model bake-off** (4 embedding models × gallery-mode × expansion) to answer: +is LVFace (455MB) actually best, or just biggest? Does cast-restriction cut false IDs? +Does per-film gallery expansion help? + +## The metric (final form — this is what to use) +`scripts/optimizer/second_score.py` — UNIFORM PER-SECOND sampling vs X-Ray: +- At each second t: GT = X-Ray scene's cast at t; Pred = actors whose window covers t. +- TPI / FPI / FN counted per second. **FPI weighted 10×** when the named actor isn't in + the film's cast at all (a true misID like naming Archie Yates in a film he's not in) vs + an in-cast timing slip. +- **FN is fair**: only counts gallery-known cast (67% of X-Ray cast have no reference + embedding, can't be recognised — see [[gallery-coverage-gap]]). +- **agreement_rate** = mean per-second Jaccard (partial credit: "% of on-screen actors we + agree with X-Ray about, over time"). NOT exact-set match. +- Objective = macro-mean per-second weighted F1. + +## Two DIFFERENT hangs — do not conflate them (corrected 2026-07-18) + +**(a) The self-inflicted 100% hang (FIXED).** replay.py's CLI briefly called +`replay(..., stop=False)` intending to `os._exit(0)` straight after, to "dodge" teardown. +That was wrong: `PyNode::stop()` is the ONLY thing that sets `stop_flag_=true`, which is the +ONLY exit condition for the source node's `run_loop()`. Skipping it meant the local `net` +destructor — which runs synchronously when `replay()` returns, BEFORE main() can reach +os._exit — joined a thread that could never stop. A **guaranteed** hang, not the driver +flake. Symptom: every solo replay timed out at 45s and DE reported flat F1=0.0%. +FIX: `replay(..., stop=True)` so `PyNode::stop()` signals the thread before the join; +removed the dead os._exit / unused os import. VERIFIED: 45s guaranteed timeout → clean ~8s +completion (3/3), and optimize.py's DE sweep returns correct non-zero metrics (F1 48-66%, +matching prior best-so-far). Only 1 isolated per-film timeout in 11 evals × 3 films. + +**(b) The genuine ROCm flake (rare, tolerated).** net.stop()→jthread.join() CAN still hang +on a KPN worker stuck mid-rocBLAS-GEMM — a KNOWN ROCm bug +(github.com/RadeonOpenCompute/ROCT-Thunk-Interface#56), NOT our code. HSA_ENABLE_SDMA=0 +makes it WORSE (breaks the matcher's DMA). It is much rarer than the ~20-30% figure quoted +earlier in this session — that number was inflated by (a). The existing subprocess + 45s +timeout absorbs it correctly. + +## Both architectures are usable +- `scripts/optimizer/optimize.py` + `replay.py` — subprocess per film, simpler, tolerates the + rare true flake via its timeout. NOT broken; good for fallback / quick single-model runs. +- `scripts/optimizer/model_server.py` + `optimize_server.py` — ONE persistent net per + (model, gallery); replay each film by SWITCHING THE SOURCE (repoint frame list + reset + index), change thresholds via runtime SETTERS, os._exit(0) at the very end (after all work, + so no destructor-join problem). Higher throughput: skips gallery/build overhead per eval. + VERIFIED: "ready", replays, emits metrics, ~30-40s/eval (GPU-bound, films serial). + Still the preferred option for the long overnight matrix. + +## Key C++ changes made (all in the KPN spec-and-tsan branch + our nodes) +1. Runtime setters: `IdentityMatcherFunc::set_prob_threshold`, `SceneTrackerFunc::set_extinction_sec` + (src/nodes/*). Exposed via sae_kpn: `set_prob_threshold(net,name,v)`, `set_extinction_sec(...)`. + Needed `ObjectVariantNodeWrapper::functor()` + `PyNetwork::node_ptr()` accessors. +2. `Channel::push_blocking()` (external/KPN/.../channel.hpp) — lossless backpressure push + (waits instead of dropping when full). Exposed on IVariantChannel/VariantChannel; PyNode's + run_loop now uses it. Reduced but did NOT fully fix a residual ~0.5% frame loss (25/5915) + — the loss is elsewhere (matcher output or reader EOF-race). DECISION: accept it, <0.5% + scattered doesn't change per-second F1 or rankings. Don't chase further. +3. `dump_embeddings` standalone exe + `--max-decode-fps` (fixes LVFace dump truncation under + parallel load). HDF5 gallery fast-load in gallery_store.cpp (18s JSON → 0.06s). + `scripts/optimizer/json_to_hdf5_gallery.py` converts; galleries are `.h5` now. + +All of KPN, matcher, scene_tracker, bindings need a rebuild: +`cmake --build build --target sae_kpn sae_gallery dump_embeddings scene_analyze` + +## Data on disk (durable, experiments/) +- `experiments/xray/` — X-Ray Zenodo dataset. `experiments/dumps//dump_.h5` — + all 9 films × 4 models, ALL FULL (LVFace re-dumped with --max-decode-fps 8). VERIFY counts + match R50 before trusting (LVFace truncated under parallel dumping earlier). +- `experiments/galleries/gallery_.h5` (+ restricted//.h5, per-film cast- + filtered to Jellyfin's ~15 top-billed — Jellyfin's hard cap, see experiments/README.md). +- `experiments/manifests/rep3__.json` — 3 REPRESENTATIVE films (Lord of War / + Scarface / Sound of Metal = clean / ensemble-lookalike / high-coverage) to keep evals fast + (~28s vs ~90s for 9). Winner should be re-scored on all 9 after. +- `experiments/manifests/films__.json` — all 9 films. + +## Salvaged partial results (per-second metric) +- R50 full +expand: **F1 66.4%** (208 evals, converged) — best so far +- R50 full noexp: 55-60% → **expansion helps ~+6-11 recall** +- MBF full noexp: 58.6% +- (older scene-metric runs, superseded: R50≈LVFace≈MBF ~85%, restricted>full, LVFace not + worth its size — but those used the OLD scene-union metric, redo with per-second.) + +## TO DO next session +1. **COMMIT everything first** (logical chunks: KPN setters+push_blocking; sae_kpn+dump exe; + HDF5 gallery; optimizer scripts; per-second metric; experiments manifests/results/docs + + tuned config.hpp defaults prob_threshold 0.76 extinction 1.5). +2. Launch the full 16-run matrix via model_server on rep3 films (write trajectories to + experiments/, NOT /tmp — /tmp gets wiped mid-session and cost us hours). ~28s/eval × + ~84 evals × 16 = ~10hr. Runner pattern: experiments/run_overnight_rep3.sh but pointing + optimize_server.py at model_server. +3. assemble table: best model + expansion effect + misID, from experiments/results/*.json. +4. Consider upstreaming to KPN++: runtime node setters, push_blocking, node_ptr/functor(). + +## Gotchas that burned time (don't repeat) +- /tmp scratch gets WIPED mid-session → lost dumps + test files repeatedly. Use experiments/. +- Verify a launched runner script EXISTS and PRODUCES evals before walking away (a heredoc + once silently failed to write; a stale-code process ran the old metric for 12h). +- pgrep/ps "survivors" are often the grep's own shell wrapper — check via /proc cmdline or ps. +- Running many DE/replay processes in parallel on one GPU → deadlock/thrash. GPU peaks ~35% + (not saturated) but concurrency>2-3 wedges. Serial-ish is safer. diff --git a/experiments/build_manifests.py b/experiments/build_manifests.py new file mode 100644 index 0000000..b967004 --- /dev/null +++ b/experiments/build_manifests.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +""" +Build per-(model, mode) manifests for the model × gallery-mode bake-off. + +Modes: + full — every film matches against the whole model gallery (2418 actors) + restricted — each film matches only its Jellyfin credited cast (~15 top-billed), + via a per-film gallery filtered from the model gallery by jellyfin_id. + +Each manifest is a list of {name, xray, slug, dump, gallery} — no "movie" path (that's +resolved locally via experiments/file-lut.json, see run_montage_all.py, to avoid +embedding source filenames in a file that gets shared as an artifact). optimize.py +reads film["gallery"] per film, so restricted mode just points each film at its own +filtered gallery — no optimizer change needed. + +Writes experiments/manifests/films__.json and the restricted galleries to +experiments/galleries/restricted//.h5. +""" +import json +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +sys.path.insert(0, str(REPO / "scripts" / "validation")) +sys.path.insert(0, str(REPO / "scripts")) +from identity import norm_name # noqa: E402 +from sae_gallery import load_gallery_hdf5, save_gallery_hdf5 # noqa: E402 + +MODELS = ["arcface_w600k_r50", "arcface_r18", "arcface_w600k_mbf", "LVFace-B_Glint360K"] + +films = json.loads((REPO / "experiments/manifests/films.json").read_text()) +casts = json.loads((REPO / "experiments/manifests/jellyfin_casts.json").read_text()) + + +def restrict_gallery(model_gallery: dict, cast_ids: set[str]) -> dict: + kept = [a for a in model_gallery["actors"] if a.get("jellyfin_id", "") in cast_ids] + return {"actors": kept} + + +for model in MODELS: + gpath = REPO / f"experiments/galleries/gallery_{model}.h5" + if not gpath.exists(): + print(f"skip {model}: gallery not built yet ({gpath})") + continue + model_gal = load_gallery_hdf5(gpath) + + # full mode + full = [{**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5", + "gallery": f"experiments/galleries/gallery_{model}.h5"} for f in films] + (REPO / f"experiments/manifests/films_{model}_full.json").write_text(json.dumps(full, indent=2, ensure_ascii=False)) + + # restricted mode — per-film filtered gallery + rdir = REPO / f"experiments/galleries/restricted/{model}" + rdir.mkdir(parents=True, exist_ok=True) + restr = [] + for f in films: + cast_ids = set(casts.get(f["name"], [])) + rg = restrict_gallery(model_gal, cast_ids) + rgpath = rdir / f"{f['slug']}.h5" + save_gallery_hdf5(rg, rgpath) + restr.append({**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5", + "gallery": str(rgpath.relative_to(REPO)), + "_cast_size": len(rg["actors"])}) + (REPO / f"experiments/manifests/films_{model}_restricted.json").write_text(json.dumps(restr, indent=2, ensure_ascii=False)) + avg = sum(r["_cast_size"] for r in restr) / len(restr) + print(f"{model}: full (2418) + restricted (avg {avg:.0f} actors/film) manifests written") diff --git a/experiments/file-lut.template.json b/experiments/file-lut.template.json new file mode 100644 index 0000000..7cc4398 --- /dev/null +++ b/experiments/file-lut.template.json @@ -0,0 +1,11 @@ +{ + "Benny___Joon": "/path/to/your/movies/.mp4", + "Café_Society": "/path/to/your/movies/.mp4", + "Downton_Abbey__A_New_Era": "/path/to/your/movies/.mp4", + "Lord_of_War": "/path/to/your/movies/.mp4", + "Lovelace": "/path/to/your/movies/.mp4", + "The_Many_Saints_of_Newark": "/path/to/your/movies/.mp4", + "Scarface": "/path/to/your/movies/.mp4", + "Sound_of_Metal": "/path/to/your/movies/.mp4", + "Valerian_and_the_City_of_a_Thousand_Plan": "/path/to/your/movies/.mp4" +} \ No newline at end of file diff --git a/experiments/run_montage_all.py b/experiments/run_montage_all.py new file mode 100644 index 0000000..22d6dd5 --- /dev/null +++ b/experiments/run_montage_all.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +""" +run_montage_all.py — replay + best/worst-per-scene montage for all films in a +manifest, using the shipped config.hpp defaults (LVFace, prob_threshold=0.754, +anneal_sec=35.54, extinction_sec=57.43, expand_gallery). Skips a film if its +montage manifest.json already exists (safe to re-run/resume). + +Manifests no longer carry a "movie" path (removed to avoid embedding source +filenames — some carry scene-release tags — in a file that gets zipped and +pushed to the artifact registry). Movie paths are resolved locally via +experiments/file-lut.json (gitignored; copy file-lut.template.json and fill +in your own paths). + +Usage: + python experiments/run_montage_all.py \ + --manifest experiments/manifests/films_LVFace-B_Glint360K_full.json \ + --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ + --out-dir experiments/results/holdout/montage +""" +import argparse +import json +import subprocess +import sys +from pathlib import Path + +REPO = Path(__file__).resolve().parent.parent +REPLAY = str(REPO / "scripts" / "optimizer" / "replay.py") +MONTAGE = str(REPO / "scripts" / "optimizer" / "dump_scene_montage.py") +FILE_LUT = REPO / "experiments" / "file-lut.json" + +CFG = {"prob_threshold": "0.7540024664611272", "anneal_sec": "35.53996030397922", + "extinction_sec": "57.43359645269811"} + + +def main(): + p = argparse.ArgumentParser() + p.add_argument("--manifest", required=True) + p.add_argument("--gallery", required=True) + p.add_argument("--out-dir", required=True) + args = p.parse_args() + + if not FILE_LUT.exists(): + sys.exit(f"{FILE_LUT} not found — copy file-lut.template.json to " + f"file-lut.json and fill in your local movie paths") + file_lut = json.loads(FILE_LUT.read_text()) + + films = json.loads(Path(args.manifest).read_text()) + out_root = Path(args.out_dir) + out_root.mkdir(parents=True, exist_ok=True) + + for f in films: + slug = f["slug"] + montage_dir = out_root / slug + manifest_path = montage_dir / "manifest.json" + if manifest_path.exists(): + print(f"[run_montage_all] skip {slug} (already done)", file=sys.stderr) + continue + + movie_path = file_lut.get(slug) + if not movie_path: + print(f"[run_montage_all] skip {slug}: no entry in {FILE_LUT}", file=sys.stderr) + continue + + print(f"[run_montage_all] === {f['name']} ({slug}) ===", file=sys.stderr) + raw_path = out_root / f"raw_{slug}.jsonl" + pred_path = out_root / f"pred_{slug}.json" + + replay_cmd = [sys.executable, REPLAY, "--dump", f["dump"], "--gallery", args.gallery, + "--out", str(pred_path), "--raw-out", str(raw_path), + "--prob-threshold", CFG["prob_threshold"], + "--anneal-sec", CFG["anneal_sec"], + "--extinction-sec", CFG["extinction_sec"], "--expand-gallery"] + r = subprocess.run(replay_cmd, capture_output=True, text=True, timeout=120) + if r.returncode != 0: + print(f"[run_montage_all] replay FAILED for {slug}: {r.stderr[-2000:]}", + file=sys.stderr) + continue + + montage_cmd = [sys.executable, MONTAGE, "--raw", str(raw_path), "--dump", f["dump"], + "--xray", f["xray"], "--movie", movie_path, "--gallery", args.gallery, + "--out-dir", str(montage_dir)] + r = subprocess.run(montage_cmd, capture_output=True, text=True, timeout=1800) + if r.returncode != 0: + print(f"[run_montage_all] montage FAILED for {slug}: {r.stderr[-2000:]}", + file=sys.stderr) + continue + print(r.stderr.strip().splitlines()[-1] if r.stderr else "(no output)", file=sys.stderr) + + print("[run_montage_all] ALL DONE", file=sys.stderr) + + +if __name__ == "__main__": + main() diff --git a/experiments/run_rep4_subprocess.sh b/experiments/run_rep4_subprocess.sh new file mode 100644 index 0000000..337561f --- /dev/null +++ b/experiments/run_rep4_subprocess.sh @@ -0,0 +1,26 @@ +#!/bin/bash +set -u +cd /home/dtourolle/Development/scene-actor-extraction +mkdir -p experiments/trajectories experiments/results +MODELS=(arcface_w600k_r50 arcface_r18 arcface_w600k_mbf LVFace-B_Glint360K) +MODES=(full restricted); EXPAND=(1 0) +for model in "${MODELS[@]}"; do + for mode in "${MODES[@]}"; do + for exp in "${EXPAND[@]}"; do + xtag=$([ "$exp" = 1 ] && echo exp || echo noexp); tag="${model}_${mode}_${xtag}" + best="experiments/results/rep4_best_${tag}.json"; traj="experiments/trajectories/rep4_${tag}.jsonl" + [ -f "$best" ] && { echo "skip $tag"; continue; } + : > "$traj"; echo "=== $tag START $(date '+%H:%M') ===" + SAE_EXPAND=$exp REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \ + --manifest "experiments/manifests/rep4_${model}_${mode}.json" \ + --gallery "experiments/galleries/gallery_${model}.h5" \ + --params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \ + --popsize 10 --maxiter 15 --seed 0 \ + --trajectory "$traj" --out "experiments/results/opt_rep4_${tag}.json" \ + > "experiments/results/rep4_de_${tag}.log" 2>&1 + python3 -c "import json;r=[json.loads(l) for l in open('$traj')];b=max(r,key=lambda x:x['f1']);json.dump({'best':b,'n_evals':len(r)},open('$best','w'))" 2>/dev/null + echo "done $tag $(date '+%H:%M'): $(python3 -c "import json;d=json.load(open('$best'));print('F1=%.1f%% misID=%d'%(d['best']['f1']*100,d['best'].get('FPI_misid',-1)))" 2>/dev/null)" + done + done +done +echo "=== REP4 MATRIX COMPLETE $(date '+%F %H:%M') ===" diff --git a/mkdocs.yml b/mkdocs.yml new file mode 100644 index 0000000..df8e2a6 --- /dev/null +++ b/mkdocs.yml @@ -0,0 +1,42 @@ +site_name: scene-actor-extraction +site_description: Face-recognition pipeline for finding on-screen actor presence in film/TV, built on KPN++ +repo_url: https://gitea.tourolle.paris/dtourolle/scene-actor-extraction +repo_name: dtourolle/scene-actor-extraction + +theme: + name: material + palette: + - scheme: slate + primary: indigo + accent: indigo + features: + - navigation.tabs + - navigation.sections + - navigation.top + - content.code.copy + - content.code.annotate + +nav: + - Home: index.md + - Rep4 Bake-off & Re-tune: rep4-optimizer-results.md + - Optimizer Experiments (prior round): optimizer-experiments.md + - Service Conversion (proposal): service-conversion.md + +markdown_extensions: + - admonition + - toc: + permalink: true + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.superfences + - pymdownx.tabbed: + alternate_style: true + - pymdownx.snippets: + base_path: ['.'] + check_paths: true + - pymdownx.details + - attr_list + - md_in_html diff --git a/models/transnetv2.onnx b/models/transnetv2.onnx new file mode 100644 index 0000000..c6771bc --- /dev/null +++ b/models/transnetv2.onnx @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c4d54a682bace32f25136ef83ca2c9d403e8f8193775efeb995172a0d95a8e0c +size 31250929