# Model bake-off + threshold re-tune — experiment log (2026-07-18/19) Follow-on to [the prior optimizer round](optimizer-experiments.md), which used an older, since-superseded scene-union metric. This round uses the **per-second** metric ([`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)) and answers three questions in one 16-run matrix: which embedding model is best, does cast-restriction cut misIDs, and does per-film gallery expansion help. (This was the fourth optimizer campaign against the X-Ray corpus, so its on-disk artifacts carry an internal `rep4_` prefix — `experiments/results/rep4_best_*.json`, `experiments/trajectories/rep4_*.jsonl`, and the manifests referenced below. The earlier campaigns used the superseded scene-union metric and were discarded.) ## Why this experiment, and what it actually delivered Four goals going in, and an honest read on each after held-out validation (see below): 1. **Find the best default parameters to ship.** Partially delivered. The DE optimum generalizes *unevenly* — strong on 3 of 5 held-out films, badly broken on 2 (one with a 974-count misID blowup). The tuned values are shipped anyway (see Caveats) because they still beat the old defaults on average, but this is not a settled, film-agnostic optimum. 2. **Find the best default model.** Delivered with more confidence. LVFace beat r50/r18/mbf across all 4 training combos, and nothing in held-out validation contradicts the model choice specifically — the held-out failures trace to `extinction_sec`/threshold interactions and gallery coverage, not the embedder. 3. **Provide insight into how the application works.** The strongest, most durable output. Found and fixed a real teardown deadlock bug (100% reproducible, not the assumed rare GPU flake), established a real concurrency ceiling (8 parallel replays, not more), and found a real parameter interaction (a strict `prob_threshold` "earns" a longer extinction window before it starts hurting). 4. **Demonstrate limitations.** Delivered, and reinforced hard by held-out validation — see the "Held-out validation" section below for concrete examples, including a screenshot of the matcher naming 15 actors, none correctly, on a completely blank title card. ## TL;DR — what changed in `src/config.hpp` | knob | old default | new default | why | | ---- | ----------- | ----------- | --- | | `arcface_model` | `arcface_w600k_r50.onnx` | **`LVFace-B_Glint360K.onnx`** | Best F1 in the full-gallery bake-off (75.3% vs r50's 68.5%). LVFace was worth its size. | | `prob_threshold` | 0.76 | **0.754** | Re-tuned for LVFace + per-second metric. | | `extinction_sec` | 1.5 | **57.4** | Reverses the earlier "short is better" finding — see below. | | `anneal_sec` | 10.0 | **35.5** | Same reversal; previously thought insensitive. | | `expand_gallery` | false | **true** | Helped recall on the full (unrestricted) gallery for the winning model — opposite of the earlier assumption. | These are the **`LVFace-B_Glint360K_full_exp`** winning values, applied to [`src/config.hpp`](https://REPOLINK/src/config.hpp) — the best result that uses only features already live in the running app (full gallery, no cast restriction; see below for why restricted mode isn't applied even though it scored higher). ## Why re-run at all [The prior round](optimizer-experiments.md)'s scene-union metric hid out-of-cast false positives behind a gallery∩cast recall mask, and the earlier 9-film benchmark was scene-level (union over a whole X-Ray scene), not a fair per-timepoint comparison. SESSION_STATE flagged the old scene-metric bake-off numbers (R50≈LVFace≈MBF ~85%) as superseded. This round uses `second_score.py`: uniform per-second sampling, GT = X-Ray scene's cast at time *t*, pred = actors whose presence window covers *t*, FPI weighted 10× when the named actor isn't in the film's cast at all (true misID) vs. an in-cast timing slip. FN only counts gallery-known cast (fair recall — 67% of X-Ray cast have no reference embedding; see [the prior round's gallery-coverage-gap analysis](optimizer-experiments.md#the-gallery-coverage-gap)). ## The deadlock that was blocking all of this Every replay in this line of work goes through [`scripts/optimizer/replay.py`](https://REPOLINK/scripts/optimizer/replay.py), which runs the real C++ tracker/matcher/scene_tracker nodes inside a Python-assembled KPN network. Before this session, every subprocess replay **timed out at 45s, 100% of the time** — not the documented ~20-30% ROCm rocBLAS-GEMM driver flake, but a plain logic bug: `replay.py`'s CLI called `replay(net, ..., stop=False)` to *skip* `net.stop()` (trying to dodge the GEMM deadlock), planning to `os._exit(0)` immediately after. But: - `PyNode::stop()` ([`include/kpn/python/bindings.hpp`](https://KPNLINK/include/kpn/python/bindings.hpp) in the KPN++ submodule) is the *only* code that sets `stop_flag_ = true` before joining the node's worker thread. - The source node's `run_loop()` has `while (!stop_flag_)` as its only exit condition (it has no input channels, so it never sees a channel-closed signal either). - Skipping `stop()` meant `stop_flag_` never became true. When `replay()` returned, its local `net` went out of scope immediately, running `~PyNetwork` → `~PyNode` → `thread_.join()` **synchronously inside `replay()`'s own call frame** — before `main()` ever got control back to run `os._exit(0)`. Root-caused via `gdb -p -batch -ex "thread apply all bt"` on a hung process: the main thread was stuck in `~PyNode`'s `jthread::join()`; the worker thread was in an ordinary `time.sleep()` inside the Python source callback, waiting for a stop signal that was never sent. The two HSA `kfd_wait_on_events` threads visible in the same trace are normal ROCm runtime housekeeping, not evidence of a wedged GPU kernel. **Fix:** `replay.py` now calls `replay(..., stop=True)` (the removed `stop=False` + `os._exit` workaround was actively harmful). Verified 3/3 clean runs at ~8s each (down from a guaranteed 45s timeout), and a full DE sweep producing real, sensible F1/precision/recall instead of flat 0.0%. ## Concurrency tuning With the deadlock fixed, [`scripts/optimizer/optimize.py`](https://REPOLINK/scripts/optimizer/optimize.py) was extended with DE-level parallelism — `differential_evolution(..., workers=ThreadPoolExecutor.map)` — so multiple population candidates evaluate concurrently, each spawning its own per-film replay subprocesses (`REPLAY_WORKERS`). Total concurrent GPU replay processes ≈ `DE_WORKERS × REPLAY_WORKERS`. | concurrent replays | result | | --- | --- | | 3 (`REPLAY_WORKERS=3`, no DE parallelism) | baseline, GPU underutilised | | 6 (`DE_WORKERS=2 × REPLAY_WORKERS=3`) | clean, real scores, ~1 isolated timeout per run | | 8 (`DE_WORKERS=2 × REPLAY_WORKERS=4`, 4-film manifest) | clean, real scores | | 9 (`DE_WORKERS=3 × REPLAY_WORKERS=3`) | **broken** — every replay blew past the 45s timeout, all scores silently degraded to 0.0% | 9 concurrent replays looks like valid output (well-formed JSON, a real number) while actually being garbage — a dangerous failure mode, not a crash. **8 concurrent is the practical ceiling** on this GPU (gfx1100) for this workload. The matrix ran at `REPLAY_WORKERS=4 DE_WORKERS=2`. ## Training films and validation set 9 films total have dumped embeddings across all 4 models. 4 were used for optimization, leaving 5 held out for validation: - **Lord of War** (64-cast, "clean") - **Scarface** (67-cast, "ensemble/lookalike") - **Sound of Metal** (14-cast, "high gallery-coverage") - **Café Society** (62-cast, added this round — similar ensemble size to Scarface but different genre/lighting; picked to add diversity, not genre-overlap, over Downton Abbey or The Many Saints of Newark) Held out: Benny & Joon, Downton Abbey: A New Era, Lovelace, The Many Saints of Newark, Valerian and the City of a Thousand Planets. The training-set numbers below are training-set fit — see "Held-out validation" further down for the real generalization test. ## Search space and DE settings `popsize=10, maxiter=15` (3 params → ≤480 evals/combo ceiling; DE's `tol` convergence usually stops earlier). `anneal_sec`/`extinction_sec` bounds were **widened from 1–30/1–15 to 1–60/1–60 mid-run** (see below) — the 4 `arcface_w600k_r50` combos finished before the widening and still use the old, narrower bounds, so they are **not directly comparable** to the other 12 on those two params. Re-running r50 with the wider bounds was deferred (diminishing-returns judgment call, not yet done). ## Results — all 16 combos (4 models × {full, restricted} × {expand, noexp}) Ranked by F1. `misid` = FPI_misid, count of true wrong-actor identifications (an actor named who isn't in the film's cast at all) — distinct from `FPI`, which includes in-cast timing slips. | combo | F1 | P | R | TPI | FPI | misid | FN | |---|---|---|---|---|---|---|---| | LVFace-B_Glint360K_restricted_exp | **78.3%** | 91.0% | 68.9% | 42830 | 3782 | 60 | 19492 | | LVFace-B_Glint360K_restricted_noexp | 76.7% | 91.5% | 66.2% | 41149 | 3400 | 59 | 21173 | | arcface_w600k_mbf_restricted_exp | 76.5% | 90.7% | 66.3% | 41270 | 4431 | **0** | 21052 | | arcface_r18_restricted_exp | 75.5% | 87.6% | 66.5% | 41399 | 5666 | 60 | 20923 | | **LVFace-B_Glint360K_full_exp** | **75.3%** | 89.7% | 65.4% | 47757 | 3407 | 232 | 26966 | | arcface_w600k_mbf_restricted_noexp | 75.0% | 91.1% | 63.9% | 39752 | 3465 | 60 | 22570 | | arcface_w600k_mbf_full_noexp | 74.2% | 87.4% | 64.4% | 12645 | 1312 | 57 | 6985 | | arcface_r18_restricted_noexp | 73.5% | 91.3% | 61.7% | 38299 | 3220 | 60 | 24023 | | LVFace-B_Glint360K_full_noexp | 72.4% | 94.2% | 58.9% | 27077 | 1725 | **0** | 19506 | | arcface_w600k_mbf_full_exp | 72.0% | 87.7% | 61.4% | 39875 | 3729 | 240 | 26338 | | arcface_w600k_r50_full_noexp † | 71.6% | 96.7% | 56.9% | 22471 | 361 | 45 | 17012 | | arcface_w600k_r50_restricted_exp † | 71.1% | 96.5% | 56.4% | 34954 | 1119 | 15 | 27368 | | arcface_w600k_r50_restricted_noexp † | 69.2% | 97.9% | 53.6% | 21146 | 327 | 15 | 18337 | | arcface_r18_full_exp | 69.1% | 87.6% | 57.7% | 37342 | 3119 | 242 | 28871 | | arcface_w600k_r50_full_exp † | 68.5% | 94.0% | 54.1% | 34982 | 903 | 150 | 31231 | | arcface_r18_full_noexp | 66.6% | 91.3% | 53.1% | 34314 | 2362 | 107 | 31899 | † old, narrower anneal/extinction bounds (see above) — not directly comparable to the other 12 on those two params. The same 16 results as a picture — the two headline effects are visible without reading a single row: filled (restricted) dots stack the top of the ranking for every model color, and yellow (LVFace) leads within both scopes: ![All 16 bake-off combos ranked by training-set F1](assets/images/rep4_matrix_f1.png) ## Calibration curves — discriminative power, independent of the threshold Each model's gallery carries a fitted Platt sigmoid `P(match | sim) = σ(a·sim + b)` (embedded directly in the gallery HDF5, see [`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)). Plotting all four side by side shows discriminative power directly, independent of whatever `prob_threshold` a particular run happened to use: ![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png) LVFace-B has both the steepest curve (`a=17.7`, vs. 15.3–16.2 for the ArcFace variants) and the lowest P=0.5 decision boundary (similarity 0.23 vs. 0.27–0.31) — it separates same-actor from different-actor pairs more confidently at a lower similarity, consistent with it winning the full-gallery F1 comparison below. Generated by [`scripts/docs/calibration_chart.py`](https://REPOLINK/scripts/docs/calibration_chart.py) (requires each gallery to have been calibrated at least once — run any replay against it first). ## Two effects in isolation: gallery scope, and pose expansion The matrix crosses two independent variables — averaging across all 4 models isolates each one from model choice: **Gallery scope (whole 2418-actor gallery vs. restricted to the film's credited cast)** — averaged over both expansion settings and all 4 models: | scope | F1 | P | R | total misID (16 evals→8 each) | |---|---|---|---|---| | full | 71.2% | 91.1% | 59.0% | 1073 | | **restricted** | **74.5%** | 92.2% | **62.9%** | **329** | Restriction wins outright on every axis — not a precision/recall trade, a clean win: **+3.3pp F1, +3.9pp recall, and less than a third the total misIDs.** Fewer candidates in the matcher's search space means fewer opportunities for a look-alike false match, and (per the recall gain) doesn't cost real detections. This is the single cleanest signal in the whole matrix — stronger than the model choice itself — which is exactly why cast-restriction becoming a real runtime feature (not just an optimizer trick) is the top item in Caveats below. **Pose expansion (promoting a confidently-identified track's novel-pose views into a per-film gallery annex — [`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp))** is smaller and interacts with scope rather than acting independently: | scope | expansion | F1 | R | misID | |---|---|---|---|---| | full | off | 71.2% | 58.3% | 209 | | full | **on** | 71.2% | 59.7% | **864** | | restricted | off | 73.6% | 61.3% | 194 | | restricted | **on** | **75.4%** | **64.5%** | 135 | In **restricted** mode, expansion is a clean win (+1.8pp F1, +3.2pp recall, misID actually *drops*) — the annex only ever competes against the film's own ~15-actor cast, so a "confidently identified, new pose" view is unlikely to be mistaken for someone else. In **full** mode, expansion buys essentially nothing on F1 (71.2% → 71.2%, recall +1.4pp) while **quadrupling misIDs** (209 → 864): a novel-pose view promoted into the annex now competes against the whole 2418-actor gallery, so a "confident" identity is confident against the wrong universe of candidates — the expansion mechanism is "learning" a pose correctly, but the enlarged evidence pool makes it easier for that learned pose to look like a plausible match for a different actor. **Practical takeaway: gallery expansion should be paired with cast restriction, not used on the full gallery** — the version currently shipped as default (`full_exp`, see TL;DR) sits in the worse of these four cells for this specific knob, even though it's the best available combo without cast-restriction support in the app yet (see Caveats). ## What the data says - **LVFace was worth its size.** It wins full-gallery mode outright (75.3% vs r50's 68.5%, r18's 69.1%, mbf's 72.0%) with the highest recall of any full-mode combo — the earlier scene-union-metric conclusion ("not worth it") doesn't survive the better metric. - **Cast-restriction is a consistent, broad win.** Every model's best combo is `restricted`. It isn't just precision-safe: `arcface_w600k_mbf_restricted_exp` and `LVFace-B_Glint360K_full_noexp` both hit **misid=0** — zero true wrong-actor identifications. But restriction is an **offline optimizer technique, not a live app feature** — it pre-filters each film's gallery to its Jellyfin-credited cast before the matcher ever runs; there's no runtime "restrict to this film's cast" switch in the app today. Implementing it for real is future work, tracked separately from this defaults update. - **Gallery expansion (`expand_gallery`) is mode-dependent.** It helps on `restricted` galleries (smaller, so novel-pose promotion adds real signal) and on LVFace's full gallery, but **hurts** r50 and mbf in full mode (compare `arcface_w600k_r50_full_exp` 68.5% vs `full_noexp` 71.6%). Don't assume it's a free win — model- and mode-dependent. - **arcface_r18 (smallest/cheapest) is last across all 4 modes** — model capacity matters here, this isn't just parameter-count padding. - **`anneal_sec`/`extinction_sec` kept pinning at the search ceiling.** With the original 1–30/1–15 bounds, 3 of 4 r50 combos landed at ~93-98% of the upper bound. Widened to 1–60/1–60 mid-run (after the r50 combos had already finished) — every subsequent combo's best config landed at ~90%+ of the *new* ceiling too (e.g. the LVFace winner: `ann=59.2, ext=59.2`, both ~99% of 60). The likely mechanism: a strict `prob_threshold` "earns" a long extinction/anneal window — once false matches are rare, a long window just bridges real presence gaps (occlusion, turned face) instead of smearing false positives into later scenes, which is what made short windows look better under the old, laxer thresholds. **Open question, not resolved**: does this keep climbing past 60s, or does it actually plateau there? Decided not to chase further this round (diminishing-returns judgment call) — flag for a future sweep if it matters. The ceiling-pinning is visible in the raw search itself. Every one of the 512 DE evaluations for the winning combo, plotted over the `prob_threshold` × `extinction_sec` plane: ![DE search landscape: 512 evaluations over prob_threshold × extinction_sec](assets/images/de_search_landscape.png) The dark band hugging the top edge *is* the finding: nearly everything scoring well sits at `extinction_sec` ≥ 50, across a wide range of thresholds, and the population converged into a dense cloud around the optimum (threshold ~0.70–0.80, extinction pinned at the 60s bound). Short extinction windows (bottom half) are uniformly pale — under a strict threshold there is simply no good configuration down there. Generated by [`scripts/docs/experiment_charts.py`](https://REPOLINK/scripts/docs/experiment_charts.py) from the DE trajectories (`experiments/trajectories/*.jsonl`, part of the `experiment-data` artifact package). ## Held-out validation — the number that actually matters The 16-combo matrix above is training-set fit. This is the real test: the shipped config (`LVFace-B_Glint360K_full_exp` — `prob_threshold=0.754, anneal_sec=35.5, extinction_sec=57.4, expand_gallery=true`) replayed against the **5 films never seen by the optimizer** (Benny & Joon, Downton Abbey: A New Era, Lovelace, The Many Saints of Newark, Valerian and the City of a Thousand Planets), scored the same way. | film | F1 | P | R | agree | TPI | FPI | misid | FN | |---|---|---|---|---|---|---|---|---| | Benny & Joon | 83.0% | 89.1% | 77.7% | 72.4% | 15125 | 1846 | 0 | 4337 | | Lovelace | 77.5% | 90.3% | 67.9% | 72.1% | 14990 | 1085 | 58 | 7085 | | Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 58.8% | 18663 | 548 | 0 | 12467 | | Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 40.6% | 52027 | 1173 | 0 | 80084 | | **The Many Saints of Newark** | **46.3%** | **54.7%** | 40.1% | 37.0% | 15922 | 4394 | **974** | 23791 | | **macro average (5 films)** | **67.4%** | 85.8% | 57.0% | 56.2% | 116727 | 9046 | 1032 | 127764 | ![Held-out per-film F1 vs. the training-set fit](assets/images/holdout_f1_by_film.png) **67.4% held out vs. 75.3% on training** — an ~8pp drop, and a much more informative number than the training-set F1 alone: a **37pp spread between best and worst film** (83.0% vs 46.3%). The config does not generalize uniformly. Two films are outright failure cases, and rendering bounding boxes + names on the extracted frames (`replay.py --raw-out` + [`dump_error_frames.py`](https://REPOLINK/scripts/optimizer/dump_error_frames.py)` --raw`, see Reproduce) turned what looked like a same-scene misidentification into something more precise and more damning: - **The Many Saints of Newark** (mob-family drama, picked as a training-adjacent genre test) has **974 true misIDs** — far more than any training combo saw at any setting. The annotated frame below shows the same mechanism as Downton Abbey, at smaller scale: **"Jon Bernthal 100%", "Joey Diaz 100%", and "Billy Magnussen 100%" are all frozen boxes over empty background — a blurred wall, hanging plates — with no face in them at all.** Only one real face in frame has a box, and it carries a *second*, colliding label ("Leslie Odom Jr." and "Michael Gandolfini" both at high confidence on the same box) — likely two tracks whose frozen bboxes happen to overlap. ![Frozen ghost boxes over background, The Many Saints of Newark](assets/images/many_saints_ghost_fpi.jpg) *Frame `many_saints/fpi/fpi_t03543.jpg` from the `montage-frames` artifact package (`scripts/artifacts/pull_artifacts.sh montage-frames Many_Saints_of_Newark`).* - **Downton Abbey: A New Era** (large ensemble, 36-cast) has high precision (97.8%) but recall collapses to 39.4% (FN=80084, by far the largest of the 5). Its starkest failure happens where there is nothing to see at all: the film's hard cut into its closing credits, where **the matcher kept reporting 15 actors — all wrong — for nearly a minute of faceless screen.** Both are the same mechanism, and it can be *measured*, not just screenshotted. Plotting the dump's own per-second `face_count` (detector output, independent of the tracker) against the number of actors the tracker reports, through Downton Abbey's cut to credits: ![Detector face_count vs. tracker-reported actors through the cut to credits](assets/images/downton_ghost_timeline.png) From the cut onward the detector sees **zero faces** — yet the tracker holds a perfectly flat plateau of 15 reported identities for 56 seconds, each with the *exact same bbox, unchanged to the pixel* (verified for Hugh Bonneville: `(1743.2, 0.0, 171.3, 317.8)` at every sampled second from 7222 through 7279+). The staircase on the right edge is the extinction window finally expiring, actor by actor. That plateau is `SceneTrackerFunc`'s `active_[actor_idx].last_bbox` ([`src/nodes/scene_tracker_node.hpp`](https://REPOLINK/src/nodes/scene_tracker_node.hpp)) being re-emitted unchanged — **the extinction state machine working exactly as coded**, not a bug in the logic. The film cuts from a packed group shot straight into ~40+ seconds of blank titles/credits with zero faces, and `extinction_sec=57.4` is comfortably long enough to bridge that entire gap without expiring, so the tracker faithfully keeps reporting "last known position" for a cast that is no longer on screen at all. This reframes the "long extinction window wins" DE-search pattern (see above): it isn't unambiguously good. It buys recall by bridging real gaps (occlusion, turned face) in some films, but on others — specifically, hard cuts into long faceless footage — it manufactures a frozen-bbox ghost the tracker has no way to verify, precisely the failure mode the *original* short-extinction-window default (`1.5s`) was chosen to avoid. The training-set films apparently didn't have a long enough faceless stretch after a confirmed identity to expose this; the held-out set did. Frames for all three films (`benny_joon`, `many_saints`, `downton_abbey` — one strong performer, two failure cases) are under `experiments/results/holdout/frames/` (not committed — pull per film with `scripts/artifacts/pull_artifacts.sh montage-frames `), each with a `manifest.json` listing the bucket (`best`/`fpi`/`fn`), timestamp, and predicted vs. ground-truth actors for every dumped frame. Frames are annotated with bounding boxes + name/confidence (green = identified, orange = unknown), matching [`src/nodes/debug_renderer_node.hpp`](https://REPOLINK/src/nodes/debug_renderer_node.hpp)'s colour convention. Generated by `scripts/optimizer/dump_error_frames.py --raw ` (see Reproduce). `dump_error_frames.py --interval-sec 600` also supports a per-N-second sweep instead of the fixed best/fpi/fn buckets: one best (highest Jaccard) and one worst (lowest Jaccard) frame per 10-minute window across the whole film, e.g. `experiments/results/holdout/frames/many_saints_intervals/` (13 windows × 2 = 26 frames for the ~2h Many Saints runtime) — a way to sample "how are we doing" evenly across a film's runtime rather than only at its most extreme seconds. ## Caveats / what this is not - **r50's 4 combos used the old, narrower search bounds** and aren't fully comparable to the other 12 on `anneal_sec`/`extinction_sec`. - **The applied defaults use `full_exp`, not the higher-scoring `restricted_exp`**, because cast-restriction isn't a real runtime feature yet (see above). The 78.3% F1 number is not what the shipped defaults will produce — 75.3% is. - **`full_exp` is the best full-gallery combo, but not the safest.** Per the isolated-effects analysis above, `expand_gallery=true` only cleanly pays off when paired with cast-restriction; on the full gallery it's flat on F1 while ~4x-ing misIDs (209→864, averaged across models). `full_noexp` scores lower (72.4% vs 75.3% for LVFace) but with **zero** true misIDs and higher precision (94.2% vs 89.7%). Kept `full_exp` as shipped since it's the highest-F1 option available without cast-restriction, but this is a real F1-vs-safety trade, not a strictly-better choice — worth revisiting if misID rate matters more than the last few points of F1 for a given deployment. - **Switching the default model is an operational change, not just a config tweak**: any existing gallery built from r50 embeddings is incompatible with LVFace embeddings and needs rebuilding. ## Reproduce ```bash # 4-film matrix, all 4 models × 2 modes × 2 expansion settings bash experiments/run_rep4_subprocess.sh # single combo SAE_EXPAND=1 REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \ --manifest experiments/manifests/rep4_LVFace-B_Glint360K_full.json \ --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ --params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \ --popsize 10 --maxiter 15 --trajectory traj.jsonl --out best.json # replay the shipped config against a held-out film — --raw-out is needed to draw # bboxes later (the merged pred.json has no per-frame bbox, only actor windows) python3 scripts/optimizer/replay.py \ --dump experiments/dumps/LVFace-B_Glint360K/dump_.h5 \ --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ --out pred.json --raw-out raw.jsonl --prob-threshold 0.754 --anneal-sec 35.54 \ --extinction-sec 57.43 --expand-gallery # regenerate the report's charts (16-combo ranking, DE landscape, held-out # per-film F1, Downton ghost timeline) from the artifacts under experiments/ python3 scripts/docs/experiment_charts.py --out-dir docs/assets/images # dump example frames (best-agreement / FPI / FN) for visual inspection, annotated # with bounding boxes + names (--raw is optional; omit for unannotated frames) python3 scripts/optimizer/dump_error_frames.py \ --pred pred.json --raw raw.jsonl --xray experiments/xray/.../ \ --movie "" \ --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ --out-dir experiments/results/holdout/frames/ --n-per-bucket 4 # or: one best + one worst frame per 10-minute window across the whole film python3 scripts/optimizer/dump_error_frames.py \ --pred pred.json --raw raw.jsonl --xray experiments/xray/.../ \ --movie "" \ --gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \ --out-dir experiments/results/holdout/frames/_intervals --interval-sec 600 ``` See also: [the prior optimizer round](optimizer-experiments.md) (superseded metric) and the session log [`experiments/SESSION_STATE.md`](https://REPOLINK/experiments/SESSION_STATE.md).