docs: rep4 bake-off write-up, MkDocs site, artifact-registry-backed experiments
docs/rep4-optimizer-results.md is the main deliverable: the model bake-off + threshold re-tune experiment log, including the ROCm teardown deadlock root cause and fix, DE concurrency tuning, the 16-combo results table, held-out validation against 5 films never seen by the optimizer (macro F1 67.4% vs. 75.3% training — a real generalization gap), the frozen-bbox "ghost track" failure mode found via annotated frame evidence, calibration curves per model, and an isolated-effects breakdown of gallery scope vs. pose expansion. MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/ pulls referenced images from the artifact registry and generates the calibration chart at build time (see the tooling commit) rather than committing images to the repo. experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories, manifests, results) moved to the Gitea package registry. film-lut.template.json is the committed placeholder for the gitignored file-lut.json (real local movie paths, never shared — some source filenames carry scene-release tags). Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for the new scene-detection path.
This commit is contained in:
@@ -0,0 +1,21 @@
|
||||
# experiments/ in git keeps only scripts, README.md, SESSION_STATE.md, and this
|
||||
# file. Every data artifact — galleries, embedding dumps, the X-Ray corpus,
|
||||
# montage/frame images, DE trajectories, film manifests, and result summaries —
|
||||
# is pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to the Gitea
|
||||
# generic package registry instead (see docs/rep4-optimizer-results.md).
|
||||
xray/
|
||||
dumps/
|
||||
galleries/
|
||||
*.h5
|
||||
manifests/
|
||||
trajectories/
|
||||
results/
|
||||
|
||||
# Raw run logs and scratch scripts (regenerated by every run).
|
||||
_scratch/
|
||||
|
||||
# Real local media paths (film slug -> path on this machine). Never committed —
|
||||
# these paths embed the specific source file names, which can include
|
||||
# scene-release tags. Only file-lut.template.json (placeholders) is tracked;
|
||||
# copy it to file-lut.json and fill in your own paths.
|
||||
file-lut.json
|
||||
@@ -0,0 +1,59 @@
|
||||
# experiments/ — X-Ray validation & optimizer artifacts
|
||||
|
||||
Durable home (in the repo tree, NOT `/tmp` scratch — a scratch wipe once cost an hour)
|
||||
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
|
||||
|
||||
## Layout
|
||||
- `xray/` — Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).
|
||||
- `dumps/` — per-model embedding dumps, one HDF5 per (model, film). Gitignored (large).
|
||||
Naming: `<model>/dump_<Film>.h5`. Regenerate with `scene_analyze --dump-embeddings`.
|
||||
- `galleries/` — per-model galleries (gitignored JSON). `gallery_<model>.json` +
|
||||
augmented variants. Regenerate with build_gallery / fetch_missing_actors.
|
||||
- `manifests/` — film manifests (committed — small, and the Jellyfin ID join is the
|
||||
authoritative record of which films/paths/X-Ray-dirs were used).
|
||||
- `trajectories/` — DE trajectories, one JSONL per run (committed — the evidence).
|
||||
- `results/` — final per-run metrics + the model comparison table (committed).
|
||||
|
||||
## Embedding-model bake-off (July 2026)
|
||||
|
||||
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest?
|
||||
Method: **optimize per model** — each model gets its own dumps + gallery + full DE run,
|
||||
then compare each model at ITS OWN optimum (fairest — no model penalised by another's
|
||||
threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see
|
||||
docs/optimizer-experiments.md).
|
||||
|
||||
Models:
|
||||
| model | file | size | MovieNet rank-1 (prior) |
|
||||
| ----- | ---- | ---- | ----------------------- |
|
||||
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
|
||||
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
|
||||
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
|
||||
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
|
||||
|
||||
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society,
|
||||
Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface,
|
||||
Sound of Metal, Valerian.
|
||||
|
||||
## Gallery-mode bake-off (full vs cast-restricted)
|
||||
|
||||
Second axis alongside the model comparison: does restricting the matcher's candidate
|
||||
set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a
|
||||
film he's not in) vs. matching against the whole 2418-actor gallery?
|
||||
|
||||
- **full** — match against the entire model gallery (2418 actors).
|
||||
- **restricted** — per film, match only against its Jellyfin credited cast, filtered
|
||||
from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
|
||||
|
||||
**LIMITATION — Jellyfin stores only ~15 actors per title.** Jellyfin's People list is
|
||||
capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface:
|
||||
Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a
|
||||
query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item
|
||||
/Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15
|
||||
top-billed leads, which caps its achievable recall at whatever fraction of on-screen
|
||||
actors are top-billed, but should drive out-of-cast misIDs toward zero. A production
|
||||
deployment wanting fuller cast restriction would need a richer cast source than Jellyfin
|
||||
(TMDB/IMDb full credits).
|
||||
|
||||
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline
|
||||
galleries (no augmentation — avoids test-set leakage on either arm). Scored by the
|
||||
duration-weighted per-scene metric with the misID split (report_rates.py).
|
||||
@@ -0,0 +1,105 @@
|
||||
# Session state — X-Ray optimizer + model bake-off (as of 2026-07-18)
|
||||
|
||||
Handoff for a fresh session. Everything below is UNCOMMITTED — commit early next session.
|
||||
|
||||
## What we're doing
|
||||
Optimizing the scene-actor pipeline's thresholds against Amazon X-Ray ground truth, and
|
||||
running a **model bake-off** (4 embedding models × gallery-mode × expansion) to answer:
|
||||
is LVFace (455MB) actually best, or just biggest? Does cast-restriction cut false IDs?
|
||||
Does per-film gallery expansion help?
|
||||
|
||||
## The metric (final form — this is what to use)
|
||||
`scripts/optimizer/second_score.py` — UNIFORM PER-SECOND sampling vs X-Ray:
|
||||
- At each second t: GT = X-Ray scene's cast at t; Pred = actors whose window covers t.
|
||||
- TPI / FPI / FN counted per second. **FPI weighted 10×** when the named actor isn't in
|
||||
the film's cast at all (a true misID like naming Archie Yates in a film he's not in) vs
|
||||
an in-cast timing slip.
|
||||
- **FN is fair**: only counts gallery-known cast (67% of X-Ray cast have no reference
|
||||
embedding, can't be recognised — see [[gallery-coverage-gap]]).
|
||||
- **agreement_rate** = mean per-second Jaccard (partial credit: "% of on-screen actors we
|
||||
agree with X-Ray about, over time"). NOT exact-set match.
|
||||
- Objective = macro-mean per-second weighted F1.
|
||||
|
||||
## Two DIFFERENT hangs — do not conflate them (corrected 2026-07-18)
|
||||
|
||||
**(a) The self-inflicted 100% hang (FIXED).** replay.py's CLI briefly called
|
||||
`replay(..., stop=False)` intending to `os._exit(0)` straight after, to "dodge" teardown.
|
||||
That was wrong: `PyNode::stop()` is the ONLY thing that sets `stop_flag_=true`, which is the
|
||||
ONLY exit condition for the source node's `run_loop()`. Skipping it meant the local `net`
|
||||
destructor — which runs synchronously when `replay()` returns, BEFORE main() can reach
|
||||
os._exit — joined a thread that could never stop. A **guaranteed** hang, not the driver
|
||||
flake. Symptom: every solo replay timed out at 45s and DE reported flat F1=0.0%.
|
||||
FIX: `replay(..., stop=True)` so `PyNode::stop()` signals the thread before the join;
|
||||
removed the dead os._exit / unused os import. VERIFIED: 45s guaranteed timeout → clean ~8s
|
||||
completion (3/3), and optimize.py's DE sweep returns correct non-zero metrics (F1 48-66%,
|
||||
matching prior best-so-far). Only 1 isolated per-film timeout in 11 evals × 3 films.
|
||||
|
||||
**(b) The genuine ROCm flake (rare, tolerated).** net.stop()→jthread.join() CAN still hang
|
||||
on a KPN worker stuck mid-rocBLAS-GEMM — a KNOWN ROCm bug
|
||||
(github.com/RadeonOpenCompute/ROCT-Thunk-Interface#56), NOT our code. HSA_ENABLE_SDMA=0
|
||||
makes it WORSE (breaks the matcher's DMA). It is much rarer than the ~20-30% figure quoted
|
||||
earlier in this session — that number was inflated by (a). The existing subprocess + 45s
|
||||
timeout absorbs it correctly.
|
||||
|
||||
## Both architectures are usable
|
||||
- `scripts/optimizer/optimize.py` + `replay.py` — subprocess per film, simpler, tolerates the
|
||||
rare true flake via its timeout. NOT broken; good for fallback / quick single-model runs.
|
||||
- `scripts/optimizer/model_server.py` + `optimize_server.py` — ONE persistent net per
|
||||
(model, gallery); replay each film by SWITCHING THE SOURCE (repoint frame list + reset
|
||||
index), change thresholds via runtime SETTERS, os._exit(0) at the very end (after all work,
|
||||
so no destructor-join problem). Higher throughput: skips gallery/build overhead per eval.
|
||||
VERIFIED: "ready", replays, emits metrics, ~30-40s/eval (GPU-bound, films serial).
|
||||
Still the preferred option for the long overnight matrix.
|
||||
|
||||
## Key C++ changes made (all in the KPN spec-and-tsan branch + our nodes)
|
||||
1. Runtime setters: `IdentityMatcherFunc::set_prob_threshold`, `SceneTrackerFunc::set_extinction_sec`
|
||||
(src/nodes/*). Exposed via sae_kpn: `set_prob_threshold(net,name,v)`, `set_extinction_sec(...)`.
|
||||
Needed `ObjectVariantNodeWrapper::functor()` + `PyNetwork::node_ptr()` accessors.
|
||||
2. `Channel::push_blocking()` (external/KPN/.../channel.hpp) — lossless backpressure push
|
||||
(waits instead of dropping when full). Exposed on IVariantChannel/VariantChannel; PyNode's
|
||||
run_loop now uses it. Reduced but did NOT fully fix a residual ~0.5% frame loss (25/5915)
|
||||
— the loss is elsewhere (matcher output or reader EOF-race). DECISION: accept it, <0.5%
|
||||
scattered doesn't change per-second F1 or rankings. Don't chase further.
|
||||
3. `dump_embeddings` standalone exe + `--max-decode-fps` (fixes LVFace dump truncation under
|
||||
parallel load). HDF5 gallery fast-load in gallery_store.cpp (18s JSON → 0.06s).
|
||||
`scripts/optimizer/json_to_hdf5_gallery.py` converts; galleries are `.h5` now.
|
||||
|
||||
All of KPN, matcher, scene_tracker, bindings need a rebuild:
|
||||
`cmake --build build --target sae_kpn sae_gallery dump_embeddings scene_analyze`
|
||||
|
||||
## Data on disk (durable, experiments/)
|
||||
- `experiments/xray/` — X-Ray Zenodo dataset. `experiments/dumps/<model>/dump_<slug>.h5` —
|
||||
all 9 films × 4 models, ALL FULL (LVFace re-dumped with --max-decode-fps 8). VERIFY counts
|
||||
match R50 before trusting (LVFace truncated under parallel dumping earlier).
|
||||
- `experiments/galleries/gallery_<model>.h5` (+ restricted/<model>/<slug>.h5, per-film cast-
|
||||
filtered to Jellyfin's ~15 top-billed — Jellyfin's hard cap, see experiments/README.md).
|
||||
- `experiments/manifests/rep3_<model>_<mode>.json` — 3 REPRESENTATIVE films (Lord of War /
|
||||
Scarface / Sound of Metal = clean / ensemble-lookalike / high-coverage) to keep evals fast
|
||||
(~28s vs ~90s for 9). Winner should be re-scored on all 9 after.
|
||||
- `experiments/manifests/films_<model>_<mode>.json` — all 9 films.
|
||||
|
||||
## Salvaged partial results (per-second metric)
|
||||
- R50 full +expand: **F1 66.4%** (208 evals, converged) — best so far
|
||||
- R50 full noexp: 55-60% → **expansion helps ~+6-11 recall**
|
||||
- MBF full noexp: 58.6%
|
||||
- (older scene-metric runs, superseded: R50≈LVFace≈MBF ~85%, restricted>full, LVFace not
|
||||
worth its size — but those used the OLD scene-union metric, redo with per-second.)
|
||||
|
||||
## TO DO next session
|
||||
1. **COMMIT everything first** (logical chunks: KPN setters+push_blocking; sae_kpn+dump exe;
|
||||
HDF5 gallery; optimizer scripts; per-second metric; experiments manifests/results/docs +
|
||||
tuned config.hpp defaults prob_threshold 0.76 extinction 1.5).
|
||||
2. Launch the full 16-run matrix via model_server on rep3 films (write trajectories to
|
||||
experiments/, NOT /tmp — /tmp gets wiped mid-session and cost us hours). ~28s/eval ×
|
||||
~84 evals × 16 = ~10hr. Runner pattern: experiments/run_overnight_rep3.sh but pointing
|
||||
optimize_server.py at model_server.
|
||||
3. assemble table: best model + expansion effect + misID, from experiments/results/*.json.
|
||||
4. Consider upstreaming to KPN++: runtime node setters, push_blocking, node_ptr/functor().
|
||||
|
||||
## Gotchas that burned time (don't repeat)
|
||||
- /tmp scratch gets WIPED mid-session → lost dumps + test files repeatedly. Use experiments/.
|
||||
- Verify a launched runner script EXISTS and PRODUCES evals before walking away (a heredoc
|
||||
once silently failed to write; a stale-code process ran the old metric for 12h).
|
||||
- pgrep/ps "survivors" are often the grep's own shell wrapper — check via /proc cmdline or ps.
|
||||
- Running many DE/replay processes in parallel on one GPU → deadlock/thrash. GPU peaks ~35%
|
||||
(not saturated) but concurrency>2-3 wedges. Serial-ish is safer.
|
||||
@@ -0,0 +1,66 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Build per-(model, mode) manifests for the model × gallery-mode bake-off.
|
||||
|
||||
Modes:
|
||||
full — every film matches against the whole model gallery (2418 actors)
|
||||
restricted — each film matches only its Jellyfin credited cast (~15 top-billed),
|
||||
via a per-film gallery filtered from the model gallery by jellyfin_id.
|
||||
|
||||
Each manifest is a list of {name, xray, slug, dump, gallery} — no "movie" path (that's
|
||||
resolved locally via experiments/file-lut.json, see run_montage_all.py, to avoid
|
||||
embedding source filenames in a file that gets shared as an artifact). optimize.py
|
||||
reads film["gallery"] per film, so restricted mode just points each film at its own
|
||||
filtered gallery — no optimizer change needed.
|
||||
|
||||
Writes experiments/manifests/films_<model>_<mode>.json and the restricted galleries to
|
||||
experiments/galleries/restricted/<model>/<slug>.h5.
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(REPO / "scripts" / "validation"))
|
||||
sys.path.insert(0, str(REPO / "scripts"))
|
||||
from identity import norm_name # noqa: E402
|
||||
from sae_gallery import load_gallery_hdf5, save_gallery_hdf5 # noqa: E402
|
||||
|
||||
MODELS = ["arcface_w600k_r50", "arcface_r18", "arcface_w600k_mbf", "LVFace-B_Glint360K"]
|
||||
|
||||
films = json.loads((REPO / "experiments/manifests/films.json").read_text())
|
||||
casts = json.loads((REPO / "experiments/manifests/jellyfin_casts.json").read_text())
|
||||
|
||||
|
||||
def restrict_gallery(model_gallery: dict, cast_ids: set[str]) -> dict:
|
||||
kept = [a for a in model_gallery["actors"] if a.get("jellyfin_id", "") in cast_ids]
|
||||
return {"actors": kept}
|
||||
|
||||
|
||||
for model in MODELS:
|
||||
gpath = REPO / f"experiments/galleries/gallery_{model}.h5"
|
||||
if not gpath.exists():
|
||||
print(f"skip {model}: gallery not built yet ({gpath})")
|
||||
continue
|
||||
model_gal = load_gallery_hdf5(gpath)
|
||||
|
||||
# full mode
|
||||
full = [{**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5",
|
||||
"gallery": f"experiments/galleries/gallery_{model}.h5"} for f in films]
|
||||
(REPO / f"experiments/manifests/films_{model}_full.json").write_text(json.dumps(full, indent=2, ensure_ascii=False))
|
||||
|
||||
# restricted mode — per-film filtered gallery
|
||||
rdir = REPO / f"experiments/galleries/restricted/{model}"
|
||||
rdir.mkdir(parents=True, exist_ok=True)
|
||||
restr = []
|
||||
for f in films:
|
||||
cast_ids = set(casts.get(f["name"], []))
|
||||
rg = restrict_gallery(model_gal, cast_ids)
|
||||
rgpath = rdir / f"{f['slug']}.h5"
|
||||
save_gallery_hdf5(rg, rgpath)
|
||||
restr.append({**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5",
|
||||
"gallery": str(rgpath.relative_to(REPO)),
|
||||
"_cast_size": len(rg["actors"])})
|
||||
(REPO / f"experiments/manifests/films_{model}_restricted.json").write_text(json.dumps(restr, indent=2, ensure_ascii=False))
|
||||
avg = sum(r["_cast_size"] for r in restr) / len(restr)
|
||||
print(f"{model}: full (2418) + restricted (avg {avg:.0f} actors/film) manifests written")
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"Benny___Joon": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Café_Society": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Downton_Abbey__A_New_Era": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Lord_of_War": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Lovelace": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"The_Many_Saints_of_Newark": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Scarface": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Sound_of_Metal": "/path/to/your/movies/<replace-with-your-file>.mp4",
|
||||
"Valerian_and_the_City_of_a_Thousand_Plan": "/path/to/your/movies/<replace-with-your-file>.mp4"
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
run_montage_all.py — replay + best/worst-per-scene montage for all films in a
|
||||
manifest, using the shipped config.hpp defaults (LVFace, prob_threshold=0.754,
|
||||
anneal_sec=35.54, extinction_sec=57.43, expand_gallery). Skips a film if its
|
||||
montage manifest.json already exists (safe to re-run/resume).
|
||||
|
||||
Manifests no longer carry a "movie" path (removed to avoid embedding source
|
||||
filenames — some carry scene-release tags — in a file that gets zipped and
|
||||
pushed to the artifact registry). Movie paths are resolved locally via
|
||||
experiments/file-lut.json (gitignored; copy file-lut.template.json and fill
|
||||
in your own paths).
|
||||
|
||||
Usage:
|
||||
python experiments/run_montage_all.py \
|
||||
--manifest experiments/manifests/films_LVFace-B_Glint360K_full.json \
|
||||
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \
|
||||
--out-dir experiments/results/holdout/montage
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO = Path(__file__).resolve().parent.parent
|
||||
REPLAY = str(REPO / "scripts" / "optimizer" / "replay.py")
|
||||
MONTAGE = str(REPO / "scripts" / "optimizer" / "dump_scene_montage.py")
|
||||
FILE_LUT = REPO / "experiments" / "file-lut.json"
|
||||
|
||||
CFG = {"prob_threshold": "0.7540024664611272", "anneal_sec": "35.53996030397922",
|
||||
"extinction_sec": "57.43359645269811"}
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser()
|
||||
p.add_argument("--manifest", required=True)
|
||||
p.add_argument("--gallery", required=True)
|
||||
p.add_argument("--out-dir", required=True)
|
||||
args = p.parse_args()
|
||||
|
||||
if not FILE_LUT.exists():
|
||||
sys.exit(f"{FILE_LUT} not found — copy file-lut.template.json to "
|
||||
f"file-lut.json and fill in your local movie paths")
|
||||
file_lut = json.loads(FILE_LUT.read_text())
|
||||
|
||||
films = json.loads(Path(args.manifest).read_text())
|
||||
out_root = Path(args.out_dir)
|
||||
out_root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
for f in films:
|
||||
slug = f["slug"]
|
||||
montage_dir = out_root / slug
|
||||
manifest_path = montage_dir / "manifest.json"
|
||||
if manifest_path.exists():
|
||||
print(f"[run_montage_all] skip {slug} (already done)", file=sys.stderr)
|
||||
continue
|
||||
|
||||
movie_path = file_lut.get(slug)
|
||||
if not movie_path:
|
||||
print(f"[run_montage_all] skip {slug}: no entry in {FILE_LUT}", file=sys.stderr)
|
||||
continue
|
||||
|
||||
print(f"[run_montage_all] === {f['name']} ({slug}) ===", file=sys.stderr)
|
||||
raw_path = out_root / f"raw_{slug}.jsonl"
|
||||
pred_path = out_root / f"pred_{slug}.json"
|
||||
|
||||
replay_cmd = [sys.executable, REPLAY, "--dump", f["dump"], "--gallery", args.gallery,
|
||||
"--out", str(pred_path), "--raw-out", str(raw_path),
|
||||
"--prob-threshold", CFG["prob_threshold"],
|
||||
"--anneal-sec", CFG["anneal_sec"],
|
||||
"--extinction-sec", CFG["extinction_sec"], "--expand-gallery"]
|
||||
r = subprocess.run(replay_cmd, capture_output=True, text=True, timeout=120)
|
||||
if r.returncode != 0:
|
||||
print(f"[run_montage_all] replay FAILED for {slug}: {r.stderr[-2000:]}",
|
||||
file=sys.stderr)
|
||||
continue
|
||||
|
||||
montage_cmd = [sys.executable, MONTAGE, "--raw", str(raw_path), "--dump", f["dump"],
|
||||
"--xray", f["xray"], "--movie", movie_path, "--gallery", args.gallery,
|
||||
"--out-dir", str(montage_dir)]
|
||||
r = subprocess.run(montage_cmd, capture_output=True, text=True, timeout=1800)
|
||||
if r.returncode != 0:
|
||||
print(f"[run_montage_all] montage FAILED for {slug}: {r.stderr[-2000:]}",
|
||||
file=sys.stderr)
|
||||
continue
|
||||
print(r.stderr.strip().splitlines()[-1] if r.stderr else "(no output)", file=sys.stderr)
|
||||
|
||||
print("[run_montage_all] ALL DONE", file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,26 @@
|
||||
#!/bin/bash
|
||||
set -u
|
||||
cd /home/dtourolle/Development/scene-actor-extraction
|
||||
mkdir -p experiments/trajectories experiments/results
|
||||
MODELS=(arcface_w600k_r50 arcface_r18 arcface_w600k_mbf LVFace-B_Glint360K)
|
||||
MODES=(full restricted); EXPAND=(1 0)
|
||||
for model in "${MODELS[@]}"; do
|
||||
for mode in "${MODES[@]}"; do
|
||||
for exp in "${EXPAND[@]}"; do
|
||||
xtag=$([ "$exp" = 1 ] && echo exp || echo noexp); tag="${model}_${mode}_${xtag}"
|
||||
best="experiments/results/rep4_best_${tag}.json"; traj="experiments/trajectories/rep4_${tag}.jsonl"
|
||||
[ -f "$best" ] && { echo "skip $tag"; continue; }
|
||||
: > "$traj"; echo "=== $tag START $(date '+%H:%M') ==="
|
||||
SAE_EXPAND=$exp REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \
|
||||
--manifest "experiments/manifests/rep4_${model}_${mode}.json" \
|
||||
--gallery "experiments/galleries/gallery_${model}.h5" \
|
||||
--params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \
|
||||
--popsize 10 --maxiter 15 --seed 0 \
|
||||
--trajectory "$traj" --out "experiments/results/opt_rep4_${tag}.json" \
|
||||
> "experiments/results/rep4_de_${tag}.log" 2>&1
|
||||
python3 -c "import json;r=[json.loads(l) for l in open('$traj')];b=max(r,key=lambda x:x['f1']);json.dump({'best':b,'n_evals':len(r)},open('$best','w'))" 2>/dev/null
|
||||
echo "done $tag $(date '+%H:%M'): $(python3 -c "import json;d=json.load(open('$best'));print('F1=%.1f%% misID=%d'%(d['best']['f1']*100,d['best'].get('FPI_misid',-1)))" 2>/dev/null)"
|
||||
done
|
||||
done
|
||||
done
|
||||
echo "=== REP4 MATRIX COMPLETE $(date '+%F %H:%M') ==="
|
||||
Reference in New Issue
Block a user