feat(tooling): X-Ray threshold optimizer, gallery utilities, artifact registry, docs build

Optimizer (scripts/optimizer/): replay.py runs the real C++ tracker/matcher/
scene_tracker chain over a dumped-embeddings HDF5 via sae_kpn, so a threshold
sweep never re-decodes video or re-embeds faces. optimize.py drives scipy's
differential_evolution over the knob space, with DE-level parallelism
(multiple population candidates evaluated concurrently via a ThreadPoolExecutor)
on top of per-film replay parallelism. second_score.py is the per-second X-Ray
scoring metric (TPI/FPI/FN, out-of-cast misID weighted 10x, fair recall masked
to gallery-known cast) that superseded an earlier scene-union metric.
dump_error_frames.py / dump_scene_montage.py extract annotated video frames
(bounding boxes, TPI/FPI/FN captions, onscreen-vs-offscreen split) for visual
review of a replay against ground truth. Gallery utilities: cast_restrict.py,
gallery_membership.py, fetch_missing_actors.py, reembed_gallery.py.

scripts/validation/: X-Ray ground-truth loading and provider-agnostic identity
matching (identity.py's keys_for — an actor is the union of every id we can
derive, since pipeline output and ground truth don't share one id space).

scripts/artifacts/: push/pull scripts for the Gitea generic package registry —
galleries, montage frames, and experiment data (manifests/trajectories/results)
are pushed there instead of committed, since none are needed to run the app,
only benchmarks. Versioned by git short-SHA.

scripts/docs/: MkDocs site build (build_site.sh) and the calibration-curve
comparison chart (calibration_chart.py, matplotlib, reads each gallery's
embedded calibration).

Gallery-building scripts (make_jellyfin_gallery.py, make_gallery.py,
filter_gallery.py, run_from_jellyfin.py, movienet_eval.py, movienet_prep.py,
sae_gallery.py) updated to read/write HDF5 galleries exclusively, matching the
engine-side format switch. run_from_jellyfin.py and the optimizer no longer
carry movie source paths in shared manifests (some source filenames include
scene-release tags) — resolved locally via a gitignored file-lut.json instead.
This commit is contained in:
2026-07-19 19:06:48 +02:00
parent 26139ffe8a
commit 6f0ad83a55
31 changed files with 3411 additions and 47 deletions
+25 -6
View File
@@ -11,18 +11,24 @@ Usage:
--jellyfin-url http://jellyfin.local:8096 \\
--api-key YOUR_API_KEY \\
--title "The Matrix" \\
--gallery whole_gallery.json \\
--gallery whole_gallery.h5 \\
-- --fps 5 --verbosity 2
Anything after "--" is passed through unchanged to scene_analyze.
Add --preview to open the live OpenCV display window: it resolves the media path
from Jellyfin exactly as normal, then launches build/scene_preview instead of the
headless binary (implies --no-push). Works with --worker or a single title:
python scripts/run_from_jellyfin.py ... --worker --preview -- --fps 5
Worker mode (--worker) polls the JRay plugin's Tasks/Pending endpoint for a
random batch of items with no truth data yet, processing each in turn:
python scripts/run_from_jellyfin.py \\
--jellyfin-url http://jellyfin.local:8096 \\
--api-key YOUR_API_KEY \\
--gallery whole_gallery.json \\
--gallery whole_gallery.h5 \\
--worker \\
-- --fps 5
"""
@@ -40,6 +46,7 @@ import requests
sys.path.insert(0, str(Path(__file__).resolve().parent))
import sae_env # noqa: F401 — loads .env into os.environ on import
from sae_gallery import load_gallery_hdf5, save_gallery_hdf5
from sae_jellyfin import (
jf_get, find_item_id, fetch_cast_person_ids, actor_jellyfin_id, fetch_episode_info,
)
@@ -171,7 +178,7 @@ def process_item(args, item_id: str, extra: list[str]) -> None:
gallery_path = args.gallery
filtered_file = None
if not args.no_filter:
gallery = json.loads(Path(args.gallery).read_text())
gallery = load_gallery_hdf5(Path(args.gallery))
actors = episode_cast_actors(args, item_id, gallery)
if actors is None:
cast_ids = fetch_cast_person_ids(args.jellyfin_url, args.api_key, item_id)
@@ -180,9 +187,9 @@ def process_item(args, item_id: str, extra: list[str]) -> None:
print(f"Filtered gallery to {len(actors)}/{len(gallery.get('actors', []))} "
f"actor(s) credited in {name!r}", file=sys.stderr)
filtered_file = tempfile.NamedTemporaryFile(
mode="w", suffix=".json", prefix="sae_gallery_", delete=False)
json.dump({"actors": actors}, filtered_file)
suffix=".h5", prefix="sae_gallery_", delete=False)
filtered_file.close()
save_gallery_hdf5({"actors": actors}, Path(filtered_file.name))
gallery_path = filtered_file.name
try:
@@ -281,7 +288,7 @@ def main():
parser.add_argument("--item-types", default="Movie,Episode",
help="Item types to search when using --title (default: Movie,Episode)")
parser.add_argument("--gallery", required=True,
help="Global gallery.json built by make_jellyfin_gallery.py")
help="Global gallery.h5 built by make_jellyfin_gallery.py")
parser.add_argument("--no-filter", action="store_true",
help="Skip per-title cast filtering and pass --gallery through as-is")
parser.add_argument("--episode-cast", choices=("tmdb", "series"), default="tmdb",
@@ -297,6 +304,12 @@ def main():
"ignored with --worker, which always uses <title>.json)")
parser.add_argument("--bin", default="build/scene_analyze",
help="Path to scene_analyze binary (default: build/scene_analyze)")
parser.add_argument("--preview", action="store_true",
help="Launch the scene_preview binary (live OpenCV display window) "
"instead of headless scene_analyze. Implies --no-push. Use "
"--preview-bin to override its path. Press q/Esc to close.")
parser.add_argument("--preview-bin", default="build/scene_preview",
help="Path to scene_preview binary (default: build/scene_preview)")
parser.add_argument("--dry-run", action="store_true",
help="Resolve and print the scene_analyze command without running it")
parser.add_argument("--no-push", action="store_true",
@@ -306,6 +319,12 @@ def main():
if extra and extra[0] == "--":
extra = extra[1:]
# --preview swaps in the display binary and disables pushing truth (a preview
# run is interactive/debug, not a truth-producing analysis).
if args.preview:
args.bin = args.preview_bin
args.no_push = True
if args.worker:
if args.output:
sys.exit("--output is incompatible with --worker (each item needs its own file)")