25 Commits
Author SHA1 Message Date
dtourolle e5885977df docs: fix Lovelace frame description, Amanda Seyfried's box is a ghost
Her bbox is frozen at identical coordinates for t=2450 and t=2451; the
dump's own per-frame detections show only one real face at t=2451, and
it matches the Chloë Sevigny box (IoU 1.0), not hers. The frame is one
ghost overlapping one fresh misidentification, not two competing fresh
identities as previously written.
2026-07-21 09:10:55 +02:00
dtourolle 0bd2747069 docs: full data-grounded rewrite of the performance report
Replaces narrative claims with verified numbers across all report pages:

- Cross-model held-out validation (LVFace/mbf/r18, all 5 held-out
  films): LVFace wins every film outright, not just "consistent with"
  the training-set pick. r50 dropped from the detailed comparison
  (gallery has ~30% fewer reference images per actor than the other
  three models on identical source photos).
- Per-film training breakdown: LVFace does not win every training
  film (mbf beats it on Lord of War); the 75.3% macro figure hides a
  10.7pp spread.
- Gallery coverage computed per film (20.3%-78.6%) instead of one
  flat 67%-missing average.
- Found and fixed a real scoring bug in optimize.py: a candidate
  whose hardest film's replay timed out was averaged over survivors
  instead of penalized, silently rewarding partial coverage. Affected
  3 of 16 training combos; corrected throughout, and optimize.py now
  scores an incomplete evaluation f1=0.0 instead of averaging over
  whichever films happened to finish.
- Every FPI frame in the deep dive now comes from the proper montage
  renderer (Onscreen/Offscreen panel, ghosts never drawn as boxes),
  never the bare-box debug overlay used earlier.
- Every distinct out-of-cast name across all 9 films gets its own
  frame at its first appearance (9 names, 4 films), not a
  single-example spot check: 2 ground-truth gaps, 1 photograph
  misread as a person, 6 genuine lookalike confusions.
- New methodology.md: the scene-level-vs-per-second scoring mismatch
  that the rest of the report assumes, written out once.
- Cut the deadlock/gdb debugging narrative from the experiment log;
  kept the one fact that matters (KPN's node/network split lets the
  expensive GPU stage run once and the cheap stage replay against
  cached embeddings).
- Plain declarative style throughout, no em dashes, no blog voice.
2026-07-21 08:55:57 +02:00
dtourolle 4b5557974b docs: montage-renderer imagery, visual polish, README screenshots
- switch report frames to the scene best/worst montage renderer
  (Onscreen/Offscreen panels + TPI/FPI/FN legend): perfect-second hero,
  wedding couple, funeral 19-of-20, polygraph bridging, crew-scene FN
  ceiling, Robert Patrick ground-truth gap, rapid-cut double label,
  Herbie Hancock on an in-fiction screen
- deep dive restructured: extinction bridging framed as designed
  behavior with a measurable cost (debug overlay draws the boxes; the
  shipped output is presence windows), plus the face-vs-presence
  ceiling and two X-Ray-is-wrong exhibits
- Material polish: light/dark palette toggle, landing-page grid cards,
  figure/caption CSS, how-to-read admonition; site_url set so 404 links
  resolve under the Pages subpath
- README: perfect-second and screen-call frames committed (gitignore
  exceptions), readme_example.jpg retired
- build_site.sh: stage_frame helper downscales montage frames to 1920px
  and pulls any missing montage-frames packages
2026-07-19 22:27:57 +02:00
dtourolle 93b6827b14 docs: richer report — data figures, success/failure frames, commit-pinned repo links
- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
2026-07-19 22:16:38 +02:00
dtourolle b1efefac6f docs: richer report — data figures, success/failure frames, commit-pinned repo links
- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
2026-07-19 22:06:56 +02:00
dtourolle 4925443e56 docs: four focused findings pages (best model, gallery scope, expansion, deep dive)
Splits the rep4 write-up's key findings into their own linkable pages:
- best-model.md: calibration curves first (discriminative power, independent
  of any threshold), then F1 on the benchmark — LVFace-B Glint360K wins both.
- gallery-scope.md: whole vs. cast-restricted gallery, isolated from model and
  expansion choice — restriction wins on every axis, but isn't a shipped
  runtime feature yet.
- pose-expansion.md: the training-set expand_gallery effect, and the held-out
  replication attempt that found it doesn't reproduce (5 films, 2 models,
  after catching and fixing a replay-timeout truncation bug and a bbox
  first-match-instead-of-best-match bug in the comparison harness itself). An
  honest null result, with the methodology errors documented since they're
  exactly the kind that manufacture a false "it works!" finding.
- lvface-deep-dive.md: the winning model's held-out generalization gap, its
  two failure modes (frozen-bbox ghost tracks), and a verified case (cross-
  checked against Jellyfin's independent cast metadata) where LVFace
  correctly identified an actor that X-Ray's ground truth failed to credit.

Adds a "report-highlights" artifact-registry package (scripts/artifacts/
push_artifacts.sh, pull_artifacts.sh) for hand-picked illustrative frames that
aren't reproducible via the automated best/worst montage selection, and wires
pulling it into scripts/docs/build_site.sh.
2026-07-19 19:40:19 +02:00
dtourolle d340da755a docs: rep4 bake-off write-up, MkDocs site, artifact-registry-backed experiments
docs/rep4-optimizer-results.md is the main deliverable: the model bake-off +
threshold re-tune experiment log, including the ROCm teardown deadlock root
cause and fix, DE concurrency tuning, the 16-combo results table, held-out
validation against 5 films never seen by the optimizer (macro F1 67.4% vs.
75.3% training — a real generalization gap), the frozen-bbox "ghost track"
failure mode found via annotated frame evidence, calibration curves per model,
and an isolated-effects breakdown of gallery scope vs. pose expansion.

MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/
pulls referenced images from the artifact registry and generates the
calibration chart at build time (see the tooling commit) rather than
committing images to the repo.

experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every
data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories,
manifests, results) moved to the Gitea package registry. film-lut.template.json
is the committed placeholder for the gitignored file-lut.json (real local
movie paths, never shared — some source filenames carry scene-release tags).

Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for
the new scene-detection path.
2026-07-19 19:12:22 +02:00
dtourolle 76df2f66aa test: add Catch2 unit test suite (gallery, calibration, tracking, similarity)
GPU-free, model-free tests for the pure logic: gallery HDF5 save/load
round-trips (actors, embeddings, embedded calibration) and legacy JSON
read back-compat; the calibration sigmoid fit, boundary inversion, and the
in-memory hash-keyed cache reuse/staleness; TrackGallery's diversity-buffer
eviction, novelty/spread safety gates, and promotion; FaceTracker's IoU/
embedding association and cross-cut track revival; and the GEMM similarity
backend (forced to CPU so the suite runs without a GPU).

Verified: all 39 test cases / 1640 assertions pass (cmake -DSAE_BUILD_TESTS=ON).
2026-07-19 19:10:57 +02:00
dtourolle 6f0ad83a55 feat(tooling): X-Ray threshold optimizer, gallery utilities, artifact registry, docs build
Optimizer (scripts/optimizer/): replay.py runs the real C++ tracker/matcher/
scene_tracker chain over a dumped-embeddings HDF5 via sae_kpn, so a threshold
sweep never re-decodes video or re-embeds faces. optimize.py drives scipy's
differential_evolution over the knob space, with DE-level parallelism
(multiple population candidates evaluated concurrently via a ThreadPoolExecutor)
on top of per-film replay parallelism. second_score.py is the per-second X-Ray
scoring metric (TPI/FPI/FN, out-of-cast misID weighted 10x, fair recall masked
to gallery-known cast) that superseded an earlier scene-union metric.
dump_error_frames.py / dump_scene_montage.py extract annotated video frames
(bounding boxes, TPI/FPI/FN captions, onscreen-vs-offscreen split) for visual
review of a replay against ground truth. Gallery utilities: cast_restrict.py,
gallery_membership.py, fetch_missing_actors.py, reembed_gallery.py.

scripts/validation/: X-Ray ground-truth loading and provider-agnostic identity
matching (identity.py's keys_for — an actor is the union of every id we can
derive, since pipeline output and ground truth don't share one id space).

scripts/artifacts/: push/pull scripts for the Gitea generic package registry —
galleries, montage frames, and experiment data (manifests/trajectories/results)
are pushed there instead of committed, since none are needed to run the app,
only benchmarks. Versioned by git short-SHA.

scripts/docs/: MkDocs site build (build_site.sh) and the calibration-curve
comparison chart (calibration_chart.py, matplotlib, reads each gallery's
embedded calibration).

Gallery-building scripts (make_jellyfin_gallery.py, make_gallery.py,
filter_gallery.py, run_from_jellyfin.py, movienet_eval.py, movienet_prep.py,
sae_gallery.py) updated to read/write HDF5 galleries exclusively, matching the
engine-side format switch. run_from_jellyfin.py and the optimizer no longer
carry movie source paths in shared manifests (some source filenames include
scene-release tags) — resolved locally via a gitignored file-lut.json instead.
2026-07-19 19:06:48 +02:00
dtourolle 26139ffe8a feat(engine): add Python replay bindings, gallery pose-expansion, scene detection, embedding dumps
New C++ sources:
- kpn_bindings.cpp (sae_kpn): assembles the real face_tracker/identity_matcher/
  scene_tracker nodes inside a Python-driven KPN network via nanobind, for
  offline threshold-sweep replay against dumped embeddings (scripts/optimizer/).
- track_gallery.hpp: per-film gallery expansion — promotes a confidently-
  identified track's novel-pose reference views into an in-memory annex so
  later frames/tracks of that actor at similar poses are recognised, without
  touching the baked gallery.
- dump_embeddings.cpp: standalone exe that runs detect→embed only (no gallery,
  no matching) and dumps per-frame face embeddings + metadata to HDF5, so a
  parameter sweep can replay the expensive half once and vary tracking/matching
  config freely downstream.
- scene_detector.hpp / scene_detector_node.hpp: TransNetV2-based shot-boundary
  detection, opt-in alongside the always-on histogram cut detector.
- camera_position_change_detector_node.hpp, embedding_dump_node.hpp: supporting
  nodes for the above.
2026-07-19 19:05:05 +02:00
dtourolle 41a277bc19 feat(engine): HDF5-native galleries with embedded calibration; TensorRT backends; scene detection
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.

Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.

Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
2026-07-19 19:04:03 +02:00
dtourolle aca6147d69 feat(scripts): add scene-gap histogram tool
scene_gap_hist.py scans scene_analyze output JSONs and, for every actor,
computes the gap (next_scene_start - prev_scene_end) between consecutive
scenes, emitting a text histogram of the distribution. Used to inform the
anneal_sec default.
2026-07-04 20:42:11 +02:00
dtourolle 65fee74585 perf(movienet): vectorise eval matching; count frames missing from Image.zip
movienet_eval: replace the per-element dot() with numpy — actor references are
loaded once as an ndarray and scored with a single matmul, keeping a
whole-library gallery fast.

movienet_prep: count and report frames referenced by annotations but absent
from Image.zip instead of skipping them silently.
2026-07-04 20:41:54 +02:00
dtourolle 1f5acc25df docs: document LVFace embedder support
LVFace-B_Glint360K.onnx shares ArcFace's I/O contract (112x112 aligned crop ->
L2-normalised 512-d) and input scaling, so it drops in via --arcface-model.
Note the caveat that galleries and calib caches must be rebuilt with the same
embedder used for analysis.
2026-07-04 20:39:43 +02:00
dtourolle 96b1c22194 feat(cameo): detect recognised actors not credited in a title
Add two cameo hunters that flag actors recognised in a title but absent from
its cast:
  - cameo_jellyfin.py — pure-Jellyfin cast-membership check (no id cross-walk)
  - cameo_hunt.py     — TMDB filmography check (actor's combined_credits)

run_from_jellyfin.py now stamps the analysed title's Jellyfin item GUID into
the output JSON as top-level 'jellyfin_item_id' (scene_analyze can't know it),
which cameo_jellyfin.py uses to look up the cast in Jellyfin's own id space.
Document that field in the result-sink output schema header.
2026-07-04 20:39:35 +02:00
dtourolle 3700c763dd tune: raise default anneal_sec from 2s to 10s
Merge actor windows separated by up to 10s into one epoch, reducing scene
fragmentation from brief detection dropouts.
2026-07-04 18:55:53 +02:00
dtourolle 66298026e2 feat(pipeline): detect node crashes and tally dropped frames
Register a KPN event handler in both scene_analyze and scene_preview:
  - Overflow events accumulate per-node dropped-frame counts, printed on exit.
  - A Closed event from any node other than result_sink at EOF means a stage
    died; trip an atomic so the main loop bails out instead of hanging on
    'done' forever, and exit non-zero.

Bumps external/KPN to the commit that exposes set_event_handler / NodeEvent.
2026-07-04 18:55:47 +02:00
dtourolle 152c34b1f4 refactor(scripts): extract shared sae_* helpers and dedupe gallery builders
Consolidate copy-pasted logic across the gallery/run scripts into shared
modules:
  - sae_env.py     — zero-dependency .env loader (populates os.environ)
  - sae_tmdb.py    — TMDB API helpers (tmdb_get, person images, id lookups)
  - sae_jellyfin.py— Jellyfin API helpers (jf_get, id/URL normalisation)
  - sae_gallery.py — image download + gallery.json writing

make_gallery, make_jellyfin_gallery and filter_gallery now import these
instead of carrying their own near-identical copies.
2026-07-04 18:55:37 +02:00
dtourolle aacaefb3dc chore(gitignore): ignore generated calib caches, cameo reports, and plots
These are regenerable per-run outputs that were polluting the worktree:
calibration caches (*.calib_cache.csv/png), cameo detection run outputs
(cameo_progress.txt, cameo_report.txt), and scene-gap analysis plots.
2026-07-04 18:54:37 +02:00
dtourolle ea92dd8150 add readme, liscence and use public KPN 2026-06-28 12:09:54 +02:00
dtourolle 0ee131a692 Add AMD support via ort alternative to trt 2026-06-28 11:50:05 +02:00
dtourolle a3ba53ddf7 improved performance 2026-06-13 22:44:44 +02:00
dtourolle fc16d4a0e1 improved jellyfin support 2026-06-12 20:57:33 +02:00
dtourolle a1d6759abc faster calibration curve generation
jellyfin intergration
2026-06-12 17:54:23 +02:00
dtourolle d753062c6c Initial commit: scene-actor-extraction pipeline
Source (KPN++ pipeline nodes, ArcFace embedders, SCRFD/YuNet detectors,
gallery builder), build scripts, and eval artifacts.

- external/KPN as a git submodule (gitea.tourolle.paris/dtourolle/KPN)
- ONNX models tracked via Git LFS (models/*.onnx)
- generated outputs, TensorRT engines, reference repos, and media ignored
2026-06-12 15:29:01 +02:00
210 changed files with 22460 additions and 17798 deletions
+1
View File
@@ -0,0 +1 @@
models/*.onnx filter=lfs diff=lfs merge=lfs -text
+115
View File
@@ -0,0 +1,115 @@
# Build
build/
cmake-build-*/
CMakeCache.txt
CMakeFiles/
*.cmake
Makefile
install_manifest.txt
compile_commands.json
# Compiled objects
*.o
*.a
*.so
*.dylib
*.json
# Exception: small, curated result summaries backing specific numbers quoted
# in docs/ (cross-model held-out scores, per-film training breakdown, gallery
# coverage). Regenerate with scripts/docs/run_holdout_all_models.py and
# scripts/docs/gallery_coverage_per_film.py.
!docs_data/*.json
# Video files
*.mp4
*.mkv
*.avi
*.mov
# ONNX models in models/ are tracked via Git LFS (see .gitattributes).
# Any stray ONNX elsewhere is generated/downloaded and not tracked.
external/*.onnx
# Generated TensorRT engines (rebuilt by ORT / scripts/build_trt_engines.sh)
trt_cache/
# ORT pre-optimized model cache (generated on first run, provider-specific)
ort_cache/
# Gallery files (generated — HDF5 only, see src/gallery/gallery_store.cpp).
# Legacy JSON galleries from before that switch are also excluded.
gallery.json
gallery_*.json
gallery.h5
gallery_*.h5
# Calibration cache (generated alongside a gallery, per-embedder)
*.calib_cache.csv
*.calib_cache.png
# Cameo detection run outputs (generated by scripts/cameo_*.py)
cameo_progress.txt
cameo_report.txt
# Analysis plot outputs (scene-gap histograms / KDEs, etc.)
scene_gap_*.png
# Per-frame debug images
images/
# Annotations output
annotations.json
*_annotations.json
# MovieNet evaluation data
movienet-ps/
# Eval probe images (data, regenerable)
eval/probe/
# Local reference repos kept for inspiration (each has its own .git)
inspiration/
# experiments/ has its own nested .gitignore for HDF5 galleries/dumps/X-Ray
# corpus (all pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to
# the Gitea generic package registry instead of committed).
# Exception to the blanket *.json rule above: the committed placeholder for
# experiments/file-lut.json (see experiments/.gitignore).
!experiments/file-lut.template.json
# Site build output (mkdocs build). Rendered site is deployed to a
# gitea-pages branch, never committed to a working branch.
site/
docs_site/
# Images staged into docs/ from the artifact registry at build time
# (scripts/docs/build_site.sh) — not committed, pulled fresh on each build.
# Exception: pipeline_topology.svg is small and hand-authored (not pulled from
# anywhere) and the README references it directly, so it needs to render on a
# plain Gitea repo view too, not just the built Pages site. (A directory-level
# ignore can't be un-ignored file-by-file below it, so this must NOT blanket-
# ignore docs/assets/ itself — only its contents, minus the one exception.)
docs/assets/images/*
!docs/assets/images/pipeline_topology.svg
# These frames are referenced directly by README.md, which renders on the
# plain Gitea repo view — committed for the same reason as the SVG above.
!docs/assets/images/lovelace_perfect_second.jpg
!docs/assets/images/valerian_screen_call.jpg
# Python
__pycache__/
*.pyc
*.pyo
.venv/
venv/
# Local secrets (API keys — never commit)
.env
# Editor / OS
.vscode/
.idea/
.claude/settings.local.json
*.swp
*.swo
.DS_Store
Thumbs.db
+4
View File
@@ -0,0 +1,4 @@
[submodule "external/KPN"]
path = external/KPN
url = https://gitea.tourolle.paris/dtourolle/KPN.git
branch = master
View File
-713
View File
@@ -1,713 +0,0 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="Face-recognition pipeline for finding on-screen actor presence in film/TV, built on KPN++">
<link rel="icon" href="/dtourolle/scene-actor-extraction/assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.7">
<title>scene-actor-extraction</title>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/main.ec1eaa64.min.css">
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/stylesheets/extra.css">
<script>__md_scope=new URL("/dtourolle/scene-actor-extraction/",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-header__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
scene-actor-extraction
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_0">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 7a5 5 0 0 1 5 5 5 5 0 0 1-5 5 5 5 0 0 1-5-5 5 5 0 0 1 5-5m0 2a3 3 0 0 0-3 3 3 3 0 0 0 3 3 3 3 0 0 0 3-3 3 3 0 0 0-3-3m0-7 2.39 3.42C13.65 5.15 12.84 5 12 5s-1.65.15-2.39.42zM3.34 7l4.16-.35A7.2 7.2 0 0 0 5.94 8.5c-.44.74-.69 1.5-.83 2.29zm.02 10 1.76-3.77a7.131 7.131 0 0 0 2.38 4.14zM20.65 7l-1.77 3.79a7.02 7.02 0 0 0-2.38-4.15zm-.01 10-4.14.36c.59-.51 1.12-1.14 1.54-1.86.42-.73.69-1.5.83-2.29zM12 22l-2.41-3.44c.74.27 1.55.44 2.41.44.82 0 1.63-.17 2.37-.44z"/></svg>
</label>
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="indigo" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_0" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m17.75 4.09-2.53 1.94.91 3.06-2.63-1.81-2.63 1.81.91-3.06-2.53-1.94L12.44 4l1.06-3 1.06 3zm3.5 6.91-1.64 1.25.59 1.98-1.7-1.17-1.7 1.17.59-1.98L15.75 11l2.06-.05L18.5 9l.69 1.95zm-2.28 4.95c.83-.08 1.72 1.1 1.19 1.85-.32.45-.66.87-1.08 1.27C15.17 23 8.84 23 4.94 19.07c-3.91-3.9-3.91-10.24 0-14.14.4-.4.82-.76 1.27-1.08.75-.53 1.93.36 1.85 1.19-.27 2.86.69 5.83 2.89 8.02a9.96 9.96 0 0 0 8.02 2.89m-1.64 2.02a12.08 12.08 0 0 1-7.8-3.47c-2.17-2.19-3.33-5-3.49-7.82-2.81 3.14-2.7 7.96.31 10.98 3.02 3.01 7.84 3.12 10.98.31"/></svg>
</label>
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-tabs__link">
How We Score Against X-Ray
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-tabs__link">
Findings
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-tabs__link">
Full Experiment Log
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-tabs__link">
Service Conversion (proposal)
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-nav__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
scene-actor-extraction
</label>
<div class="md-nav__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-nav__link">
<span class="md-ellipsis">
How We Score Against X-Ray
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Findings
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Findings
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-nav__link">
<span class="md-ellipsis">
Best Model
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/gallery-scope/" class="md-nav__link">
<span class="md-ellipsis">
Gallery Scope (Full vs. Limited)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/pose-expansion/" class="md-nav__link">
<span class="md-ellipsis">
Pose Expansion
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/lvface-deep-dive/" class="md-nav__link">
<span class="md-ellipsis">
LVFace Deep Dive
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-nav__link">
<span class="md-ellipsis">
Full Experiment Log
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-nav__link">
<span class="md-ellipsis">
Service Conversion (proposal)
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "/dtourolle/scene-actor-extraction/", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "content.code.copy", "content.code.annotate"], "search": "/dtourolle/scene-actor-extraction/assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="/dtourolle/scene-actor-extraction/assets/javascripts/bundle.d7400e89.min.js"></script>
</body>
</html>
+318
View File
@@ -0,0 +1,318 @@
cmake_minimum_required(VERSION 3.21)
project(scene_actor_extraction VERSION 0.1.0 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
# ── Dependencies ──────────────────────────────────────────────────────────────
# KPN++ (pipeline backbone)
set(KPN_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_PYTHON OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
option(SAE_WEB_DEBUG "Enable KPN web debug UI (localhost:9090)" OFF)
if(SAE_WEB_DEBUG)
set(KPN_WEB_DEBUG ON CACHE BOOL "" FORCE)
endif()
add_subdirectory(external/KPN)
# OpenCV (video decode, image ops, DNN inference, face detection)
find_package(OpenCV 4 REQUIRED COMPONENTS
core imgproc imgcodecs videoio dnn objdetect highgui)
# ── Model paths ───────────────────────────────────────────────────────────────
# Defined early so the backend object libraries below can embed it.
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Backend selection ─────────────────────────────────────────────────────────
# Two independent compile-time axes. The core application is agnostic to both:
# only the matching backend .cpp (in src/backends/) is compiled, and the backend
# headers (onnxruntime / NvInfer.h / cublas / rocblas) never reach core TUs.
#
# SAE_INFERENCE_BACKEND ORT → SCRFD + ArcFace via ONNX Runtime (.onnx models)
# TRT → SCRFD + ArcFace via raw TensorRT (.engine files)
# SAE_GEMM_BACKEND ROCM → gallery similarity GEMM via rocBLAS / HIP
# CUDA → gallery similarity GEMM via cuBLAS / CUDA
# CPU → portable reference GEMM (no GPU; CI / testing)
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "Inference backend: ORT | TRT")
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "Gallery GEMM backend: ROCM | CUDA | CPU")
set_property(CACHE SAE_INFERENCE_BACKEND PROPERTY STRINGS ORT TRT)
set_property(CACHE SAE_GEMM_BACKEND PROPERTY STRINGS ROCM CUDA CPU)
# Enable the ORT TensorRT/CUDA execution providers inside the ORT inference
# backend (only meaningful when ORT was built with the TensorRT EP). Off by
# default so ROCm/CPU builds don't reference unavailable EPs.
option(SAE_ORT_TRT_EP "ORT backend: enable TensorRT/CUDA execution providers" OFF)
# Back-compat: a legacy -DSAE_WITH_TRT=ON/OFF seeds the new vars (ON⇒TRT+CUDA,
# OFF⇒ORT+ROCM) unless the user set them explicitly.
if(DEFINED SAE_WITH_TRT)
if(SAE_WITH_TRT)
set(SAE_INFERENCE_BACKEND "TRT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "CUDA" CACHE STRING "" FORCE)
else()
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "" FORCE)
endif()
message(STATUS "SAE_WITH_TRT=${SAE_WITH_TRT} (legacy) → "
"SAE_INFERENCE_BACKEND=${SAE_INFERENCE_BACKEND} "
"SAE_GEMM_BACKEND=${SAE_GEMM_BACKEND}")
endif()
if(NOT SAE_INFERENCE_BACKEND MATCHES "^(ORT|TRT)$")
message(FATAL_ERROR "SAE_INFERENCE_BACKEND must be ORT or TRT (got '${SAE_INFERENCE_BACKEND}')")
endif()
if(NOT SAE_GEMM_BACKEND MATCHES "^(ROCM|CUDA|CPU)$")
message(FATAL_ERROR "SAE_GEMM_BACKEND must be ROCM, CUDA or CPU (got '${SAE_GEMM_BACKEND}')")
endif()
# CUDA runtime is needed by both TRT inference and CUDA GEMM — find it once.
function(sae_find_cudart)
if(TARGET cudart_dep)
return()
endif()
find_library(CUDART_LIB cudart
HINTS /opt/cuda/lib64 /usr/local/cuda/lib64 /usr/lib)
find_path(CUDART_INCLUDE cuda_runtime_api.h
HINTS /opt/cuda/targets/x86_64-linux/include /opt/cuda/include
/usr/local/cuda/include /usr/include)
if(NOT (CUDART_LIB AND CUDART_INCLUDE))
message(FATAL_ERROR "CUDA runtime not found (cudart=${CUDART_LIB} headers=${CUDART_INCLUDE}).")
endif()
add_library(cudart_dep INTERFACE)
target_include_directories(cudart_dep INTERFACE "${CUDART_INCLUDE}")
target_link_libraries(cudart_dep INTERFACE "${CUDART_LIB}")
set_property(GLOBAL PROPERTY sae_cudart_found TRUE)
endfunction()
# ── Inference backend dependency: builds the `inference_backend` object lib ────
if(SAE_INFERENCE_BACKEND STREQUAL "ORT")
find_library(ORT_LIB onnxruntime REQUIRED
HINTS /usr/lib64/rocm/lib /usr/lib /usr/local/lib)
find_path(ORT_INCLUDE onnxruntime_cxx_api.h
PATH_SUFFIXES onnxruntime
HINTS /usr/lib64/rocm/include/onnxruntime /usr/include/onnxruntime /usr/local/include/onnxruntime
/usr/lib64/rocm/include /usr/include /usr/local/include
REQUIRED)
# The include directive is <onnxruntime/onnxruntime_cxx_api.h>, so we need the
# parent of the onnxruntime/ subdirectory on the include path.
get_filename_component(ORT_INCLUDE_PARENT "${ORT_INCLUDE}" DIRECTORY)
if(NOT EXISTS "${ORT_INCLUDE_PARENT}/onnxruntime")
set(ORT_INCLUDE_PARENT "${ORT_INCLUDE}")
endif()
message(STATUS "Inference backend: ORT (${ORT_LIB} headers: ${ORT_INCLUDE_PARENT})")
add_library(inference_backend OBJECT src/backends/ort_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${ORT_INCLUDE_PARENT}")
target_link_libraries(inference_backend PRIVATE ${OpenCV_LIBS} "${ORT_LIB}")
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
$<$<BOOL:${SAE_ORT_TRT_EP}>:SAE_ORT_WITH_TRT_EP>)
else() # TRT
find_library(NVINFER_LIB nvinfer
HINTS /usr/lib /usr/local/lib /opt/tensorrt/lib)
find_path(NVINFER_INCLUDE NvInfer.h
HINTS /usr/include /usr/local/include /opt/tensorrt/include)
if(NOT (NVINFER_LIB AND NVINFER_INCLUDE))
message(FATAL_ERROR
"TensorRT not found (nvinfer=${NVINFER_LIB} headers=${NVINFER_INCLUDE}). "
"Pass -DSAE_INFERENCE_BACKEND=ORT to load .onnx models without TensorRT.")
endif()
sae_find_cudart()
message(STATUS "Inference backend: TRT (${NVINFER_LIB})")
add_library(inference_backend OBJECT src/backends/trt_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${NVINFER_INCLUDE}")
target_link_libraries(inference_backend PRIVATE
${OpenCV_LIBS} "${NVINFER_LIB}" cudart_dep)
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}")
endif()
# ── GEMM backend dependency: builds the `gemm_backend` object lib ──────────────
if(SAE_GEMM_BACKEND STREQUAL "CPU")
# Portable reference GEMM: no GPU libraries, no headers. Used for CI and as
# the correctness oracle for the CUDA/ROCm backends.
message(STATUS "GEMM backend: CPU (portable reference, no GPU)")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CPU)
elseif(SAE_GEMM_BACKEND STREQUAL "CUDA")
find_library(CUBLAS_LIB cublas
HINTS /opt/cuda/targets/x86_64-linux/lib /opt/cuda/lib64
/usr/local/cuda/lib64 /usr/lib)
if(NOT CUBLAS_LIB)
message(FATAL_ERROR "cuBLAS not found (cublas=${CUBLAS_LIB}).")
endif()
sae_find_cudart()
message(STATUS "GEMM backend: CUDA (${CUBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_link_libraries(gemm_backend PRIVATE "${CUBLAS_LIB}" cudart_dep)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CUDA)
else() # ROCM
find_library(ROCBLAS_LIB rocblas
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(ROCBLAS_INCLUDE rocblas/rocblas.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
find_library(HIP_LIB amdhip64
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(HIP_INCLUDE hip/hip_runtime_api.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
if(NOT (ROCBLAS_LIB AND ROCBLAS_INCLUDE AND HIP_LIB AND HIP_INCLUDE))
message(FATAL_ERROR
"rocBLAS or HIP runtime not found "
"(rocblas=${ROCBLAS_LIB} headers=${ROCBLAS_INCLUDE} "
"hip=${HIP_LIB} headers=${HIP_INCLUDE}). "
"Install rocblas-devel and hip-devel (or pass -DSAE_GEMM_BACKEND=CUDA).")
endif()
message(STATUS "GEMM backend: ROCM (${ROCBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src "${ROCBLAS_INCLUDE}" "${HIP_INCLUDE}")
target_link_libraries(gemm_backend PRIVATE "${ROCBLAS_LIB}" "${HIP_LIB}")
# HIP headers require the platform to be declared explicitly when compiled with g++.
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_ROCM __HIP_PLATFORM_AMD__)
endif()
# FFmpeg (hwaccel video decode: CUDA/VAAPI, runtime-detected + swscale colour
# conversion). Hwaccel support is built into libavcodec/libavutil; no extra
# libraries are needed here.
find_package(PkgConfig REQUIRED)
pkg_check_modules(AVFORMAT REQUIRED libavformat)
pkg_check_modules(AVCODEC REQUIRED libavcodec)
pkg_check_modules(AVUTIL REQUIRED libavutil)
pkg_check_modules(SWSCALE REQUIRED libswscale)
add_library(ffmpeg_libs INTERFACE)
target_compile_options(ffmpeg_libs INTERFACE
${AVFORMAT_CFLAGS_OTHER} ${AVCODEC_CFLAGS_OTHER}
${AVUTIL_CFLAGS_OTHER} ${SWSCALE_CFLAGS_OTHER})
target_include_directories(ffmpeg_libs INTERFACE
${AVFORMAT_INCLUDE_DIRS} ${AVCODEC_INCLUDE_DIRS}
${AVUTIL_INCLUDE_DIRS} ${SWSCALE_INCLUDE_DIRS})
target_link_libraries(ffmpeg_libs INTERFACE
${AVFORMAT_LIBRARIES} ${AVCODEC_LIBRARIES}
${AVUTIL_LIBRARIES} ${SWSCALE_LIBRARIES})
message(STATUS "FFmpeg: avformat=${AVFORMAT_VERSION} avcodec=${AVCODEC_VERSION}")
# nlohmann/json (gallery + output serialisation)
include(FetchContent)
FetchContent_Declare(
nlohmann_json
GIT_REPOSITORY https://github.com/nlohmann/json.git
GIT_TAG v3.11.3
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nlohmann_json)
# nanobind (Python bindings for the sae_embed module)
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
FetchContent_Declare(
nanobind
GIT_REPOSITORY https://github.com/wjakob/nanobind.git
GIT_TAG v2.4.0
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nanobind)
# ── Model paths ───────────────────────────────────────────────────────────────
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Shared library: gallery store + compiled-in backends ──────────────────────
# The backend object libraries carry their own ORT/TRT/CUDA/ROCm linkage and
# headers; sae_gallery re-exports those object files so every binary that links
# sae_gallery gets the chosen backend without ever seeing its headers.
# HDF5 (C++) — gallery fast-load path + embedding dump. Found here so sae_gallery
# (gallery_store.cpp) can link it; scene_analyze/dump_embeddings reuse the same vars.
find_package(HDF5 REQUIRED COMPONENTS CXX)
add_library(sae_gallery STATIC
src/gallery/gallery_store.cpp
src/gallery/gallery_builder.cpp
)
set_target_properties(sae_gallery PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(sae_gallery PUBLIC src ${HDF5_INCLUDE_DIRS})
target_link_libraries(sae_gallery PUBLIC
kpn
${OpenCV_LIBS}
nlohmann_json::nlohmann_json
inference_backend
gemm_backend
ffmpeg_libs
${HDF5_CXX_LIBRARIES}
)
target_compile_definitions(sae_gallery PUBLIC
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
)
# ── embed_faces — image → embedding JSON (used by gallery builder scripts) ────
add_executable(embed_faces src/embed_faces.cpp)
target_link_libraries(embed_faces PRIVATE sae_gallery)
# ── sae_embed — Python module: load SCRFD+ArcFace once, embed many images ───
nanobind_add_module(sae_embed src/python_bindings.cpp)
target_link_libraries(sae_embed PRIVATE sae_gallery)
# ── sae_kpn — Python module: run the real downstream nodes over dumped embeddings ─
# Assembles face_tracker/identity_matcher/scene_tracker in a Python-driven KPN
# network (KPN_BUILD_PYTHON is enabled per-TU inside the .cpp). Powers the
# threshold-sweep optimizer in scripts/optimizer/.
nanobind_add_module(sae_kpn src/kpn_bindings.cpp)
target_link_libraries(sae_kpn PRIVATE sae_gallery)
# HDF5 already found above (before sae_gallery); vars HDF5_CXX_LIBRARIES / _INCLUDE_DIRS
# are reused by scene_analyze / dump_embeddings below.
# ── analyze — main analysis binary ───────────────────────────────────────────
add_executable(scene_analyze src/main.cpp)
target_link_libraries(scene_analyze PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze PRIVATE ${HDF5_INCLUDE_DIRS})
# ── analyze_debug — same binary with debug frame/crop output ─────────────────
add_executable(scene_analyze_debug src/main.cpp)
target_link_libraries(scene_analyze_debug PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze_debug PRIVATE ${HDF5_INCLUDE_DIRS})
target_compile_definitions(scene_analyze_debug PRIVATE SAE_DEBUG=1)
# ── dump_embeddings — standalone embedding dumper, NO gallery/matcher ─────────
# Front-half only (decode→detect→align→embed→HDF5) for the optimizer replay corpus
# and model bake-off. Skips gallery load + calibration (~24s/run faster).
add_executable(dump_embeddings src/dump_embeddings.cpp)
target_link_libraries(dump_embeddings PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(dump_embeddings PRIVATE ${HDF5_INCLUDE_DIRS})
# ── scene_preview — live annotated display while analysing ───────────────────
add_executable(scene_preview src/scene_preview.cpp)
target_link_libraries(scene_preview PRIVATE sae_gallery)
# ── build_gallery — offline gallery construction tool ────────────────────────
add_executable(build_gallery src/build_gallery.cpp)
target_link_libraries(build_gallery PRIVATE sae_gallery)
# ── Optional: web debug UI for pipeline introspection ────────────────────────
if(SAE_WEB_DEBUG)
kpn_target_enable_web_debug(scene_analyze)
kpn_target_enable_web_debug(scene_analyze_debug)
kpn_target_enable_web_debug(scene_preview)
endif()
# ── Tests ─────────────────────────────────────────────────────────────────────
option(SAE_BUILD_TESTS "Build unit tests (GPU-free)" OFF)
if(SAE_BUILD_TESTS)
enable_testing()
add_subdirectory(tests)
endif()
message(STATUS "OpenCV ${OpenCV_VERSION} found")
message(STATUS "Models dir: ${SAE_MODELS_DIR}")
+41
View File
@@ -0,0 +1,41 @@
MIT License
Copyright (c) 2026 Duncan Tourolle
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
---
NOTE ON MODELS AND THIRD-PARTY COMPONENTS
The MIT license above applies only to the source code in this repository. It
does NOT cover:
* Machine-learning model weights (the ONNX files in models/). These weights
are the property of their respective authors and are governed by their own
licenses, not by the MIT license above. The models — including the
InsightFace "buffalo" packs (ArcFace / SCRFD), YuNet, and LVFace — are
redistributed here for convenience under those upstream licenses. Several,
notably the InsightFace models, are licensed for NON-COMMERCIAL RESEARCH
USE ONLY. You are responsible for reviewing and complying with each model's
license before use.
* Third-party libraries this software links against (OpenCV, ONNX Runtime,
FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind, and others), each of which
carries its own license.
+259
View File
@@ -0,0 +1,259 @@
# Scene Actor Extraction
Identifies actors in movie files and produces X-ray-style scene annotations compatible with [Jellyfin](https://jellyfin.org/). Built on a KPN++ pipeline with ArcFace/LVFace embeddings and a tracked-identity matcher.
**67.4% macro-F1 against Amazon X-Ray ground truth**, on 5 films never seen by
the optimizer (89.7% P / 65.4% R training-set; see the generalization-gap
discussion in the [deep dive](https://pages.tourolle.paris/dtourolle/scene-actor-extraction/lvface-deep-dive/)).
Full benchmark write-up, model comparison, and failure-mode analysis:
**https://pages.tourolle.paris/dtourolle/scene-actor-extraction/**
![A perfect X-Ray second on a held-out film](docs/assets/images/lovelace_perfect_second.jpg)
*A perfect X-Ray second on a held-out film (never used for threshold tuning):
every visible face named at 100%, the background extra honestly left unnamed,
and the two credited cast without a visible face correctly carried as present
off-screen. Bottom panels show the per-second verdict against Amazon X-Ray
(green = correct, orange = wrong, blue = missed).*
It also doesn't care whether the face is in the room:
![Herbie Hancock identified on an in-fiction video-call screen](docs/assets/images/valerian_screen_call.jpg)
*Herbie Hancock at 98% — as a face on a screen inside the movie, under a
sci-fi HUD overlay.*
## How it works
1. **Build a gallery** — download actor headshots from TMDB/IMDB, embed them with ArcFace or LVFace (`build_gallery` / `scripts/make_gallery.py`).
2. **Analyze a movie**`scene_analyze` decodes frames at configurable FPS, detects faces (SCRFD), tracks them across cuts, matches identities against the gallery using calibrated similarity, and writes time-window JSON.
3. **Output** — minimal mode produces Jellyfin-ready actor name + time-window JSON; standard mode adds per-frame bbox, similarity, and track data.
![Pipeline topology](docs/assets/images/pipeline_topology.svg)
## Dependencies
| Dependency | Role |
|---|---|
| KPN++ | Pipeline backbone (nodes, networks) |
| OpenCV 4 | Video decode, image ops, DNN inference |
| ONNX Runtime | SCRFD face detector (dynamic shape nodes unsupported by cv::dnn) |
| TensorRT + CUDA runtime + cuBLAS | Optional TRT engines for SCRFD/ArcFace (`--detector-engine`/`--arcface-engine`); identity_matcher's GPU gallery scan |
| FFmpeg (libav*) | NVDEC hardware video decode + colour conversion |
| nlohmann/json | JSON I/O |
| nanobind | Python bindings for `sae_embed` |
## Build
```bash
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
```
This also builds `sae_embed`, a Python module (via nanobind) that loads the
SCRFD detector and ArcFace embedder once and exposes a reusable `embed()`
method. The gallery-builder scripts (`make_gallery.py`,
`make_jellyfin_gallery.py`, `movienet_eval.py`) import it directly — there is
no subprocess fallback, so if it's missing they exit with a build instruction:
```bash
cmake --build build --target sae_embed
```
Optional flags:
| Flag | Default | Effect |
|---|---|---|
| `-DSAE_WEB_DEBUG=ON` | OFF | Enables KPN web debug UI at `localhost:9090` |
## Models
The ONNX model weights live in `models/` (tracked via Git LFS):
- `LVFace-B_Glint360K.onnx` — LVFace embedder (ViT backbone, ICCV 2025), the default
(best F1 in the rep4 model bake-off, see `docs/rep4-optimizer-results.md`)
- `arcface_w600k_r50.onnx` — ArcFace embedder, previous default
- `arcface_w600k_mbf.onnx`, `arcface_r18.onnx` — lighter ArcFace alternatives
- `face_detection_yunet_2023mar.onnx` — YuNet face detector
- `scrfd_500m_bnkps.onnx` — SCRFD face detector
### LVFace
[LVFace](https://github.com/bytedance/LVFace) is a Vision-Transformer face
recognition model. The `LVFace-B_Glint360K.onnx` export shares ArcFace's I/O
contract (112×112 aligned BGR crop → L2-normalised 512-d embedding) and its
`(x 127.5)/128` input scaling, so it slots straight into the existing embedder
— just point `--arcface-model` at it:
```bash
./build/scene_analyze --arcface-model models/LVFace-B_Glint360K.onnx \
--gallery gallery.h5 --movie movie.mp4
```
> **Important:** embeddings from different recognition models are not
> interchangeable. A gallery (and its calibration cache) must be built with the
> **same** embedder used for analysis — rebuild the gallery with
> `--arcface models/LVFace-B_Glint360K.onnx` before analysing with LVFace.
If they are missing (e.g. LFS not fetched), re-download them with:
```bash
bash scripts/download_models.sh
```
> **Model licensing:** the model weights carry their own licenses, separate
> from this project's MIT license, and are redistributed here under those
> upstream terms. Several — notably the InsightFace "buffalo" models (ArcFace /
> SCRFD) — are licensed for **non-commercial research use only**. Review and
> comply with each model's license before use.
## Binaries
| Binary | Description |
|---|---|
| `scene_analyze` | Main analysis pipeline, writes JSON output |
| `scene_analyze_debug` | Same as above + per-frame annotated JPEGs (`SAE_DEBUG=1`) |
| `scene_preview` | Live OpenCV display window while analysing |
| `build_gallery` | Offline gallery builder from a directory of images |
| `embed_faces` | CLI: image(s) → embedding JSON, used by gallery-builder scripts |
| `sae_embed` | Python module (nanobind) used by gallery-builder scripts — loads SCRFD+ArcFace once |
### `scene_analyze`
```bash
./build/scene_analyze --gallery gallery.h5 --movie movie.mp4 [options]
```
Key options:
| Flag | Default | Description |
|---|---|---|
| `--fps` | 1 | Frames per second to sample (510 recommended for tracking) |
| `--prob-threshold` | 0.5 | Minimum calibrated match probability |
| `--match-threshold` | — | Raw cosine similarity threshold (fallback) |
| `--extinction` | 5s | How long a track persists after last detection |
| `--track-alpha` | — | IoU vs. embedding weight in Hungarian assignment |
| `--track-min-iou` | — | Minimum IoU gate for spatial assignment |
| `--track-max-embed` | — | Maximum embedding distance gate |
| `--track-max-missing` | — | Frames a track survives without a detection |
### Gallery builders
**Per-movie (TMDB):**
```bash
python3 scripts/make_gallery.py --tmdb-key <TMDB_KEY> --movie-id <TMDB_ID> --output gallery.h5
```
Fetches cast images from TMDB and embeds them via `sae_embed`.
**Whole-library (Jellyfin):**
```bash
python3 scripts/make_jellyfin_gallery.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--output gallery.h5
```
Scans every Movie/Series in Jellyfin, collects the unique cast across the
whole library, downloads each actor's headshot directly from Jellyfin (no
TMDB key needed), and embeds them via `sae_embed` into one global
gallery.h5. Since `identity_matcher` scores faces against the entire
gallery, `scene_analyze` can then recognise any actor from your library in
any film — not just the cast listed for that one title. Pass `--merge` on
later runs to only embed actors newly added to the library. Pass
`--tmdb-key` to fall back to TMDB profile images for actors with no usable
image cached in Jellyfin.
Jellyfin/TMDB lookups and image downloads for different actors run
concurrently (`--workers`, default 8). Embedding is GPU-bound, so it's
gated separately via `--embed-concurrency` (default 1) — only that many
embed calls run at once while other actors' downloads continue in the
background.
To restrict a single-title run to that title's credited cast (faster, fewer
look-alike mismatches), filter the global gallery first:
```bash
python3 scripts/filter_gallery.py \
--gallery gallery.h5 \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--output gallery_matrix.h5
```
## Running directly from Jellyfin
`scripts/run_from_jellyfin.py` resolves a title to its media file via the
Jellyfin API, filters the gallery to that title's cast, and runs
`scene_analyze` in one step. Requires this tool to run on a host that shares
Jellyfin's media mount (it uses the item's on-disk `Path`, not a stream URL):
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--gallery gallery.h5 \
-- --fps 5 --verbosity 2
```
Anything after `--` is passed through to `scene_analyze` unchanged. Pass
`--no-filter` to use the gallery as-is (skip per-title cast filtering), or
`--item-id` instead of `--title` to skip the search.
After a successful run, the output JSON is pushed to the [JRay Jellyfin
plugin](https://gitea.tourolle.paris/dtourolle/jRay)'s Truth endpoint
(`PUT /Plugins/JRay/Items/{itemId}/Truth`) so
Jellyfin picks it up immediately, using `--api-key` (must be an
**Administrator** key for the push to succeed). Pass `--no-push` to skip
this and only write `--output` locally (e.g. for local debugging).
### Worker mode
Pass `--worker` instead of `--item-id`/`--title` to run this as an extraction
worker: it polls the JRay plugin's `GET /Plugins/JRay/Tasks/Pending` endpoint
for a random batch of items with no truth data yet, processes each one, and
pushes the result back. The endpoint's sampling spreads work across the
backlog without any server-side task tracking, so any number of workers can
poll the same library concurrently.
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <ADMIN_API_KEY> \
--gallery whole_gallery.h5 \
--worker \
-- --fps 5
```
- `--poll-limit` — batch size requested from `Tasks/Pending` (default 10, max 100)
- `--poll-interval` — seconds to sleep between polls when the backlog is empty (default 60)
- `--once` — process a single batch and exit instead of looping forever
A failure on one item (bad path, push rejected, etc.) is logged and the
worker moves on to the next item rather than exiting.
## Output format
**Minimal** (default) — Jellyfin-ready:
```json
[
{ "actor": "Name", "start": 12.0, "end": 45.5 }
]
```
**Standard** — per-frame detail with bounding boxes, similarity scores, and track IDs.
## Evaluation
Scripts in `eval/` and `scripts/movienet_*.py` support benchmarking against the MovieNet dataset.
## License
The source code in this repository is licensed under the [MIT License](LICENSE).
The MIT license covers **only the code**. The model weights in `models/` (see
[Models](#models)) are redistributed under their own licenses — several for
non-commercial research use only. Third-party libraries this software links
against (OpenCV, ONNX Runtime, FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind,
and others) likewise carry their own licenses.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 137 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 164 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 201 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 311 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 155 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 161 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 142 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 174 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 141 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Danish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){var r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){function e(){var e,r=f.cursor+3;if(d=f.limit,0<=r&&r<=f.limit){for(a=r;;){if(e=f.cursor,f.in_grouping(w,97,248)){f.cursor=e;break}if(f.cursor=e,e>=f.limit)return;f.cursor++}for(;!f.out_grouping(w,97,248);){if(f.cursor>=f.limit)return;f.cursor++}d=f.cursor,d<a&&(d=a)}}function n(){var e,r;if(f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(c,32),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del();break;case 2:f.in_grouping_b(p,97,229)&&f.slice_del()}}function t(){var e,r=f.limit-f.cursor;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.find_among_b(l,4)?(f.bra=f.cursor,f.limit_backward=e,f.cursor=f.limit-r,f.cursor>f.limit_backward&&(f.cursor--,f.bra=f.cursor,f.slice_del())):f.limit_backward=e)}function s(){var e,r,i,n=f.limit-f.cursor;if(f.ket=f.cursor,f.eq_s_b(2,"st")&&(f.bra=f.cursor,f.eq_s_b(2,"ig")&&f.slice_del()),f.cursor=f.limit-n,f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(m,5),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del(),i=f.limit-f.cursor,t(),f.cursor=f.limit-i;break;case 2:f.slice_from("løs")}}function o(){var e;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.out_grouping_b(w,97,248)?(f.bra=f.cursor,u=f.slice_to(u),f.limit_backward=e,f.eq_v_b(u)&&f.slice_del()):f.limit_backward=e)}var a,d,u,c=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],l=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],w=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],p=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],f=new i;this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var r=f.cursor;return e(),f.limit_backward=r,f.cursor=f.limit,n(),f.cursor=f.limit,t(),f.cursor=f.limit,s(),f.cursor=f.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hi=function(){this.pipeline.reset(),this.pipeline.add(e.hi.trimmer,e.hi.stopWordFilter,e.hi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hi.stemmer))},e.hi.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿa-zA-Z-zA-0-9-",e.hi.trimmer=e.trimmerSupport.generateTrimmer(e.hi.wordCharacters),e.Pipeline.registerFunction(e.hi.trimmer,"trimmer-hi"),e.hi.stopWordFilter=e.generateStopWordFilter("अत अपना अपनी अपने अभी अंदर आदि आप इत्यादि इन इनका इन्हीं इन्हें इन्हों इस इसका इसकी इसके इसमें इसी इसे उन उनका उनकी उनके उनको उन्हीं उन्हें उन्हों उस उसके उसी उसे एक एवं एस ऐसे और कई कर करता करते करना करने करें कहते कहा का काफ़ी कि कितना किन्हें किन्हों किया किर किस किसी किसे की कुछ कुल के को कोई कौन कौनसा गया घर जब जहाँ जा जितना जिन जिन्हें जिन्हों जिस जिसे जीधर जैसा जैसे जो तक तब तरह तिन तिन्हें तिन्हों तिस तिसे तो था थी थे दबारा दिया दुसरा दूसरे दो द्वारा न नके नहीं ना निहायत नीचे ने पर पहले पूरा पे फिर बनी बही बहुत बाद बाला बिलकुल भी भीतर मगर मानो मे में यदि यह यहाँ यही या यिह ये रखें रहा रहे ऱ्वासा लिए लिये लेकिन व वग़ैरह वर्ग वह वहाँ वहीं वाले वुह वे वो सकता सकते सबसे सभी साथ साबुत साभ सारा से सो संग ही हुआ हुई हुए है हैं हो होता होती होते होना होने".split(" ")),e.hi.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.hi.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var t=i.toString().toLowerCase().replace(/^\s+/,"");return r.cut(t).split("|")},e.Pipeline.registerFunction(e.hi.stemmer,"stemmer-hi"),e.Pipeline.registerFunction(e.hi.stopWordFilter,"stopWordFilter-hi")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hy=function(){this.pipeline.reset(),this.pipeline.add(e.hy.trimmer,e.hy.stopWordFilter)},e.hy.wordCharacters="[A-Za-z԰-֏ff-ﭏ]",e.hy.trimmer=e.trimmerSupport.generateTrimmer(e.hy.wordCharacters),e.Pipeline.registerFunction(e.hy.trimmer,"trimmer-hy"),e.hy.stopWordFilter=e.generateStopWordFilter("դու և եք էիր էիք հետո նաև նրանք որը վրա է որ պիտի են այս մեջ ն իր ու ի այդ որոնք այն կամ էր մի ես համար այլ իսկ էին ենք հետ ին թ էինք մենք նրա նա դուք եմ էի ըստ որպես ում".split(" ")),e.Pipeline.registerFunction(e.hy.stopWordFilter,"stopWordFilter-hy"),e.hy.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}(),e.Pipeline.registerFunction(e.hy.stemmer,"stemmer-hy")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.ja=function(){this.pipeline.reset(),this.pipeline.add(e.ja.trimmer,e.ja.stopWordFilter,e.ja.stemmer),r?this.tokenizer=e.ja.tokenizer:(e.tokenizer&&(e.tokenizer=e.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.ja.tokenizer))};var t=new e.TinySegmenter;e.ja.tokenizer=function(i){var n,o,s,p,a,u,m,l,c,f;if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(o=i.toString().toLowerCase().replace(/^\s+/,""),n=o.length-1;n>=0;n--)if(/\S/.test(o.charAt(n))){o=o.substring(0,n+1);break}for(a=[],s=o.length,c=0,l=0;c<=s;c++)if(u=o.charAt(c),m=c-l,u.match(/\s/)||c==s){if(m>0)for(p=t.segment(o.slice(l,c)).filter(function(e){return!!e}),f=l,n=0;n<p.length;n++)r?a.push(new e.Token(p[n],{position:[f,p[n].length],index:a.length})):a.push(p[n]),f+=p[n].length;l=c+1}return a},e.ja.stemmer=function(){return function(e){return e}}(),e.Pipeline.registerFunction(e.ja.stemmer,"stemmer-ja"),e.ja.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーア-ン゙a-zA-Z-zA-0-9-",e.ja.trimmer=e.trimmerSupport.generateTrimmer(e.ja.wordCharacters),e.Pipeline.registerFunction(e.ja.trimmer,"trimmer-ja"),e.ja.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.ja.stopWordFilter,"stopWordFilter-ja"),e.jp=e.ja,e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.Pipeline.registerFunction(e.jp.trimmer,"trimmer-jp"),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});
-1
View File
@@ -1 +0,0 @@
module.exports=require("./lunr.ja");
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.kn=function(){this.pipeline.reset(),this.pipeline.add(e.kn.trimmer,e.kn.stopWordFilter,e.kn.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.kn.stemmer))},e.kn.wordCharacters="ಀ-಄ಅ-ಔಕ-ಹಾ-ೌ಼-ಽೕ-ೖೝ-ೞೠ-ೡೢ-ೣ೤೥೦-೯ೱ-ೳ",e.kn.trimmer=e.trimmerSupport.generateTrimmer(e.kn.wordCharacters),e.Pipeline.registerFunction(e.kn.trimmer,"trimmer-kn"),e.kn.stopWordFilter=e.generateStopWordFilter("ಮತ್ತು ಈ ಒಂದು ರಲ್ಲಿ ಹಾಗೂ ಎಂದು ಅಥವಾ ಇದು ರ ಅವರು ಎಂಬ ಮೇಲೆ ಅವರ ತನ್ನ ಆದರೆ ತಮ್ಮ ನಂತರ ಮೂಲಕ ಹೆಚ್ಚು ನ ಆ ಕೆಲವು ಅನೇಕ ಎರಡು ಹಾಗು ಪ್ರಮುಖ ಇದನ್ನು ಇದರ ಸುಮಾರು ಅದರ ಅದು ಮೊದಲ ಬಗ್ಗೆ ನಲ್ಲಿ ರಂದು ಇತರ ಅತ್ಯಂತ ಹೆಚ್ಚಿನ ಸಹ ಸಾಮಾನ್ಯವಾಗಿ ನೇ ಹಲವಾರು ಹೊಸ ದಿ ಕಡಿಮೆ ಯಾವುದೇ ಹೊಂದಿದೆ ದೊಡ್ಡ ಅನ್ನು ಇವರು ಪ್ರಕಾರ ಇದೆ ಮಾತ್ರ ಕೂಡ ಇಲ್ಲಿ ಎಲ್ಲಾ ವಿವಿಧ ಅದನ್ನು ಹಲವು ರಿಂದ ಕೇವಲ ದ ದಕ್ಷಿಣ ಗೆ ಅವನ ಅತಿ ನೆಯ ಬಹಳ ಕೆಲಸ ಎಲ್ಲ ಪ್ರತಿ ಇತ್ಯಾದಿ ಇವು ಬೇರೆ ಹೀಗೆ ನಡುವೆ ಇದಕ್ಕೆ ಎಸ್ ಇವರ ಮೊದಲು ಶ್ರೀ ಮಾಡುವ ಇದರಲ್ಲಿ ರೀತಿಯ ಮಾಡಿದ ಕಾಲ ಅಲ್ಲಿ ಮಾಡಲು ಅದೇ ಈಗ ಅವು ಗಳು ಎ ಎಂಬುದು ಅವನು ಅಂದರೆ ಅವರಿಗೆ ಇರುವ ವಿಶೇಷ ಮುಂದೆ ಅವುಗಳ ಮುಂತಾದ ಮೂಲ ಬಿ ಮೀ ಒಂದೇ ಇನ್ನೂ ಹೆಚ್ಚಾಗಿ ಮಾಡಿ ಅವರನ್ನು ಇದೇ ಯ ರೀತಿಯಲ್ಲಿ ಜೊತೆ ಅದರಲ್ಲಿ ಮಾಡಿದರು ನಡೆದ ಆಗ ಮತ್ತೆ ಪೂರ್ವ ಆತ ಬಂದ ಯಾವ ಒಟ್ಟು ಇತರೆ ಹಿಂದೆ ಪ್ರಮಾಣದ ಗಳನ್ನು ಕುರಿತು ಯು ಆದ್ದರಿಂದ ಅಲ್ಲದೆ ನಗರದ ಮೇಲಿನ ಏಕೆಂದರೆ ರಷ್ಟು ಎಂಬುದನ್ನು ಬಾರಿ ಎಂದರೆ ಹಿಂದಿನ ಆದರೂ ಆದ ಸಂಬಂಧಿಸಿದ ಮತ್ತೊಂದು ಸಿ ಆತನ ".split(" ")),e.kn.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.kn.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var n=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(n).split("|")},e.Pipeline.registerFunction(e.kn.stemmer,"stemmer-kn"),e.Pipeline.registerFunction(e.kn.stopWordFilter,"stopWordFilter-kn")}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var t=Array.prototype.slice.call(arguments),i=t.join("-"),r="",n=[],s=[],p=0;p<t.length;++p)"en"==t[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[t[p]].wordCharacters,e[t[p]].stopWordFilter&&n.unshift(e[t[p]].stopWordFilter),e[t[p]].stemmer&&(n.push(e[t[p]].stemmer),s.push(e[t[p]].stemmer)));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+i),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});
File diff suppressed because one or more lines are too long
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Norwegian` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){function e(){var e,r=w.cursor+3;if(a=w.limit,0<=r||r<=w.limit){for(s=r;;){if(e=w.cursor,w.in_grouping(d,97,248)){w.cursor=e;break}if(e>=w.limit)return;w.cursor=e+1}for(;!w.out_grouping(d,97,248);){if(w.cursor>=w.limit)return;w.cursor++}a=w.cursor,a<s&&(a=s)}}function i(){var e,r,n;if(w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(m,29),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:n=w.limit-w.cursor,w.in_grouping_b(c,98,122)?w.slice_del():(w.cursor=w.limit-n,w.eq_s_b(1,"k")&&w.out_grouping_b(d,97,248)&&w.slice_del());break;case 3:w.slice_from("er")}}function t(){var e,r=w.limit-w.cursor;w.cursor>=a&&(e=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,w.find_among_b(u,2)?(w.bra=w.cursor,w.limit_backward=e,w.cursor=w.limit-r,w.cursor>w.limit_backward&&(w.cursor--,w.bra=w.cursor,w.slice_del())):w.limit_backward=e)}function o(){var e,r;w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(l,11),e?(w.bra=w.cursor,w.limit_backward=r,1==e&&w.slice_del()):w.limit_backward=r)}var s,a,m=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],u=[new r("dt",-1,-1),new r("vt",-1,-1)],l=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],c=[119,125,149,1],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,i(),w.cursor=w.limit,t(),w.cursor=w.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}}(),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sa=function(){this.pipeline.reset(),this.pipeline.add(e.sa.trimmer,e.sa.stopWordFilter,e.sa.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sa.stemmer))},e.sa.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿ꣠-꣱ꣲ-ꣷ꣸-ꣻ꣼-ꣽꣾ-ꣿᆰ0-ᆰ9",e.sa.trimmer=e.trimmerSupport.generateTrimmer(e.sa.wordCharacters),e.Pipeline.registerFunction(e.sa.trimmer,"trimmer-sa"),e.sa.stopWordFilter=e.generateStopWordFilter('तथा अयम्‌ एकम्‌ इत्यस्मिन्‌ तथा तत्‌ वा अयम्‌ इत्यस्य ते आहूत उपरि तेषाम्‌ किन्तु तेषाम्‌ तदा इत्यनेन अधिकः इत्यस्य तत्‌ केचन बहवः द्वि तथा महत्वपूर्णः अयम्‌ अस्य विषये अयं अस्ति तत्‌ प्रथमः विषये इत्युपरि इत्युपरि इतर अधिकतमः अधिकः अपि सामान्यतया ठ इतरेतर नूतनम्‌ द न्यूनम्‌ कश्चित्‌ वा विशालः द सः अस्ति तदनुसारम् तत्र अस्ति केवलम्‌ अपि अत्र सर्वे विविधाः तत्‌ बहवः यतः इदानीम्‌ द दक्षिण इत्यस्मै तस्य उपरि नथ अतीव कार्यम्‌ सर्वे एकैकम्‌ इत्यादि। एते सन्ति उत इत्थम्‌ मध्ये एतदर्थं . स कस्य प्रथमः श्री. करोति अस्मिन् प्रकारः निर्मिता कालः तत्र कर्तुं समान अधुना ते सन्ति स एकः अस्ति सः अर्थात् तेषां कृते . स्थितम् विशेषः अग्रिम तेषाम्‌ समान स्रोतः ख म समान इदानीमपि अधिकतया करोतु ते समान इत्यस्य वीथी सह यस्मिन् कृतवान्‌ धृतः तदा पुनः पूर्वं सः आगतः किम्‌ कुल इतर पुरा मात्रा स विषये उ अतएव अपि नगरस्य उपरि यतः प्रतिशतं कतरः कालः साधनानि भूत तथापि जात सम्बन्धि अन्यत्‌ ग अतः अस्माकं स्वकीयाः अस्माकं इदानीं अन्तः इत्यादयः भवन्तः इत्यादयः एते एताः तस्य अस्य इदम् एते तेषां तेषां तेषां तान् तेषां तेषां तेषां समानः सः एकः च तादृशाः बहवः अन्ये च वदन्ति यत् कियत् कस्मै कस्मै यस्मै यस्मै यस्मै यस्मै न अतिनीचः किन्तु प्रथमं सम्पूर्णतया ततः चिरकालानन्तरं पुस्तकं सम्पूर्णतया अन्तः किन्तु अत्र वा इह इव श्रद्धाय अवशिष्यते परन्तु अन्ये वर्गाः सन्ति ते सन्ति शक्नुवन्ति सर्वे मिलित्वा सर्वे एकत्र"'.split(" ")),e.sa.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.sa.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var i=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(i).split("|")},e.Pipeline.registerFunction(e.sa.stemmer,"stemmer-sa"),e.Pipeline.registerFunction(e.sa.stopWordFilter,"stopWordFilter-sa")}});
@@ -1 +0,0 @@
!function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s<t;s++)i[s]=r.charCodeAt(s);return i},!r&&""!=r||!t&&0!=t||!i)throw"Bad Among initialisation: s:"+r+", substring_i: "+t+", result: "+i;this.s_size=r.length,this.s=this.toCharArray(r),this.substring_i=t,this.result=i,this.method=s},SnowballProgram:function(){var r;return{bra:0,ket:0,limit:0,cursor:0,limit_backward:0,setCurrent:function(t){r=t,this.cursor=0,this.limit=t.length,this.limit_backward=0,this.bra=this.cursor,this.ket=this.limit},getCurrent:function(){var t=r;return r=null,t},in_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},in_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},out_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e>s||e<i)return this.cursor++,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},out_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e>s||e<i)return this.cursor--,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},eq_s:function(t,i){if(this.limit-this.cursor<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor+s)!=i.charCodeAt(s))return!1;return this.cursor+=t,!0},eq_s_b:function(t,i){if(this.cursor-this.limit_backward<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor-t+s)!=i.charCodeAt(s))return!1;return this.cursor-=t,!0},find_among:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=l;m<_.s_size;m++){if(n+l==u){f=-1;break}if(f=r.charCodeAt(n+l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n+_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n+_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},find_among_b:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit_backward,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=_.s_size-1-l;m>=0;m--){if(n-l==u){f=-1;break}if(f=r.charCodeAt(n-1-l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n-_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n-_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},replace_s:function(t,i,s){var e=s.length-(i-t),n=r.substring(0,t),u=r.substring(i);return r=n+s+u,this.limit+=e,this.cursor>=i?this.cursor+=e:this.cursor>t&&(this.cursor=t),e},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>r.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),r.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}});
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Swedish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=new function(){function e(){var e,r=w.cursor+3;if(o=w.limit,0<=r||r<=w.limit){for(a=r;;){if(e=w.cursor,w.in_grouping(l,97,246)){w.cursor=e;break}if(w.cursor=e,w.cursor>=w.limit)return;w.cursor++}for(;!w.out_grouping(l,97,246);){if(w.cursor>=w.limit)return;w.cursor++}o=w.cursor,o<a&&(o=a)}}function t(){var e,r=w.limit_backward;if(w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(u,37),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.in_grouping_b(d,98,121)&&w.slice_del()}}function i(){var e=w.limit_backward;w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.find_among_b(c,7)&&(w.cursor=w.limit,w.ket=w.cursor,w.cursor>w.limit_backward&&(w.bra=--w.cursor,w.slice_del())),w.limit_backward=e)}function s(){var e,r;if(w.cursor>=o){if(r=w.limit_backward,w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(m,5))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.slice_from("lös");break;case 3:w.slice_from("full")}w.limit_backward=r}}var a,o,u=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],c=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],l=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],d=[119,127,149],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,t(),w.cursor=w.limit,i(),w.cursor=w.limit,s(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return t.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}}(),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.ta=function(){this.pipeline.reset(),this.pipeline.add(e.ta.trimmer,e.ta.stopWordFilter,e.ta.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ta.stemmer))},e.ta.wordCharacters="஀-உஊ-ஏஐ-ஙச-ட஠-னப-யர-ஹ஺-ிீ-௉ொ-௏ௐ-௙௚-௟௠-௩௪-௯௰-௹௺-௿a-zA-Z-zA-0-9-",e.ta.trimmer=e.trimmerSupport.generateTrimmer(e.ta.wordCharacters),e.Pipeline.registerFunction(e.ta.trimmer,"trimmer-ta"),e.ta.stopWordFilter=e.generateStopWordFilter("அங்கு அங்கே அது அதை அந்த அவர் அவர்கள் அவள் அவன் அவை ஆக ஆகவே ஆகையால் ஆதலால் ஆதலினால் ஆனாலும் ஆனால் இங்கு இங்கே இது இதை இந்த இப்படி இவர் இவர்கள் இவள் இவன் இவை இவ்வளவு உனக்கு உனது உன் உன்னால் எங்கு எங்கே எது எதை எந்த எப்படி எவர் எவர்கள் எவள் எவன் எவை எவ்வளவு எனக்கு எனது எனவே என் என்ன என்னால் ஏது ஏன் தனது தன்னால் தானே தான் நாங்கள் நாம் நான் நீ நீங்கள்".split(" ")),e.ta.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.ta.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.ta.stemmer,"stemmer-ta"),e.Pipeline.registerFunction(e.ta.stopWordFilter,"stopWordFilter-ta")}});
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.te=function(){this.pipeline.reset(),this.pipeline.add(e.te.trimmer,e.te.stopWordFilter,e.te.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.te.stemmer))},e.te.wordCharacters="ఀ-ఄఅ-ఔక-హా-ౌౕ-ౖౘ-ౚౠ-ౡౢ-ౣ౦-౯౸-౿఼ఽ్ౝ౷౤౥",e.te.trimmer=e.trimmerSupport.generateTrimmer(e.te.wordCharacters),e.Pipeline.registerFunction(e.te.trimmer,"trimmer-te"),e.te.stopWordFilter=e.generateStopWordFilter("అందరూ అందుబాటులో అడగండి అడగడం అడ్డంగా అనుగుణంగా అనుమతించు అనుమతిస్తుంది అయితే ఇప్పటికే ఉన్నారు ఎక్కడైనా ఎప్పుడు ఎవరైనా ఎవరో ఏ ఏదైనా ఏమైనప్పటికి ఒక ఒకరు కనిపిస్తాయి కాదు కూడా గా గురించి చుట్టూ చేయగలిగింది తగిన తర్వాత దాదాపు దూరంగా నిజంగా పై ప్రకారం ప్రక్కన మధ్య మరియు మరొక మళ్ళీ మాత్రమే మెచ్చుకో వద్ద వెంట వేరుగా వ్యతిరేకంగా సంబంధం".split(" ")),e.te.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.te.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.te.stemmer,"stemmer-te"),e.Pipeline.registerFunction(e.te.stopWordFilter,"stopWordFilter-te")}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.th=function(){this.pipeline.reset(),this.pipeline.add(e.th.trimmer),r?this.tokenizer=e.th.tokenizer:(e.tokenizer&&(e.tokenizer=e.th.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.th.tokenizer))},e.th.wordCharacters="[฀-๿]",e.th.trimmer=e.trimmerSupport.generateTrimmer(e.th.wordCharacters),e.Pipeline.registerFunction(e.th.trimmer,"trimmer-th");var t=e.wordcut;t.init(),e.th.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t):t});var n=i.toString().replace(/^\s+/,"");return t.cut(n).split("|")}}});
File diff suppressed because one or more lines are too long
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.vi=function(){this.pipeline.reset(),this.pipeline.add(e.vi.stopWordFilter,e.vi.trimmer)},e.vi.wordCharacters="[A-Za-ẓ̀͐́͑̉̃̓ÂâÊêÔôĂ-ăĐ-đƠ-ơƯ-ư]",e.vi.trimmer=e.trimmerSupport.generateTrimmer(e.vi.wordCharacters),e.Pipeline.registerFunction(e.vi.trimmer,"trimmer-vi"),e.vi.stopWordFilter=e.generateStopWordFilter("là cái nhưng mà".split(" "))}});
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r(require("@node-rs/jieba")):r()(e.lunr)}(this,function(e){return function(r,t){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i="2"==r.version[0];r.zh=function(){this.pipeline.reset(),this.pipeline.add(r.zh.trimmer,r.zh.stopWordFilter,r.zh.stemmer),i?this.tokenizer=r.zh.tokenizer:(r.tokenizer&&(r.tokenizer=r.zh.tokenizer),this.tokenizerFn&&(this.tokenizerFn=r.zh.tokenizer))},r.zh.tokenizer=function(n){if(!arguments.length||null==n||void 0==n)return[];if(Array.isArray(n))return n.map(function(e){return i?new r.Token(e.toLowerCase()):e.toLowerCase()});t&&e.load(t);var o=n.toString().trim().toLowerCase(),s=[];e.cut(o,!0).forEach(function(e){s=s.concat(e.split(" "))}),s=s.filter(function(e){return!!e});var u=0;return s.map(function(e,t){if(i){var n=o.indexOf(e,u),s={};return s.position=[n,e.length],s.index=t,u=n,new r.Token(e,s)}return e})},r.zh.wordCharacters="\\w一-龥",r.zh.trimmer=r.trimmerSupport.generateTrimmer(r.zh.wordCharacters),r.Pipeline.registerFunction(r.zh.trimmer,"trimmer-zh"),r.zh.stemmer=function(){return function(e){return e}}(),r.Pipeline.registerFunction(r.zh.stemmer,"stemmer-zh"),r.zh.stopWordFilter=r.generateStopWordFilter("的 一 不 在 人 有 是 为 為 以 于 於 上 他 而 后 後 之 来 來 及 了 因 下 可 到 由 这 這 与 與 也 此 但 并 並 个 個 其 已 无 無 小 我 们 們 起 最 再 今 去 好 只 又 或 很 亦 某 把 那 你 乃 它 吧 被 比 别 趁 当 當 从 從 得 打 凡 儿 兒 尔 爾 该 該 各 给 給 跟 和 何 还 還 即 几 幾 既 看 据 據 距 靠 啦 另 么 麽 每 嘛 拿 哪 您 凭 憑 且 却 卻 让 讓 仍 啥 如 若 使 谁 誰 虽 雖 随 隨 同 所 她 哇 嗡 往 些 向 沿 哟 喲 用 咱 则 則 怎 曾 至 致 着 著 诸 諸 自".split(" ")),r.Pipeline.registerFunction(r.zh.stopWordFilter,"stopWordFilter-zh")}});
-206
View File
@@ -1,206 +0,0 @@
/**
* export the module via AMD, CommonJS or as a browser global
* Export code from https://github.com/umdjs/umd/blob/master/returnExports.js
*/
;(function (root, factory) {
if (typeof define === 'function' && define.amd) {
// AMD. Register as an anonymous module.
define(factory)
} else if (typeof exports === 'object') {
/**
* Node. Does not work with strict CommonJS, but
* only CommonJS-like environments that support module.exports,
* like Node.
*/
module.exports = factory()
} else {
// Browser globals (root is window)
factory()(root.lunr);
}
}(this, function () {
/**
* Just return a value to define the module export.
* This example returns an object, but the module
* can return a function as the exported value.
*/
return function(lunr) {
// TinySegmenter 0.1 -- Super compact Japanese tokenizer in Javascript
// (c) 2008 Taku Kudo <taku@chasen.org>
// TinySegmenter is freely distributable under the terms of a new BSD licence.
// For details, see http://chasen.org/~taku/software/TinySegmenter/LICENCE.txt
function TinySegmenter() {
var patterns = {
"[一二三四五六七八九十百千万億兆]":"M",
"[一-龠々〆ヵヶ]":"H",
"[ぁ-ん]":"I",
"[ァ-ヴーア-ン゙ー]":"K",
"[a-zA-Z-zA-]":"A",
"[0-9-]":"N"
}
this.chartype_ = [];
for (var i in patterns) {
var regexp = new RegExp(i);
this.chartype_.push([regexp, patterns[i]]);
}
this.BIAS__ = -332
this.BC1__ = {"HH":6,"II":2461,"KH":406,"OH":-1378};
this.BC2__ = {"AA":-3267,"AI":2744,"AN":-878,"HH":-4070,"HM":-1711,"HN":4012,"HO":3761,"IA":1327,"IH":-1184,"II":-1332,"IK":1721,"IO":5492,"KI":3831,"KK":-8741,"MH":-3132,"MK":3334,"OO":-2920};
this.BC3__ = {"HH":996,"HI":626,"HK":-721,"HN":-1307,"HO":-836,"IH":-301,"KK":2762,"MK":1079,"MM":4034,"OA":-1652,"OH":266};
this.BP1__ = {"BB":295,"OB":304,"OO":-125,"UB":352};
this.BP2__ = {"BO":60,"OO":-1762};
this.BQ1__ = {"BHH":1150,"BHM":1521,"BII":-1158,"BIM":886,"BMH":1208,"BNH":449,"BOH":-91,"BOO":-2597,"OHI":451,"OIH":-296,"OKA":1851,"OKH":-1020,"OKK":904,"OOO":2965};
this.BQ2__ = {"BHH":118,"BHI":-1159,"BHM":466,"BIH":-919,"BKK":-1720,"BKO":864,"OHH":-1139,"OHM":-181,"OIH":153,"UHI":-1146};
this.BQ3__ = {"BHH":-792,"BHI":2664,"BII":-299,"BKI":419,"BMH":937,"BMM":8335,"BNN":998,"BOH":775,"OHH":2174,"OHM":439,"OII":280,"OKH":1798,"OKI":-793,"OKO":-2242,"OMH":-2402,"OOO":11699};
this.BQ4__ = {"BHH":-3895,"BIH":3761,"BII":-4654,"BIK":1348,"BKK":-1806,"BMI":-3385,"BOO":-12396,"OAH":926,"OHH":266,"OHK":-2036,"ONN":-973};
this.BW1__ = {",と":660,",同":727,"B1あ":1404,"B1同":542,"、と":660,"、同":727,"」と":1682,"あっ":1505,"いう":1743,"いっ":-2055,"いる":672,"うし":-4817,"うん":665,"から":3472,"がら":600,"こう":-790,"こと":2083,"こん":-1262,"さら":-4143,"さん":4573,"した":2641,"して":1104,"すで":-3399,"そこ":1977,"それ":-871,"たち":1122,"ため":601,"った":3463,"つい":-802,"てい":805,"てき":1249,"でき":1127,"です":3445,"では":844,"とい":-4915,"とみ":1922,"どこ":3887,"ない":5713,"なっ":3015,"など":7379,"なん":-1113,"にし":2468,"には":1498,"にも":1671,"に対":-912,"の一":-501,"の中":741,"ませ":2448,"まで":1711,"まま":2600,"まる":-2155,"やむ":-1947,"よっ":-2565,"れた":2369,"れで":-913,"をし":1860,"を見":731,"亡く":-1886,"京都":2558,"取り":-2784,"大き":-2604,"大阪":1497,"平方":-2314,"引き":-1336,"日本":-195,"本当":-2423,"毎日":-2113,"目指":-724,"B1あ":1404,"B1同":542,"」と":1682};
this.BW2__ = {"..":-11822,"11":-669,"――":-5730,"−−":-13175,"いう":-1609,"うか":2490,"かし":-1350,"かも":-602,"から":-7194,"かれ":4612,"がい":853,"がら":-3198,"きた":1941,"くな":-1597,"こと":-8392,"この":-4193,"させ":4533,"され":13168,"さん":-3977,"しい":-1819,"しか":-545,"した":5078,"して":972,"しな":939,"その":-3744,"たい":-1253,"たた":-662,"ただ":-3857,"たち":-786,"たと":1224,"たは":-939,"った":4589,"って":1647,"っと":-2094,"てい":6144,"てき":3640,"てく":2551,"ては":-3110,"ても":-3065,"でい":2666,"でき":-1528,"でし":-3828,"です":-4761,"でも":-4203,"とい":1890,"とこ":-1746,"とと":-2279,"との":720,"とみ":5168,"とも":-3941,"ない":-2488,"なが":-1313,"など":-6509,"なの":2614,"なん":3099,"にお":-1615,"にし":2748,"にな":2454,"によ":-7236,"に対":-14943,"に従":-4688,"に関":-11388,"のか":2093,"ので":-7059,"のに":-6041,"のの":-6125,"はい":1073,"はが":-1033,"はず":-2532,"ばれ":1813,"まし":-1316,"まで":-6621,"まれ":5409,"めて":-3153,"もい":2230,"もの":-10713,"らか":-944,"らし":-1611,"らに":-1897,"りし":651,"りま":1620,"れた":4270,"れて":849,"れば":4114,"ろう":6067,"われ":7901,"を通":-11877,"んだ":728,"んな":-4115,"一人":602,"一方":-1375,"一日":970,"一部":-1051,"上が":-4479,"会社":-1116,"出て":2163,"分の":-7758,"同党":970,"同日":-913,"大阪":-2471,"委員":-1250,"少な":-1050,"年度":-8669,"年間":-1626,"府県":-2363,"手権":-1982,"新聞":-4066,"日新":-722,"日本":-7068,"日米":3372,"曜日":-601,"朝鮮":-2355,"本人":-2697,"東京":-1543,"然と":-1384,"社会":-1276,"立て":-990,"第に":-1612,"米国":-4268,"11":-669};
this.BW3__ = {"あた":-2194,"あり":719,"ある":3846,"い.":-1185,"い。":-1185,"いい":5308,"いえ":2079,"いく":3029,"いた":2056,"いっ":1883,"いる":5600,"いわ":1527,"うち":1117,"うと":4798,"えと":1454,"か.":2857,"か。":2857,"かけ":-743,"かっ":-4098,"かに":-669,"から":6520,"かり":-2670,"が,":1816,"が、":1816,"がき":-4855,"がけ":-1127,"がっ":-913,"がら":-4977,"がり":-2064,"きた":1645,"けど":1374,"こと":7397,"この":1542,"ころ":-2757,"さい":-714,"さを":976,"し,":1557,"し、":1557,"しい":-3714,"した":3562,"して":1449,"しな":2608,"しま":1200,"す.":-1310,"す。":-1310,"する":6521,"ず,":3426,"ず、":3426,"ずに":841,"そう":428,"た.":8875,"た。":8875,"たい":-594,"たの":812,"たり":-1183,"たる":-853,"だ.":4098,"だ。":4098,"だっ":1004,"った":-4748,"って":300,"てい":6240,"てお":855,"ても":302,"です":1437,"でに":-1482,"では":2295,"とう":-1387,"とし":2266,"との":541,"とも":-3543,"どう":4664,"ない":1796,"なく":-903,"など":2135,"に,":-1021,"に、":-1021,"にし":1771,"にな":1906,"には":2644,"の,":-724,"の、":-724,"の子":-1000,"は,":1337,"は、":1337,"べき":2181,"まし":1113,"ます":6943,"まっ":-1549,"まで":6154,"まれ":-793,"らし":1479,"られ":6820,"るる":3818,"れ,":854,"れ、":854,"れた":1850,"れて":1375,"れば":-3246,"れる":1091,"われ":-605,"んだ":606,"んで":798,"カ月":990,"会議":860,"入り":1232,"大会":2217,"始め":1681,"市":965,"新聞":-5055,"日,":974,"日、":974,"社会":2024,"カ月":990};
this.TC1__ = {"AAA":1093,"HHH":1029,"HHM":580,"HII":998,"HOH":-390,"HOM":-331,"IHI":1169,"IOH":-142,"IOI":-1015,"IOM":467,"MMH":187,"OOI":-1832};
this.TC2__ = {"HHO":2088,"HII":-1023,"HMM":-1154,"IHI":-1965,"KKH":703,"OII":-2649};
this.TC3__ = {"AAA":-294,"HHH":346,"HHI":-341,"HII":-1088,"HIK":731,"HOH":-1486,"IHH":128,"IHI":-3041,"IHO":-1935,"IIH":-825,"IIM":-1035,"IOI":-542,"KHH":-1216,"KKA":491,"KKH":-1217,"KOK":-1009,"MHH":-2694,"MHM":-457,"MHO":123,"MMH":-471,"NNH":-1689,"NNO":662,"OHO":-3393};
this.TC4__ = {"HHH":-203,"HHI":1344,"HHK":365,"HHM":-122,"HHN":182,"HHO":669,"HIH":804,"HII":679,"HOH":446,"IHH":695,"IHO":-2324,"IIH":321,"III":1497,"IIO":656,"IOO":54,"KAK":4845,"KKA":3386,"KKK":3065,"MHH":-405,"MHI":201,"MMH":-241,"MMM":661,"MOM":841};
this.TQ1__ = {"BHHH":-227,"BHHI":316,"BHIH":-132,"BIHH":60,"BIII":1595,"BNHH":-744,"BOHH":225,"BOOO":-908,"OAKK":482,"OHHH":281,"OHIH":249,"OIHI":200,"OIIH":-68};
this.TQ2__ = {"BIHH":-1401,"BIII":-1033,"BKAK":-543,"BOOO":-5591};
this.TQ3__ = {"BHHH":478,"BHHM":-1073,"BHIH":222,"BHII":-504,"BIIH":-116,"BIII":-105,"BMHI":-863,"BMHM":-464,"BOMH":620,"OHHH":346,"OHHI":1729,"OHII":997,"OHMH":481,"OIHH":623,"OIIH":1344,"OKAK":2792,"OKHH":587,"OKKA":679,"OOHH":110,"OOII":-685};
this.TQ4__ = {"BHHH":-721,"BHHM":-3604,"BHII":-966,"BIIH":-607,"BIII":-2181,"OAAA":-2763,"OAKK":180,"OHHH":-294,"OHHI":2446,"OHHO":480,"OHIH":-1573,"OIHH":1935,"OIHI":-493,"OIIH":626,"OIII":-4007,"OKAK":-8156};
this.TW1__ = {"につい":-4681,"東京都":2026};
this.TW2__ = {"ある程":-2049,"いった":-1256,"ころが":-2434,"しょう":3873,"その後":-4430,"だって":-1049,"ていた":1833,"として":-4657,"ともに":-4517,"もので":1882,"一気に":-792,"初めて":-1512,"同時に":-8097,"大きな":-1255,"対して":-2721,"社会党":-3216};
this.TW3__ = {"いただ":-1734,"してい":1314,"として":-4314,"につい":-5483,"にとっ":-5989,"に当た":-6247,"ので,":-727,"ので、":-727,"のもの":-600,"れから":-3752,"十二月":-2287};
this.TW4__ = {"いう.":8576,"いう。":8576,"からな":-2348,"してい":2958,"たが,":1516,"たが、":1516,"ている":1538,"という":1349,"ました":5543,"ません":1097,"ようと":-4258,"よると":5865};
this.UC1__ = {"A":484,"K":93,"M":645,"O":-505};
this.UC2__ = {"A":819,"H":1059,"I":409,"M":3987,"N":5775,"O":646};
this.UC3__ = {"A":-1370,"I":2311};
this.UC4__ = {"A":-2643,"H":1809,"I":-1032,"K":-3450,"M":3565,"N":3876,"O":6646};
this.UC5__ = {"H":313,"I":-1238,"K":-799,"M":539,"O":-831};
this.UC6__ = {"H":-506,"I":-253,"K":87,"M":247,"O":-387};
this.UP1__ = {"O":-214};
this.UP2__ = {"B":69,"O":935};
this.UP3__ = {"B":189};
this.UQ1__ = {"BH":21,"BI":-12,"BK":-99,"BN":142,"BO":-56,"OH":-95,"OI":477,"OK":410,"OO":-2422};
this.UQ2__ = {"BH":216,"BI":113,"OK":1759};
this.UQ3__ = {"BA":-479,"BH":42,"BI":1913,"BK":-7198,"BM":3160,"BN":6427,"BO":14761,"OI":-827,"ON":-3212};
this.UW1__ = {",":156,"、":156,"「":-463,"あ":-941,"う":-127,"が":-553,"き":121,"こ":505,"で":-201,"と":-547,"ど":-123,"に":-789,"の":-185,"は":-847,"も":-466,"や":-470,"よ":182,"ら":-292,"り":208,"れ":169,"を":-446,"ん":-137,"・":-135,"主":-402,"京":-268,"区":-912,"午":871,"国":-460,"大":561,"委":729,"市":-411,"日":-141,"理":361,"生":-408,"県":-386,"都":-718,"「":-463,"・":-135};
this.UW2__ = {",":-829,"、":-829,"":892,"「":-645,"」":3145,"あ":-538,"い":505,"う":134,"お":-502,"か":1454,"が":-856,"く":-412,"こ":1141,"さ":878,"ざ":540,"し":1529,"す":-675,"せ":300,"そ":-1011,"た":188,"だ":1837,"つ":-949,"て":-291,"で":-268,"と":-981,"ど":1273,"な":1063,"に":-1764,"の":130,"は":-409,"ひ":-1273,"べ":1261,"ま":600,"も":-1263,"や":-402,"よ":1639,"り":-579,"る":-694,"れ":571,"を":-2516,"ん":2095,"ア":-587,"カ":306,"キ":568,"ッ":831,"三":-758,"不":-2150,"世":-302,"中":-968,"主":-861,"事":492,"人":-123,"会":978,"保":362,"入":548,"初":-3025,"副":-1566,"北":-3414,"区":-422,"大":-1769,"天":-865,"太":-483,"子":-1519,"学":760,"実":1023,"小":-2009,"市":-813,"年":-1060,"強":1067,"手":-1519,"揺":-1033,"政":1522,"文":-1355,"新":-1682,"日":-1815,"明":-1462,"最":-630,"朝":-1843,"本":-1650,"東":-931,"果":-665,"次":-2378,"民":-180,"気":-1740,"理":752,"発":529,"目":-1584,"相":-242,"県":-1165,"立":-763,"第":810,"米":509,"自":-1353,"行":838,"西":-744,"見":-3874,"調":1010,"議":1198,"込":3041,"開":1758,"間":-1257,"「":-645,"」":3145,"ッ":831,"ア":-587,"カ":306,"キ":568};
this.UW3__ = {",":4889,"1":-800,"":-1723,"、":4889,"々":-2311,"":5827,"」":2670,"〓":-3573,"あ":-2696,"い":1006,"う":2342,"え":1983,"お":-4864,"か":-1163,"が":3271,"く":1004,"け":388,"げ":401,"こ":-3552,"ご":-3116,"さ":-1058,"し":-395,"す":584,"せ":3685,"そ":-5228,"た":842,"ち":-521,"っ":-1444,"つ":-1081,"て":6167,"で":2318,"と":1691,"ど":-899,"な":-2788,"に":2745,"の":4056,"は":4555,"ひ":-2171,"ふ":-1798,"へ":1199,"ほ":-5516,"ま":-4384,"み":-120,"め":1205,"も":2323,"や":-788,"よ":-202,"ら":727,"り":649,"る":5905,"れ":2773,"わ":-1207,"を":6620,"ん":-518,"ア":551,"グ":1319,"ス":874,"ッ":-1350,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278,"・":-3794,"一":-1619,"下":-1759,"世":-2087,"両":3815,"中":653,"主":-758,"予":-1193,"二":974,"人":2742,"今":792,"他":1889,"以":-1368,"低":811,"何":4265,"作":-361,"保":-2439,"元":4858,"党":3593,"全":1574,"公":-3030,"六":755,"共":-1880,"円":5807,"再":3095,"分":457,"初":2475,"別":1129,"前":2286,"副":4437,"力":365,"動":-949,"務":-1872,"化":1327,"北":-1038,"区":4646,"千":-2309,"午":-783,"協":-1006,"口":483,"右":1233,"各":3588,"合":-241,"同":3906,"和":-837,"員":4513,"国":642,"型":1389,"場":1219,"外":-241,"妻":2016,"学":-1356,"安":-423,"実":-1008,"家":1078,"小":-513,"少":-3102,"州":1155,"市":3197,"平":-1804,"年":2416,"広":-1030,"府":1605,"度":1452,"建":-2352,"当":-3885,"得":1905,"思":-1291,"性":1822,"戸":-488,"指":-3973,"政":-2013,"教":-1479,"数":3222,"文":-1489,"新":1764,"日":2099,"旧":5792,"昨":-661,"時":-1248,"曜":-951,"最":-937,"月":4125,"期":360,"李":3094,"村":364,"東":-805,"核":5156,"森":2438,"業":484,"氏":2613,"民":-1694,"決":-1073,"法":1868,"海":-495,"無":979,"物":461,"特":-3850,"生":-273,"用":914,"町":1215,"的":7313,"直":-1835,"省":792,"県":6293,"知":-1528,"私":4231,"税":401,"立":-960,"第":1201,"米":7767,"系":3066,"約":3663,"級":1384,"統":-4229,"総":1163,"線":1255,"者":6457,"能":725,"自":-2869,"英":785,"見":1044,"調":-562,"財":-733,"費":1777,"車":1835,"軍":1375,"込":-1504,"通":-1136,"選":-681,"郎":1026,"郡":4404,"部":1200,"金":2163,"長":421,"開":-1432,"間":1302,"関":-1282,"雨":2009,"電":-1045,"非":2066,"駅":1620,"":-800,"」":2670,"・":-3794,"ッ":-1350,"ア":551,"グ":1319,"ス":874,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278};
this.UW4__ = {",":3930,".":3508,"―":-4841,"、":3930,"。":3508,"":4999,"「":1895,"」":3798,"〓":-5156,"あ":4752,"い":-3435,"う":-640,"え":-2514,"お":2405,"か":530,"が":6006,"き":-4482,"ぎ":-3821,"く":-3788,"け":-4376,"げ":-4734,"こ":2255,"ご":1979,"さ":2864,"し":-843,"じ":-2506,"す":-731,"ず":1251,"せ":181,"そ":4091,"た":5034,"だ":5408,"ち":-3654,"っ":-5882,"つ":-1659,"て":3994,"で":7410,"と":4547,"な":5433,"に":6499,"ぬ":1853,"ね":1413,"の":7396,"は":8578,"ば":1940,"ひ":4249,"び":-4134,"ふ":1345,"へ":6665,"べ":-744,"ほ":1464,"ま":1051,"み":-2082,"む":-882,"め":-5046,"も":4169,"ゃ":-2666,"や":2795,"ょ":-1544,"よ":3351,"ら":-2922,"り":-9726,"る":-14896,"れ":-2613,"ろ":-4570,"わ":-1783,"を":13150,"ん":-2352,"カ":2145,"コ":1789,"セ":1287,"ッ":-724,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637,"・":-4371,"ー":-11870,"一":-2069,"中":2210,"予":782,"事":-190,"井":-1768,"人":1036,"以":544,"会":950,"体":-1286,"作":530,"側":4292,"先":601,"党":-2006,"共":-1212,"内":584,"円":788,"初":1347,"前":1623,"副":3879,"力":-302,"動":-740,"務":-2715,"化":776,"区":4517,"協":1013,"参":1555,"合":-1834,"和":-681,"員":-910,"器":-851,"回":1500,"国":-619,"園":-1200,"地":866,"場":-1410,"塁":-2094,"士":-1413,"多":1067,"大":571,"子":-4802,"学":-1397,"定":-1057,"寺":-809,"小":1910,"屋":-1328,"山":-1500,"島":-2056,"川":-2667,"市":2771,"年":374,"庁":-4556,"後":456,"性":553,"感":916,"所":-1566,"支":856,"改":787,"政":2182,"教":704,"文":522,"方":-856,"日":1798,"時":1829,"最":845,"月":-9066,"木":-485,"来":-442,"校":-360,"業":-1043,"氏":5388,"民":-2716,"気":-910,"沢":-939,"済":-543,"物":-735,"率":672,"球":-1267,"生":-1286,"産":-1101,"田":-2900,"町":1826,"的":2586,"目":922,"省":-3485,"県":2997,"空":-867,"立":-2112,"第":788,"米":2937,"系":786,"約":2171,"経":1146,"統":-1169,"総":940,"線":-994,"署":749,"者":2145,"能":-730,"般":-852,"行":-792,"規":792,"警":-1184,"議":-244,"谷":-1000,"賞":730,"車":-1481,"軍":1158,"輪":-1433,"込":-3370,"近":929,"道":-1291,"選":2596,"郎":-4866,"都":1192,"野":-1100,"銀":-2213,"長":357,"間":-2344,"院":-2297,"際":-2604,"電":-878,"領":-1659,"題":-792,"館":-1984,"首":1749,"高":2120,"「":1895,"」":3798,"・":-4371,"ッ":-724,"ー":-11870,"カ":2145,"コ":1789,"セ":1287,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637};
this.UW5__ = {",":465,".":-299,"1":-514,"E2":-32768,"]":-2762,"、":465,"。":-299,"「":363,"あ":1655,"い":331,"う":-503,"え":1199,"お":527,"か":647,"が":-421,"き":1624,"ぎ":1971,"く":312,"げ":-983,"さ":-1537,"し":-1371,"す":-852,"だ":-1186,"ち":1093,"っ":52,"つ":921,"て":-18,"で":-850,"と":-127,"ど":1682,"な":-787,"に":-1224,"の":-635,"は":-578,"べ":1001,"み":502,"め":865,"ゃ":3350,"ょ":854,"り":-208,"る":429,"れ":504,"わ":419,"を":-1264,"ん":327,"イ":241,"ル":451,"ン":-343,"中":-871,"京":722,"会":-1153,"党":-654,"務":3519,"区":-901,"告":848,"員":2104,"大":-1296,"学":-548,"定":1785,"嵐":-1304,"市":-2991,"席":921,"年":1763,"思":872,"所":-814,"挙":1618,"新":-1682,"日":218,"月":-4353,"査":932,"格":1356,"機":-1508,"氏":-1347,"田":240,"町":-3912,"的":-3149,"相":1319,"省":-1052,"県":-4003,"研":-997,"社":-278,"空":-813,"統":1955,"者":-2233,"表":663,"語":-1073,"議":1219,"選":-1018,"郎":-368,"長":786,"間":1191,"題":2368,"館":-689,"":-514,"E2":-32768,"「":363,"イ":241,"ル":451,"ン":-343};
this.UW6__ = {",":227,".":808,"1":-270,"E1":306,"、":227,"。":808,"あ":-307,"う":189,"か":241,"が":-73,"く":-121,"こ":-200,"じ":1782,"す":383,"た":-428,"っ":573,"て":-1014,"で":101,"と":-105,"な":-253,"に":-149,"の":-417,"は":-236,"も":-206,"り":187,"る":-135,"を":195,"ル":-673,"ン":-496,"一":-277,"中":201,"件":-800,"会":624,"前":302,"区":1792,"員":-1212,"委":798,"学":-960,"市":887,"広":-695,"後":535,"業":-697,"相":753,"社":-507,"福":974,"空":-822,"者":1811,"連":463,"郎":1082,"":-270,"E1":306,"ル":-673,"ン":-496};
return this;
}
TinySegmenter.prototype.ctype_ = function(str) {
for (var i in this.chartype_) {
if (str.match(this.chartype_[i][0])) {
return this.chartype_[i][1];
}
}
return "O";
}
TinySegmenter.prototype.ts_ = function(v) {
if (v) { return v; }
return 0;
}
TinySegmenter.prototype.segment = function(input) {
if (input == null || input == undefined || input == "") {
return [];
}
var result = [];
var seg = ["B3","B2","B1"];
var ctype = ["O","O","O"];
var o = input.split("");
for (i = 0; i < o.length; ++i) {
seg.push(o[i]);
ctype.push(this.ctype_(o[i]))
}
seg.push("E1");
seg.push("E2");
seg.push("E3");
ctype.push("O");
ctype.push("O");
ctype.push("O");
var word = seg[3];
var p1 = "U";
var p2 = "U";
var p3 = "U";
for (var i = 4; i < seg.length - 3; ++i) {
var score = this.BIAS__;
var w1 = seg[i-3];
var w2 = seg[i-2];
var w3 = seg[i-1];
var w4 = seg[i];
var w5 = seg[i+1];
var w6 = seg[i+2];
var c1 = ctype[i-3];
var c2 = ctype[i-2];
var c3 = ctype[i-1];
var c4 = ctype[i];
var c5 = ctype[i+1];
var c6 = ctype[i+2];
score += this.ts_(this.UP1__[p1]);
score += this.ts_(this.UP2__[p2]);
score += this.ts_(this.UP3__[p3]);
score += this.ts_(this.BP1__[p1 + p2]);
score += this.ts_(this.BP2__[p2 + p3]);
score += this.ts_(this.UW1__[w1]);
score += this.ts_(this.UW2__[w2]);
score += this.ts_(this.UW3__[w3]);
score += this.ts_(this.UW4__[w4]);
score += this.ts_(this.UW5__[w5]);
score += this.ts_(this.UW6__[w6]);
score += this.ts_(this.BW1__[w2 + w3]);
score += this.ts_(this.BW2__[w3 + w4]);
score += this.ts_(this.BW3__[w4 + w5]);
score += this.ts_(this.TW1__[w1 + w2 + w3]);
score += this.ts_(this.TW2__[w2 + w3 + w4]);
score += this.ts_(this.TW3__[w3 + w4 + w5]);
score += this.ts_(this.TW4__[w4 + w5 + w6]);
score += this.ts_(this.UC1__[c1]);
score += this.ts_(this.UC2__[c2]);
score += this.ts_(this.UC3__[c3]);
score += this.ts_(this.UC4__[c4]);
score += this.ts_(this.UC5__[c5]);
score += this.ts_(this.UC6__[c6]);
score += this.ts_(this.BC1__[c2 + c3]);
score += this.ts_(this.BC2__[c3 + c4]);
score += this.ts_(this.BC3__[c4 + c5]);
score += this.ts_(this.TC1__[c1 + c2 + c3]);
score += this.ts_(this.TC2__[c2 + c3 + c4]);
score += this.ts_(this.TC3__[c3 + c4 + c5]);
score += this.ts_(this.TC4__[c4 + c5 + c6]);
// score += this.ts_(this.TC5__[c4 + c5 + c6]);
score += this.ts_(this.UQ1__[p1 + c1]);
score += this.ts_(this.UQ2__[p2 + c2]);
score += this.ts_(this.UQ3__[p3 + c3]);
score += this.ts_(this.BQ1__[p2 + c2 + c3]);
score += this.ts_(this.BQ2__[p2 + c3 + c4]);
score += this.ts_(this.BQ3__[p3 + c2 + c3]);
score += this.ts_(this.BQ4__[p3 + c3 + c4]);
score += this.ts_(this.TQ1__[p2 + c1 + c2 + c3]);
score += this.ts_(this.TQ2__[p2 + c2 + c3 + c4]);
score += this.ts_(this.TQ3__[p3 + c1 + c2 + c3]);
score += this.ts_(this.TQ4__[p3 + c2 + c3 + c4]);
var p = "O";
if (score > 0) {
result.push(word);
word = "";
p = "B";
}
p1 = p2;
p2 = p3;
p3 = p;
word += seg[i];
}
result.push(word);
return result;
}
lunr.TinySegmenter = TinySegmenter;
};
}));
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -1 +0,0 @@
{"version":3,"sources":["src/templates/assets/stylesheets/palette/_scheme.scss","../../../../src/templates/assets/stylesheets/palette.scss","src/templates/assets/stylesheets/palette/_accent.scss","src/templates/assets/stylesheets/palette/_primary.scss","src/templates/assets/stylesheets/utilities/_break.scss"],"names":[],"mappings":"AA2BA,cAGE,6BAME,sDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CACA,mDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CAGA,mDAAA,CACA,gDAAA,CACA,yDAAA,CACA,4DAAA,CAGA,0BAAA,CACA,mCAAA,CAGA,iCAAA,CACA,kCAAA,CACA,mCAAA,CACA,mCAAA,CACA,kCAAA,CACA,iCAAA,CACA,+CAAA,CACA,6DAAA,CACA,gEAAA,CACA,4DAAA,CACA,4DAAA,CACA,6DAAA,CAGA,6CAAA,CAGA,+CAAA,CAGA,uDAAA,CACA,6DAAA,CACA,2DAAA,CAGA,iCAAA,CAGA,yDAAA,CACA,iEAAA,CAGA,mDAAA,CACA,mDAAA,CAGA,qDAAA,CACA,uDAAA,CAGA,8DAAA,CAKA,8DAAA,CAKA,0DAAA,CAzEA,iBCiBF,CD6DE,kHAEE,YC3DJ,CDkFE,yDACE,4BChFJ,CD+EE,2DACE,4BC7EJ,CD4EE,gEACE,4BC1EJ,CDyEE,2DACE,4BCvEJ,CDsEE,yDACE,4BCpEJ,CDmEE,0DACE,4BCjEJ,CDgEE,gEACE,4BC9DJ,CD6DE,0DACE,4BC3DJ,CD0DE,2OACE,4BC/CJ,CDsDA,+FAGE,iCCpDF,CACF,CCjDE,2BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD6CN,CCvDE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDoDN,CC9DE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD2DN,CCrEE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDkEN,CC5EE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDyEN,CCnFE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDgFN,CC1FE,kCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDuFN,CCjGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD8FN,CCxGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDqGN,CC/GE,6BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD4GN,CCtHE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDmHN,CC7HE,4BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD6HN,CCpIE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDoIN,CC3IE,6BACE,yBAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD2IN,CClJE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDkJN,CCzJE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDsJN,CE3JE,4BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwJN,CEnKE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgKN,CE3KE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwKN,CEnLE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgLN,CE3LE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwLN,CEnME,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgMN,CE3ME,mCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwMN,CEnNE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgNN,CE3NE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwNN,CEnOE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgON,CE3OE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwON,CEnPE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmPN,CE3PE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2PN,CEnQE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmQN,CE3QE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2QN,CEnRE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgRN,CE3RE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwRN,CEnSE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BF4RN,CE5SE,kCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BFqSN,CEtRE,sEACE,4BFyRJ,CE1RE,+DACE,4BF6RJ,CE9RE,iEACE,4BFiSJ,CElSE,gEACE,4BFqSJ,CEtSE,iEACE,4BFySJ,CEhSA,8BACE,mDAAA,CACA,4DAAA,CACA,0DAAA,CACA,oDAAA,CACA,2DAAA,CAGA,4BFiSF,CE9RE,yCACE,+BFgSJ,CE7RI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCFiSN,CG7MI,mCD1EA,+CACE,8CF0RJ,CEvRI,qDACE,8CFyRN,CEpRE,iEACE,mCFsRJ,CACF,CGxNI,sCDvDA,uCACE,oCFkRJ,CACF,CEzQA,8BACE,kDAAA,CACA,4DAAA,CACA,wDAAA,CACA,oDAAA,CACA,6DAAA,CAGA,4BF0QF,CEvQE,yCACE,+BFyQJ,CEtQI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCF0QN,CEnQE,yCACE,6CFqQJ,CG9NI,0CDhCA,8CACE,gDFiQJ,CACF,CGnOI,0CDvBA,iFACE,6CF6PJ,CACF,CG3PI,sCDKA,uCACE,6CFyPJ,CACF","file":"palette.css"}
File diff suppressed because it is too large Load Diff

Before

Width:  |  Height:  |  Size: 239 KiB

After

Width:  |  Height:  |  Size: 239 KiB

Before

Width:  |  Height:  |  Size: 2.8 KiB

After

Width:  |  Height:  |  Size: 2.8 KiB

Before

Width:  |  Height:  |  Size: 165 KiB

After

Width:  |  Height:  |  Size: 165 KiB

+103
View File
@@ -0,0 +1,103 @@
# Which embedding model is best?
Three ArcFace variants (w600k-R50, R18, w600k-MBF) and LVFace-B (Glint360K,
455MB) were compared. r50 is excluded from the training/held-out comparison
below; its gallery has roughly 30% fewer reference images per actor than the
other three on the identical source photos, which confounds a direct score
comparison (see [the full experiment log](model-bakeoff.md) for detail). It
remains in the calibration comparison, which does not depend on the gallery
image count.
## First signal: calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | cosine similarity) =
σ(a·sim + b)`, stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This is a property of the embedding space alone, computed from intra- and
inter-actor reference-image pairs with no tracking or scene logic involved,
so it is a clean first read on discriminative power before running a
benchmark.
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
| model | a (steepness) | boundary at P=0.5 |
|---|---|---|
| LVFace-B Glint360K | 17.7 | sim 0.228 |
| ArcFace w600k-MBF | 16.2 | sim 0.267 |
| ArcFace w600k-R50 | 15.4 | sim 0.301 |
| ArcFace R18 | 15.3 | sim 0.309 |
LVFace has both the steepest transition and the lowest decision boundary,
separating same-actor from different-actor reference pairs more confidently
at a lower similarity than any ArcFace variant.
## Second signal: held-out F1
Each model's own tuned `full_exp` config, replayed against the 5 films the
optimizer never saw and scored the same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on all 5 held-out films; the ranking never flips
between models. Total misID count across the 5 films: LVFace 1032, mbf
2197, r18 1224. LVFace has less than half mbf's misID total and still
scores higher on every film.
Held-out results are stronger evidence than training results, because
training numbers can reflect what the optimizer was tuned to fit rather
than general performance. On training data, the ordering is not as clean:
| film | LVFace F1 | mbf F1 | r18 F1 | best |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
mbf beats LVFace on Lord of War (77.2% vs 75.6%), the only film in either
table where LVFace does not score highest. LVFace's training-set macro
average (75.3%, see [the full experiment log](model-bakeoff.md)) is not a
uniform win across every film it contributes to; the held-out result, where
LVFace wins all 5 films outright, is the stronger claim.
This reverses an earlier, superseded benchmarking pass that used a
scene-union metric and found the three models statistically
indistinguishable (around 85% each), concluding LVFace was not worth its
size. That metric masked out-of-cast false positives behind a
gallery-intersect-cast recall filter; the per-second metric used here does
not.
## Full training-matrix picture
![All 12 combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
Best full-gallery combo per model (all three are `full_exp`), from the
training matrix in [the full experiment log](model-bakeoff.md):
| model | F1 | P | R | misID |
|---|---|---|---|---|
| LVFace-B Glint360K | 75.3% | 89.7% | 65.4% | 232 |
| ArcFace w600k-MBF | 72.0% | 87.7% | 61.4% | 240 |
| ArcFace R18 | 69.1% | 87.6% | 57.7% | 242 |
LVFace leads within both the restricted and full gallery modes, visible
directly in the chart above without reading the table. The three models'
misID counts on the full gallery are nearly identical (232/240/242); LVFace's
lead here is a precision-and-recall lead, not a misID one.
## Operational note
Switching the default embedder is not a config change alone; the gallery
is model-specific, since embeddings from different models are not
comparable. Any existing gallery built against a different model must be
rebuilt from source images before the new default takes effect.
[`scripts/optimizer/reembed_gallery.py`](https://REPOLINK/scripts/optimizer/reembed_gallery.py)
does this from a reference gallery's cached source images without
re-downloading anything.
+69
View File
@@ -0,0 +1,69 @@
# Whole gallery vs. cast-restricted gallery
Two ways to run the matcher. Full mode scores every detected face against
the entire 2418-actor gallery. Restricted mode pre-filters each film's
gallery down to just its Jellyfin-credited cast (typically around 15
top-billed actors) before the matcher runs.
## Result
Averaged across the 3 compared models (r50 excluded, see
[the full experiment log](model-bakeoff.md)) and both expansion settings, on
the 4 training films:
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once, not a precision/recall trade:
+4.8pp F1, +6.0pp recall, roughly a quarter the total misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections.
Every model's best-scoring combo in the training matrix uses the
restricted gallery:
![All combos ranked by training-set F1, filled dots are restricted](assets/images/rep4_matrix_f1.png)
See [the full experiment log](model-bakeoff.md) for the complete table. One
combo reaches zero true out-of-cast misidentifications,
`arcface_w600k_mbf_restricted_exp` (F1 76.2%), and it is a restricted one,
consistent with restriction, not expansion, being what suppresses cross-film
confusions.
The restriction effect (+4.8pp averaged across models) is larger than the
model-choice effect: LVFace beats r18 by 6.2pp in full mode but beats mbf by
3.3pp. Restriction is the single strongest lever in the matrix.
## Why this is not the shipped default
Cast restriction is implemented today only as an offline optimizer
technique
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
it pre-builds a filtered gallery file per film using Jellyfin's cast list
before the benchmark calls the matcher. There is no runtime "restrict to
this title's credited cast" switch in the shipped application;
`scene_analyze` always matches against whatever single gallery file it is
given.
Building this as a real feature requires:
- A live Jellyfin cast lookup at analysis time. The title is already known,
and [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
already performs this lookup for its own `filter_gallery`-based
restriction path; it is not wired into `scene_analyze` as a first-class
option.
- A decision on the fallback case: what happens to a real, uncredited
cameo (see the Germar Terrell Gardner and Talia Balsam cases in the
[LVFace deep dive](lvface-deep-dive.md#where-lvface-beat-x-ray)) if the
restricted gallery never includes them at all.
- Regenerating the restricted-gallery cache whenever a title's Jellyfin
cast list changes.
The shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults use
the full-mode winner (`LVFace-B_Glint360K_full_exp`, F1 75.3% training,
67.4% held-out macro) rather than the higher-scoring `restricted_exp`
(78.3%), because 78.3% describes a capability the application does not
have yet.
+91
View File
@@ -0,0 +1,91 @@
# scene-actor-extraction
A face-recognition pipeline that finds when each actor appears on screen in
a film or TV episode, built on [KPN++](https://gitea.tourolle.paris/dtourolle/KPN)
(a C++20 Kahn Process Network library) for the detect, track, match, and
scene pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated
optimizer.
This is a correctly scored second from a held-out film, one the optimizer
never saw during tuning:
![A perfect X-Ray second: three faces named at 100%, two more correctly carried off-screen](assets/images/lovelace_perfect_second.jpg)
Every visible face is named at 100% confidence (Chris Noth, Hank Azaria,
Bobby Cannavale), the background extra is correctly left unnamed, and the
two credited cast members without a visible face are correctly reported
present but not visible. This matches Amazon X-Ray's own record for this
second exactly.
Results are not uniform across films. The hardest held-out film scores 46%
F1. This report documents why: one tunable trade (extinction bridging at
hard cuts), one structural limit (X-Ray credits people whose faces never
appear on screen), and a small number of cases where the pipeline is
correct and X-Ray's ground truth is not. Read
[how we score against X-Ray](methodology.md) first. X-Ray's ground truth is
scene-level; the pipeline's output is per-second. That difference shapes
every finding below.
## Findings
<div class="grid cards" markdown>
- :material-trophy:{ .lg .middle } **[Which model is best?](best-model.md)**
---
Calibration curves first, independent of any threshold, then held-out
F1 across three models. LVFace-B Glint360K wins both, and wins on every
held-out film.
- :material-filter:{ .lg .middle } **[Whole vs. cast-restricted gallery](gallery-scope.md)**
---
Restricting the matcher to a film's credited cast improves F1,
recall, and misID rate at once, but is not a shipped runtime feature
yet.
- :material-account-convert:{ .lg .middle } **[Does pose expansion help?](pose-expansion.md)**
---
A training-set effect that did not reproduce on 5 held-out films once
two methodology bugs in the comparison harness were found and fixed.
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
---
The held-out generalization gap, the two mechanisms behind its errors,
and every distinct case where it names someone outside the film's
credited cast.
</div>
## Full experiment log
- **[Full experiment log](model-bakeoff.md)**: the complete log behind the
four pages above, including how replaying against cached embeddings
inside the same KPN network makes a full model and configuration
comparison practical, the full results table, and every caveat. This is
where the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
defaults come from.
- **[Service conversion (proposal)](service-conversion.md)**: design
sketch for a native idle-GPU worker gated on screen lock, not yet built.
## Reproducing the benchmarks
Gallery `.h5` files, embedding dumps, the X-Ray corpus, montage frame
images, and DE trajectories are not committed to this repository. They are
pushed to the Gitea package registry and pulled on demand:
```bash
scripts/artifacts/pull_artifacts.sh galleries
scripts/artifacts/pull_artifacts.sh experiment-data
scripts/artifacts/pull_artifacts.sh montage-frames <film-slug>
```
See [`scripts/artifacts/push_artifacts.sh`](https://REPOLINK/scripts/artifacts/push_artifacts.sh)
for the upload side, which requires a `GITEA_TOKEN` with package write
scope.
+303
View File
@@ -0,0 +1,303 @@
# Deep dive: LVFace-B Glint360K
LVFace won the model comparison (see [Which model is best?](best-model.md))
and is the shipped default embedder. This page reports how it performs in
detail: a baseline of correct output, the two mechanisms behind its errors,
and every distinct case where it names someone who is not in the film's
credited cast.
Read [How we score against X-Ray](methodology.md) first. X-Ray's ground truth
is scene-level, not per-frame. A name marked correct in the Offscreen column
below is the pipeline correctly reporting scene membership, not a workaround.
!!! note "How to read the frames on this page"
The top of each image is the film frame, with a box and name on every
face the pipeline matched to a real detection. The panels below are the
per-second result against X-Ray. **Onscreen** lists names attached to a
visible face this second. **Offscreen** lists names the pipeline reports
present without a currently visible face. Colors mark the verdict:
<span style="color:#0ca30c">**green**</span> correct (TPI),
<span style="color:#eb6834">**orange**</span> wrong (FPI),
<span style="color:#3987e5">**blue**</span> missed (FN).
## Baseline: correctly scored seconds
![Wedding couple correctly identified, Downton Abbey: A New Era](assets/images/downton_wedding_couple.jpg)
Six faces on screen, all six named correctly, including Penelope Wilton at
the edge of the pews and a partly occluded Michelle Dockery. Thirteen more
cast members X-Ray lists as present in the scene are correctly reported
Offscreen. One miss: Maggie Smith (blue). Score for this second: 0.86.
![19 of 20 correct in the funeral crowd](assets/images/downton_funeral_19of20.jpg)
The same film's funeral scene: dark clothing, hats, half the faces turned
away. Nineteen of the twenty cast members X-Ray lists for this scene score
correct: seven named on screen at up to 100% confidence, twelve more reported
correctly as present but not visible.
![Herbie Hancock identified on an in-fiction video call](assets/images/valerian_screen_call.jpg)
The pipeline does not require a live face. This is Herbie Hancock at 98%
confidence, identified from a face displayed on a screen inside the film, on
a video call under a science-fiction HUD overlay.
## Training vs. held-out: the generalization gap
The shipped config (`prob_threshold=0.754`, `anneal_sec=35.54`,
`extinction_sec=57.43`, `expand_gallery=true`) was tuned on 4 films. Scored
on the 5 films the optimizer never saw:
![Held-out per-film F1 vs. the training-set fit](assets/images/holdout_f1_by_film.png)
| film | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| Benny & Joon | 83.0% | 89.1% | 77.7% | 15125 | 1846 | 0 | 4337 |
| Lovelace | 77.5% | 90.3% | 67.9% | 14990 | 1085 | 58 | 7085 |
| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 18663 | 548 | 0 | 12467 |
| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 52027 | 1173 | 0 | 80084 |
| The Many Saints of Newark | 46.3% | 54.7% | 40.1% | 15922 | 4394 | 974 | 23791 |
| macro average | 67.4% | 85.8% | 57.0% | | | | |
The `P` column is misID-weighted (each out-of-film name counts 10x in the
denominator; see [methodology](methodology.md#precision-recall-and-the-misid-weighting)).
That weighting is why Many Saints reads 54.7% here despite naming mostly real,
present faces: its raw (unweighted) precision is **78.4%**, and the gap is
entirely its 974 misIDs paying the 10x penalty. The three zero-misID films
(Benny & Joon, Downton, Valerian) have identical weighted and raw precision;
Lovelace, with 58 misIDs, sits 3pp below its raw 93.3%.
Held-out F1 is 67.4%, against 75.3% on training, an 8pp drop. The spread
between the best and worst held-out film is 37pp. This is not unique to
LVFace: [the full experiment log](model-bakeoff.md#held-out-validation-all-3-models)
shows mbf and r18 with the same shape of spread on the same films, at a
uniformly lower level. Two mechanisms explain the spread. Both are shown
below with frame-level evidence.
## Mechanism 1: extinction bridging
The extinction window keeps a name reported as present for up to
`extinction_sec` after its last real detection. This is deliberate: most
gaps in face visibility are short (a turned head, an occlusion, a cut to a
reaction shot), and the window bridges them.
![Two faces on screen, six more correctly bridged](assets/images/lovelace_polygraph_bridged.jpg)
Lovelace's polygraph scene: only Eric Roberts and Amanda Seyfried have
visible faces. X-Ray lists eight cast members present. All eight score
correct; the other six are reported Offscreen through a stretch where the
camera never shows them. The extinction window is why.
The same mechanism fails at a hard cut into a long stretch with no faces at
all. Downton Abbey's recall (39.4%, the worst of the five held-out films) is
dominated by this failure. It is verified directly against the raw
per-frame stream and the dump's own detection counts, not inferred from the
score. Plotting the dump's per-second `face_count` (detector output,
independent of the tracker) against what the tracker reports, through
Downton Abbey's hard cut into its closing credits:
![Detector vs. tracker through Downton Abbey's cut to credits](assets/images/downton_ghost_timeline.png)
From the cut onward the detector reports zero faces for close to a minute.
The tracker continues reporting the previous shot's 15 identities for the
same span (verified for Hugh Bonneville: bbox `(1743.2, 0.0, 171.3, 317.8)`,
unchanged to the pixel, at every sampled second for 57 seconds). The
staircase at the right edge is the extinction window expiring, actor by
actor. This is `SceneTrackerFunc::active_[actor_idx].last_bbox`
([`src/nodes/scene_tracker_node.hpp`](https://REPOLINK/src/nodes/scene_tracker_node.hpp))
re-emitted as designed. `extinction_sec=57.4` was tuned long because
bridging is correct on most footage, as in the polygraph scene above. The
training films did not contain a faceless stretch long enough to expose the
cost side; the held-out set did.
The extinction window is a scoring concept, not something drawn on screen.
The shipped output is presence windows with no bounding boxes. Even the
debug overlay used for this report never draws a box for a bridged name: a
name inside its extinction window with no current detection appears only as
a name in the Offscreen column, the same as every correctly bridged name
above.
A related, smaller effect shows up at rapid cuts:
![Two labels on one face after a shot/reverse-shot cut](assets/images/cafe_society_rapid_cut.jpg)
Café Society (a training film), a shot/reverse-shot dialog. The box on Steve
Carell's face carries two labels: his own, and Jesse Eisenberg's, left over
from the counter-shot a moment earlier. Both names score correct, because
both actors are present in this scene per X-Ray. The box position is
briefly wrong; the presence claim, which is what the pipeline ships, is
right.
## Mechanism 2: the face-vs-presence ceiling
Downton Abbey's recall did not collapse because faces were misread. It
collapsed because for most of its 80084 false-negative seconds there was no
face to read.
![22 cast credited, nobody facing the camera](assets/images/downton_crew_fn.jpg)
A newsreel crew moves equipment through the hall. X-Ray credits 22 cast
members as present in this scene. None face the camera. Eight still score
correct, carried by presence windows from adjacent shots. The other fourteen
are missed, and no face-recognition system can recover them, because there
is no face in the frame. X-Ray records scene membership; the pipeline
measures visible faces. In ensemble scenes these two quantities diverge, and
that gap accounts for most of the false-negative count.
## Every distinct out-of-cast name
Many Saints of Newark has the largest misID count of any held-out film: 974
seconds, weighted. Rather than characterize this from a single frame, the
raw replay stream was searched directly for every name the pipeline reports
that is not in the film's credited cast. The same search was run on all 9
films in the benchmark, one rule applied uniformly: **find the first second
each distinct out-of-cast name appears, and render that exact second.**
Five films produce no such name anywhere in their runtime: Benny & Joon,
Café Society, Downton Abbey, Sound of Metal, Valerian. Zero out-of-cast
names across their entire length. Four films produce nine distinct names
between them, shown below in full, not a sample.
### The Many Saints of Newark: 4 names
![Germar Terrell Gardner, first out-of-cast name in Many Saints](assets/images/many_saints_fpi_gardner.jpg)
Germar Terrell Gardner, t=848s, 78% confidence. A real, clearly visible
background actor. He is not in X-Ray's cast list for this film, but he is
credited in Jellyfin's independent cast metadata (see
[Where LVFace beat X-Ray](#where-lvface-beat-x-ray) below). This is a
ground-truth gap, not a model error.
![Archie Yates, second out-of-cast name in Many Saints](assets/images/many_saints_fpi_yates.jpg)
Archie Yates, t=2521s, 78% confidence. A real detected face, a genuine
lookalike confusion.
![Zooey Deschanel, third out-of-cast name in Many Saints](assets/images/many_saints_fpi_deschanel.jpg)
Zooey Deschanel, t=2819s, 99% confidence. A real detected face at a dinner
table, high-confidence lookalike confusion.
![Talia Balsam, fourth out-of-cast name in Many Saints](assets/images/many_saints_fpi_balsam.jpg)
Talia Balsam, t=4551s, 93% confidence. A real detected face. Talia Balsam
plays Mrs. Jarecki, a guidance counselor, in this film; she is confirmed
on screen by direct inspection of the frame. She does not appear in X-Ray's
`people.csv` for this title. This is a second ground-truth gap in the same
film, not a model error.
Two of these four names are ground-truth gaps (Gardner, Balsam), not
misidentifications. The other two (Yates, Deschanel) are genuine embedding
errors on real faces.
### Lord of War: 3 names
![David Shumbris, first out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_shumbris.jpg)
David Shumbris, t=418s, 81% confidence. A real face in a dim, low-detail
shot under a train track. A genuine lookalike confusion in poor lighting.
![Ronald Reagan, second out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reagan_photo.jpg)
Ronald Reagan, t=1003s, 100% confidence. This is not a lookalike confusion.
The detected face is a photograph of Reagan appearing within the shot, not a
living actor. The detector and matcher both did their job correctly on the
image content in front of them; the error is that a photograph inside the
scene is not the same thing as an actor present in the scene, and the
pipeline has no way to draw that distinction from a face crop alone.
![Lance Reddick, third out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reddick.jpg)
Lance Reddick, t=6424s, 78% confidence. A small, distant, low-detail face at
the edge of frame. A marginal, low-confidence lookalike confusion.
### Lovelace: 1 name
![Chloë Sevigny, out-of-cast name in Lovelace](assets/images/lovelace_fpi_sevigny.jpg)
Chloë Sevigny, t=2451s, 100% confidence. Amanda Seyfried's track is real and
well-tracked through most of this shot, but her bbox is frozen at the exact
same coordinates for t=2450 and t=2451, one second where her box stopped
updating from a fresh detection. Only one real face is detected at t=2451
(confirmed against the dump's own per-frame detections), and it is a tight
IoU-1.0 fit under the Chloë Sevigny box, not the Seyfried one. So the green
Seyfried box in this frame is a ghost, re-emitting her last known position
for that one second, and the fresh, wrong detection is Sevigny, landing on
top of it. Not two competing fresh identities on one crop: one ghost and
one fresh misidentification happening to overlap.
### Scarface: 1 name
![Kirstie Alley, out-of-cast name in Scarface](assets/images/scarface_fpi_alley.jpg)
Kirstie Alley, t=2451s, 89% confidence. Al Pacino is correctly identified in
the foreground at 100%; a background face in the same shot is wrongly
labeled Kirstie Alley. (The t=2451s here and the Lovelace Chloë Sevigny case
above landing on the identical second is a genuine coincidence, verified from
each film's raw stream by [`first_fpi_frames.py`](https://REPOLINK/scripts/docs/first_fpi_frames.py),
not a transcription slip, two unrelated films whose *first* out-of-cast name
happens to fall at the same timestamp.)
### Summary of the nine
| film | name | t (s) | confidence | classification |
|---|---|---|---|---|
| Many Saints of Newark | Germar Terrell Gardner | 848 | 78% | ground-truth gap |
| Many Saints of Newark | Archie Yates | 2521 | 78% | lookalike confusion |
| Many Saints of Newark | Zooey Deschanel | 2819 | 99% | lookalike confusion |
| Many Saints of Newark | Talia Balsam | 4551 | 93% | ground-truth gap |
| Lord of War | David Shumbris | 418 | 81% | lookalike confusion |
| Lord of War | Ronald Reagan | 1003 | 100% | photo-in-frame |
| Lord of War | Lance Reddick | 6424 | 78% | lookalike confusion, marginal |
| Lovelace | Chloë Sevigny | 2451 | 100% | lookalike confusion |
| Scarface | Kirstie Alley | 2451 | 89% | lookalike confusion |
Of nine distinct out-of-cast names across four films, two are ground-truth
gaps, one is a photograph misread as a person, and six are genuine
embedding-space confusions on real detected faces. None trace to extinction
bridging: every one of these nine is a fresh detection on a real face crop
at the second it first appears.
## Where LVFace beat X-Ray
Not every name marked wrong is actually wrong.
[`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)
scores strictly against X-Ray, and X-Ray has gaps of its own.
![LVFace correctly identifies Germar Terrell Gardner, uncredited by X-Ray](assets/images/germar_beats_xray.jpg)
Germar Terrell Gardner, the same name from the table above, does not appear
in X-Ray's `people.csv` for The Many Saints of Newark. Jellyfin's
independent cast metadata does credit him for this film (cross-checked
against `experiments/manifests/jellyfin_casts.json` from the
`experiment-data` artifact package, a data source entirely separate from
X-Ray). Talia Balsam is the same case: confirmed on screen, absent from
X-Ray's cast list for this title.
![Robert Patrick, clearly on screen, scored wrong by a ground-truth gap](assets/images/lovelace_robert_patrick_fpi.jpg)
This extends past uncredited background actors. This is Robert Patrick,
top-billed in Lovelace, clearly on screen reading a newspaper, identified at
100%. The frame is scored wrong because X-Ray's people-in-scene list for
this specific scene omits him, despite crediting him elsewhere in the film.
The identification is correct; the ground truth is missing an entry.
X-Ray is a large, convenient ground truth. It is not a complete one. The
misID and FPI counts reported throughout this document include some fixed
amount of noise from gaps in X-Ray itself, in both directions.
## Summary
LVFace wins the model comparison on every held-out film. It correctly names
19 of 20 people in a crowded funeral scene and correctly identifies a face
displayed on a screen inside the film. Its errors resolve into two
mechanisms: extinction bridging, which is correct on most footage and fails
specifically at hard cuts into long faceless stretches, and the
face-versus-presence ceiling, where X-Ray credits scene membership for
people whose faces never appear on screen. Of the nine distinct
out-of-cast identifications found across the benchmark, two trace to gaps in
X-Ray's own cast data, one is a photograph misread as a person, and six are
genuine lookalike confusions on real faces. The held-out generalization gap,
75.3% training to 67.4% held-out, is real and should be treated as the
expected operating point, not the training-set figure.
+134
View File
@@ -0,0 +1,134 @@
# How we score against X-Ray
Every number in this report, every F1 and misID count, comes from one
comparison. The comparison has a mismatch at its core that shapes nearly
every finding in this report: the ground truth is scene-level, the
pipeline's output is per-second, and the two do not mean the same thing.
This page documents that comparison once, so the findings pages can rely on
it without re-explaining it.
## What Amazon X-Ray records
X-Ray ships three tables per film: `scenes.csv` (a list of `[start, end]`
timespans), `people_in_scenes.csv` (which actors are credited in each
scene), and `people.csv` (actor identities). There is no per-frame or
per-second annotation anywhere in X-Ray. A scene might run 45 seconds, and
X-Ray records one cast list for the entire span, not "on screen from
second 12 to second 30."
To compare this against per-second predictions, `second_score.py` expands
every scene into per-second ground truth by copying the whole scene's cast
list onto every second inside it:
```python
for sn, (t0, t1) in spans.items():
cast = scene_cast.get(sn, [])
for t in range(int(t0), int(t1)):
timeline[t] = cast
```
That is the entire mechanism. If X-Ray credits five actors to a 30-second
scene, all five count as ground truth present for all 30 seconds, including
seconds where only one of them is on screen. This is not a simplification
introduced by the pipeline; it is the only reading of X-Ray's data that is
possible, because X-Ray itself does not record anything finer-grained.
## Why an offscreen name can be scored correct
A name listed under Offscreen with a correct (green) label is not the
pipeline guessing or padding its score. It is the pipeline correctly
answering the question X-Ray actually asks: is this actor part of this
scene. It answers that question using a presence window (`[start, end]`,
held open across cuts by `anneal_sec` and `extinction_sec`), which matches
X-Ray's scene-level semantics more closely than a raw per-frame detection
would.
A system that only reported "this actor is visible in this exact frame"
would score worse against X-Ray's scene-level ground truth, producing a
false negative every time the camera cuts away from a character who is
still present in the scene. Not because it is wrong about the world, but
because it would be answering a stricter, different question than the one
X-Ray's data supports. The presence-window design exists specifically to
answer X-Ray's actual question.
## What this resolves and what it does not
This resolves the semantic mismatch between a scene and an instant. It does
not resolve two other limitations, both discussed in the
[LVFace deep dive](lvface-deep-dive.md).
**The face-vs-presence ceiling.** X-Ray credits scene membership regardless
of whether a face is ever visible: background crew, characters shot from
behind, voice-only presence. No amount of bridging recovers a face that
never appears on screen. This is a hard ceiling on recall, not a defect.
**Extinction bridging can overshoot.** The same presence-window mechanism
that correctly answers "still in this scene" during a normal cut can also
bridge across a scene boundary it has no way to detect. A hard cut into a
different scene with no faces, such as closing credits, carries the
previous scene's identities forward until the window expires. This is the
mechanism behind Downton Abbey's recall collapse, documented in the deep
dive.
## Precision, recall, and the misID weighting
Per sampled second `t`:
**TPI** (true positive instances): actors both X-Ray and the pipeline agree
are present.
**FPI** (false positive instances): actors the pipeline reports that are
not in X-Ray's cast for this second. Split into two categories:
- **FPI_incast**: the actor is in the film's cast, just not credited to
this particular scene. A timing or boundary slip.
- **FPI_misid**: the actor is not in the film's cast at all. A genuine
wrong-identity error, weighted 10x in the precision objective, because
naming someone who is not even in the film is a categorically worse
error than a few seconds of scene-boundary slop.
!!! note "Every headline `P` and `F1` is misID-weighted"
The precision reported throughout this report, and therefore the F1
derived from it, puts each `FPI_misid` into the denominator **10 times**
(`precision = TPI / (TPI + FPI_incast + 10·FPI_misid)`,
[`second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
This is deliberate: the whole point is to punish naming an out-of-film
actor far harder than a scene-boundary slip. But it means the `P` column
is not raw precision, and a misID-heavy film's `P` is depressed
super-linearly. `second_score.py` also emits an unweighted `precision_raw`
(always ≥ the weighted `P`); where the gap matters, The Many Saints of
Newark, weighted `P` 54.7% vs. raw 78.4%, the [LVFace deep dive](lvface-deep-dive.md)
reports both. When comparing `P` across films, remember you are comparing a
quantity that penalizes misIDs, not just a hit rate.
**FN** (false negatives): actors X-Ray lists that the pipeline never
reports, counted only for actors who have a gallery reference embedding.
Across the 9-film benchmark, coverage of X-Ray's credited cast ranges from
20% to 79% by film (see
[the full experiment log](model-bakeoff.md#gallery-coverage-per-film)); an
actor with no reference photo can never be recognized regardless of model
quality, and counting them as a miss would penalize gallery coverage, not
recognition accuracy.
Two further numbers are reported alongside F1:
**agreement_rate**: mean per-second Jaccard overlap
(`|Pred ∩ GT| / |Pred GT|`), partial credit. Naming 2 of 3 present actors
scores 2/3, not 0.
**exact_match_rate**: the fraction of sampled seconds where the pipeline's
named set exactly equals X-Ray's, no partial credit. Far harsher, and
dominated by recall, since any single missed actor zeroes that second.
## Reproduce
```bash
python3 scripts/optimizer/second_score.py \
--pred pred.json --xray experiments/xray/.../<xray_dir> \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5
```
See also [the full experiment log](model-bakeoff.md) for how `pred.json` is
produced, and the [LVFace deep dive](lvface-deep-dive.md) for what these
mechanisms look like frame by frame.
+346
View File
@@ -0,0 +1,346 @@
# Full experiment log
This page reports how the pipeline performs across three questions: which
embedding model is best, whether restricting the gallery to a film's
credited cast helps, and whether promoting confidently identified poses into
a per-film gallery annex helps. It also documents the replay architecture
that made testing all three questions in one pass practical, and every
caveat needed to trust the numbers.
Read [How we score against X-Ray](methodology.md) first for what F1,
precision, recall, and misID mean in this report. All numbers below use the
per-second metric
([`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
r50 (ArcFace w600k-R50) is excluded from the detailed comparison below. Its
gallery was built with roughly 30% fewer reference images per actor than the
other three models on the identical source photos (10808 vs 15055 total
embeddings across the same 2418 actors), which confounds any direct
comparison of its scores against the others. It remains in the
[calibration curve comparison](best-model.md#first-signal-calibration-curves),
which does not depend on the training benchmark.
## Why replay makes this affordable
Decoding video and running face detection, alignment, and embedding is the
expensive part of this pipeline. Everything downstream of that (tracking,
identity matching, scene aggregation) is cheap. KPN++'s node/network
structure means those two stages are separate components connected by
typed channels, so the expensive stage can run once per film, cache its
output, and the cheap stage can be re-run against that cache as many times
as needed with different Config values.
`scene_analyze --dump-embeddings out.h5` runs the expensive half once per
film and writes per-frame face detections and embeddings to HDF5
([`scripts/optimizer/SCHEMA.md`](https://REPOLINK/scripts/optimizer/SCHEMA.md)).
[`scripts/optimizer/replay.py`](https://REPOLINK/scripts/optimizer/replay.py)
then re-assembles the real C++ `face_tracker`, `identity_matcher`, and
`scene_tracker` nodes into a Python-driven KPN network and replays a
film's cached embeddings through them, varying `prob_threshold`,
`anneal_sec`, `extinction_sec`, and `expand_gallery` freely. No GPU
inference and no video decode happen during a replay; each one completes
in seconds. This is what makes a 512-evaluation differential-evolution
search per model, per gallery mode, per expansion setting, tractable, and
what made the full held-out validation across three models in this report
possible in one session rather than requiring three full re-encodes of the
benchmark set.
`optimize.py` runs `differential_evolution` over this replay function as its
objective, with DE-level parallelism (multiple candidate configs evaluated
concurrently, each spawning its own replay subprocesses) on top of it. The
practical ceiling on this machine's GPU was 8 concurrent replay processes;
9 silently degraded every score to 0.0% (well-formed output, wrong numbers,
not a crash), so `optimize.py` was run at `REPLAY_WORKERS=4 DE_WORKERS=2`.
## Search space
`popsize=10, maxiter=15` per combo (3 parameters, up to 512 evaluations,
usually stopping earlier on DE's convergence tolerance).
`anneal_sec`/`extinction_sec` bounds were widened from 1-30/1-15 to 1-60/1-60
partway through the sweep. r50's 4 combos finished before the widening and
used the old, narrower bounds; this is one more reason r50 is excluded from
direct comparison here.
## Training films and held-out films
9 films have dumped embeddings across all 4 models. 4 were used for
optimization:
- Café Society (62-cast)
- Lord of War (64-cast)
- Scarface (67-cast)
- Sound of Metal (14-cast)
5 were held out, never seen by any optimizer run:
- Benny & Joon
- Downton Abbey: A New Era
- Lovelace
- The Many Saints of Newark
- Valerian and the City of a Thousand Planets
## Gallery coverage per film
The gallery has reference embeddings for 2418 actors, but coverage of any
given film's credited cast varies widely. This was previously reported as
one flat number (67% of X-Ray cast lacking a reference embedding, averaged
across the whole benchmark); the per-film breakdown is:
| film | cast credited | in gallery | coverage |
|---|---|---|---|
| Lord of War | 64 | 13 | 20.3% |
| Scarface | 67 | 15 | 22.4% |
| The Many Saints of Newark | 48 | 13 | 27.1% |
| Café Society | 62 | 17 | 27.4% |
| Lovelace | 42 | 15 | 35.7% |
| Valerian and the City of a Thousand Planets | 36 | 13 | 36.1% |
| Benny & Joon | 23 | 12 | 52.2% |
| Downton Abbey: A New Era | 36 | 22 | 61.1% |
| Sound of Metal | 14 | 11 | 78.6% |
Two training films (Lord of War, Scarface) have the worst coverage in the
set, 20-22%. Their training-set F1 numbers below are partly capped by
missing references, not purely by model quality. Downton Abbey has 61%
coverage, the second-best in the benchmark, yet the worst held-out recall
of any film (39.4%, LVFace). Its recall problem is not primarily a coverage
problem; it is the extinction-bridging failure documented in the
[LVFace deep dive](lvface-deep-dive.md#mechanism-1-extinction-bridging).
Reproduce with `scripts/docs/gallery_coverage_per_film.py`.
## Training results, 3 models × 2 gallery modes × 2 expansion settings
Ranked by F1. misid = FPI_misid, the count of true wrong-actor
identifications (naming someone not in the film's cast at all), distinct
from FPI, which also includes in-cast timing slips.
Each combo's row is its best **full-coverage** evaluation: the highest-F1 DE
evaluation in which all 4 training films replayed without a timeout (see
[Dropped-film scoring](#a-scoring-bug-worth-recording-dropped-film-evaluations)
below for why this qualifier is load-bearing and not the same as `argmax F1`
over the raw sweep).
| combo | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| LVFace-B_Glint360K_restricted_exp | 78.3% | 91.0% | 68.9% | 42830 | 3782 | 60 | 19492 |
| LVFace-B_Glint360K_restricted_noexp | 76.7% | 91.5% | 66.2% | 41149 | 3400 | 59 | 21173 |
| arcface_w600k_mbf_restricted_exp | 76.2% | 90.0% | 66.2% | 64328 | 7480 | 0 | 33234 |
| arcface_r18_restricted_exp | 75.5% | 87.6% | 66.5% | 41399 | 5666 | 60 | 20923 |
| LVFace-B_Glint360K_full_exp | 75.3% | 89.7% | 65.4% | 47757 | 3407 | 232 | 26966 |
| arcface_w600k_mbf_restricted_noexp | 75.0% | 91.1% | 63.9% | 39752 | 3465 | 60 | 22570 |
| arcface_r18_restricted_noexp | 73.5% | 91.3% | 61.7% | 38299 | 3220 | 60 | 24023 |
| LVFace-B_Glint360K_full_noexp | 72.3% | 88.3% | 61.8% | 40363 | 3503 | 244 | 25850 |
| arcface_w600k_mbf_full_exp | 72.0% | 87.7% | 61.4% | 39875 | 3729 | 240 | 26338 |
| arcface_w600k_mbf_full_noexp | 71.0% | 93.2% | 57.9% | 41699 | 2472 | 56 | 33024 |
| arcface_r18_full_exp | 69.1% | 87.6% | 57.7% | 37342 | 3119 | 242 | 28871 |
| arcface_r18_full_noexp | 66.6% | 91.3% | 53.1% | 34314 | 2362 | 107 | 31899 |
![All combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
The two clearest patterns: every model's best-scoring combo uses the
restricted gallery, and LVFace leads within both gallery modes. `full_exp`
(the shipped combination) is the best-scoring option that uses only
features the running application currently supports; restriction is not
wired into the application yet (see
[Whole vs. cast-restricted gallery](gallery-scope.md)).
### A scoring bug worth recording: dropped-film evaluations
The numbers above are corrected ones. The raw `rep4_best_*.json` files, and an
earlier version of this table, reported a different `arcface_w600k_mbf_full_noexp`
row: **74.2% F1 at TPI 12645**, a third the TPI of every sibling combo. That was
not a better config; it was an artifact of how the optimizer aggregates.
`optimize.py` builds each candidate's score from only the films whose replay
subprocess returned (`per_film = [m for m in ex.map(_one, films) if m is not
None]`), then **averages** F1/precision/recall and **sums** TPI/FPI/misID over
just those survivors. When a film's replay times out (the sweep ran near the
8-process concurrency ceiling, so this happened intermittently), that film
silently drops from both. A candidate whose hardest film timed out is therefore
scored on an easier subset, and differential evolution, maximizing that score,
will happily converge onto exactly such a candidate. For `mbf_full_noexp` the
reported winner was one of 7 evaluations (out of 512) whose TPI had collapsed to
a partial-film subset; its median-coverage evaluations sit around 51686 TPI.
The fix here was to re-derive each combo's best row from its DE trajectory
(`experiments/trajectories/rep4_*.jsonl`), keeping only evaluations within 30% of
that combo's median TPI (full 4-film coverage) before taking the best F1. This
needs no re-running, the honest best configuration was already in the sweep,
just not the one `argmax F1` selected. Three combos moved: `mbf_full_noexp`
74.2% → **71.0%**, `LVFace_full_noexp` 72.4% → **72.3%** (and its misID, 0 → 244,
was itself a dropped-film artifact), `mbf_restricted_exp` 76.5% → **76.2%**. The
shipped LVFace `full_exp` winner was unaffected, its reported evaluation already
had full coverage (TPI 47757 ≈ median). `experiment_charts.py` applies the same
`clean_best` filter, so every figure on this page matches the corrected table.
The underlying `optimize.py` aggregation is also being fixed so a dropped-film
evaluation can never be selected as a winner again.
### Per-film training breakdown
The 75.3% LVFace training figure is a macro average across 4 films, not a
uniform result:
| film | LVFace F1 | mbf F1 | r18 F1 | best model |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
LVFace does not win every training film. mbf scores higher on Lord of War
(77.2% vs 75.6%). LVFace's own training-film range is 68.1% to 78.8%, a
10.7pp spread, smaller than the 37pp spread seen on held-out films but real.
Reproduce with `scripts/docs/run_holdout_all_models.py --films training`.
## Held-out validation, all 3 models
The training matrix above is training-set fit. Each model's own tuned
`full_exp` config was replayed against the 5 held-out films, scored the
same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on every one of the 5 held-out films; the ranking
never flips. Total misIDs across the 5 films: LVFace 1032, mbf 2197, r18
1224. LVFace has less than half mbf's misID count while also scoring
higher on every film. This directly confirms the model choice out of
sample; it is not inferred from the training numbers alone. See the
[LVFace deep dive](lvface-deep-dive.md) for frame-level detail on where and
why LVFace still fails on the two worst films. Reproduce with
`scripts/docs/run_holdout_all_models.py`.
## Two effects in isolation: gallery scope and pose expansion
Averaging across the 3 compared models (r50 excluded) isolates each variable
from model choice.
**Gallery scope**, averaged over both expansion settings and all 3 models
(6 evaluations per row):
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once. This is not a precision/recall
trade: +4.8pp F1, +6.0pp recall, and roughly a quarter the misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections. Restriction is currently an offline optimizer technique, not a
runtime feature of the application; see
[Whole vs. cast-restricted gallery](gallery-scope.md) for what building it
into the application would require.
**Pose expansion** (promoting a confidently identified track's novel-pose
views into a per-film gallery annex,
[`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)):
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion is a clean win: +1.6pp F1, +3.3pp recall,
misID drops. The annex only competes against the film's own roughly 15-actor
cast, so a new pose of a known actor is unlikely to be confused with someone
else. In full mode, expansion buys +2.1pp F1 and +3.9pp recall but at a real
cost: misID rises from 407 to 714 as the same new-pose view now competes
against the full 2418-actor gallery, where a confidently learned pose is more
likely to match the wrong person. On the full gallery it is a recall-vs-misID
trade, not a free gain. This training-set effect
did not reproduce on held-out data; see
[Does pose expansion help?](pose-expansion.md) for the full held-out test
and the two methodology bugs caught while checking it.
## Calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | sim) = σ(a·sim + b)`,
stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This measures discriminative power independent of whatever
`prob_threshold` a given run used:
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
LVFace has the steepest curve (`a=17.7` vs 15.3-16.2 for the ArcFace
variants) and the lowest P=0.5 decision boundary (similarity 0.23 vs
0.27-0.31), separating same-actor from different-actor pairs more
confidently at a lower similarity than any ArcFace variant tested,
including r50. Generated by
[`scripts/docs/calibration_chart.py`](https://REPOLINK/scripts/docs/calibration_chart.py).
## Extinction and anneal window search
Every one of the 512 DE evaluations for the winning LVFace `full_exp`
combo, plotted over the `prob_threshold` × `extinction_sec` plane:
![DE search landscape: 512 evaluations over prob_threshold × extinction_sec](assets/images/de_search_landscape.png)
Nearly everything scoring well sits at `extinction_sec` above 50, across a
wide range of thresholds. Short extinction windows are uniformly weaker:
under a strict threshold, there is no good configuration in that region of
the search space. The optimizer converged with `anneal_sec=59.2,
extinction_sec=59.2`, about 99% of the widened 60s bound, which raises an
open question not resolved in this round: does performance keep improving
past 60s, or does it plateau there. Not chased further this pass.
## Caveats
- r50's 4 combos used the older, narrower search bounds (1-30/1-15 instead
of 1-60/1-60) and are further confounded by its thinner gallery. Excluded
from all comparisons above except calibration.
- The shipped defaults use `full_exp` (75.3% training F1), not the
higher-scoring `restricted_exp` (78.3%), because cast restriction is not
a runtime feature of the application yet.
- `expand_gallery` is mode-dependent, not a free win. Averaged across models
on the full gallery it trades misIDs for recall (see the pose-expansion
table). For LVFace specifically, though, `full_exp` beats `full_noexp` on
every axis at once (F1 75.3 vs 72.3, precision 89.7 vs 88.3, recall 65.4 vs
61.8, misID 232 vs 244), so the shipped `full_exp` is a clean choice for
this model, not an F1-vs-safety trade. (An earlier version of this page
reported `full_noexp` at 72.4% with zero misIDs and higher precision, which
made it look like the safer option; that was the dropped-film artifact
described above, not a real property of the config.)
- Switching the default model is an operational change: any gallery built
from a different model's embeddings must be rebuilt before the new
default takes effect.
## Reproduce
```bash
# 4-film training matrix, all 4 models × 2 gallery modes × 2 expansion settings
bash experiments/run_rep4_subprocess.sh
# single combo
SAE_EXPAND=1 REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \
--manifest experiments/manifests/rep4_LVFace-B_Glint360K_full.json \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \
--params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \
--popsize 10 --maxiter 15 --trajectory traj.jsonl --out best.json
# held-out validation, all 3 models, 5 films
python3 scripts/docs/run_holdout_all_models.py --out docs_data/holdout_all_models.json
# per-film training breakdown, all 3 models, 4 films
python3 scripts/docs/run_holdout_all_models.py --films training --out docs_data/training_per_film.json
# gallery coverage per film
python3 scripts/docs/gallery_coverage_per_film.py --out docs_data/gallery_coverage_per_film.json
# regenerate this page's charts from experiments/ artifacts
python3 scripts/docs/experiment_charts.py --out-dir docs/assets/images
# one frame per distinct out-of-cast name across all 9 films (used in the deep dive)
python3 scripts/docs/first_fpi_frames.py
```
See also the session log
[`experiments/SESSION_STATE.md`](https://REPOLINK/experiments/SESSION_STATE.md).
+110
View File
@@ -0,0 +1,110 @@
# Pose expansion: does promoting new poses mid-film help?
`expand_gallery`
([`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp))
promotes a confidently identified track's novel-pose reference views into a
per-film, in-memory gallery annex. The idea: once the pipeline is confident
about an identity, a pose it has not seen before (turned head, different
lighting) becomes an extra reference for recognizing that actor again later
in the same film, without touching the baked gallery.
## Training-set signal
Averaged across the 3 compared models (r50 excluded), on the 4 films used
for optimization. These are the corrected, full-coverage figures, see the
[dropped-film note](model-bakeoff.md#a-scoring-bug-worth-recording-dropped-film-evaluations)
in the experiment log for why an earlier version of this table overstated the
full-mode misID jump (209 → 864) that was itself partly a truncation artifact:
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion looks like a clean win: +1.6pp F1, +3.3pp
recall, lower misID. In full mode it looks like a recall-for-misID trade:
+2.1pp F1, +3.9pp recall, but misID rises from 407 to 714. See
[the full experiment log](model-bakeoff.md) for the per-model breakdown.
This asymmetry motivated the question below: does turning expansion on
change what gets recognized frame by frame, or is the aggregate F1 shift
coming from something else.
## Held-out test
Same model, same tuned config, `expand_gallery` toggled on vs. off, nothing
else changed, full gallery mode, per-second scoring against X-Ray. This
isolates expansion from every other variable that differs between the
training-set rows above.
LVFace-B Glint360K, all 5 held-out films:
| film | F1 (exp) | F1 (noexp) | TPI delta | FN delta |
|---|---|---|---|---|
| Benny & Joon | 83.0% | 83.0% | -2 | +2 |
| Downton Abbey: A New Era | 56.1% | 56.2% | -7 | +7 |
| Lovelace | 77.5% | 77.4% | +33 | -33 |
| The Many Saints of Newark | 46.3% | 46.3% | +2 | -2 |
| Valerian and the City of a Thousand Planets | 74.1% | 74.1% | +2 | -2 |
ArcFace R18, Benny & Joon, r18's own tuned config: F1 77.1% for both, TPI
and FN identical, FPI differs by 2.
Every film, both models tested: F1 differs by 0.1-0.2pp, TPI/FN swings are
in the tens out of tens of thousands. This is noise, not a signal.
Expansion made no measurable difference to per-second on-screen
identification on any held-out film tested.
## Two methodology bugs caught during this check
Getting to the table above required catching two wrong turns, both worth
recording because they are exactly the kind of error that produces a false
positive "expansion helped" finding.
1. **Timeout truncation.** The first Downton Abbey `exp` replay was cut off
by a 60-second subprocess timeout at about 76% through the film (5589 of
7368 expected seconds). This silent data loss produced a large,
convincing-looking TPI gap (47938 vs 52032) purely because one run was
missing a quarter of the film. Caught by comparing `n_seconds` between
runs before trusting any score delta; fixed by re-running with a longer
timeout.
2. **Bbox-matching bug.** An early per-second raw-annotation diff matched
each `exp` detection to the first `noexp` detection with IoU above 0.5,
not the best-overlapping one. With 3 faces close together in frame, this
produced spurious disagreements (for example "exp says Aidan Quinn,
noexp says Johnny Depp" at the same second) that vanished once the match
used the best-IoU candidate instead of the first one. Both configs had
actually output the same three names at the same three boxes.
Both bugs independently pointed toward "expansion is doing something," and
both were artifacts of the comparison harness, not the pipeline. Before
trusting a dramatic before/after diff, check that both runs cover the same
seconds and that entities are matched by best overlap, not first found.
## Conclusion
The training-set aggregate effect, particularly the full-mode misID
increase, does not reproduce on held-out data. At minimum it
is far smaller than the training-set numbers suggested; it may be sampling
variation from only 4 training films rather than a generalizable
mechanism. Note the same *class* of harness bug appears twice in this
investigation, the timeout truncation in bug #1 above, and the dropped-film
aggregation that inflated the raw training-set misID figures. Both make an
inert config look consequential; both are reasons to distrust a dramatic
training-set delta until it survives on held-out films, which this one did
not. This does not mean `expand_gallery` never does anything: the
mechanism is real, and
[`track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)'s
promotion logging confirms tracks get confirmed and views get promoted
into the annex on every film tested. It means whatever effect expansion
has on final per-second identification was too small to detect against 5
held-out films with this scoring method. A cleaner test would need either
more held-out films or a metric that can see the annex's direct
contribution, such as tagging which reference embedding won each match;
neither was in scope for this pass.
Do not treat the training-set exp/noexp numbers in
[the full experiment log](model-bakeoff.md) as proof that expansion changes
real-world behavior in either direction. On the evidence gathered so far,
it does not move the needle enough to see.
+215
View File
@@ -0,0 +1,215 @@
# Conversion to service — a native idle-GPU worker
Status: **design / proposal**. Nothing here is built yet.
## The idea
Turn the CLI tools into a **turnkey batch worker that uses the machine's idle
GPU**: it analyses newly-added Jellyfin media when you're not using the computer
(screen locked), and stops the instant you come back. It's an overnight job on
your own Linux box.
**No Docker.** This runs on your own machine with your own drivers, so a container
buys little and costs a lot: GPU passthrough (nvidia-container-toolkit, or
`/dev/kfd`+`/dev/dri`+`video` group for ROCm) is the single most fragile part of a
containerised setup, and it exists *only* because of the container. Natively, the
GPU just works with the drivers you already have, and the media paths Jellyfin
reports are just real paths — no re-mounting. So we ship a **native installer**
instead of an image builder.
Two deliverables:
1. **An installer**`scripts/build_install.py`. Detects your distro, ensures the
GPU/build dependencies are present (via `dnf`/`pacman`), compiles `scene_analyze`
for your GPU, and installs the binary + Python glue + two systemd **user**
units under `~/.local`.
2. **A screen-lock gate** — one of those systemd units watches logind lock/unlock
and starts/stops the worker. Lock → analyse. Unlock → stop.
## What already exists (reuse, don't rebuild)
The processing loop is already implemented — this is packaging, building, and
lock-gating, not new pipeline logic.
| Piece | Where | What it does |
|---|---|---|
| Analysis engine | `build/scene_analyze` | Video → face detect/align/embed → gallery match → result JSON |
| Backend selection | [`CMakeLists.txt`](https://REPOLINK/CMakeLists.txt) (`SAE_INFERENCE_BACKEND`, `SAE_GEMM_BACKEND`) | ORT/TRT + ROCm/CUDA, chosen **at build time** |
| New-media queue | JRay plugin → `GET /Plugins/JRay/Tasks/Pending` | Backlog of items with no results yet |
| Worker loop | [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)` --worker` | Poll Pending → run `scene_analyze` → push results |
| Result push | `PUT /Plugins/JRay/Items/{id}/Truth` | Stores per-actor scene windows back in Jellyfin |
| Incremental gallery | [`scripts/make_jellyfin_gallery.py`](https://REPOLINK/scripts/make_jellyfin_gallery.py)` --merge` | Embeds only cast not already in the gallery |
| Secrets loader | `.env` via [`scripts/sae_env.py`](https://REPOLINK/scripts/sae_env.py) | `JELLYFIN_URL`, `JELLYFIN_API_KEY`, `TMDB_API_KEY` |
## Installer config
One file. Build-time settings (fixed when we compile) vs. run-time settings (in the
worker's `.env`, editable without recompiling).
```yaml
# install.yaml — consumed by scripts/build_install.py
platform: nvidia # nvidia | amd | cpu → picks the cmake backend
model:
arcface: LVFace-B_Glint360K.onnx # embedder compiled against; gallery MUST match
schedule:
gallery_scan_interval: 24h # incremental --merge cadence; 0 disables the scanner
prefix: ~/.local # install root (bin, share, systemd user units)
# runtime (written to the worker .env, not compiled in):
runtime:
jellyfin_url: http://localhost:8096
# JELLYFIN_API_KEY / TMDB_API_KEY are filled into .env by hand after install
```
**Secrets never go in the repo or a build artifact** — the installer writes a
`.env` under the install prefix with blanks for the keys, and you fill them in
once. `sae_env.py` already loads it.
**Model ⇄ gallery coupling (guard, don't just document):** embeddings from
different recognition models aren't interchangeable. We compile against one
embedder; the gallery must be built with the same one. Stamp the embedder name
into `gallery.json`, and have the worker **refuse to start** if the gallery's
embedder ≠ the configured `model.arcface`, rather than silently mismatching.
## Dependencies via the system package manager
The heavy build/runtime deps (OpenCV, ffmpeg, the GPU stack) are best provided by
the distro, not vendored. The installer ships a per-distro dependency list and
either installs them or prints the exact command. Targets: **Fedora (dnf)** and
**Arch (pacman)** first.
| Dependency | Fedora (dnf) | Arch (pacman) |
|---|---|---|
| OpenCV | `opencv-devel` | `opencv` |
| ffmpeg | `ffmpeg-free`/`ffmpeg` (RPM Fusion) | `ffmpeg` |
| CMake / toolchain | `cmake gcc-c++` | `cmake gcc` |
| CUDA + TensorRT (nvidia) | NVIDIA CUDA repo + `libnvinfer-*` | `cuda`, `tensorrt` |
| ROCm (amd) | `rocm-hip-sdk` / `rocblas-devel` | `rocm-hip-sdk`, `rocblas` |
| ONNX Runtime | **not packaged** — installer fetches a pinned release tarball into the prefix | AUR `onnxruntime` (or same pinned-tarball fallback) |
So the flow is: **detect distro → check each package → install via the native
manager (or print `sudo dnf install …` / `sudo pacman -S …`)**, with ONNX Runtime
as the one known gap the installer fills itself (a pinned upstream release
extracted under the install prefix, so it doesn't depend on a system package that
may not exist). CUDA/ROCm being present is *assumed* — you already run a GPU
desktop; the installer verifies and points you at the vendor repo if not.
## What `build_install.py` does
```
build_install.py install.yaml
├─ detect distro (dnf vs pacman) and platform from config
├─ ensure deps: install via manager, or print the exact command; fetch ONNX Runtime if needed
├─ cmake + build scene_analyze with the platform's backend flags:
│ nvidia → -DSAE_INFERENCE_BACKEND=TRT -DSAE_GEMM_BACKEND=CUDA
│ amd → -DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=ROCM
│ cpu → -DSAE_INFERENCE_BACKEND=ORT (CPU EP; slow, for smoke tests)
├─ install into <prefix>:
│ bin/sae-scene-analyze the compiled binary
│ share/sae-worker/ Python glue + a venv (requests, etc.), models/
│ share/sae-worker/.env runtime config (keys blank, url from config)
├─ install systemd --user units:
│ sae-worker.service runs the worker + gallery-scan supervisor
│ sae-lock-gate.service watches logind lock/unlock, start/stops the worker
└─ print next steps (edit .env, `systemctl --user enable --now sae-lock-gate`)
```
## The worker service (supervisor)
`sae-worker.service` runs a small Python supervisor as its main process:
- starts the **worker loop** (`run_from_jellyfin.py --worker`) — the hot path,
- starts a **gallery-scan timer** — sleeps `gallery_scan_interval`, runs
`make_jellyfin_gallery.py --merge`, repeats,
- exits cleanly on SIGTERM (see re-queue below).
## The lock gate
`sae-lock-gate.service` runs a tiny watcher that subscribes to logind
lock/unlock signals and drives the worker service:
```
screen locks → systemctl --user start sae-worker.service
screen unlocks → systemctl --user stop sae-worker.service (SIGTERM)
```
**Screen-lock is the only signal — deliberately.** We don't also gate on GPU/CPU
load, because our own worker *is* the load: a load threshold would form a feedback
loop (worker starts → GPU spikes → threshold trips → worker stops → load drops →
restart → …). Lock state is external to what the worker does, so it can't
oscillate.
Signal source is desktop-dependent: logind `Lock`/`Unlock` (GNOME/KDE via
`loginctl`/D-Bus) covers most setups; a `swayidle`/`xss-lock` hook is the fallback
for wlroots/X-only compositors. The installer picks based on what's present.
## On resume: hard stop + re-queue (it's free)
Stopping the worker mid-analysis costs nothing to reschedule, because of how the
JRay queue works: **an item only leaves `/Tasks/Pending` once its results are
pushed** (`push_truth`). A worker stopped mid-`scene_analyze` simply leaves that
item Pending — next lock picks it up again. No re-queue bookkeeping.
Two small correctness requirements (the only worker changes needed):
1. **Never push a partial result.** Already true — `push_truth` runs only after
`scene_analyze` returns; a killed run pushes nothing. ✓ (keep it that way).
2. **Clean up on signal.** `process_item` writes a temp filtered-gallery file and
unlinks it in a `finally`; a SIGKILL skips `finally`. Fix: write temps under a
dir the worker wipes on start, and/or a SIGTERM handler that unlinks before
exit. Minor.
Accepted trade-off: a partially-analysed title restarts from scratch next lock.
Fine for an overnight/idle workload; no mid-video checkpointing.
## The end-to-end UX
```bash
# once: build + install for your GPU + model
./scripts/build_install.py install.yaml
# detects Fedora/Arch, ensures deps, compiles, installs units under ~/.local
# once: set your keys, enable the gate
$EDITOR ~/.local/share/sae-worker/.env # JELLYFIN_API_KEY, TMDB_API_KEY
systemctl --user enable --now sae-lock-gate.service
# from then on: nothing. Lock your screen → it analyses. Unlock → it stops.
```
No Docker, no GPU passthrough config, no media re-mounting — the worker sees the
same filesystem and GPU as everything else on the box.
## Implementation plan (follow-up commits)
Ordered so each step stands alone:
1. **installer skeleton**`scripts/build_install.py`: parse `install.yaml`,
distro detect, dependency check/print (start with cpu platform so it builds
without a GPU), cmake+build, copy into prefix.
2. **supervisor + cleanup**`scripts/service.py` (worker loop + gallery-scan
timer + SIGTERM); temp-file cleanup fix in `run_from_jellyfin.py`.
3. **systemd units + lock gate** — generate/install `sae-worker.service`,
`sae-lock-gate.service`, and the logind lock watcher.
4. **gallery/model guard** — stamp embedder into `gallery.json`; startup mismatch
check.
5. **platform + distro matrix** — nvidia/amd backends; dnf/pacman dep lists; ONNX
Runtime fetch fallback.
6. **docs** — README "Run on your idle GPU" section.
## Settled decisions
- **ONNX Runtime build** — the installer fetches the **ROCm ORT** release. It
serves the `amd` platform, and its CPU execution provider covers the `cpu`
smoke-test fallback too, so one download handles both. (nvidia uses raw TRT and
doesn't need ORT.)
- **`dnf`/`pacman` invocation** — **auto-install.** The installer runs `sudo dnf
install …` / `sudo pacman -S …` itself (prompting for sudo), rather than only
printing the command. It still prints what it's about to install first.
- **Distro coverage** — **Fedora + Arch only** for now. Debian/Ubuntu (`apt`) is
out of scope.
## Open questions
*(none blocking — the spec above is buildable as-is.)*
+269
View File
@@ -0,0 +1,269 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15119,
"FPI": 1845,
"FPI_misid": 0,
"FPI_incast": 1845,
"FN": 4343,
"precision": 0.8912402735203961,
"precision_raw": 0.8912402735203961,
"recall": 0.7768471893947179,
"f1": 0.8301213418986437,
"agreement_rate": 0.7239983093829193,
"exact_match_rate": 0.41098901098901097,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 52022,
"FPI": 1160,
"FPI_misid": 0,
"FPI_incast": 1160,
"FN": 80089,
"precision": 0.9781881087586025,
"precision_raw": 0.9781881087586025,
"recall": 0.39377493168623356,
"f1": 0.5615106884771687,
"agreement_rate": 0.4057686401759256,
"exact_match_rate": 0.033084311632870865,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14988,
"FPI": 1086,
"FPI_misid": 58,
"FPI_incast": 1028,
"FN": 7087,
"precision": 0.9031091829356471,
"precision_raw": 0.9324374766703994,
"recall": 0.6789580973952435,
"f1": 0.775154508546456,
"agreement_rate": 0.7204967829586512,
"exact_match_rate": 0.3597703211914588,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15928,
"FPI": 4394,
"FPI_misid": 974,
"FPI_incast": 3420,
"FN": 23785,
"precision": 0.5475797579757976,
"precision_raw": 0.7837811239051274,
"recall": 0.40107773273235464,
"f1": 0.46301652592258835,
"agreement_rate": 0.3705156874642392,
"exact_match_rate": 0.04588936642173853,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18658,
"FPI": 548,
"FPI_misid": 0,
"FPI_incast": 548,
"FN": 12472,
"precision": 0.9714672498177653,
"precision_raw": 0.9714672498177653,
"recall": 0.5993575329264376,
"f1": 0.7413382072472983,
"agreement_rate": 0.5877853464704299,
"exact_match_rate": 0.21980294368081743,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15219,
"FPI": 2463,
"FPI_misid": 180,
"FPI_incast": 2283,
"FN": 4243,
"precision": 0.7884675163195524,
"precision_raw": 0.8607058025110281,
"recall": 0.7819854074606927,
"f1": 0.7852130843050252,
"agreement_rate": 0.7072306082196138,
"exact_match_rate": 0.34911242603550297,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 53043,
"FPI": 2383,
"FPI_misid": 604,
"FPI_incast": 1779,
"FN": 79068,
"precision": 0.8715290328940882,
"precision_raw": 0.9570057373795692,
"recall": 0.4015032813316075,
"f1": 0.5497453011561202,
"agreement_rate": 0.4094114144765671,
"exact_match_rate": 0.032817502668089645,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14606,
"FPI": 1337,
"FPI_misid": 180,
"FPI_incast": 1157,
"FN": 7469,
"precision": 0.8316346865569664,
"precision_raw": 0.916138744276485,
"recall": 0.6616534541336353,
"f1": 0.7369695746505879,
"agreement_rate": 0.6907677036961077,
"exact_match_rate": 0.31742329086667864,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15223,
"FPI": 4574,
"FPI_misid": 994,
"FPI_incast": 3580,
"FN": 24490,
"precision": 0.52962460425147,
"precision_raw": 0.768954892155377,
"recall": 0.383325359454083,
"f1": 0.44475283393712756,
"agreement_rate": 0.3554753116932427,
"exact_match_rate": 0.03715513655899071,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18472,
"FPI": 853,
"FPI_misid": 239,
"FPI_incast": 614,
"FN": 12658,
"precision": 0.860122927919538,
"precision_raw": 0.9558602846054334,
"recall": 0.5933825891423065,
"f1": 0.7022773067710907,
"agreement_rate": 0.5795914643682625,
"exact_match_rate": 0.18817662084904513,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 13580,
"FPI": 1666,
"FPI_misid": 60,
"FPI_incast": 1606,
"FN": 5882,
"precision": 0.86025592296972,
"precision_raw": 0.8907254361799817,
"recall": 0.697770013359367,
"f1": 0.7705401724920563,
"agreement_rate": 0.6547675401521545,
"exact_match_rate": 0.32578191039729504,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 48545,
"FPI": 1066,
"FPI_misid": 180,
"FPI_incast": 886,
"FN": 83566,
"precision": 0.9475708067381078,
"precision_raw": 0.9785128298159682,
"recall": 0.3674561542944948,
"f1": 0.5295567845883649,
"agreement_rate": 0.3815368792000116,
"exact_match_rate": 0.032950907150480255,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 13615,
"FPI": 963,
"FPI_misid": 120,
"FPI_incast": 843,
"FN": 8460,
"precision": 0.8695235662281262,
"precision_raw": 0.933941555768967,
"recall": 0.6167610419026047,
"f1": 0.7216494845360825,
"agreement_rate": 0.6506356469257915,
"exact_match_rate": 0.2894311860757222,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 13489,
"FPI": 3757,
"FPI_misid": 796,
"FPI_incast": 2961,
"FN": 26224,
"precision": 0.5526013928717739,
"precision_raw": 0.7821523831613127,
"recall": 0.3396620753909299,
"f1": 0.42072267361165266,
"agreement_rate": 0.3229817885335633,
"exact_match_rate": 0.04422570359073894,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 17692,
"FPI": 397,
"FPI_misid": 68,
"FPI_incast": 329,
"FN": 13438,
"precision": 0.9460456660071654,
"precision_raw": 0.9780529603626513,
"recall": 0.5683263732733698,
"f1": 0.710080070638759,
"agreement_rate": 0.5633540120828806,
"exact_match_rate": 0.13404695292543486,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
}
}
+221
View File
@@ -0,0 +1,221 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 14499,
"FPI": 1380,
"FPI_misid": 0,
"FPI_incast": 1380,
"FN": 12231,
"precision": 0.9130927640279615,
"precision_raw": 0.9130927640279615,
"recall": 0.5424242424242425,
"f1": 0.6805604449764134,
"agreement_rate": 0.57285804629501,
"exact_match_rate": 0.18947003810183582,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13893,
"FPI": 1654,
"FPI_misid": 174,
"FPI_incast": 1480,
"FN": 5737,
"precision": 0.811838952842868,
"precision_raw": 0.8936129156750499,
"recall": 0.7077432501273561,
"f1": 0.7562256756388972,
"agreement_rate": 0.7005158404089996,
"exact_match_rate": 0.38715420432758146,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 20518,
"FPI": 1078,
"FPI_misid": 58,
"FPI_incast": 1020,
"FN": 14722,
"precision": 0.9276607288181572,
"precision_raw": 0.9500833487682904,
"recall": 0.5822360953461975,
"f1": 0.7154363820216882,
"agreement_rate": 0.6297735703976657,
"exact_match_rate": 0.25910733470065433,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 13349,
"FPI": 677,
"FPI_misid": 0,
"FPI_incast": 677,
"FN": 6504,
"precision": 0.9517324967916726,
"precision_raw": 0.9517324967916726,
"recall": 0.6723920818012391,
"f1": 0.7880397886596416,
"agreement_rate": 0.7112222835587533,
"exact_match_rate": 0.40311896218603366,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 13456,
"FPI": 1434,
"FPI_misid": 180,
"FPI_incast": 1254,
"FN": 13274,
"precision": 0.8150211992731677,
"precision_raw": 0.9036937541974479,
"recall": 0.5034044145155256,
"f1": 0.622386679000925,
"agreement_rate": 0.5463565775524695,
"exact_match_rate": 0.19154832005542086,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13778,
"FPI": 1740,
"FPI_misid": 60,
"FPI_incast": 1680,
"FN": 5852,
"precision": 0.8580146967243741,
"precision_raw": 0.8878721484727413,
"recall": 0.7018848700967907,
"f1": 0.7721362923111411,
"agreement_rate": 0.6881858851455998,
"exact_match_rate": 0.36469460421802247,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 18863,
"FPI": 862,
"FPI_misid": 0,
"FPI_incast": 862,
"FN": 16377,
"precision": 0.956299112801014,
"precision_raw": 0.956299112801014,
"recall": 0.535272417707151,
"f1": 0.6863640498499044,
"agreement_rate": 0.594178111391709,
"exact_match_rate": 0.2357652114464303,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12642,
"FPI": 554,
"FPI_misid": 0,
"FPI_incast": 554,
"FN": 7211,
"precision": 0.9580175810851773,
"precision_raw": 0.9580175810851773,
"recall": 0.6367803354656727,
"f1": 0.7650458410239341,
"agreement_rate": 0.687468948385309,
"exact_match_rate": 0.3789677063207287,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 12207,
"FPI": 1119,
"FPI_misid": 60,
"FPI_incast": 1059,
"FN": 14523,
"precision": 0.88035482475119,
"precision_raw": 0.9160288158487168,
"recall": 0.45667789001122333,
"f1": 0.6013892994383683,
"agreement_rate": 0.5125626845924863,
"exact_match_rate": 0.1674748874263942,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13122,
"FPI": 1409,
"FPI_misid": 60,
"FPI_incast": 1349,
"FN": 6508,
"precision": 0.870678787074514,
"precision_raw": 0.9030348909228546,
"recall": 0.6684666327050433,
"f1": 0.7562894441082388,
"agreement_rate": 0.6698963754222389,
"exact_match_rate": 0.3389482333607231,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 16961,
"FPI": 685,
"FPI_misid": 0,
"FPI_incast": 685,
"FN": 18279,
"precision": 0.961181004193585,
"precision_raw": 0.961181004193585,
"recall": 0.48129965947786607,
"f1": 0.6414173883447416,
"agreement_rate": 0.5440709115628456,
"exact_match_rate": 0.2017775173356773,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12017,
"FPI": 591,
"FPI_misid": 122,
"FPI_incast": 469,
"FN": 7836,
"precision": 0.8767692981176127,
"precision_raw": 0.953125,
"recall": 0.6052989472623784,
"f1": 0.7161715188176049,
"agreement_rate": 0.6594534915815532,
"exact_match_rate": 0.3573005796301408,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
}
}
+362
View File
@@ -0,0 +1,362 @@
[
{
"crop": "eval/probe/nm0001589_0000.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0119_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0001.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0125_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0002.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0128_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0003.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0225_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0004.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0306_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0005.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0308_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0006.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0311_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0007.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0315_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0008.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0317_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0009.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0319_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0000.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0001.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0074_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0002.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0078_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0003.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0079_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0004.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0080_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0005.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0638_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0006.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0641_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0007.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0017_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0008.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0022_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0009.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0024_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0000.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0097_img_2.jpg"
},
{
"crop": "eval/probe/nm0005042_0001.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0100_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0002.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0108_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0003.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0110_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0004.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0121_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0005.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0123_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0006.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0151_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0007.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0158_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0008.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0178_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0009.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0180_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0000.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0154_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0001.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0157_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0002.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0161_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0003.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0163_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0004.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0164_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0005.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0167_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0006.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0199_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0007.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0200_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0008.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0201_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0009.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0202_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0000.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0005_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0001.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0009_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0002.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0010_img_2.jpg"
},
{
"crop": "eval/probe/nm1385871_0003.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0013_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0004.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0014_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0005.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0543_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0006.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0585_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0007.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0628_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0008.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0631_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0009.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0635_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0000.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0019_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0001.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0032_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0002.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0055_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0003.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0057_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0004.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0059_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0005.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0063_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0006.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0007.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0074_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0008.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0075_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0009.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0080_img_0.jpg"
}
]
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+20
View File
@@ -0,0 +1,20 @@
# MovieNet Validation Report
## Summary
| Model | Rank-1 | Det.Fail | Mean-sim | Probes | Size |
| ----- | ------ | -------- | -------- | ------ | ----- |
| R50 | 85.2% | 10.0% | 0.470 | 60 | 167MB |
| R18 | 72.2% | 10.0% | 0.453 | 60 | 46MB |
| MBF | 83.3% | 10.0% | 0.383 | 60 | 13MB |
## Per-Actor Recall
| Actor | R50 | R18 | MBF |
| ------------------ | ------------ | ------------ | ------------ |
| Steve Buscemi | 100% (9/9) | 100% (9/9) | 100% (9/9) |
| Michael Palin | 88% (7/8) | 50% (4/8) | 62% (5/8) |
| Jason Isaacs | 100% (10/10) | 100% (10/10) | 100% (10/10) |
| Paddy Considine | 50% (5/10) | 50% (5/10) | 60% (6/10) |
| Olga Kurylenko | 89% (8/9) | 67% (6/9) | 100% (9/9) |
| Andrea Riseborough | 88% (7/8) | 62% (5/8) | 75% (6/8) |
+21
View File
@@ -0,0 +1,21 @@
# experiments/ in git keeps only scripts, README.md, SESSION_STATE.md, and this
# file. Every data artifact — galleries, embedding dumps, the X-Ray corpus,
# montage/frame images, DE trajectories, film manifests, and result summaries —
# is pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to the Gitea
# generic package registry instead (see docs/rep4-optimizer-results.md).
xray/
dumps/
galleries/
*.h5
manifests/
trajectories/
results/
# Raw run logs and scratch scripts (regenerated by every run).
_scratch/
# Real local media paths (film slug -> path on this machine). Never committed —
# these paths embed the specific source file names, which can include
# scene-release tags. Only file-lut.template.json (placeholders) is tracked;
# copy it to file-lut.json and fill in your own paths.
file-lut.json
+59
View File
@@ -0,0 +1,59 @@
# experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT `/tmp` scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
## Layout
- `xray/` — Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).
- `dumps/` — per-model embedding dumps, one HDF5 per (model, film). Gitignored (large).
Naming: `<model>/dump_<Film>.h5`. Regenerate with `scene_analyze --dump-embeddings`.
- `galleries/` — per-model galleries (gitignored JSON). `gallery_<model>.json` +
augmented variants. Regenerate with build_gallery / fetch_missing_actors.
- `manifests/` — film manifests (committed — small, and the Jellyfin ID join is the
authoritative record of which films/paths/X-Ray-dirs were used).
- `trajectories/` — DE trajectories, one JSONL per run (committed — the evidence).
- `results/` — final per-run metrics + the model comparison table (committed).
## Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest?
Method: **optimize per model** — each model gets its own dumps + gallery + full DE run,
then compare each model at ITS OWN optimum (fairest — no model penalised by another's
threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see
docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
| ----- | ---- | ---- | ----------------------- |
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society,
Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface,
Sound of Metal, Valerian.
## Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate
set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a
film he's not in) vs. matching against the whole 2418-actor gallery?
- **full** — match against the entire model gallery (2418 actors).
- **restricted** — per film, match only against its Jellyfin credited cast, filtered
from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
**LIMITATION — Jellyfin stores only ~15 actors per title.** Jellyfin's People list is
capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface:
Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a
query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item
/Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15
top-billed leads, which caps its achievable recall at whatever fraction of on-screen
actors are top-billed, but should drive out-of-cast misIDs toward zero. A production
deployment wanting fuller cast restriction would need a richer cast source than Jellyfin
(TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline
galleries (no augmentation — avoids test-set leakage on either arm). Scored by the
duration-weighted per-scene metric with the misID split (report_rates.py).
+105
View File
@@ -0,0 +1,105 @@
# Session state — X-Ray optimizer + model bake-off (as of 2026-07-18)
Handoff for a fresh session. Everything below is UNCOMMITTED — commit early next session.
## What we're doing
Optimizing the scene-actor pipeline's thresholds against Amazon X-Ray ground truth, and
running a **model bake-off** (4 embedding models × gallery-mode × expansion) to answer:
is LVFace (455MB) actually best, or just biggest? Does cast-restriction cut false IDs?
Does per-film gallery expansion help?
## The metric (final form — this is what to use)
`scripts/optimizer/second_score.py` — UNIFORM PER-SECOND sampling vs X-Ray:
- At each second t: GT = X-Ray scene's cast at t; Pred = actors whose window covers t.
- TPI / FPI / FN counted per second. **FPI weighted 10×** when the named actor isn't in
the film's cast at all (a true misID like naming Archie Yates in a film he's not in) vs
an in-cast timing slip.
- **FN is fair**: only counts gallery-known cast (67% of X-Ray cast have no reference
embedding, can't be recognised — see [[gallery-coverage-gap]]).
- **agreement_rate** = mean per-second Jaccard (partial credit: "% of on-screen actors we
agree with X-Ray about, over time"). NOT exact-set match.
- Objective = macro-mean per-second weighted F1.
## Two DIFFERENT hangs — do not conflate them (corrected 2026-07-18)
**(a) The self-inflicted 100% hang (FIXED).** replay.py's CLI briefly called
`replay(..., stop=False)` intending to `os._exit(0)` straight after, to "dodge" teardown.
That was wrong: `PyNode::stop()` is the ONLY thing that sets `stop_flag_=true`, which is the
ONLY exit condition for the source node's `run_loop()`. Skipping it meant the local `net`
destructor — which runs synchronously when `replay()` returns, BEFORE main() can reach
os._exit — joined a thread that could never stop. A **guaranteed** hang, not the driver
flake. Symptom: every solo replay timed out at 45s and DE reported flat F1=0.0%.
FIX: `replay(..., stop=True)` so `PyNode::stop()` signals the thread before the join;
removed the dead os._exit / unused os import. VERIFIED: 45s guaranteed timeout → clean ~8s
completion (3/3), and optimize.py's DE sweep returns correct non-zero metrics (F1 48-66%,
matching prior best-so-far). Only 1 isolated per-film timeout in 11 evals × 3 films.
**(b) The genuine ROCm flake (rare, tolerated).** net.stop()→jthread.join() CAN still hang
on a KPN worker stuck mid-rocBLAS-GEMM — a KNOWN ROCm bug
(github.com/RadeonOpenCompute/ROCT-Thunk-Interface#56), NOT our code. HSA_ENABLE_SDMA=0
makes it WORSE (breaks the matcher's DMA). It is much rarer than the ~20-30% figure quoted
earlier in this session — that number was inflated by (a). The existing subprocess + 45s
timeout absorbs it correctly.
## Both architectures are usable
- `scripts/optimizer/optimize.py` + `replay.py` — subprocess per film, simpler, tolerates the
rare true flake via its timeout. NOT broken; good for fallback / quick single-model runs.
- `scripts/optimizer/model_server.py` + `optimize_server.py` — ONE persistent net per
(model, gallery); replay each film by SWITCHING THE SOURCE (repoint frame list + reset
index), change thresholds via runtime SETTERS, os._exit(0) at the very end (after all work,
so no destructor-join problem). Higher throughput: skips gallery/build overhead per eval.
VERIFIED: "ready", replays, emits metrics, ~30-40s/eval (GPU-bound, films serial).
Still the preferred option for the long overnight matrix.
## Key C++ changes made (all in the KPN spec-and-tsan branch + our nodes)
1. Runtime setters: `IdentityMatcherFunc::set_prob_threshold`, `SceneTrackerFunc::set_extinction_sec`
(src/nodes/*). Exposed via sae_kpn: `set_prob_threshold(net,name,v)`, `set_extinction_sec(...)`.
Needed `ObjectVariantNodeWrapper::functor()` + `PyNetwork::node_ptr()` accessors.
2. `Channel::push_blocking()` (external/KPN/.../channel.hpp) — lossless backpressure push
(waits instead of dropping when full). Exposed on IVariantChannel/VariantChannel; PyNode's
run_loop now uses it. Reduced but did NOT fully fix a residual ~0.5% frame loss (25/5915)
— the loss is elsewhere (matcher output or reader EOF-race). DECISION: accept it, <0.5%
scattered doesn't change per-second F1 or rankings. Don't chase further.
3. `dump_embeddings` standalone exe + `--max-decode-fps` (fixes LVFace dump truncation under
parallel load). HDF5 gallery fast-load in gallery_store.cpp (18s JSON → 0.06s).
`scripts/optimizer/json_to_hdf5_gallery.py` converts; galleries are `.h5` now.
All of KPN, matcher, scene_tracker, bindings need a rebuild:
`cmake --build build --target sae_kpn sae_gallery dump_embeddings scene_analyze`
## Data on disk (durable, experiments/)
- `experiments/xray/` — X-Ray Zenodo dataset. `experiments/dumps/<model>/dump_<slug>.h5`
all 9 films × 4 models, ALL FULL (LVFace re-dumped with --max-decode-fps 8). VERIFY counts
match R50 before trusting (LVFace truncated under parallel dumping earlier).
- `experiments/galleries/gallery_<model>.h5` (+ restricted/<model>/<slug>.h5, per-film cast-
filtered to Jellyfin's ~15 top-billed — Jellyfin's hard cap, see experiments/README.md).
- `experiments/manifests/rep3_<model>_<mode>.json` — 3 REPRESENTATIVE films (Lord of War /
Scarface / Sound of Metal = clean / ensemble-lookalike / high-coverage) to keep evals fast
(~28s vs ~90s for 9). Winner should be re-scored on all 9 after.
- `experiments/manifests/films_<model>_<mode>.json` — all 9 films.
## Salvaged partial results (per-second metric)
- R50 full +expand: **F1 66.4%** (208 evals, converged) — best so far
- R50 full noexp: 55-60% → **expansion helps ~+6-11 recall**
- MBF full noexp: 58.6%
- (older scene-metric runs, superseded: R50≈LVFace≈MBF ~85%, restricted>full, LVFace not
worth its size — but those used the OLD scene-union metric, redo with per-second.)
## TO DO next session
1. **COMMIT everything first** (logical chunks: KPN setters+push_blocking; sae_kpn+dump exe;
HDF5 gallery; optimizer scripts; per-second metric; experiments manifests/results/docs +
tuned config.hpp defaults prob_threshold 0.76 extinction 1.5).
2. Launch the full 16-run matrix via model_server on rep3 films (write trajectories to
experiments/, NOT /tmp — /tmp gets wiped mid-session and cost us hours). ~28s/eval ×
~84 evals × 16 = ~10hr. Runner pattern: experiments/run_overnight_rep3.sh but pointing
optimize_server.py at model_server.
3. assemble table: best model + expansion effect + misID, from experiments/results/*.json.
4. Consider upstreaming to KPN++: runtime node setters, push_blocking, node_ptr/functor().
## Gotchas that burned time (don't repeat)
- /tmp scratch gets WIPED mid-session → lost dumps + test files repeatedly. Use experiments/.
- Verify a launched runner script EXISTS and PRODUCES evals before walking away (a heredoc
once silently failed to write; a stale-code process ran the old metric for 12h).
- pgrep/ps "survivors" are often the grep's own shell wrapper — check via /proc cmdline or ps.
- Running many DE/replay processes in parallel on one GPU → deadlock/thrash. GPU peaks ~35%
(not saturated) but concurrency>2-3 wedges. Serial-ish is safer.
+66
View File
@@ -0,0 +1,66 @@
#!/usr/bin/env python3
"""
Build per-(model, mode) manifests for the model × gallery-mode bake-off.
Modes:
full — every film matches against the whole model gallery (2418 actors)
restricted — each film matches only its Jellyfin credited cast (~15 top-billed),
via a per-film gallery filtered from the model gallery by jellyfin_id.
Each manifest is a list of {name, xray, slug, dump, gallery} — no "movie" path (that's
resolved locally via experiments/file-lut.json, see run_montage_all.py, to avoid
embedding source filenames in a file that gets shared as an artifact). optimize.py
reads film["gallery"] per film, so restricted mode just points each film at its own
filtered gallery — no optimizer change needed.
Writes experiments/manifests/films_<model>_<mode>.json and the restricted galleries to
experiments/galleries/restricted/<model>/<slug>.h5.
"""
import json
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(REPO / "scripts" / "validation"))
sys.path.insert(0, str(REPO / "scripts"))
from identity import norm_name # noqa: E402
from sae_gallery import load_gallery_hdf5, save_gallery_hdf5 # noqa: E402
MODELS = ["arcface_w600k_r50", "arcface_r18", "arcface_w600k_mbf", "LVFace-B_Glint360K"]
films = json.loads((REPO / "experiments/manifests/films.json").read_text())
casts = json.loads((REPO / "experiments/manifests/jellyfin_casts.json").read_text())
def restrict_gallery(model_gallery: dict, cast_ids: set[str]) -> dict:
kept = [a for a in model_gallery["actors"] if a.get("jellyfin_id", "") in cast_ids]
return {"actors": kept}
for model in MODELS:
gpath = REPO / f"experiments/galleries/gallery_{model}.h5"
if not gpath.exists():
print(f"skip {model}: gallery not built yet ({gpath})")
continue
model_gal = load_gallery_hdf5(gpath)
# full mode
full = [{**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5",
"gallery": f"experiments/galleries/gallery_{model}.h5"} for f in films]
(REPO / f"experiments/manifests/films_{model}_full.json").write_text(json.dumps(full, indent=2, ensure_ascii=False))
# restricted mode — per-film filtered gallery
rdir = REPO / f"experiments/galleries/restricted/{model}"
rdir.mkdir(parents=True, exist_ok=True)
restr = []
for f in films:
cast_ids = set(casts.get(f["name"], []))
rg = restrict_gallery(model_gal, cast_ids)
rgpath = rdir / f"{f['slug']}.h5"
save_gallery_hdf5(rg, rgpath)
restr.append({**f, "dump": f"experiments/dumps/{model}/dump_{f['slug']}.h5",
"gallery": str(rgpath.relative_to(REPO)),
"_cast_size": len(rg["actors"])})
(REPO / f"experiments/manifests/films_{model}_restricted.json").write_text(json.dumps(restr, indent=2, ensure_ascii=False))
avg = sum(r["_cast_size"] for r in restr) / len(restr)
print(f"{model}: full (2418) + restricted (avg {avg:.0f} actors/film) manifests written")
+11
View File
@@ -0,0 +1,11 @@
{
"Benny___Joon": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Café_Society": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Downton_Abbey__A_New_Era": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Lord_of_War": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Lovelace": "/path/to/your/movies/<replace-with-your-file>.mp4",
"The_Many_Saints_of_Newark": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Scarface": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Sound_of_Metal": "/path/to/your/movies/<replace-with-your-file>.mp4",
"Valerian_and_the_City_of_a_Thousand_Plan": "/path/to/your/movies/<replace-with-your-file>.mp4"
}
+93
View File
@@ -0,0 +1,93 @@
#!/usr/bin/env python3
"""
run_montage_all.py — replay + best/worst-per-scene montage for all films in a
manifest, using the shipped config.hpp defaults (LVFace, prob_threshold=0.754,
anneal_sec=35.54, extinction_sec=57.43, expand_gallery). Skips a film if its
montage manifest.json already exists (safe to re-run/resume).
Manifests no longer carry a "movie" path (removed to avoid embedding source
filenames — some carry scene-release tags — in a file that gets zipped and
pushed to the artifact registry). Movie paths are resolved locally via
experiments/file-lut.json (gitignored; copy file-lut.template.json and fill
in your own paths).
Usage:
python experiments/run_montage_all.py \
--manifest experiments/manifests/films_LVFace-B_Glint360K_full.json \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \
--out-dir experiments/results/holdout/montage
"""
import argparse
import json
import subprocess
import sys
from pathlib import Path
REPO = Path(__file__).resolve().parent.parent
REPLAY = str(REPO / "scripts" / "optimizer" / "replay.py")
MONTAGE = str(REPO / "scripts" / "optimizer" / "dump_scene_montage.py")
FILE_LUT = REPO / "experiments" / "file-lut.json"
CFG = {"prob_threshold": "0.7540024664611272", "anneal_sec": "35.53996030397922",
"extinction_sec": "57.43359645269811"}
def main():
p = argparse.ArgumentParser()
p.add_argument("--manifest", required=True)
p.add_argument("--gallery", required=True)
p.add_argument("--out-dir", required=True)
args = p.parse_args()
if not FILE_LUT.exists():
sys.exit(f"{FILE_LUT} not found — copy file-lut.template.json to "
f"file-lut.json and fill in your local movie paths")
file_lut = json.loads(FILE_LUT.read_text())
films = json.loads(Path(args.manifest).read_text())
out_root = Path(args.out_dir)
out_root.mkdir(parents=True, exist_ok=True)
for f in films:
slug = f["slug"]
montage_dir = out_root / slug
manifest_path = montage_dir / "manifest.json"
if manifest_path.exists():
print(f"[run_montage_all] skip {slug} (already done)", file=sys.stderr)
continue
movie_path = file_lut.get(slug)
if not movie_path:
print(f"[run_montage_all] skip {slug}: no entry in {FILE_LUT}", file=sys.stderr)
continue
print(f"[run_montage_all] === {f['name']} ({slug}) ===", file=sys.stderr)
raw_path = out_root / f"raw_{slug}.jsonl"
pred_path = out_root / f"pred_{slug}.json"
replay_cmd = [sys.executable, REPLAY, "--dump", f["dump"], "--gallery", args.gallery,
"--out", str(pred_path), "--raw-out", str(raw_path),
"--prob-threshold", CFG["prob_threshold"],
"--anneal-sec", CFG["anneal_sec"],
"--extinction-sec", CFG["extinction_sec"], "--expand-gallery"]
r = subprocess.run(replay_cmd, capture_output=True, text=True, timeout=120)
if r.returncode != 0:
print(f"[run_montage_all] replay FAILED for {slug}: {r.stderr[-2000:]}",
file=sys.stderr)
continue
montage_cmd = [sys.executable, MONTAGE, "--raw", str(raw_path), "--dump", f["dump"],
"--xray", f["xray"], "--movie", movie_path, "--gallery", args.gallery,
"--out-dir", str(montage_dir)]
r = subprocess.run(montage_cmd, capture_output=True, text=True, timeout=1800)
if r.returncode != 0:
print(f"[run_montage_all] montage FAILED for {slug}: {r.stderr[-2000:]}",
file=sys.stderr)
continue
print(r.stderr.strip().splitlines()[-1] if r.stderr else "(no output)", file=sys.stderr)
print("[run_montage_all] ALL DONE", file=sys.stderr)
if __name__ == "__main__":
main()
+26
View File
@@ -0,0 +1,26 @@
#!/bin/bash
set -u
cd /home/dtourolle/Development/scene-actor-extraction
mkdir -p experiments/trajectories experiments/results
MODELS=(arcface_w600k_r50 arcface_r18 arcface_w600k_mbf LVFace-B_Glint360K)
MODES=(full restricted); EXPAND=(1 0)
for model in "${MODELS[@]}"; do
for mode in "${MODES[@]}"; do
for exp in "${EXPAND[@]}"; do
xtag=$([ "$exp" = 1 ] && echo exp || echo noexp); tag="${model}_${mode}_${xtag}"
best="experiments/results/rep4_best_${tag}.json"; traj="experiments/trajectories/rep4_${tag}.jsonl"
[ -f "$best" ] && { echo "skip $tag"; continue; }
: > "$traj"; echo "=== $tag START $(date '+%H:%M') ==="
SAE_EXPAND=$exp REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \
--manifest "experiments/manifests/rep4_${model}_${mode}.json" \
--gallery "experiments/galleries/gallery_${model}.h5" \
--params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \
--popsize 10 --maxiter 15 --seed 0 \
--trajectory "$traj" --out "experiments/results/opt_rep4_${tag}.json" \
> "experiments/results/rep4_de_${tag}.log" 2>&1
python3 -c "import json;r=[json.loads(l) for l in open('$traj')];b=max(r,key=lambda x:x['f1']);json.dump({'best':b,'n_evals':len(r)},open('$best','w'))" 2>/dev/null
echo "done $tag $(date '+%H:%M'): $(python3 -c "import json;d=json.load(open('$best'));print('F1=%.1f%% misID=%d'%(d['best']['f1']*100,d['best'].get('FPI_misid',-1)))" 2>/dev/null)"
done
done
done
echo "=== REP4 MATRIX COMPLETE $(date '+%F %H:%M') ==="

Some files were not shown because too many files have changed in this diff Show More