Author SHA1 Message Date
dtourolle 9c029d43ca docs: deploy from e588597 2026-07-21 09:12:46 +02:00
dtourolle 34b1f2c58f docs: deploy from 0bd2747 2026-07-21 08:56:42 +02:00
dtourolle 36f75ba199 docs: deploy from 4b55579 2026-07-19 22:28:14 +02:00
dtourolle 252625330f docs: deploy from 4925443 2026-07-19 19:42:29 +02:00
254 changed files with 17798 additions and 36926 deletions
-1
View File
@@ -1 +0,0 @@
models/*.onnx filter=lfs diff=lfs merge=lfs -text
-145
View File
@@ -1,145 +0,0 @@
name: Traceability Validation
# Mirrors JellyTau's .gitea/workflows/traceability-check.yml. The extractor is
# stdlib Python, so there is no toolchain install step and no jq.
#
# This workflow is component-agnostic: every repo-specific setting - which ID
# prefixes count, which file suffixes are source, which directories to scan,
# the threshold - lives in traceability.toml at the repo root, and the same
# extractor is shared by all three JRay components. Copying this file into
# another component needs no edits.
#
# NOTE: the runner here is an Intel N100 with no discrete GPU. This job is only
# ever static analysis of source comments plus markdown parsing, so it is cheap;
# the requirements it reports as "tagged but unexecuted" are the ones that need
# a GPU host, and they are deliberately never counted as covered.
on:
push:
branches:
- main
- master
- develop
pull_request:
branches:
- main
- master
- develop
jobs:
validate-traces:
runs-on: linux/amd64
name: Check requirement traces
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Check Python is available
run: |
set -e
command -v python3 >/dev/null 2>&1 || {
echo "python3 is missing from the runner image."
echo "The traceability tooling is stdlib-only Python;"
echo "3.9+ with CLI flags, 3.11+ to read traceability.toml."
exit 1
}
python3 --version
# The gate's own arithmetic is the thing being trusted, so its tests run
# before it does. JellyTau's gate was believed for months while it was
# dividing by frozen literals; untested gate logic is how that happens.
- name: Test the extractor
run: python3 scripts/vendor/jray-project/scripts/traceability/test_extract_traces.py
# Threshold policy and every other repo-specific setting live in
# traceability.toml, not here, so local runs and CI runs cannot disagree
# about what "passing" means. Denominators come from docs/requirements.md
# at run time and are never hardcoded -- in this file or anywhere else.
#
# A misconfigured run (zero requirements parsed, zero files scanned) is a
# hard failure rather than a plausible-looking 0%.
- name: Traceability gate
run: sh scripts/vendor/jray-project/scripts/traceability/traceability-gate.sh
- name: Check modified files for traces
if: github.event_name == 'pull_request'
run: |
set -e
echo "Checking modified sources for TRACES tags..."
# The extensions come from the report the gate just wrote, which got
# them from traceability.toml. Restating them here would be a second
# place for the source-file definition to live, and the two would
# drift the first time a language is added.
PATTERN=$(python3 -c "
import json, re, sys
suffixes = json.load(open('traces-report.json'))['config']['sourceSuffixes']
print('(' + '|'.join(re.escape(s) + '\$' for s in suffixes) + ')')
")
echo "Source suffixes from traceability.toml: $PATTERN"
CHANGED=$(git diff --name-only "origin/${{ github.base_ref }}...HEAD" \
| grep -E "$PATTERN" || true)
if [ -z "$CHANGED" ]; then
echo "No source files changed."
exit 0
fi
echo "Changed files:"
echo "$CHANGED" | sed 's/^/ /'
echo ""
# Advisory by design: not every file implements a requirement, and a
# tag on every function is noise that rots faster than it helps
# (CLAUDE.md: tag the unit that decides). This step exists to prompt,
# not to block. The blocking checks are in the gate step above.
#
# Piped into the loop rather than a here-string, and `case` rather
# than `[[ == ]]`, so this works under dash as well as bash. The loop
# body runs in a subshell, so misses are recorded in a file.
MISSING=$(mktemp)
echo "$CHANGED" | while IFS= read -r file; do
case "$file" in
*/test_*.py|*_test.py|*Tests.cs|tests/*|*/tests/*) continue ;;
esac
[ -f "$file" ] || continue
if ! grep -q 'TRACES:' "$file"; then
echo " no TRACES tag: $file"
echo "$file" >> "$MISSING"
fi
done
COUNT=$(wc -l < "$MISSING" | tr -d ' ')
rm -f "$MISSING"
if [ "$COUNT" -gt 0 ]; then
echo ""
echo "$COUNT changed file(s) carry no requirement tag."
echo "Format: // TRACES: AR-012, AR-013 | SR-002"
echo " (pipe separates requirement types, comma separates IDs)"
echo "A deliberate invariant exception is tagged separately:"
echo " // EXCEPTION: AR-024 <reason>"
echo "See CLAUDE.md and SPEC.md section 6."
fi
- name: Report summary
if: always()
run: |
echo "Traceability matrix: docs/traceability.md"
echo ""
head -40 docs/traceability.md || true
- name: Save reports
if: always()
uses: actions/upload-artifact@v3
with:
name: traceability-reports
path: |
traces-report.json
docs/traceability.md
retention-days: 30
-119
View File
@@ -1,119 +0,0 @@
# Build
build/
cmake-build-*/
CMakeCache.txt
CMakeFiles/
*.cmake
Makefile
install_manifest.txt
compile_commands.json
# Compiled objects
*.o
*.a
*.so
*.dylib
*.json
# Exception: small, curated result summaries backing specific numbers quoted
# in docs/ (cross-model held-out scores, per-film training breakdown, gallery
# coverage). Regenerate with scripts/docs/run_holdout_all_models.py and
# scripts/docs/gallery_coverage_per_film.py.
!docs_data/*.json
# Exception: test fixtures are inputs, not build output. The audio golden
# vector (IR-005) is shared verbatim with the jRay plugin repo, so it has to be
# tracked. Regenerate the media with tests/fixtures/audio/make_fixture.py.
!tests/fixtures/**
# Video files
*.mp4
*.mkv
*.avi
*.mov
# ONNX models in models/ are tracked via Git LFS (see .gitattributes).
# Any stray ONNX elsewhere is generated/downloaded and not tracked.
external/*.onnx
# Generated TensorRT engines (rebuilt by ORT / scripts/build_trt_engines.sh)
trt_cache/
# ORT pre-optimized model cache (generated on first run, provider-specific)
ort_cache/
# Gallery files (generated — HDF5 only, see src/gallery/gallery_store.cpp).
# Legacy JSON galleries from before that switch are also excluded.
gallery.json
gallery_*.json
gallery.h5
gallery_*.h5
# Calibration cache (generated alongside a gallery, per-embedder)
*.calib_cache.csv
*.calib_cache.png
# Cameo detection run outputs (generated by scripts/cameo_*.py)
cameo_progress.txt
cameo_report.txt
# Analysis plot outputs (scene-gap histograms / KDEs, etc.)
scene_gap_*.png
# Per-frame debug images
images/
# Annotations output
annotations.json
*_annotations.json
# MovieNet evaluation data
movienet-ps/
# Eval probe images (data, regenerable)
eval/probe/
# Local reference repos kept for inspiration (each has its own .git)
inspiration/
# experiments/ has its own nested .gitignore for HDF5 galleries/dumps/X-Ray
# corpus (all pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to
# the Gitea generic package registry instead of committed).
# Exception to the blanket *.json rule above: the committed placeholder for
# experiments/file-lut.json (see experiments/.gitignore).
!experiments/file-lut.template.json
# Site build output (mkdocs build). Rendered site is deployed to a
# gitea-pages branch, never committed to a working branch.
site/
docs_site/
# Images staged into docs/ from the artifact registry at build time
# (scripts/docs/build_site.sh) — not committed, pulled fresh on each build.
# Exception: pipeline_topology.svg is small and hand-authored (not pulled from
# anywhere) and the README references it directly, so it needs to render on a
# plain Gitea repo view too, not just the built Pages site. (A directory-level
# ignore can't be un-ignored file-by-file below it, so this must NOT blanket-
# ignore docs/assets/ itself — only its contents, minus the one exception.)
docs/assets/images/*
!docs/assets/images/pipeline_topology.svg
# These frames are referenced directly by README.md, which renders on the
# plain Gitea repo view — committed for the same reason as the SVG above.
!docs/assets/images/lovelace_perfect_second.jpg
!docs/assets/images/valerian_screen_call.jpg
# Python
__pycache__/
*.pyc
*.pyo
.venv/
venv/
# Local secrets (API keys — never commit)
.env
# Editor / OS
.vscode/
.idea/
.claude/settings.local.json
*.swp
*.swo
.DS_Store
Thumbs.db
-7
View File
@@ -1,7 +0,0 @@
[submodule "external/KPN"]
path = external/KPN
url = https://gitea.tourolle.paris/dtourolle/KPN.git
branch = master
[submodule "jray-project"]
path = scripts/vendor/jray-project
url = git@gitea.tourolle.paris:dtourolle/jray-project.git
View File
+713
View File
@@ -0,0 +1,713 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="Face-recognition pipeline for finding on-screen actor presence in film/TV, built on KPN++">
<link rel="icon" href="/dtourolle/scene-actor-extraction/assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.7">
<title>scene-actor-extraction</title>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/main.ec1eaa64.min.css">
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/stylesheets/extra.css">
<script>__md_scope=new URL("/dtourolle/scene-actor-extraction/",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-header__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
scene-actor-extraction
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_0">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 7a5 5 0 0 1 5 5 5 5 0 0 1-5 5 5 5 0 0 1-5-5 5 5 0 0 1 5-5m0 2a3 3 0 0 0-3 3 3 3 0 0 0 3 3 3 3 0 0 0 3-3 3 3 0 0 0-3-3m0-7 2.39 3.42C13.65 5.15 12.84 5 12 5s-1.65.15-2.39.42zM3.34 7l4.16-.35A7.2 7.2 0 0 0 5.94 8.5c-.44.74-.69 1.5-.83 2.29zm.02 10 1.76-3.77a7.131 7.131 0 0 0 2.38 4.14zM20.65 7l-1.77 3.79a7.02 7.02 0 0 0-2.38-4.15zm-.01 10-4.14.36c.59-.51 1.12-1.14 1.54-1.86.42-.73.69-1.5.83-2.29zM12 22l-2.41-3.44c.74.27 1.55.44 2.41.44.82 0 1.63-.17 2.37-.44z"/></svg>
</label>
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="indigo" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_0" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m17.75 4.09-2.53 1.94.91 3.06-2.63-1.81-2.63 1.81.91-3.06-2.53-1.94L12.44 4l1.06-3 1.06 3zm3.5 6.91-1.64 1.25.59 1.98-1.7-1.17-1.7 1.17.59-1.98L15.75 11l2.06-.05L18.5 9l.69 1.95zm-2.28 4.95c.83-.08 1.72 1.1 1.19 1.85-.32.45-.66.87-1.08 1.27C15.17 23 8.84 23 4.94 19.07c-3.91-3.9-3.91-10.24 0-14.14.4-.4.82-.76 1.27-1.08.75-.53 1.93.36 1.85 1.19-.27 2.86.69 5.83 2.89 8.02a9.96 9.96 0 0 0 8.02 2.89m-1.64 2.02a12.08 12.08 0 0 1-7.8-3.47c-2.17-2.19-3.33-5-3.49-7.82-2.81 3.14-2.7 7.96.31 10.98 3.02 3.01 7.84 3.12 10.98.31"/></svg>
</label>
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-tabs__link">
How We Score Against X-Ray
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-tabs__link">
Findings
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-tabs__link">
Full Experiment Log
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-tabs__link">
Service Conversion (proposal)
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-nav__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
scene-actor-extraction
</label>
<div class="md-nav__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-nav__link">
<span class="md-ellipsis">
How We Score Against X-Ray
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Findings
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Findings
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-nav__link">
<span class="md-ellipsis">
Best Model
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/gallery-scope/" class="md-nav__link">
<span class="md-ellipsis">
Gallery Scope (Full vs. Limited)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/pose-expansion/" class="md-nav__link">
<span class="md-ellipsis">
Pose Expansion
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/lvface-deep-dive/" class="md-nav__link">
<span class="md-ellipsis">
LVFace Deep Dive
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-nav__link">
<span class="md-ellipsis">
Full Experiment Log
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-nav__link">
<span class="md-ellipsis">
Service Conversion (proposal)
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "/dtourolle/scene-actor-extraction/", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "content.code.copy", "content.code.annotate"], "search": "/dtourolle/scene-actor-extraction/assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="/dtourolle/scene-actor-extraction/assets/javascripts/bundle.d7400e89.min.js"></script>
</body>
</html>
-365
View File
@@ -1,365 +0,0 @@
cmake_minimum_required(VERSION 3.21)
project(scene_actor_extraction VERSION 0.1.0 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
# ── Dependencies ──────────────────────────────────────────────────────────────
# KPN++ (pipeline backbone)
set(KPN_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_PYTHON OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
option(SAE_WEB_DEBUG "Enable KPN web debug UI (localhost:9090)" OFF)
if(SAE_WEB_DEBUG)
set(KPN_WEB_DEBUG ON CACHE BOOL "" FORCE)
endif()
add_subdirectory(external/KPN)
# OpenCV (video decode, image ops, DNN inference, face detection)
# Accept 4 or 5: the APIs used here are stable across both, and distros have
# begun shipping 5.x as the default (Arch/CachyOS). find_package's version
# argument is a minimum, but OpenCV's config rejects a 5.x install when 4 is
# requested, so probe for 5 first and fall back to 4.
find_package(OpenCV 5 QUIET COMPONENTS
core imgproc imgcodecs videoio dnn objdetect highgui)
if(NOT OpenCV_FOUND)
find_package(OpenCV 4 REQUIRED COMPONENTS
core imgproc imgcodecs videoio dnn objdetect highgui)
endif()
message(STATUS "OpenCV: ${OpenCV_VERSION}")
# ── Model paths ───────────────────────────────────────────────────────────────
# Defined early so the backend object libraries below can embed it.
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Backend selection ─────────────────────────────────────────────────────────
# Two independent compile-time axes. The core application is agnostic to both:
# only the matching backend .cpp (in src/backends/) is compiled, and the backend
# headers (onnxruntime / NvInfer.h / cublas / rocblas) never reach core TUs.
#
# SAE_INFERENCE_BACKEND ORT → SCRFD + ArcFace via ONNX Runtime (.onnx models)
# TRT → SCRFD + ArcFace via raw TensorRT (.engine files)
# SAE_GEMM_BACKEND ROCM → gallery similarity GEMM via rocBLAS / HIP
# CUDA → gallery similarity GEMM via cuBLAS / CUDA
# CPU → portable reference GEMM (no GPU; CI / testing)
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "Inference backend: ORT | TRT")
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "Gallery GEMM backend: ROCM | CUDA | CPU")
set_property(CACHE SAE_INFERENCE_BACKEND PROPERTY STRINGS ORT TRT)
set_property(CACHE SAE_GEMM_BACKEND PROPERTY STRINGS ROCM CUDA CPU)
# Enable the ORT TensorRT/CUDA execution providers inside the ORT inference
# backend (only meaningful when ORT was built with the TensorRT EP). Off by
# default so ROCm/CPU builds don't reference unavailable EPs.
option(SAE_ORT_TRT_EP "ORT backend: enable TensorRT/CUDA execution providers" OFF)
# Back-compat: a legacy -DSAE_WITH_TRT=ON/OFF seeds the new vars (ON⇒TRT+CUDA,
# OFF⇒ORT+ROCM) unless the user set them explicitly.
if(DEFINED SAE_WITH_TRT)
if(SAE_WITH_TRT)
set(SAE_INFERENCE_BACKEND "TRT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "CUDA" CACHE STRING "" FORCE)
else()
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "" FORCE)
endif()
message(STATUS "SAE_WITH_TRT=${SAE_WITH_TRT} (legacy) → "
"SAE_INFERENCE_BACKEND=${SAE_INFERENCE_BACKEND} "
"SAE_GEMM_BACKEND=${SAE_GEMM_BACKEND}")
endif()
if(NOT SAE_INFERENCE_BACKEND MATCHES "^(ORT|TRT)$")
message(FATAL_ERROR "SAE_INFERENCE_BACKEND must be ORT or TRT (got '${SAE_INFERENCE_BACKEND}')")
endif()
if(NOT SAE_GEMM_BACKEND MATCHES "^(ROCM|CUDA|CPU)$")
message(FATAL_ERROR "SAE_GEMM_BACKEND must be ROCM, CUDA or CPU (got '${SAE_GEMM_BACKEND}')")
endif()
# CUDA runtime is needed by both TRT inference and CUDA GEMM — find it once.
function(sae_find_cudart)
if(TARGET cudart_dep)
return()
endif()
find_library(CUDART_LIB cudart
HINTS /opt/cuda/lib64 /usr/local/cuda/lib64 /usr/lib)
find_path(CUDART_INCLUDE cuda_runtime_api.h
HINTS /opt/cuda/targets/x86_64-linux/include /opt/cuda/include
/usr/local/cuda/include /usr/include)
if(NOT (CUDART_LIB AND CUDART_INCLUDE))
message(FATAL_ERROR "CUDA runtime not found (cudart=${CUDART_LIB} headers=${CUDART_INCLUDE}).")
endif()
add_library(cudart_dep INTERFACE)
target_include_directories(cudart_dep INTERFACE "${CUDART_INCLUDE}")
target_link_libraries(cudart_dep INTERFACE "${CUDART_LIB}")
set_property(GLOBAL PROPERTY sae_cudart_found TRUE)
endfunction()
# ── Inference backend dependency: builds the `inference_backend` object lib ────
if(SAE_INFERENCE_BACKEND STREQUAL "ORT")
find_library(ORT_LIB onnxruntime REQUIRED
HINTS /usr/lib64/rocm/lib /usr/lib /usr/local/lib)
find_path(ORT_INCLUDE onnxruntime_cxx_api.h
PATH_SUFFIXES onnxruntime
HINTS /usr/lib64/rocm/include/onnxruntime /usr/include/onnxruntime /usr/local/include/onnxruntime
/usr/lib64/rocm/include /usr/include /usr/local/include
REQUIRED)
# The include directive is <onnxruntime/onnxruntime_cxx_api.h>, so we need the
# parent of the onnxruntime/ subdirectory on the include path.
get_filename_component(ORT_INCLUDE_PARENT "${ORT_INCLUDE}" DIRECTORY)
if(NOT EXISTS "${ORT_INCLUDE_PARENT}/onnxruntime")
set(ORT_INCLUDE_PARENT "${ORT_INCLUDE}")
endif()
message(STATUS "Inference backend: ORT (${ORT_LIB} headers: ${ORT_INCLUDE_PARENT})")
add_library(inference_backend OBJECT src/backends/ort_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${ORT_INCLUDE_PARENT}")
target_link_libraries(inference_backend PRIVATE ${OpenCV_LIBS} "${ORT_LIB}")
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
$<$<BOOL:${SAE_ORT_TRT_EP}>:SAE_ORT_WITH_TRT_EP>)
else() # TRT
find_library(NVINFER_LIB nvinfer
HINTS /usr/lib /usr/local/lib /opt/tensorrt/lib)
find_path(NVINFER_INCLUDE NvInfer.h
HINTS /usr/include /usr/local/include /opt/tensorrt/include)
if(NOT (NVINFER_LIB AND NVINFER_INCLUDE))
message(FATAL_ERROR
"TensorRT not found (nvinfer=${NVINFER_LIB} headers=${NVINFER_INCLUDE}). "
"Pass -DSAE_INFERENCE_BACKEND=ORT to load .onnx models without TensorRT.")
endif()
sae_find_cudart()
message(STATUS "Inference backend: TRT (${NVINFER_LIB})")
add_library(inference_backend OBJECT src/backends/trt_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${NVINFER_INCLUDE}")
target_link_libraries(inference_backend PRIVATE
${OpenCV_LIBS} "${NVINFER_LIB}" cudart_dep)
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}")
endif()
# ── GEMM backend dependency: builds the `gemm_backend` object lib ──────────────
if(SAE_GEMM_BACKEND STREQUAL "CPU")
# Portable reference GEMM: no GPU libraries, no headers. Used for CI and as
# the correctness oracle for the CUDA/ROCm backends.
message(STATUS "GEMM backend: CPU (portable reference, no GPU)")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CPU)
# AR-026/AR-027: back the CPU path with OpenBLAS when present. Optional, so
# the build gains no hard dependency — but without it the fallback is a
# scalar loop, which does not hold up against a library-scale gallery, and
# the CPU path is exactly what CI (no GPU) and the cpu builder image use.
find_package(PkgConfig QUIET)
if(PkgConfig_FOUND)
pkg_check_modules(OPENBLAS QUIET openblas)
endif()
if(OPENBLAS_FOUND)
message(STATUS "GEMM backend: CPU + OpenBLAS ${OPENBLAS_VERSION}")
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CBLAS)
target_include_directories(gemm_backend PRIVATE ${OPENBLAS_INCLUDE_DIRS})
target_link_libraries(gemm_backend PRIVATE ${OPENBLAS_LINK_LIBRARIES})
else()
message(WARNING "GEMM backend: CPU scalar fallback — OpenBLAS not found. "
"Correct, but slow on a large gallery (AR-027).")
endif()
elseif(SAE_GEMM_BACKEND STREQUAL "CUDA")
find_library(CUBLAS_LIB cublas
HINTS /opt/cuda/targets/x86_64-linux/lib /opt/cuda/lib64
/usr/local/cuda/lib64 /usr/lib)
if(NOT CUBLAS_LIB)
message(FATAL_ERROR "cuBLAS not found (cublas=${CUBLAS_LIB}).")
endif()
sae_find_cudart()
message(STATUS "GEMM backend: CUDA (${CUBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_link_libraries(gemm_backend PRIVATE "${CUBLAS_LIB}" cudart_dep)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CUDA)
else() # ROCM
find_library(ROCBLAS_LIB rocblas
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(ROCBLAS_INCLUDE rocblas/rocblas.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
find_library(HIP_LIB amdhip64
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(HIP_INCLUDE hip/hip_runtime_api.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
if(NOT (ROCBLAS_LIB AND ROCBLAS_INCLUDE AND HIP_LIB AND HIP_INCLUDE))
message(FATAL_ERROR
"rocBLAS or HIP runtime not found "
"(rocblas=${ROCBLAS_LIB} headers=${ROCBLAS_INCLUDE} "
"hip=${HIP_LIB} headers=${HIP_INCLUDE}). "
"Install rocblas-devel and hip-devel (or pass -DSAE_GEMM_BACKEND=CUDA).")
endif()
message(STATUS "GEMM backend: ROCM (${ROCBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src "${ROCBLAS_INCLUDE}" "${HIP_INCLUDE}")
target_link_libraries(gemm_backend PRIVATE "${ROCBLAS_LIB}" "${HIP_LIB}")
# HIP headers require the platform to be declared explicitly when compiled with g++.
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_ROCM __HIP_PLATFORM_AMD__)
endif()
# FFmpeg (hwaccel video decode: CUDA/VAAPI, runtime-detected + swscale colour
# conversion). Hwaccel support is built into libavcodec/libavutil; no extra
# libraries are needed here.
# libswresample is the audio side of the same dependency — downmix + resample
# for the audio signature (IR-004, src/audio_signature.cpp). Not a new project
# dependency: it ships with the libav* set already required above.
find_package(PkgConfig REQUIRED)
pkg_check_modules(AVFORMAT REQUIRED libavformat)
pkg_check_modules(AVCODEC REQUIRED libavcodec)
pkg_check_modules(AVUTIL REQUIRED libavutil)
pkg_check_modules(SWSCALE REQUIRED libswscale)
pkg_check_modules(SWRESAMPLE REQUIRED libswresample)
add_library(ffmpeg_libs INTERFACE)
target_compile_options(ffmpeg_libs INTERFACE
${AVFORMAT_CFLAGS_OTHER} ${AVCODEC_CFLAGS_OTHER}
${AVUTIL_CFLAGS_OTHER} ${SWSCALE_CFLAGS_OTHER}
${SWRESAMPLE_CFLAGS_OTHER})
target_include_directories(ffmpeg_libs INTERFACE
${AVFORMAT_INCLUDE_DIRS} ${AVCODEC_INCLUDE_DIRS}
${AVUTIL_INCLUDE_DIRS} ${SWSCALE_INCLUDE_DIRS}
${SWRESAMPLE_INCLUDE_DIRS})
target_link_libraries(ffmpeg_libs INTERFACE
${AVFORMAT_LIBRARIES} ${AVCODEC_LIBRARIES}
${AVUTIL_LIBRARIES} ${SWSCALE_LIBRARIES}
${SWRESAMPLE_LIBRARIES})
message(STATUS "FFmpeg: avformat=${AVFORMAT_VERSION} avcodec=${AVCODEC_VERSION} "
"swresample=${SWRESAMPLE_VERSION}")
# nlohmann/json (gallery + output serialisation)
include(FetchContent)
FetchContent_Declare(
nlohmann_json
GIT_REPOSITORY https://github.com/nlohmann/json.git
GIT_TAG v3.11.3
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nlohmann_json)
# nanobind (Python bindings for the sae_embed module)
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
FetchContent_Declare(
nanobind
GIT_REPOSITORY https://github.com/wjakob/nanobind.git
GIT_TAG v2.4.0
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nanobind)
# ── Model paths ───────────────────────────────────────────────────────────────
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Shared library: gallery store + compiled-in backends ──────────────────────
# The backend object libraries carry their own ORT/TRT/CUDA/ROCm linkage and
# headers; sae_gallery re-exports those object files so every binary that links
# sae_gallery gets the chosen backend without ever seeing its headers.
# HDF5 (C++) — gallery fast-load path + embedding dump. Found here so sae_gallery
# (gallery_store.cpp) can link it; scene_analyze/dump_embeddings reuse the same vars.
find_package(HDF5 REQUIRED COMPONENTS CXX)
add_library(sae_gallery STATIC
src/gallery/gallery_store.cpp
src/gallery/gallery_builder.cpp
src/audio_signature.cpp # IR-004 — content-derived audio signature
src/gallery/embedder_stamp.cpp # GR-004 — gallery/embedder binding
)
set_target_properties(sae_gallery PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(sae_gallery PUBLIC src ${HDF5_INCLUDE_DIRS})
target_link_libraries(sae_gallery PUBLIC
kpn
${OpenCV_LIBS}
nlohmann_json::nlohmann_json
inference_backend
gemm_backend
ffmpeg_libs
${HDF5_CXX_LIBRARIES}
)
target_compile_definitions(sae_gallery PUBLIC
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
)
# ── embed_faces — image → embedding JSON (used by gallery builder scripts) ────
add_executable(embed_faces src/embed_faces.cpp)
target_link_libraries(embed_faces PRIVATE sae_gallery)
# ── sae_embed — Python module: load SCRFD+ArcFace once, embed many images ───
nanobind_add_module(sae_embed src/python_bindings.cpp)
target_link_libraries(sae_embed PRIVATE sae_gallery)
# ── sae_kpn — Python module: run the real downstream nodes over dumped embeddings ─
# Assembles face_tracker/identity_matcher/scene_tracker in a Python-driven KPN
# network (KPN_BUILD_PYTHON is enabled per-TU inside the .cpp). Powers the
# threshold-sweep optimizer in scripts/optimizer/.
nanobind_add_module(sae_kpn src/kpn_bindings.cpp)
target_link_libraries(sae_kpn PRIVATE sae_gallery)
# ── sae_audio — Python module: the v1 audio signature (IR-004) ────────────────
# Compiles audio_signature.cpp directly and links only FFmpeg, rather than
# linking sae_gallery: the signature needs no model, no OpenCV and no HDF5, and
# a module that dragged all three in would make `import sae_audio` depend on a
# GPU-capable build of a repo whose audio path is pure CPU DSP. tests/ compiles
# the same source the same way, for the same reason.
nanobind_add_module(sae_audio src/audio_bindings.cpp src/audio_signature.cpp)
target_include_directories(sae_audio PRIVATE src)
target_link_libraries(sae_audio PRIVATE ffmpeg_libs)
# HDF5 already found above (before sae_gallery); vars HDF5_CXX_LIBRARIES / _INCLUDE_DIRS
# are reused by scene_analyze / dump_embeddings below.
# ── analyze — main analysis binary ───────────────────────────────────────────
add_executable(scene_analyze src/main.cpp)
target_link_libraries(scene_analyze PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze PRIVATE ${HDF5_INCLUDE_DIRS})
# ── analyze_debug — same binary with debug frame/crop output ─────────────────
add_executable(scene_analyze_debug src/main.cpp)
target_link_libraries(scene_analyze_debug PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze_debug PRIVATE ${HDF5_INCLUDE_DIRS})
target_compile_definitions(scene_analyze_debug PRIVATE SAE_DEBUG=1)
# ── dump_embeddings — standalone embedding dumper, NO gallery/matcher ─────────
# Front-half only (decode→detect→align→embed→HDF5) for the optimizer replay corpus
# and model bake-off. Skips gallery load + calibration (~24s/run faster).
add_executable(dump_embeddings src/dump_embeddings.cpp)
target_link_libraries(dump_embeddings PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(dump_embeddings PRIVATE ${HDF5_INCLUDE_DIRS})
# ── scene_preview — live annotated display while analysing ───────────────────
add_executable(scene_preview src/scene_preview.cpp)
target_link_libraries(scene_preview PRIVATE sae_gallery)
# ── build_gallery — offline gallery construction tool ────────────────────────
add_executable(build_gallery src/build_gallery.cpp)
target_link_libraries(build_gallery PRIVATE sae_gallery)
# ── Optional: web debug UI for pipeline introspection ────────────────────────
if(SAE_WEB_DEBUG)
kpn_target_enable_web_debug(scene_analyze)
kpn_target_enable_web_debug(scene_analyze_debug)
kpn_target_enable_web_debug(scene_preview)
endif()
# ── Tests ─────────────────────────────────────────────────────────────────────
option(SAE_BUILD_TESTS "Build unit tests (GPU-free)" OFF)
if(SAE_BUILD_TESTS)
enable_testing()
add_subdirectory(tests)
endif()
message(STATUS "OpenCV ${OpenCV_VERSION} found")
message(STATUS "Models dir: ${SAE_MODELS_DIR}")
-41
View File
@@ -1,41 +0,0 @@
MIT License
Copyright (c) 2026 Duncan Tourolle
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
---
NOTE ON MODELS AND THIRD-PARTY COMPONENTS
The MIT license above applies only to the source code in this repository. It
does NOT cover:
* Machine-learning model weights (the ONNX files in models/). These weights
are the property of their respective authors and are governed by their own
licenses, not by the MIT license above. The models — including the
InsightFace "buffalo" packs (ArcFace / SCRFD), YuNet, and LVFace — are
redistributed here for convenience under those upstream licenses. Several,
notably the InsightFace models, are licensed for NON-COMMERCIAL RESEARCH
USE ONLY. You are responsible for reviewing and complying with each model's
license before use.
* Third-party libraries this software links against (OpenCV, ONNX Runtime,
FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind, and others), each of which
carries its own license.
-259
View File
@@ -1,259 +0,0 @@
# Scene Actor Extraction
Identifies actors in movie files and produces X-ray-style scene annotations compatible with [Jellyfin](https://jellyfin.org/). Built on a KPN++ pipeline with ArcFace/LVFace embeddings and a tracked-identity matcher.
**67.4% macro-F1 against Amazon X-Ray ground truth**, on 5 films never seen by
the optimizer (89.7% P / 65.4% R training-set; see the generalization-gap
discussion in the [deep dive](https://pages.tourolle.paris/dtourolle/scene-actor-extraction/lvface-deep-dive/)).
Full benchmark write-up, model comparison, and failure-mode analysis:
**https://pages.tourolle.paris/dtourolle/scene-actor-extraction/**
![A perfect X-Ray second on a held-out film](docs/assets/images/lovelace_perfect_second.jpg)
*A perfect X-Ray second on a held-out film (never used for threshold tuning):
every visible face named at 100%, the background extra honestly left unnamed,
and the two credited cast without a visible face correctly carried as present
off-screen. Bottom panels show the per-second verdict against Amazon X-Ray
(green = correct, orange = wrong, blue = missed).*
It also doesn't care whether the face is in the room:
![Herbie Hancock identified on an in-fiction video-call screen](docs/assets/images/valerian_screen_call.jpg)
*Herbie Hancock at 98% — as a face on a screen inside the movie, under a
sci-fi HUD overlay.*
## How it works
1. **Build a gallery** — download actor headshots from TMDB/IMDB, embed them with ArcFace or LVFace (`build_gallery` / `scripts/make_gallery.py`).
2. **Analyze a movie**`scene_analyze` decodes frames at configurable FPS, detects faces (SCRFD), tracks them across cuts, matches identities against the gallery using calibrated similarity, and writes time-window JSON.
3. **Output** — minimal mode produces Jellyfin-ready actor name + time-window JSON; standard mode adds per-frame bbox, similarity, and track data.
![Pipeline topology](docs/assets/images/pipeline_topology.svg)
## Dependencies
| Dependency | Role |
|---|---|
| KPN++ | Pipeline backbone (nodes, networks) |
| OpenCV 4 | Video decode, image ops, DNN inference |
| ONNX Runtime | SCRFD face detector (dynamic shape nodes unsupported by cv::dnn) |
| TensorRT + CUDA runtime + cuBLAS | Optional TRT engines for SCRFD/ArcFace (`--detector-engine`/`--arcface-engine`); identity_matcher's GPU gallery scan |
| FFmpeg (libav*) | NVDEC hardware video decode + colour conversion |
| nlohmann/json | JSON I/O |
| nanobind | Python bindings for `sae_embed` |
## Build
```bash
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
```
This also builds `sae_embed`, a Python module (via nanobind) that loads the
SCRFD detector and ArcFace embedder once and exposes a reusable `embed()`
method. The gallery-builder scripts (`make_gallery.py`,
`make_jellyfin_gallery.py`, `movienet_eval.py`) import it directly — there is
no subprocess fallback, so if it's missing they exit with a build instruction:
```bash
cmake --build build --target sae_embed
```
Optional flags:
| Flag | Default | Effect |
|---|---|---|
| `-DSAE_WEB_DEBUG=ON` | OFF | Enables KPN web debug UI at `localhost:9090` |
## Models
The ONNX model weights live in `models/` (tracked via Git LFS):
- `LVFace-B_Glint360K.onnx` — LVFace embedder (ViT backbone, ICCV 2025), the default
(best F1 in the rep4 model bake-off, see `docs/rep4-optimizer-results.md`)
- `arcface_w600k_r50.onnx` — ArcFace embedder, previous default
- `arcface_w600k_mbf.onnx`, `arcface_r18.onnx` — lighter ArcFace alternatives
- `face_detection_yunet_2023mar.onnx` — YuNet face detector
- `scrfd_500m_bnkps.onnx` — SCRFD face detector
### LVFace
[LVFace](https://github.com/bytedance/LVFace) is a Vision-Transformer face
recognition model. The `LVFace-B_Glint360K.onnx` export shares ArcFace's I/O
contract (112×112 aligned BGR crop → L2-normalised 512-d embedding) and its
`(x 127.5)/128` input scaling, so it slots straight into the existing embedder
— just point `--arcface-model` at it:
```bash
./build/scene_analyze --arcface-model models/LVFace-B_Glint360K.onnx \
--gallery gallery.h5 --movie movie.mp4
```
> **Important:** embeddings from different recognition models are not
> interchangeable. A gallery (and its calibration cache) must be built with the
> **same** embedder used for analysis — rebuild the gallery with
> `--arcface models/LVFace-B_Glint360K.onnx` before analysing with LVFace.
If they are missing (e.g. LFS not fetched), re-download them with:
```bash
bash scripts/download_models.sh
```
> **Model licensing:** the model weights carry their own licenses, separate
> from this project's MIT license, and are redistributed here under those
> upstream terms. Several — notably the InsightFace "buffalo" models (ArcFace /
> SCRFD) — are licensed for **non-commercial research use only**. Review and
> comply with each model's license before use.
## Binaries
| Binary | Description |
|---|---|
| `scene_analyze` | Main analysis pipeline, writes JSON output |
| `scene_analyze_debug` | Same as above + per-frame annotated JPEGs (`SAE_DEBUG=1`) |
| `scene_preview` | Live OpenCV display window while analysing |
| `build_gallery` | Offline gallery builder from a directory of images |
| `embed_faces` | CLI: image(s) → embedding JSON, used by gallery-builder scripts |
| `sae_embed` | Python module (nanobind) used by gallery-builder scripts — loads SCRFD+ArcFace once |
### `scene_analyze`
```bash
./build/scene_analyze --gallery gallery.h5 --movie movie.mp4 [options]
```
Key options:
| Flag | Default | Description |
|---|---|---|
| `--fps` | 1 | Frames per second to sample (510 recommended for tracking) |
| `--prob-threshold` | 0.5 | Minimum calibrated match probability |
| `--match-threshold` | — | Raw cosine similarity threshold (fallback) |
| `--extinction` | 5s | How long a track persists after last detection |
| `--track-alpha` | — | IoU vs. embedding weight in Hungarian assignment |
| `--track-min-iou` | — | Minimum IoU gate for spatial assignment |
| `--track-max-embed` | — | Maximum embedding distance gate |
| `--track-max-missing` | — | Frames a track survives without a detection |
### Gallery builders
**Per-movie (TMDB):**
```bash
python3 scripts/make_gallery.py --tmdb-key <TMDB_KEY> --movie-id <TMDB_ID> --output gallery.h5
```
Fetches cast images from TMDB and embeds them via `sae_embed`.
**Whole-library (Jellyfin):**
```bash
python3 scripts/make_jellyfin_gallery.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--output gallery.h5
```
Scans every Movie/Series in Jellyfin, collects the unique cast across the
whole library, downloads each actor's headshot directly from Jellyfin (no
TMDB key needed), and embeds them via `sae_embed` into one global
gallery.h5. Since `identity_matcher` scores faces against the entire
gallery, `scene_analyze` can then recognise any actor from your library in
any film — not just the cast listed for that one title. Pass `--merge` on
later runs to only embed actors newly added to the library. Pass
`--tmdb-key` to fall back to TMDB profile images for actors with no usable
image cached in Jellyfin.
Jellyfin/TMDB lookups and image downloads for different actors run
concurrently (`--workers`, default 8). Embedding is GPU-bound, so it's
gated separately via `--embed-concurrency` (default 1) — only that many
embed calls run at once while other actors' downloads continue in the
background.
To restrict a single-title run to that title's credited cast (faster, fewer
look-alike mismatches), filter the global gallery first:
```bash
python3 scripts/filter_gallery.py \
--gallery gallery.h5 \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--output gallery_matrix.h5
```
## Running directly from Jellyfin
`scripts/run_from_jellyfin.py` resolves a title to its media file via the
Jellyfin API, filters the gallery to that title's cast, and runs
`scene_analyze` in one step. Requires this tool to run on a host that shares
Jellyfin's media mount (it uses the item's on-disk `Path`, not a stream URL):
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--gallery gallery.h5 \
-- --fps 5 --verbosity 2
```
Anything after `--` is passed through to `scene_analyze` unchanged. Pass
`--no-filter` to use the gallery as-is (skip per-title cast filtering), or
`--item-id` instead of `--title` to skip the search.
After a successful run, the output JSON is pushed to the [JRay Jellyfin
plugin](https://gitea.tourolle.paris/dtourolle/jRay)'s Truth endpoint
(`PUT /Plugins/JRay/Items/{itemId}/Truth`) so
Jellyfin picks it up immediately, using `--api-key` (must be an
**Administrator** key for the push to succeed). Pass `--no-push` to skip
this and only write `--output` locally (e.g. for local debugging).
### Worker mode
Pass `--worker` instead of `--item-id`/`--title` to run this as an extraction
worker: it polls the JRay plugin's `GET /Plugins/JRay/Tasks/Pending` endpoint
for a random batch of items with no truth data yet, processes each one, and
pushes the result back. The endpoint's sampling spreads work across the
backlog without any server-side task tracking, so any number of workers can
poll the same library concurrently.
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <ADMIN_API_KEY> \
--gallery whole_gallery.h5 \
--worker \
-- --fps 5
```
- `--poll-limit` — batch size requested from `Tasks/Pending` (default 10, max 100)
- `--poll-interval` — seconds to sleep between polls when the backlog is empty (default 60)
- `--once` — process a single batch and exit instead of looping forever
A failure on one item (bad path, push rejected, etc.) is logged and the
worker moves on to the next item rather than exiting.
## Output format
**Minimal** (default) — Jellyfin-ready:
```json
[
{ "actor": "Name", "start": 12.0, "end": 45.5 }
]
```
**Standard** — per-frame detail with bounding boxes, similarity scores, and track IDs.
## Evaluation
Scripts in `eval/` and `scripts/movienet_*.py` support benchmarking against the MovieNet dataset.
## License
The source code in this repository is licensed under the [MIT License](LICENSE).
The MIT license covers **only the code**. The model weights in `models/` (see
[Models](#models)) are redistributed under their own licenses — several for
non-commercial research use only. Third-party libraries this software links
against (OpenCV, ONNX Runtime, FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind,
and others) likewise carry their own licenses.
Binary file not shown.

After

Width:  |  Height:  |  Size: 200 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 137 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 219 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 164 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 201 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 311 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 83 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 155 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 176 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 161 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 142 KiB

Before

Width:  |  Height:  |  Size: 239 KiB

After

Width:  |  Height:  |  Size: 239 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 163 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 174 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 111 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 134 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 141 KiB

Before

Width:  |  Height:  |  Size: 2.8 KiB

After

Width:  |  Height:  |  Size: 2.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 103 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 40 KiB

Before

Width:  |  Height:  |  Size: 165 KiB

After

Width:  |  Height:  |  Size: 165 KiB

File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+18
View File
@@ -0,0 +1,18 @@
/*!
* Lunr languages, `Danish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){var r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){function e(){var e,r=f.cursor+3;if(d=f.limit,0<=r&&r<=f.limit){for(a=r;;){if(e=f.cursor,f.in_grouping(w,97,248)){f.cursor=e;break}if(f.cursor=e,e>=f.limit)return;f.cursor++}for(;!f.out_grouping(w,97,248);){if(f.cursor>=f.limit)return;f.cursor++}d=f.cursor,d<a&&(d=a)}}function n(){var e,r;if(f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(c,32),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del();break;case 2:f.in_grouping_b(p,97,229)&&f.slice_del()}}function t(){var e,r=f.limit-f.cursor;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.find_among_b(l,4)?(f.bra=f.cursor,f.limit_backward=e,f.cursor=f.limit-r,f.cursor>f.limit_backward&&(f.cursor--,f.bra=f.cursor,f.slice_del())):f.limit_backward=e)}function s(){var e,r,i,n=f.limit-f.cursor;if(f.ket=f.cursor,f.eq_s_b(2,"st")&&(f.bra=f.cursor,f.eq_s_b(2,"ig")&&f.slice_del()),f.cursor=f.limit-n,f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(m,5),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del(),i=f.limit-f.cursor,t(),f.cursor=f.limit-i;break;case 2:f.slice_from("løs")}}function o(){var e;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.out_grouping_b(w,97,248)?(f.bra=f.cursor,u=f.slice_to(u),f.limit_backward=e,f.eq_v_b(u)&&f.slice_del()):f.limit_backward=e)}var a,d,u,c=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],l=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],w=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],p=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],f=new i;this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var r=f.cursor;return e(),f.limit_backward=r,f.cursor=f.limit,n(),f.cursor=f.limit,t(),f.cursor=f.limit,s(),f.cursor=f.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hi=function(){this.pipeline.reset(),this.pipeline.add(e.hi.trimmer,e.hi.stopWordFilter,e.hi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hi.stemmer))},e.hi.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿa-zA-Z-zA-0-9-",e.hi.trimmer=e.trimmerSupport.generateTrimmer(e.hi.wordCharacters),e.Pipeline.registerFunction(e.hi.trimmer,"trimmer-hi"),e.hi.stopWordFilter=e.generateStopWordFilter("अत अपना अपनी अपने अभी अंदर आदि आप इत्यादि इन इनका इन्हीं इन्हें इन्हों इस इसका इसकी इसके इसमें इसी इसे उन उनका उनकी उनके उनको उन्हीं उन्हें उन्हों उस उसके उसी उसे एक एवं एस ऐसे और कई कर करता करते करना करने करें कहते कहा का काफ़ी कि कितना किन्हें किन्हों किया किर किस किसी किसे की कुछ कुल के को कोई कौन कौनसा गया घर जब जहाँ जा जितना जिन जिन्हें जिन्हों जिस जिसे जीधर जैसा जैसे जो तक तब तरह तिन तिन्हें तिन्हों तिस तिसे तो था थी थे दबारा दिया दुसरा दूसरे दो द्वारा न नके नहीं ना निहायत नीचे ने पर पहले पूरा पे फिर बनी बही बहुत बाद बाला बिलकुल भी भीतर मगर मानो मे में यदि यह यहाँ यही या यिह ये रखें रहा रहे ऱ्वासा लिए लिये लेकिन व वग़ैरह वर्ग वह वहाँ वहीं वाले वुह वे वो सकता सकते सबसे सभी साथ साबुत साभ सारा से सो संग ही हुआ हुई हुए है हैं हो होता होती होते होना होने".split(" ")),e.hi.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.hi.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var t=i.toString().toLowerCase().replace(/^\s+/,"");return r.cut(t).split("|")},e.Pipeline.registerFunction(e.hi.stemmer,"stemmer-hi"),e.Pipeline.registerFunction(e.hi.stopWordFilter,"stopWordFilter-hi")}});
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hy=function(){this.pipeline.reset(),this.pipeline.add(e.hy.trimmer,e.hy.stopWordFilter)},e.hy.wordCharacters="[A-Za-z԰-֏ff-ﭏ]",e.hy.trimmer=e.trimmerSupport.generateTrimmer(e.hy.wordCharacters),e.Pipeline.registerFunction(e.hy.trimmer,"trimmer-hy"),e.hy.stopWordFilter=e.generateStopWordFilter("դու և եք էիր էիք հետո նաև նրանք որը վրա է որ պիտի են այս մեջ ն իր ու ի այդ որոնք այն կամ էր մի ես համար այլ իսկ էին ենք հետ ին թ էինք մենք նրա նա դուք եմ էի ըստ որպես ում".split(" ")),e.Pipeline.registerFunction(e.hy.stopWordFilter,"stopWordFilter-hy"),e.hy.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}(),e.Pipeline.registerFunction(e.hy.stemmer,"stemmer-hy")}});
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.ja=function(){this.pipeline.reset(),this.pipeline.add(e.ja.trimmer,e.ja.stopWordFilter,e.ja.stemmer),r?this.tokenizer=e.ja.tokenizer:(e.tokenizer&&(e.tokenizer=e.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.ja.tokenizer))};var t=new e.TinySegmenter;e.ja.tokenizer=function(i){var n,o,s,p,a,u,m,l,c,f;if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(o=i.toString().toLowerCase().replace(/^\s+/,""),n=o.length-1;n>=0;n--)if(/\S/.test(o.charAt(n))){o=o.substring(0,n+1);break}for(a=[],s=o.length,c=0,l=0;c<=s;c++)if(u=o.charAt(c),m=c-l,u.match(/\s/)||c==s){if(m>0)for(p=t.segment(o.slice(l,c)).filter(function(e){return!!e}),f=l,n=0;n<p.length;n++)r?a.push(new e.Token(p[n],{position:[f,p[n].length],index:a.length})):a.push(p[n]),f+=p[n].length;l=c+1}return a},e.ja.stemmer=function(){return function(e){return e}}(),e.Pipeline.registerFunction(e.ja.stemmer,"stemmer-ja"),e.ja.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーア-ン゙a-zA-Z-zA-0-9-",e.ja.trimmer=e.trimmerSupport.generateTrimmer(e.ja.wordCharacters),e.Pipeline.registerFunction(e.ja.trimmer,"trimmer-ja"),e.ja.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.ja.stopWordFilter,"stopWordFilter-ja"),e.jp=e.ja,e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.Pipeline.registerFunction(e.jp.trimmer,"trimmer-jp"),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});
+1
View File
@@ -0,0 +1 @@
module.exports=require("./lunr.ja");
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.kn=function(){this.pipeline.reset(),this.pipeline.add(e.kn.trimmer,e.kn.stopWordFilter,e.kn.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.kn.stemmer))},e.kn.wordCharacters="ಀ-಄ಅ-ಔಕ-ಹಾ-ೌ಼-ಽೕ-ೖೝ-ೞೠ-ೡೢ-ೣ೤೥೦-೯ೱ-ೳ",e.kn.trimmer=e.trimmerSupport.generateTrimmer(e.kn.wordCharacters),e.Pipeline.registerFunction(e.kn.trimmer,"trimmer-kn"),e.kn.stopWordFilter=e.generateStopWordFilter("ಮತ್ತು ಈ ಒಂದು ರಲ್ಲಿ ಹಾಗೂ ಎಂದು ಅಥವಾ ಇದು ರ ಅವರು ಎಂಬ ಮೇಲೆ ಅವರ ತನ್ನ ಆದರೆ ತಮ್ಮ ನಂತರ ಮೂಲಕ ಹೆಚ್ಚು ನ ಆ ಕೆಲವು ಅನೇಕ ಎರಡು ಹಾಗು ಪ್ರಮುಖ ಇದನ್ನು ಇದರ ಸುಮಾರು ಅದರ ಅದು ಮೊದಲ ಬಗ್ಗೆ ನಲ್ಲಿ ರಂದು ಇತರ ಅತ್ಯಂತ ಹೆಚ್ಚಿನ ಸಹ ಸಾಮಾನ್ಯವಾಗಿ ನೇ ಹಲವಾರು ಹೊಸ ದಿ ಕಡಿಮೆ ಯಾವುದೇ ಹೊಂದಿದೆ ದೊಡ್ಡ ಅನ್ನು ಇವರು ಪ್ರಕಾರ ಇದೆ ಮಾತ್ರ ಕೂಡ ಇಲ್ಲಿ ಎಲ್ಲಾ ವಿವಿಧ ಅದನ್ನು ಹಲವು ರಿಂದ ಕೇವಲ ದ ದಕ್ಷಿಣ ಗೆ ಅವನ ಅತಿ ನೆಯ ಬಹಳ ಕೆಲಸ ಎಲ್ಲ ಪ್ರತಿ ಇತ್ಯಾದಿ ಇವು ಬೇರೆ ಹೀಗೆ ನಡುವೆ ಇದಕ್ಕೆ ಎಸ್ ಇವರ ಮೊದಲು ಶ್ರೀ ಮಾಡುವ ಇದರಲ್ಲಿ ರೀತಿಯ ಮಾಡಿದ ಕಾಲ ಅಲ್ಲಿ ಮಾಡಲು ಅದೇ ಈಗ ಅವು ಗಳು ಎ ಎಂಬುದು ಅವನು ಅಂದರೆ ಅವರಿಗೆ ಇರುವ ವಿಶೇಷ ಮುಂದೆ ಅವುಗಳ ಮುಂತಾದ ಮೂಲ ಬಿ ಮೀ ಒಂದೇ ಇನ್ನೂ ಹೆಚ್ಚಾಗಿ ಮಾಡಿ ಅವರನ್ನು ಇದೇ ಯ ರೀತಿಯಲ್ಲಿ ಜೊತೆ ಅದರಲ್ಲಿ ಮಾಡಿದರು ನಡೆದ ಆಗ ಮತ್ತೆ ಪೂರ್ವ ಆತ ಬಂದ ಯಾವ ಒಟ್ಟು ಇತರೆ ಹಿಂದೆ ಪ್ರಮಾಣದ ಗಳನ್ನು ಕುರಿತು ಯು ಆದ್ದರಿಂದ ಅಲ್ಲದೆ ನಗರದ ಮೇಲಿನ ಏಕೆಂದರೆ ರಷ್ಟು ಎಂಬುದನ್ನು ಬಾರಿ ಎಂದರೆ ಹಿಂದಿನ ಆದರೂ ಆದ ಸಂಬಂಧಿಸಿದ ಮತ್ತೊಂದು ಸಿ ಆತನ ".split(" ")),e.kn.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.kn.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var n=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(n).split("|")},e.Pipeline.registerFunction(e.kn.stemmer,"stemmer-kn"),e.Pipeline.registerFunction(e.kn.stopWordFilter,"stopWordFilter-kn")}});
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var t=Array.prototype.slice.call(arguments),i=t.join("-"),r="",n=[],s=[],p=0;p<t.length;++p)"en"==t[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[t[p]].wordCharacters,e[t[p]].stopWordFilter&&n.unshift(e[t[p]].stopWordFilter),e[t[p]].stemmer&&(n.push(e[t[p]].stemmer),s.push(e[t[p]].stemmer)));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+i),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});
File diff suppressed because one or more lines are too long
+18
View File
@@ -0,0 +1,18 @@
/*!
* Lunr languages, `Norwegian` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){function e(){var e,r=w.cursor+3;if(a=w.limit,0<=r||r<=w.limit){for(s=r;;){if(e=w.cursor,w.in_grouping(d,97,248)){w.cursor=e;break}if(e>=w.limit)return;w.cursor=e+1}for(;!w.out_grouping(d,97,248);){if(w.cursor>=w.limit)return;w.cursor++}a=w.cursor,a<s&&(a=s)}}function i(){var e,r,n;if(w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(m,29),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:n=w.limit-w.cursor,w.in_grouping_b(c,98,122)?w.slice_del():(w.cursor=w.limit-n,w.eq_s_b(1,"k")&&w.out_grouping_b(d,97,248)&&w.slice_del());break;case 3:w.slice_from("er")}}function t(){var e,r=w.limit-w.cursor;w.cursor>=a&&(e=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,w.find_among_b(u,2)?(w.bra=w.cursor,w.limit_backward=e,w.cursor=w.limit-r,w.cursor>w.limit_backward&&(w.cursor--,w.bra=w.cursor,w.slice_del())):w.limit_backward=e)}function o(){var e,r;w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(l,11),e?(w.bra=w.cursor,w.limit_backward=r,1==e&&w.slice_del()):w.limit_backward=r)}var s,a,m=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],u=[new r("dt",-1,-1),new r("vt",-1,-1)],l=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],c=[119,125,149,1],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,i(),w.cursor=w.limit,t(),w.cursor=w.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}}(),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sa=function(){this.pipeline.reset(),this.pipeline.add(e.sa.trimmer,e.sa.stopWordFilter,e.sa.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sa.stemmer))},e.sa.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿ꣠-꣱ꣲ-ꣷ꣸-ꣻ꣼-ꣽꣾ-ꣿᆰ0-ᆰ9",e.sa.trimmer=e.trimmerSupport.generateTrimmer(e.sa.wordCharacters),e.Pipeline.registerFunction(e.sa.trimmer,"trimmer-sa"),e.sa.stopWordFilter=e.generateStopWordFilter('तथा अयम्‌ एकम्‌ इत्यस्मिन्‌ तथा तत्‌ वा अयम्‌ इत्यस्य ते आहूत उपरि तेषाम्‌ किन्तु तेषाम्‌ तदा इत्यनेन अधिकः इत्यस्य तत्‌ केचन बहवः द्वि तथा महत्वपूर्णः अयम्‌ अस्य विषये अयं अस्ति तत्‌ प्रथमः विषये इत्युपरि इत्युपरि इतर अधिकतमः अधिकः अपि सामान्यतया ठ इतरेतर नूतनम्‌ द न्यूनम्‌ कश्चित्‌ वा विशालः द सः अस्ति तदनुसारम् तत्र अस्ति केवलम्‌ अपि अत्र सर्वे विविधाः तत्‌ बहवः यतः इदानीम्‌ द दक्षिण इत्यस्मै तस्य उपरि नथ अतीव कार्यम्‌ सर्वे एकैकम्‌ इत्यादि। एते सन्ति उत इत्थम्‌ मध्ये एतदर्थं . स कस्य प्रथमः श्री. करोति अस्मिन् प्रकारः निर्मिता कालः तत्र कर्तुं समान अधुना ते सन्ति स एकः अस्ति सः अर्थात् तेषां कृते . स्थितम् विशेषः अग्रिम तेषाम्‌ समान स्रोतः ख म समान इदानीमपि अधिकतया करोतु ते समान इत्यस्य वीथी सह यस्मिन् कृतवान्‌ धृतः तदा पुनः पूर्वं सः आगतः किम्‌ कुल इतर पुरा मात्रा स विषये उ अतएव अपि नगरस्य उपरि यतः प्रतिशतं कतरः कालः साधनानि भूत तथापि जात सम्बन्धि अन्यत्‌ ग अतः अस्माकं स्वकीयाः अस्माकं इदानीं अन्तः इत्यादयः भवन्तः इत्यादयः एते एताः तस्य अस्य इदम् एते तेषां तेषां तेषां तान् तेषां तेषां तेषां समानः सः एकः च तादृशाः बहवः अन्ये च वदन्ति यत् कियत् कस्मै कस्मै यस्मै यस्मै यस्मै यस्मै न अतिनीचः किन्तु प्रथमं सम्पूर्णतया ततः चिरकालानन्तरं पुस्तकं सम्पूर्णतया अन्तः किन्तु अत्र वा इह इव श्रद्धाय अवशिष्यते परन्तु अन्ये वर्गाः सन्ति ते सन्ति शक्नुवन्ति सर्वे मिलित्वा सर्वे एकत्र"'.split(" ")),e.sa.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.sa.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var i=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(i).split("|")},e.Pipeline.registerFunction(e.sa.stemmer,"stemmer-sa"),e.Pipeline.registerFunction(e.sa.stopWordFilter,"stopWordFilter-sa")}});
@@ -0,0 +1 @@
!function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s<t;s++)i[s]=r.charCodeAt(s);return i},!r&&""!=r||!t&&0!=t||!i)throw"Bad Among initialisation: s:"+r+", substring_i: "+t+", result: "+i;this.s_size=r.length,this.s=this.toCharArray(r),this.substring_i=t,this.result=i,this.method=s},SnowballProgram:function(){var r;return{bra:0,ket:0,limit:0,cursor:0,limit_backward:0,setCurrent:function(t){r=t,this.cursor=0,this.limit=t.length,this.limit_backward=0,this.bra=this.cursor,this.ket=this.limit},getCurrent:function(){var t=r;return r=null,t},in_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},in_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},out_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e>s||e<i)return this.cursor++,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},out_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e>s||e<i)return this.cursor--,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},eq_s:function(t,i){if(this.limit-this.cursor<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor+s)!=i.charCodeAt(s))return!1;return this.cursor+=t,!0},eq_s_b:function(t,i){if(this.cursor-this.limit_backward<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor-t+s)!=i.charCodeAt(s))return!1;return this.cursor-=t,!0},find_among:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=l;m<_.s_size;m++){if(n+l==u){f=-1;break}if(f=r.charCodeAt(n+l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n+_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n+_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},find_among_b:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit_backward,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=_.s_size-1-l;m>=0;m--){if(n-l==u){f=-1;break}if(f=r.charCodeAt(n-1-l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n-_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n-_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},replace_s:function(t,i,s){var e=s.length-(i-t),n=r.substring(0,t),u=r.substring(i);return r=n+s+u,this.limit+=e,this.cursor>=i?this.cursor+=e:this.cursor>t&&(this.cursor=t),e},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>r.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),r.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}});
+18
View File
@@ -0,0 +1,18 @@
/*!
* Lunr languages, `Swedish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stA-Za-z",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=new function(){function e(){var e,r=w.cursor+3;if(o=w.limit,0<=r||r<=w.limit){for(a=r;;){if(e=w.cursor,w.in_grouping(l,97,246)){w.cursor=e;break}if(w.cursor=e,w.cursor>=w.limit)return;w.cursor++}for(;!w.out_grouping(l,97,246);){if(w.cursor>=w.limit)return;w.cursor++}o=w.cursor,o<a&&(o=a)}}function t(){var e,r=w.limit_backward;if(w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(u,37),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.in_grouping_b(d,98,121)&&w.slice_del()}}function i(){var e=w.limit_backward;w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.find_among_b(c,7)&&(w.cursor=w.limit,w.ket=w.cursor,w.cursor>w.limit_backward&&(w.bra=--w.cursor,w.slice_del())),w.limit_backward=e)}function s(){var e,r;if(w.cursor>=o){if(r=w.limit_backward,w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(m,5))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.slice_from("lös");break;case 3:w.slice_from("full")}w.limit_backward=r}}var a,o,u=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],c=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],l=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],d=[119,127,149],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,t(),w.cursor=w.limit,i(),w.cursor=w.limit,s(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return t.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}}(),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
+1
View File
@@ -0,0 +1 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.ta=function(){this.pipeline.reset(),this.pipeline.add(e.ta.trimmer,e.ta.stopWordFilter,e.ta.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ta.stemmer))},e.ta.wordCharacters="஀-உஊ-ஏஐ-ஙச-ட஠-னப-யர-ஹ஺-ிீ-௉ொ-௏ௐ-௙௚-௟௠-௩௪-௯௰-௹௺-௿a-zA-Z-zA-0-9-",e.ta.trimmer=e.trimmerSupport.generateTrimmer(e.ta.wordCharacters),e.Pipeline.registerFunction(e.ta.trimmer,"trimmer-ta"),e.ta.stopWordFilter=e.generateStopWordFilter("அங்கு அங்கே அது அதை அந்த அவர் அவர்கள் அவள் அவன் அவை ஆக ஆகவே ஆகையால் ஆதலால் ஆதலினால் ஆனாலும் ஆனால் இங்கு இங்கே இது இதை இந்த இப்படி இவர் இவர்கள் இவள் இவன் இவை இவ்வளவு உனக்கு உனது உன் உன்னால் எங்கு எங்கே எது எதை எந்த எப்படி எவர் எவர்கள் எவள் எவன் எவை எவ்வளவு எனக்கு எனது எனவே என் என்ன என்னால் ஏது ஏன் தனது தன்னால் தானே தான் நாங்கள் நாம் நான் நீ நீங்கள்".split(" ")),e.ta.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.ta.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.ta.stemmer,"stemmer-ta"),e.Pipeline.registerFunction(e.ta.stopWordFilter,"stopWordFilter-ta")}});
+1
View File
@@ -0,0 +1 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.te=function(){this.pipeline.reset(),this.pipeline.add(e.te.trimmer,e.te.stopWordFilter,e.te.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.te.stemmer))},e.te.wordCharacters="ఀ-ఄఅ-ఔక-హా-ౌౕ-ౖౘ-ౚౠ-ౡౢ-ౣ౦-౯౸-౿఼ఽ్ౝ౷౤౥",e.te.trimmer=e.trimmerSupport.generateTrimmer(e.te.wordCharacters),e.Pipeline.registerFunction(e.te.trimmer,"trimmer-te"),e.te.stopWordFilter=e.generateStopWordFilter("అందరూ అందుబాటులో అడగండి అడగడం అడ్డంగా అనుగుణంగా అనుమతించు అనుమతిస్తుంది అయితే ఇప్పటికే ఉన్నారు ఎక్కడైనా ఎప్పుడు ఎవరైనా ఎవరో ఏ ఏదైనా ఏమైనప్పటికి ఒక ఒకరు కనిపిస్తాయి కాదు కూడా గా గురించి చుట్టూ చేయగలిగింది తగిన తర్వాత దాదాపు దూరంగా నిజంగా పై ప్రకారం ప్రక్కన మధ్య మరియు మరొక మళ్ళీ మాత్రమే మెచ్చుకో వద్ద వెంట వేరుగా వ్యతిరేకంగా సంబంధం".split(" ")),e.te.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.te.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.te.stemmer,"stemmer-te"),e.Pipeline.registerFunction(e.te.stopWordFilter,"stopWordFilter-te")}});
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.th=function(){this.pipeline.reset(),this.pipeline.add(e.th.trimmer),r?this.tokenizer=e.th.tokenizer:(e.tokenizer&&(e.tokenizer=e.th.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.th.tokenizer))},e.th.wordCharacters="[฀-๿]",e.th.trimmer=e.trimmerSupport.generateTrimmer(e.th.wordCharacters),e.Pipeline.registerFunction(e.th.trimmer,"trimmer-th");var t=e.wordcut;t.init(),e.th.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t):t});var n=i.toString().replace(/^\s+/,"");return t.cut(n).split("|")}}});
File diff suppressed because one or more lines are too long
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.vi=function(){this.pipeline.reset(),this.pipeline.add(e.vi.stopWordFilter,e.vi.trimmer)},e.vi.wordCharacters="[A-Za-ẓ̀͐́͑̉̃̓ÂâÊêÔôĂ-ăĐ-đƠ-ơƯ-ư]",e.vi.trimmer=e.trimmerSupport.generateTrimmer(e.vi.wordCharacters),e.Pipeline.registerFunction(e.vi.trimmer,"trimmer-vi"),e.vi.stopWordFilter=e.generateStopWordFilter("là cái nhưng mà".split(" "))}});
+1
View File
@@ -0,0 +1 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r(require("@node-rs/jieba")):r()(e.lunr)}(this,function(e){return function(r,t){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i="2"==r.version[0];r.zh=function(){this.pipeline.reset(),this.pipeline.add(r.zh.trimmer,r.zh.stopWordFilter,r.zh.stemmer),i?this.tokenizer=r.zh.tokenizer:(r.tokenizer&&(r.tokenizer=r.zh.tokenizer),this.tokenizerFn&&(this.tokenizerFn=r.zh.tokenizer))},r.zh.tokenizer=function(n){if(!arguments.length||null==n||void 0==n)return[];if(Array.isArray(n))return n.map(function(e){return i?new r.Token(e.toLowerCase()):e.toLowerCase()});t&&e.load(t);var o=n.toString().trim().toLowerCase(),s=[];e.cut(o,!0).forEach(function(e){s=s.concat(e.split(" "))}),s=s.filter(function(e){return!!e});var u=0;return s.map(function(e,t){if(i){var n=o.indexOf(e,u),s={};return s.position=[n,e.length],s.index=t,u=n,new r.Token(e,s)}return e})},r.zh.wordCharacters="\\w一-龥",r.zh.trimmer=r.trimmerSupport.generateTrimmer(r.zh.wordCharacters),r.Pipeline.registerFunction(r.zh.trimmer,"trimmer-zh"),r.zh.stemmer=function(){return function(e){return e}}(),r.Pipeline.registerFunction(r.zh.stemmer,"stemmer-zh"),r.zh.stopWordFilter=r.generateStopWordFilter("的 一 不 在 人 有 是 为 為 以 于 於 上 他 而 后 後 之 来 來 及 了 因 下 可 到 由 这 這 与 與 也 此 但 并 並 个 個 其 已 无 無 小 我 们 們 起 最 再 今 去 好 只 又 或 很 亦 某 把 那 你 乃 它 吧 被 比 别 趁 当 當 从 從 得 打 凡 儿 兒 尔 爾 该 該 各 给 給 跟 和 何 还 還 即 几 幾 既 看 据 據 距 靠 啦 另 么 麽 每 嘛 拿 哪 您 凭 憑 且 却 卻 让 讓 仍 啥 如 若 使 谁 誰 虽 雖 随 隨 同 所 她 哇 嗡 往 些 向 沿 哟 喲 用 咱 则 則 怎 曾 至 致 着 著 诸 諸 自".split(" ")),r.Pipeline.registerFunction(r.zh.stopWordFilter,"stopWordFilter-zh")}});
+206
View File
@@ -0,0 +1,206 @@
/**
* export the module via AMD, CommonJS or as a browser global
* Export code from https://github.com/umdjs/umd/blob/master/returnExports.js
*/
;(function (root, factory) {
if (typeof define === 'function' && define.amd) {
// AMD. Register as an anonymous module.
define(factory)
} else if (typeof exports === 'object') {
/**
* Node. Does not work with strict CommonJS, but
* only CommonJS-like environments that support module.exports,
* like Node.
*/
module.exports = factory()
} else {
// Browser globals (root is window)
factory()(root.lunr);
}
}(this, function () {
/**
* Just return a value to define the module export.
* This example returns an object, but the module
* can return a function as the exported value.
*/
return function(lunr) {
// TinySegmenter 0.1 -- Super compact Japanese tokenizer in Javascript
// (c) 2008 Taku Kudo <taku@chasen.org>
// TinySegmenter is freely distributable under the terms of a new BSD licence.
// For details, see http://chasen.org/~taku/software/TinySegmenter/LICENCE.txt
function TinySegmenter() {
var patterns = {
"[一二三四五六七八九十百千万億兆]":"M",
"[一-龠々〆ヵヶ]":"H",
"[ぁ-ん]":"I",
"[ァ-ヴーア-ン゙ー]":"K",
"[a-zA-Z-zA-]":"A",
"[0-9-]":"N"
}
this.chartype_ = [];
for (var i in patterns) {
var regexp = new RegExp(i);
this.chartype_.push([regexp, patterns[i]]);
}
this.BIAS__ = -332
this.BC1__ = {"HH":6,"II":2461,"KH":406,"OH":-1378};
this.BC2__ = {"AA":-3267,"AI":2744,"AN":-878,"HH":-4070,"HM":-1711,"HN":4012,"HO":3761,"IA":1327,"IH":-1184,"II":-1332,"IK":1721,"IO":5492,"KI":3831,"KK":-8741,"MH":-3132,"MK":3334,"OO":-2920};
this.BC3__ = {"HH":996,"HI":626,"HK":-721,"HN":-1307,"HO":-836,"IH":-301,"KK":2762,"MK":1079,"MM":4034,"OA":-1652,"OH":266};
this.BP1__ = {"BB":295,"OB":304,"OO":-125,"UB":352};
this.BP2__ = {"BO":60,"OO":-1762};
this.BQ1__ = {"BHH":1150,"BHM":1521,"BII":-1158,"BIM":886,"BMH":1208,"BNH":449,"BOH":-91,"BOO":-2597,"OHI":451,"OIH":-296,"OKA":1851,"OKH":-1020,"OKK":904,"OOO":2965};
this.BQ2__ = {"BHH":118,"BHI":-1159,"BHM":466,"BIH":-919,"BKK":-1720,"BKO":864,"OHH":-1139,"OHM":-181,"OIH":153,"UHI":-1146};
this.BQ3__ = {"BHH":-792,"BHI":2664,"BII":-299,"BKI":419,"BMH":937,"BMM":8335,"BNN":998,"BOH":775,"OHH":2174,"OHM":439,"OII":280,"OKH":1798,"OKI":-793,"OKO":-2242,"OMH":-2402,"OOO":11699};
this.BQ4__ = {"BHH":-3895,"BIH":3761,"BII":-4654,"BIK":1348,"BKK":-1806,"BMI":-3385,"BOO":-12396,"OAH":926,"OHH":266,"OHK":-2036,"ONN":-973};
this.BW1__ = {",と":660,",同":727,"B1あ":1404,"B1同":542,"、と":660,"、同":727,"」と":1682,"あっ":1505,"いう":1743,"いっ":-2055,"いる":672,"うし":-4817,"うん":665,"から":3472,"がら":600,"こう":-790,"こと":2083,"こん":-1262,"さら":-4143,"さん":4573,"した":2641,"して":1104,"すで":-3399,"そこ":1977,"それ":-871,"たち":1122,"ため":601,"った":3463,"つい":-802,"てい":805,"てき":1249,"でき":1127,"です":3445,"では":844,"とい":-4915,"とみ":1922,"どこ":3887,"ない":5713,"なっ":3015,"など":7379,"なん":-1113,"にし":2468,"には":1498,"にも":1671,"に対":-912,"の一":-501,"の中":741,"ませ":2448,"まで":1711,"まま":2600,"まる":-2155,"やむ":-1947,"よっ":-2565,"れた":2369,"れで":-913,"をし":1860,"を見":731,"亡く":-1886,"京都":2558,"取り":-2784,"大き":-2604,"大阪":1497,"平方":-2314,"引き":-1336,"日本":-195,"本当":-2423,"毎日":-2113,"目指":-724,"B1あ":1404,"B1同":542,"」と":1682};
this.BW2__ = {"..":-11822,"11":-669,"――":-5730,"−−":-13175,"いう":-1609,"うか":2490,"かし":-1350,"かも":-602,"から":-7194,"かれ":4612,"がい":853,"がら":-3198,"きた":1941,"くな":-1597,"こと":-8392,"この":-4193,"させ":4533,"され":13168,"さん":-3977,"しい":-1819,"しか":-545,"した":5078,"して":972,"しな":939,"その":-3744,"たい":-1253,"たた":-662,"ただ":-3857,"たち":-786,"たと":1224,"たは":-939,"った":4589,"って":1647,"っと":-2094,"てい":6144,"てき":3640,"てく":2551,"ては":-3110,"ても":-3065,"でい":2666,"でき":-1528,"でし":-3828,"です":-4761,"でも":-4203,"とい":1890,"とこ":-1746,"とと":-2279,"との":720,"とみ":5168,"とも":-3941,"ない":-2488,"なが":-1313,"など":-6509,"なの":2614,"なん":3099,"にお":-1615,"にし":2748,"にな":2454,"によ":-7236,"に対":-14943,"に従":-4688,"に関":-11388,"のか":2093,"ので":-7059,"のに":-6041,"のの":-6125,"はい":1073,"はが":-1033,"はず":-2532,"ばれ":1813,"まし":-1316,"まで":-6621,"まれ":5409,"めて":-3153,"もい":2230,"もの":-10713,"らか":-944,"らし":-1611,"らに":-1897,"りし":651,"りま":1620,"れた":4270,"れて":849,"れば":4114,"ろう":6067,"われ":7901,"を通":-11877,"んだ":728,"んな":-4115,"一人":602,"一方":-1375,"一日":970,"一部":-1051,"上が":-4479,"会社":-1116,"出て":2163,"分の":-7758,"同党":970,"同日":-913,"大阪":-2471,"委員":-1250,"少な":-1050,"年度":-8669,"年間":-1626,"府県":-2363,"手権":-1982,"新聞":-4066,"日新":-722,"日本":-7068,"日米":3372,"曜日":-601,"朝鮮":-2355,"本人":-2697,"東京":-1543,"然と":-1384,"社会":-1276,"立て":-990,"第に":-1612,"米国":-4268,"11":-669};
this.BW3__ = {"あた":-2194,"あり":719,"ある":3846,"い.":-1185,"い。":-1185,"いい":5308,"いえ":2079,"いく":3029,"いた":2056,"いっ":1883,"いる":5600,"いわ":1527,"うち":1117,"うと":4798,"えと":1454,"か.":2857,"か。":2857,"かけ":-743,"かっ":-4098,"かに":-669,"から":6520,"かり":-2670,"が,":1816,"が、":1816,"がき":-4855,"がけ":-1127,"がっ":-913,"がら":-4977,"がり":-2064,"きた":1645,"けど":1374,"こと":7397,"この":1542,"ころ":-2757,"さい":-714,"さを":976,"し,":1557,"し、":1557,"しい":-3714,"した":3562,"して":1449,"しな":2608,"しま":1200,"す.":-1310,"す。":-1310,"する":6521,"ず,":3426,"ず、":3426,"ずに":841,"そう":428,"た.":8875,"た。":8875,"たい":-594,"たの":812,"たり":-1183,"たる":-853,"だ.":4098,"だ。":4098,"だっ":1004,"った":-4748,"って":300,"てい":6240,"てお":855,"ても":302,"です":1437,"でに":-1482,"では":2295,"とう":-1387,"とし":2266,"との":541,"とも":-3543,"どう":4664,"ない":1796,"なく":-903,"など":2135,"に,":-1021,"に、":-1021,"にし":1771,"にな":1906,"には":2644,"の,":-724,"の、":-724,"の子":-1000,"は,":1337,"は、":1337,"べき":2181,"まし":1113,"ます":6943,"まっ":-1549,"まで":6154,"まれ":-793,"らし":1479,"られ":6820,"るる":3818,"れ,":854,"れ、":854,"れた":1850,"れて":1375,"れば":-3246,"れる":1091,"われ":-605,"んだ":606,"んで":798,"カ月":990,"会議":860,"入り":1232,"大会":2217,"始め":1681,"市":965,"新聞":-5055,"日,":974,"日、":974,"社会":2024,"カ月":990};
this.TC1__ = {"AAA":1093,"HHH":1029,"HHM":580,"HII":998,"HOH":-390,"HOM":-331,"IHI":1169,"IOH":-142,"IOI":-1015,"IOM":467,"MMH":187,"OOI":-1832};
this.TC2__ = {"HHO":2088,"HII":-1023,"HMM":-1154,"IHI":-1965,"KKH":703,"OII":-2649};
this.TC3__ = {"AAA":-294,"HHH":346,"HHI":-341,"HII":-1088,"HIK":731,"HOH":-1486,"IHH":128,"IHI":-3041,"IHO":-1935,"IIH":-825,"IIM":-1035,"IOI":-542,"KHH":-1216,"KKA":491,"KKH":-1217,"KOK":-1009,"MHH":-2694,"MHM":-457,"MHO":123,"MMH":-471,"NNH":-1689,"NNO":662,"OHO":-3393};
this.TC4__ = {"HHH":-203,"HHI":1344,"HHK":365,"HHM":-122,"HHN":182,"HHO":669,"HIH":804,"HII":679,"HOH":446,"IHH":695,"IHO":-2324,"IIH":321,"III":1497,"IIO":656,"IOO":54,"KAK":4845,"KKA":3386,"KKK":3065,"MHH":-405,"MHI":201,"MMH":-241,"MMM":661,"MOM":841};
this.TQ1__ = {"BHHH":-227,"BHHI":316,"BHIH":-132,"BIHH":60,"BIII":1595,"BNHH":-744,"BOHH":225,"BOOO":-908,"OAKK":482,"OHHH":281,"OHIH":249,"OIHI":200,"OIIH":-68};
this.TQ2__ = {"BIHH":-1401,"BIII":-1033,"BKAK":-543,"BOOO":-5591};
this.TQ3__ = {"BHHH":478,"BHHM":-1073,"BHIH":222,"BHII":-504,"BIIH":-116,"BIII":-105,"BMHI":-863,"BMHM":-464,"BOMH":620,"OHHH":346,"OHHI":1729,"OHII":997,"OHMH":481,"OIHH":623,"OIIH":1344,"OKAK":2792,"OKHH":587,"OKKA":679,"OOHH":110,"OOII":-685};
this.TQ4__ = {"BHHH":-721,"BHHM":-3604,"BHII":-966,"BIIH":-607,"BIII":-2181,"OAAA":-2763,"OAKK":180,"OHHH":-294,"OHHI":2446,"OHHO":480,"OHIH":-1573,"OIHH":1935,"OIHI":-493,"OIIH":626,"OIII":-4007,"OKAK":-8156};
this.TW1__ = {"につい":-4681,"東京都":2026};
this.TW2__ = {"ある程":-2049,"いった":-1256,"ころが":-2434,"しょう":3873,"その後":-4430,"だって":-1049,"ていた":1833,"として":-4657,"ともに":-4517,"もので":1882,"一気に":-792,"初めて":-1512,"同時に":-8097,"大きな":-1255,"対して":-2721,"社会党":-3216};
this.TW3__ = {"いただ":-1734,"してい":1314,"として":-4314,"につい":-5483,"にとっ":-5989,"に当た":-6247,"ので,":-727,"ので、":-727,"のもの":-600,"れから":-3752,"十二月":-2287};
this.TW4__ = {"いう.":8576,"いう。":8576,"からな":-2348,"してい":2958,"たが,":1516,"たが、":1516,"ている":1538,"という":1349,"ました":5543,"ません":1097,"ようと":-4258,"よると":5865};
this.UC1__ = {"A":484,"K":93,"M":645,"O":-505};
this.UC2__ = {"A":819,"H":1059,"I":409,"M":3987,"N":5775,"O":646};
this.UC3__ = {"A":-1370,"I":2311};
this.UC4__ = {"A":-2643,"H":1809,"I":-1032,"K":-3450,"M":3565,"N":3876,"O":6646};
this.UC5__ = {"H":313,"I":-1238,"K":-799,"M":539,"O":-831};
this.UC6__ = {"H":-506,"I":-253,"K":87,"M":247,"O":-387};
this.UP1__ = {"O":-214};
this.UP2__ = {"B":69,"O":935};
this.UP3__ = {"B":189};
this.UQ1__ = {"BH":21,"BI":-12,"BK":-99,"BN":142,"BO":-56,"OH":-95,"OI":477,"OK":410,"OO":-2422};
this.UQ2__ = {"BH":216,"BI":113,"OK":1759};
this.UQ3__ = {"BA":-479,"BH":42,"BI":1913,"BK":-7198,"BM":3160,"BN":6427,"BO":14761,"OI":-827,"ON":-3212};
this.UW1__ = {",":156,"、":156,"「":-463,"あ":-941,"う":-127,"が":-553,"き":121,"こ":505,"で":-201,"と":-547,"ど":-123,"に":-789,"の":-185,"は":-847,"も":-466,"や":-470,"よ":182,"ら":-292,"り":208,"れ":169,"を":-446,"ん":-137,"・":-135,"主":-402,"京":-268,"区":-912,"午":871,"国":-460,"大":561,"委":729,"市":-411,"日":-141,"理":361,"生":-408,"県":-386,"都":-718,"「":-463,"・":-135};
this.UW2__ = {",":-829,"、":-829,"":892,"「":-645,"」":3145,"あ":-538,"い":505,"う":134,"お":-502,"か":1454,"が":-856,"く":-412,"こ":1141,"さ":878,"ざ":540,"し":1529,"す":-675,"せ":300,"そ":-1011,"た":188,"だ":1837,"つ":-949,"て":-291,"で":-268,"と":-981,"ど":1273,"な":1063,"に":-1764,"の":130,"は":-409,"ひ":-1273,"べ":1261,"ま":600,"も":-1263,"や":-402,"よ":1639,"り":-579,"る":-694,"れ":571,"を":-2516,"ん":2095,"ア":-587,"カ":306,"キ":568,"ッ":831,"三":-758,"不":-2150,"世":-302,"中":-968,"主":-861,"事":492,"人":-123,"会":978,"保":362,"入":548,"初":-3025,"副":-1566,"北":-3414,"区":-422,"大":-1769,"天":-865,"太":-483,"子":-1519,"学":760,"実":1023,"小":-2009,"市":-813,"年":-1060,"強":1067,"手":-1519,"揺":-1033,"政":1522,"文":-1355,"新":-1682,"日":-1815,"明":-1462,"最":-630,"朝":-1843,"本":-1650,"東":-931,"果":-665,"次":-2378,"民":-180,"気":-1740,"理":752,"発":529,"目":-1584,"相":-242,"県":-1165,"立":-763,"第":810,"米":509,"自":-1353,"行":838,"西":-744,"見":-3874,"調":1010,"議":1198,"込":3041,"開":1758,"間":-1257,"「":-645,"」":3145,"ッ":831,"ア":-587,"カ":306,"キ":568};
this.UW3__ = {",":4889,"1":-800,"":-1723,"、":4889,"々":-2311,"":5827,"」":2670,"〓":-3573,"あ":-2696,"い":1006,"う":2342,"え":1983,"お":-4864,"か":-1163,"が":3271,"く":1004,"け":388,"げ":401,"こ":-3552,"ご":-3116,"さ":-1058,"し":-395,"す":584,"せ":3685,"そ":-5228,"た":842,"ち":-521,"っ":-1444,"つ":-1081,"て":6167,"で":2318,"と":1691,"ど":-899,"な":-2788,"に":2745,"の":4056,"は":4555,"ひ":-2171,"ふ":-1798,"へ":1199,"ほ":-5516,"ま":-4384,"み":-120,"め":1205,"も":2323,"や":-788,"よ":-202,"ら":727,"り":649,"る":5905,"れ":2773,"わ":-1207,"を":6620,"ん":-518,"ア":551,"グ":1319,"ス":874,"ッ":-1350,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278,"・":-3794,"一":-1619,"下":-1759,"世":-2087,"両":3815,"中":653,"主":-758,"予":-1193,"二":974,"人":2742,"今":792,"他":1889,"以":-1368,"低":811,"何":4265,"作":-361,"保":-2439,"元":4858,"党":3593,"全":1574,"公":-3030,"六":755,"共":-1880,"円":5807,"再":3095,"分":457,"初":2475,"別":1129,"前":2286,"副":4437,"力":365,"動":-949,"務":-1872,"化":1327,"北":-1038,"区":4646,"千":-2309,"午":-783,"協":-1006,"口":483,"右":1233,"各":3588,"合":-241,"同":3906,"和":-837,"員":4513,"国":642,"型":1389,"場":1219,"外":-241,"妻":2016,"学":-1356,"安":-423,"実":-1008,"家":1078,"小":-513,"少":-3102,"州":1155,"市":3197,"平":-1804,"年":2416,"広":-1030,"府":1605,"度":1452,"建":-2352,"当":-3885,"得":1905,"思":-1291,"性":1822,"戸":-488,"指":-3973,"政":-2013,"教":-1479,"数":3222,"文":-1489,"新":1764,"日":2099,"旧":5792,"昨":-661,"時":-1248,"曜":-951,"最":-937,"月":4125,"期":360,"李":3094,"村":364,"東":-805,"核":5156,"森":2438,"業":484,"氏":2613,"民":-1694,"決":-1073,"法":1868,"海":-495,"無":979,"物":461,"特":-3850,"生":-273,"用":914,"町":1215,"的":7313,"直":-1835,"省":792,"県":6293,"知":-1528,"私":4231,"税":401,"立":-960,"第":1201,"米":7767,"系":3066,"約":3663,"級":1384,"統":-4229,"総":1163,"線":1255,"者":6457,"能":725,"自":-2869,"英":785,"見":1044,"調":-562,"財":-733,"費":1777,"車":1835,"軍":1375,"込":-1504,"通":-1136,"選":-681,"郎":1026,"郡":4404,"部":1200,"金":2163,"長":421,"開":-1432,"間":1302,"関":-1282,"雨":2009,"電":-1045,"非":2066,"駅":1620,"":-800,"」":2670,"・":-3794,"ッ":-1350,"ア":551,"グ":1319,"ス":874,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278};
this.UW4__ = {",":3930,".":3508,"―":-4841,"、":3930,"。":3508,"":4999,"「":1895,"」":3798,"〓":-5156,"あ":4752,"い":-3435,"う":-640,"え":-2514,"お":2405,"か":530,"が":6006,"き":-4482,"ぎ":-3821,"く":-3788,"け":-4376,"げ":-4734,"こ":2255,"ご":1979,"さ":2864,"し":-843,"じ":-2506,"す":-731,"ず":1251,"せ":181,"そ":4091,"た":5034,"だ":5408,"ち":-3654,"っ":-5882,"つ":-1659,"て":3994,"で":7410,"と":4547,"な":5433,"に":6499,"ぬ":1853,"ね":1413,"の":7396,"は":8578,"ば":1940,"ひ":4249,"び":-4134,"ふ":1345,"へ":6665,"べ":-744,"ほ":1464,"ま":1051,"み":-2082,"む":-882,"め":-5046,"も":4169,"ゃ":-2666,"や":2795,"ょ":-1544,"よ":3351,"ら":-2922,"り":-9726,"る":-14896,"れ":-2613,"ろ":-4570,"わ":-1783,"を":13150,"ん":-2352,"カ":2145,"コ":1789,"セ":1287,"ッ":-724,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637,"・":-4371,"ー":-11870,"一":-2069,"中":2210,"予":782,"事":-190,"井":-1768,"人":1036,"以":544,"会":950,"体":-1286,"作":530,"側":4292,"先":601,"党":-2006,"共":-1212,"内":584,"円":788,"初":1347,"前":1623,"副":3879,"力":-302,"動":-740,"務":-2715,"化":776,"区":4517,"協":1013,"参":1555,"合":-1834,"和":-681,"員":-910,"器":-851,"回":1500,"国":-619,"園":-1200,"地":866,"場":-1410,"塁":-2094,"士":-1413,"多":1067,"大":571,"子":-4802,"学":-1397,"定":-1057,"寺":-809,"小":1910,"屋":-1328,"山":-1500,"島":-2056,"川":-2667,"市":2771,"年":374,"庁":-4556,"後":456,"性":553,"感":916,"所":-1566,"支":856,"改":787,"政":2182,"教":704,"文":522,"方":-856,"日":1798,"時":1829,"最":845,"月":-9066,"木":-485,"来":-442,"校":-360,"業":-1043,"氏":5388,"民":-2716,"気":-910,"沢":-939,"済":-543,"物":-735,"率":672,"球":-1267,"生":-1286,"産":-1101,"田":-2900,"町":1826,"的":2586,"目":922,"省":-3485,"県":2997,"空":-867,"立":-2112,"第":788,"米":2937,"系":786,"約":2171,"経":1146,"統":-1169,"総":940,"線":-994,"署":749,"者":2145,"能":-730,"般":-852,"行":-792,"規":792,"警":-1184,"議":-244,"谷":-1000,"賞":730,"車":-1481,"軍":1158,"輪":-1433,"込":-3370,"近":929,"道":-1291,"選":2596,"郎":-4866,"都":1192,"野":-1100,"銀":-2213,"長":357,"間":-2344,"院":-2297,"際":-2604,"電":-878,"領":-1659,"題":-792,"館":-1984,"首":1749,"高":2120,"「":1895,"」":3798,"・":-4371,"ッ":-724,"ー":-11870,"カ":2145,"コ":1789,"セ":1287,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637};
this.UW5__ = {",":465,".":-299,"1":-514,"E2":-32768,"]":-2762,"、":465,"。":-299,"「":363,"あ":1655,"い":331,"う":-503,"え":1199,"お":527,"か":647,"が":-421,"き":1624,"ぎ":1971,"く":312,"げ":-983,"さ":-1537,"し":-1371,"す":-852,"だ":-1186,"ち":1093,"っ":52,"つ":921,"て":-18,"で":-850,"と":-127,"ど":1682,"な":-787,"に":-1224,"の":-635,"は":-578,"べ":1001,"み":502,"め":865,"ゃ":3350,"ょ":854,"り":-208,"る":429,"れ":504,"わ":419,"を":-1264,"ん":327,"イ":241,"ル":451,"ン":-343,"中":-871,"京":722,"会":-1153,"党":-654,"務":3519,"区":-901,"告":848,"員":2104,"大":-1296,"学":-548,"定":1785,"嵐":-1304,"市":-2991,"席":921,"年":1763,"思":872,"所":-814,"挙":1618,"新":-1682,"日":218,"月":-4353,"査":932,"格":1356,"機":-1508,"氏":-1347,"田":240,"町":-3912,"的":-3149,"相":1319,"省":-1052,"県":-4003,"研":-997,"社":-278,"空":-813,"統":1955,"者":-2233,"表":663,"語":-1073,"議":1219,"選":-1018,"郎":-368,"長":786,"間":1191,"題":2368,"館":-689,"":-514,"E2":-32768,"「":363,"イ":241,"ル":451,"ン":-343};
this.UW6__ = {",":227,".":808,"1":-270,"E1":306,"、":227,"。":808,"あ":-307,"う":189,"か":241,"が":-73,"く":-121,"こ":-200,"じ":1782,"す":383,"た":-428,"っ":573,"て":-1014,"で":101,"と":-105,"な":-253,"に":-149,"の":-417,"は":-236,"も":-206,"り":187,"る":-135,"を":195,"ル":-673,"ン":-496,"一":-277,"中":201,"件":-800,"会":624,"前":302,"区":1792,"員":-1212,"委":798,"学":-960,"市":887,"広":-695,"後":535,"業":-697,"相":753,"社":-507,"福":974,"空":-822,"者":1811,"連":463,"郎":1082,"":-270,"E1":306,"ル":-673,"ン":-496};
return this;
}
TinySegmenter.prototype.ctype_ = function(str) {
for (var i in this.chartype_) {
if (str.match(this.chartype_[i][0])) {
return this.chartype_[i][1];
}
}
return "O";
}
TinySegmenter.prototype.ts_ = function(v) {
if (v) { return v; }
return 0;
}
TinySegmenter.prototype.segment = function(input) {
if (input == null || input == undefined || input == "") {
return [];
}
var result = [];
var seg = ["B3","B2","B1"];
var ctype = ["O","O","O"];
var o = input.split("");
for (i = 0; i < o.length; ++i) {
seg.push(o[i]);
ctype.push(this.ctype_(o[i]))
}
seg.push("E1");
seg.push("E2");
seg.push("E3");
ctype.push("O");
ctype.push("O");
ctype.push("O");
var word = seg[3];
var p1 = "U";
var p2 = "U";
var p3 = "U";
for (var i = 4; i < seg.length - 3; ++i) {
var score = this.BIAS__;
var w1 = seg[i-3];
var w2 = seg[i-2];
var w3 = seg[i-1];
var w4 = seg[i];
var w5 = seg[i+1];
var w6 = seg[i+2];
var c1 = ctype[i-3];
var c2 = ctype[i-2];
var c3 = ctype[i-1];
var c4 = ctype[i];
var c5 = ctype[i+1];
var c6 = ctype[i+2];
score += this.ts_(this.UP1__[p1]);
score += this.ts_(this.UP2__[p2]);
score += this.ts_(this.UP3__[p3]);
score += this.ts_(this.BP1__[p1 + p2]);
score += this.ts_(this.BP2__[p2 + p3]);
score += this.ts_(this.UW1__[w1]);
score += this.ts_(this.UW2__[w2]);
score += this.ts_(this.UW3__[w3]);
score += this.ts_(this.UW4__[w4]);
score += this.ts_(this.UW5__[w5]);
score += this.ts_(this.UW6__[w6]);
score += this.ts_(this.BW1__[w2 + w3]);
score += this.ts_(this.BW2__[w3 + w4]);
score += this.ts_(this.BW3__[w4 + w5]);
score += this.ts_(this.TW1__[w1 + w2 + w3]);
score += this.ts_(this.TW2__[w2 + w3 + w4]);
score += this.ts_(this.TW3__[w3 + w4 + w5]);
score += this.ts_(this.TW4__[w4 + w5 + w6]);
score += this.ts_(this.UC1__[c1]);
score += this.ts_(this.UC2__[c2]);
score += this.ts_(this.UC3__[c3]);
score += this.ts_(this.UC4__[c4]);
score += this.ts_(this.UC5__[c5]);
score += this.ts_(this.UC6__[c6]);
score += this.ts_(this.BC1__[c2 + c3]);
score += this.ts_(this.BC2__[c3 + c4]);
score += this.ts_(this.BC3__[c4 + c5]);
score += this.ts_(this.TC1__[c1 + c2 + c3]);
score += this.ts_(this.TC2__[c2 + c3 + c4]);
score += this.ts_(this.TC3__[c3 + c4 + c5]);
score += this.ts_(this.TC4__[c4 + c5 + c6]);
// score += this.ts_(this.TC5__[c4 + c5 + c6]);
score += this.ts_(this.UQ1__[p1 + c1]);
score += this.ts_(this.UQ2__[p2 + c2]);
score += this.ts_(this.UQ3__[p3 + c3]);
score += this.ts_(this.BQ1__[p2 + c2 + c3]);
score += this.ts_(this.BQ2__[p2 + c3 + c4]);
score += this.ts_(this.BQ3__[p3 + c2 + c3]);
score += this.ts_(this.BQ4__[p3 + c3 + c4]);
score += this.ts_(this.TQ1__[p2 + c1 + c2 + c3]);
score += this.ts_(this.TQ2__[p2 + c2 + c3 + c4]);
score += this.ts_(this.TQ3__[p3 + c1 + c2 + c3]);
score += this.ts_(this.TQ4__[p3 + c2 + c3 + c4]);
var p = "O";
if (score > 0) {
result.push(word);
word = "";
p = "B";
}
p1 = p2;
p2 = p3;
p3 = p;
word += seg[i];
}
result.push(word);
return result;
}
lunr.TinySegmenter = TinySegmenter;
};
}));
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
{"version":3,"sources":["src/templates/assets/stylesheets/palette/_scheme.scss","../../../../src/templates/assets/stylesheets/palette.scss","src/templates/assets/stylesheets/palette/_accent.scss","src/templates/assets/stylesheets/palette/_primary.scss","src/templates/assets/stylesheets/utilities/_break.scss"],"names":[],"mappings":"AA2BA,cAGE,6BAME,sDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CACA,mDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CAGA,mDAAA,CACA,gDAAA,CACA,yDAAA,CACA,4DAAA,CAGA,0BAAA,CACA,mCAAA,CAGA,iCAAA,CACA,kCAAA,CACA,mCAAA,CACA,mCAAA,CACA,kCAAA,CACA,iCAAA,CACA,+CAAA,CACA,6DAAA,CACA,gEAAA,CACA,4DAAA,CACA,4DAAA,CACA,6DAAA,CAGA,6CAAA,CAGA,+CAAA,CAGA,uDAAA,CACA,6DAAA,CACA,2DAAA,CAGA,iCAAA,CAGA,yDAAA,CACA,iEAAA,CAGA,mDAAA,CACA,mDAAA,CAGA,qDAAA,CACA,uDAAA,CAGA,8DAAA,CAKA,8DAAA,CAKA,0DAAA,CAzEA,iBCiBF,CD6DE,kHAEE,YC3DJ,CDkFE,yDACE,4BChFJ,CD+EE,2DACE,4BC7EJ,CD4EE,gEACE,4BC1EJ,CDyEE,2DACE,4BCvEJ,CDsEE,yDACE,4BCpEJ,CDmEE,0DACE,4BCjEJ,CDgEE,gEACE,4BC9DJ,CD6DE,0DACE,4BC3DJ,CD0DE,2OACE,4BC/CJ,CDsDA,+FAGE,iCCpDF,CACF,CCjDE,2BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD6CN,CCvDE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDoDN,CC9DE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD2DN,CCrEE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDkEN,CC5EE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDyEN,CCnFE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDgFN,CC1FE,kCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDuFN,CCjGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD8FN,CCxGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDqGN,CC/GE,6BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD4GN,CCtHE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDmHN,CC7HE,4BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD6HN,CCpIE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDoIN,CC3IE,6BACE,yBAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD2IN,CClJE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDkJN,CCzJE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDsJN,CE3JE,4BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwJN,CEnKE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgKN,CE3KE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwKN,CEnLE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgLN,CE3LE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwLN,CEnME,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgMN,CE3ME,mCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwMN,CEnNE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgNN,CE3NE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwNN,CEnOE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgON,CE3OE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwON,CEnPE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmPN,CE3PE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2PN,CEnQE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmQN,CE3QE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2QN,CEnRE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgRN,CE3RE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwRN,CEnSE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BF4RN,CE5SE,kCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BFqSN,CEtRE,sEACE,4BFyRJ,CE1RE,+DACE,4BF6RJ,CE9RE,iEACE,4BFiSJ,CElSE,gEACE,4BFqSJ,CEtSE,iEACE,4BFySJ,CEhSA,8BACE,mDAAA,CACA,4DAAA,CACA,0DAAA,CACA,oDAAA,CACA,2DAAA,CAGA,4BFiSF,CE9RE,yCACE,+BFgSJ,CE7RI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCFiSN,CG7MI,mCD1EA,+CACE,8CF0RJ,CEvRI,qDACE,8CFyRN,CEpRE,iEACE,mCFsRJ,CACF,CGxNI,sCDvDA,uCACE,oCFkRJ,CACF,CEzQA,8BACE,kDAAA,CACA,4DAAA,CACA,wDAAA,CACA,oDAAA,CACA,6DAAA,CAGA,4BF0QF,CEvQE,yCACE,+BFyQJ,CEtQI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCF0QN,CEnQE,yCACE,6CFqQJ,CG9NI,0CDhCA,8CACE,gDFiQJ,CACF,CGnOI,0CDvBA,iFACE,6CF6PJ,CACF,CG3PI,sCDKA,uCACE,6CFyPJ,CACF","file":"palette.css"}
File diff suppressed because it is too large Load Diff
-1785
View File
File diff suppressed because it is too large Load Diff
-103
View File
@@ -1,103 +0,0 @@
# Which embedding model is best?
Three ArcFace variants (w600k-R50, R18, w600k-MBF) and LVFace-B (Glint360K,
455MB) were compared. r50 is excluded from the training/held-out comparison
below; its gallery has roughly 30% fewer reference images per actor than the
other three on the identical source photos, which confounds a direct score
comparison (see [the full experiment log](model-bakeoff.md) for detail). It
remains in the calibration comparison, which does not depend on the gallery
image count.
## First signal: calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | cosine similarity) =
σ(a·sim + b)`, stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This is a property of the embedding space alone, computed from intra- and
inter-actor reference-image pairs with no tracking or scene logic involved,
so it is a clean first read on discriminative power before running a
benchmark.
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
| model | a (steepness) | boundary at P=0.5 |
|---|---|---|
| LVFace-B Glint360K | 17.7 | sim 0.228 |
| ArcFace w600k-MBF | 16.2 | sim 0.267 |
| ArcFace w600k-R50 | 15.4 | sim 0.301 |
| ArcFace R18 | 15.3 | sim 0.309 |
LVFace has both the steepest transition and the lowest decision boundary,
separating same-actor from different-actor reference pairs more confidently
at a lower similarity than any ArcFace variant.
## Second signal: held-out F1
Each model's own tuned `full_exp` config, replayed against the 5 films the
optimizer never saw and scored the same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on all 5 held-out films; the ranking never flips
between models. Total misID count across the 5 films: LVFace 1032, mbf
2197, r18 1224. LVFace has less than half mbf's misID total and still
scores higher on every film.
Held-out results are stronger evidence than training results, because
training numbers can reflect what the optimizer was tuned to fit rather
than general performance. On training data, the ordering is not as clean:
| film | LVFace F1 | mbf F1 | r18 F1 | best |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
mbf beats LVFace on Lord of War (77.2% vs 75.6%), the only film in either
table where LVFace does not score highest. LVFace's training-set macro
average (75.3%, see [the full experiment log](model-bakeoff.md)) is not a
uniform win across every film it contributes to; the held-out result, where
LVFace wins all 5 films outright, is the stronger claim.
This reverses an earlier, superseded benchmarking pass that used a
scene-union metric and found the three models statistically
indistinguishable (around 85% each), concluding LVFace was not worth its
size. That metric masked out-of-cast false positives behind a
gallery-intersect-cast recall filter; the per-second metric used here does
not.
## Full training-matrix picture
![All 12 combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
Best full-gallery combo per model (all three are `full_exp`), from the
training matrix in [the full experiment log](model-bakeoff.md):
| model | F1 | P | R | misID |
|---|---|---|---|---|
| LVFace-B Glint360K | 75.3% | 89.7% | 65.4% | 232 |
| ArcFace w600k-MBF | 72.0% | 87.7% | 61.4% | 240 |
| ArcFace R18 | 69.1% | 87.6% | 57.7% | 242 |
LVFace leads within both the restricted and full gallery modes, visible
directly in the chart above without reading the table. The three models'
misID counts on the full gallery are nearly identical (232/240/242); LVFace's
lead here is a precision-and-recall lead, not a misID one.
## Operational note
Switching the default embedder is not a config change alone; the gallery
is model-specific, since embeddings from different models are not
comparable. Any existing gallery built against a different model must be
rebuilt from source images before the new default takes effect.
[`scripts/optimizer/reembed_gallery.py`](https://REPOLINK/scripts/optimizer/reembed_gallery.py)
does this from a reference gallery's cached source images without
re-downloading anything.
-69
View File
@@ -1,69 +0,0 @@
# Whole gallery vs. cast-restricted gallery
Two ways to run the matcher. Full mode scores every detected face against
the entire 2418-actor gallery. Restricted mode pre-filters each film's
gallery down to just its Jellyfin-credited cast (typically around 15
top-billed actors) before the matcher runs.
## Result
Averaged across the 3 compared models (r50 excluded, see
[the full experiment log](model-bakeoff.md)) and both expansion settings, on
the 4 training films:
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once, not a precision/recall trade:
+4.8pp F1, +6.0pp recall, roughly a quarter the total misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections.
Every model's best-scoring combo in the training matrix uses the
restricted gallery:
![All combos ranked by training-set F1, filled dots are restricted](assets/images/rep4_matrix_f1.png)
See [the full experiment log](model-bakeoff.md) for the complete table. One
combo reaches zero true out-of-cast misidentifications,
`arcface_w600k_mbf_restricted_exp` (F1 76.2%), and it is a restricted one,
consistent with restriction, not expansion, being what suppresses cross-film
confusions.
The restriction effect (+4.8pp averaged across models) is larger than the
model-choice effect: LVFace beats r18 by 6.2pp in full mode but beats mbf by
3.3pp. Restriction is the single strongest lever in the matrix.
## Why this is not the shipped default
Cast restriction is implemented today only as an offline optimizer
technique
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
it pre-builds a filtered gallery file per film using Jellyfin's cast list
before the benchmark calls the matcher. There is no runtime "restrict to
this title's credited cast" switch in the shipped application;
`scene_analyze` always matches against whatever single gallery file it is
given.
Building this as a real feature requires:
- A live Jellyfin cast lookup at analysis time. The title is already known,
and [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
already performs this lookup for its own `filter_gallery`-based
restriction path; it is not wired into `scene_analyze` as a first-class
option.
- A decision on the fallback case: what happens to a real, uncredited
cameo (see the Germar Terrell Gardner and Talia Balsam cases in the
[LVFace deep dive](lvface-deep-dive.md#where-lvface-beat-x-ray)) if the
restricted gallery never includes them at all.
- Regenerating the restricted-gallery cache whenever a title's Jellyfin
cast list changes.
The shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults use
the full-mode winner (`LVFace-B_Glint360K_full_exp`, F1 75.3% training,
67.4% held-out macro) rather than the higher-scoring `restricted_exp`
(78.3%), because 78.3% describes a capability the application does not
have yet.
-99
View File
@@ -1,99 +0,0 @@
# scene-actor-extraction
A face-recognition pipeline that finds when each actor appears on screen in
a film or TV episode, built on [KPN++](https://gitea.tourolle.paris/dtourolle/KPN)
(a C++20 Kahn Process Network library) for the detect, track, match, and
scene pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated
optimizer.
This is a correctly scored second from a held-out film, one the optimizer
never saw during tuning:
![A perfect X-Ray second: three faces named at 100%, two more correctly carried off-screen](assets/images/lovelace_perfect_second.jpg)
Every visible face is named at 100% confidence (Chris Noth, Hank Azaria,
Bobby Cannavale), the background extra is correctly left unnamed, and the
two credited cast members without a visible face are correctly reported
present but not visible. This matches Amazon X-Ray's own record for this
second exactly.
Results are not uniform across films. The hardest held-out film scores 46%
F1. This report documents why: one tunable trade (extinction bridging at
hard cuts), one structural limit (X-Ray credits people whose faces never
appear on screen), and a small number of cases where the pipeline is
correct and X-Ray's ground truth is not. Read
[how we score against X-Ray](methodology.md) first. X-Ray's ground truth is
scene-level; the pipeline's output is per-second. That difference shapes
every finding below.
## Findings
<div class="grid cards" markdown>
- :material-trophy:{ .lg .middle } **[Which model is best?](best-model.md)**
---
Calibration curves first, independent of any threshold, then held-out
F1 across three models. LVFace-B Glint360K wins both, and wins on every
held-out film.
- :material-filter:{ .lg .middle } **[Whole vs. cast-restricted gallery](gallery-scope.md)**
---
Restricting the matcher to a film's credited cast improves F1,
recall, and misID rate at once, but is not a shipped runtime feature
yet.
- :material-account-convert:{ .lg .middle } **[Does pose expansion help?](pose-expansion.md)**
---
A training-set effect that did not reproduce on 5 held-out films once
two methodology bugs in the comparison harness were found and fixed.
- :material-blur:{ .lg .middle } **[What does blur cost?](quality-knee.md)**
---
Sharpness is not a sufficient statistic for identity loss, blur breaks
confidence rather than ranking, and variance-of-Laplacian is
anti-predictive at fixed resolution.
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
---
The held-out generalization gap, the two mechanisms behind its errors,
and every distinct case where it names someone outside the film's
credited cast.
</div>
## Full experiment log
- **[Full experiment log](model-bakeoff.md)**: the complete log behind the
four pages above, including how replaying against cached embeddings
inside the same KPN network makes a full model and configuration
comparison practical, the full results table, and every caveat. This is
where the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
defaults come from.
- **[Service conversion (proposal)](service-conversion.md)**: design
sketch for a native idle-GPU worker gated on screen lock, not yet built.
## Reproducing the benchmarks
Gallery `.h5` files, embedding dumps, the X-Ray corpus, montage frame
images, and DE trajectories are not committed to this repository. They are
pushed to the Gitea package registry and pulled on demand:
```bash
scripts/artifacts/pull_artifacts.sh galleries
scripts/artifacts/pull_artifacts.sh experiment-data
scripts/artifacts/pull_artifacts.sh montage-frames <film-slug>
```
See [`scripts/artifacts/push_artifacts.sh`](https://REPOLINK/scripts/artifacts/push_artifacts.sh)
for the upload side, which requires a `GITEA_TOKEN` with package write
scope.
-303
View File
@@ -1,303 +0,0 @@
# Deep dive: LVFace-B Glint360K
LVFace won the model comparison (see [Which model is best?](best-model.md))
and is the shipped default embedder. This page reports how it performs in
detail: a baseline of correct output, the two mechanisms behind its errors,
and every distinct case where it names someone who is not in the film's
credited cast.
Read [How we score against X-Ray](methodology.md) first. X-Ray's ground truth
is scene-level, not per-frame. A name marked correct in the Offscreen column
below is the pipeline correctly reporting scene membership, not a workaround.
!!! note "How to read the frames on this page"
The top of each image is the film frame, with a box and name on every
face the pipeline matched to a real detection. The panels below are the
per-second result against X-Ray. **Onscreen** lists names attached to a
visible face this second. **Offscreen** lists names the pipeline reports
present without a currently visible face. Colors mark the verdict:
<span style="color:#0ca30c">**green**</span> correct (TPI),
<span style="color:#eb6834">**orange**</span> wrong (FPI),
<span style="color:#3987e5">**blue**</span> missed (FN).
## Baseline: correctly scored seconds
![Wedding couple correctly identified, Downton Abbey: A New Era](assets/images/downton_wedding_couple.jpg)
Six faces on screen, all six named correctly, including Penelope Wilton at
the edge of the pews and a partly occluded Michelle Dockery. Thirteen more
cast members X-Ray lists as present in the scene are correctly reported
Offscreen. One miss: Maggie Smith (blue). Score for this second: 0.86.
![19 of 20 correct in the funeral crowd](assets/images/downton_funeral_19of20.jpg)
The same film's funeral scene: dark clothing, hats, half the faces turned
away. Nineteen of the twenty cast members X-Ray lists for this scene score
correct: seven named on screen at up to 100% confidence, twelve more reported
correctly as present but not visible.
![Herbie Hancock identified on an in-fiction video call](assets/images/valerian_screen_call.jpg)
The pipeline does not require a live face. This is Herbie Hancock at 98%
confidence, identified from a face displayed on a screen inside the film, on
a video call under a science-fiction HUD overlay.
## Training vs. held-out: the generalization gap
The shipped config (`prob_threshold=0.754`, `anneal_sec=35.54`,
`extinction_sec=57.43`, `expand_gallery=true`) was tuned on 4 films. Scored
on the 5 films the optimizer never saw:
![Held-out per-film F1 vs. the training-set fit](assets/images/holdout_f1_by_film.png)
| film | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| Benny & Joon | 83.0% | 89.1% | 77.7% | 15125 | 1846 | 0 | 4337 |
| Lovelace | 77.5% | 90.3% | 67.9% | 14990 | 1085 | 58 | 7085 |
| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 18663 | 548 | 0 | 12467 |
| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 52027 | 1173 | 0 | 80084 |
| The Many Saints of Newark | 46.3% | 54.7% | 40.1% | 15922 | 4394 | 974 | 23791 |
| macro average | 67.4% | 85.8% | 57.0% | | | | |
The `P` column is misID-weighted (each out-of-film name counts 10x in the
denominator; see [methodology](methodology.md#precision-recall-and-the-misid-weighting)).
That weighting is why Many Saints reads 54.7% here despite naming mostly real,
present faces: its raw (unweighted) precision is **78.4%**, and the gap is
entirely its 974 misIDs paying the 10x penalty. The three zero-misID films
(Benny & Joon, Downton, Valerian) have identical weighted and raw precision;
Lovelace, with 58 misIDs, sits 3pp below its raw 93.3%.
Held-out F1 is 67.4%, against 75.3% on training, an 8pp drop. The spread
between the best and worst held-out film is 37pp. This is not unique to
LVFace: [the full experiment log](model-bakeoff.md#held-out-validation-all-3-models)
shows mbf and r18 with the same shape of spread on the same films, at a
uniformly lower level. Two mechanisms explain the spread. Both are shown
below with frame-level evidence.
## Mechanism 1: extinction bridging
The extinction window keeps a name reported as present for up to
`extinction_sec` after its last real detection. This is deliberate: most
gaps in face visibility are short (a turned head, an occlusion, a cut to a
reaction shot), and the window bridges them.
![Two faces on screen, six more correctly bridged](assets/images/lovelace_polygraph_bridged.jpg)
Lovelace's polygraph scene: only Eric Roberts and Amanda Seyfried have
visible faces. X-Ray lists eight cast members present. All eight score
correct; the other six are reported Offscreen through a stretch where the
camera never shows them. The extinction window is why.
The same mechanism fails at a hard cut into a long stretch with no faces at
all. Downton Abbey's recall (39.4%, the worst of the five held-out films) is
dominated by this failure. It is verified directly against the raw
per-frame stream and the dump's own detection counts, not inferred from the
score. Plotting the dump's per-second `face_count` (detector output,
independent of the tracker) against what the tracker reports, through
Downton Abbey's hard cut into its closing credits:
![Detector vs. tracker through Downton Abbey's cut to credits](assets/images/downton_ghost_timeline.png)
From the cut onward the detector reports zero faces for close to a minute.
The tracker continues reporting the previous shot's 15 identities for the
same span (verified for Hugh Bonneville: bbox `(1743.2, 0.0, 171.3, 317.8)`,
unchanged to the pixel, at every sampled second for 57 seconds). The
staircase at the right edge is the extinction window expiring, actor by
actor. This is `SceneTrackerFunc::active_[actor_idx].last_bbox`
([`src/nodes/scene_tracker_node.hpp`](https://REPOLINK/src/nodes/scene_tracker_node.hpp))
re-emitted as designed. `extinction_sec=57.4` was tuned long because
bridging is correct on most footage, as in the polygraph scene above. The
training films did not contain a faceless stretch long enough to expose the
cost side; the held-out set did.
The extinction window is a scoring concept, not something drawn on screen.
The shipped output is presence windows with no bounding boxes. Even the
debug overlay used for this report never draws a box for a bridged name: a
name inside its extinction window with no current detection appears only as
a name in the Offscreen column, the same as every correctly bridged name
above.
A related, smaller effect shows up at rapid cuts:
![Two labels on one face after a shot/reverse-shot cut](assets/images/cafe_society_rapid_cut.jpg)
Café Society (a training film), a shot/reverse-shot dialog. The box on Steve
Carell's face carries two labels: his own, and Jesse Eisenberg's, left over
from the counter-shot a moment earlier. Both names score correct, because
both actors are present in this scene per X-Ray. The box position is
briefly wrong; the presence claim, which is what the pipeline ships, is
right.
## Mechanism 2: the face-vs-presence ceiling
Downton Abbey's recall did not collapse because faces were misread. It
collapsed because for most of its 80084 false-negative seconds there was no
face to read.
![22 cast credited, nobody facing the camera](assets/images/downton_crew_fn.jpg)
A newsreel crew moves equipment through the hall. X-Ray credits 22 cast
members as present in this scene. None face the camera. Eight still score
correct, carried by presence windows from adjacent shots. The other fourteen
are missed, and no face-recognition system can recover them, because there
is no face in the frame. X-Ray records scene membership; the pipeline
measures visible faces. In ensemble scenes these two quantities diverge, and
that gap accounts for most of the false-negative count.
## Every distinct out-of-cast name
Many Saints of Newark has the largest misID count of any held-out film: 974
seconds, weighted. Rather than characterize this from a single frame, the
raw replay stream was searched directly for every name the pipeline reports
that is not in the film's credited cast. The same search was run on all 9
films in the benchmark, one rule applied uniformly: **find the first second
each distinct out-of-cast name appears, and render that exact second.**
Five films produce no such name anywhere in their runtime: Benny & Joon,
Café Society, Downton Abbey, Sound of Metal, Valerian. Zero out-of-cast
names across their entire length. Four films produce nine distinct names
between them, shown below in full, not a sample.
### The Many Saints of Newark: 4 names
![Germar Terrell Gardner, first out-of-cast name in Many Saints](assets/images/many_saints_fpi_gardner.jpg)
Germar Terrell Gardner, t=848s, 78% confidence. A real, clearly visible
background actor. He is not in X-Ray's cast list for this film, but he is
credited in Jellyfin's independent cast metadata (see
[Where LVFace beat X-Ray](#where-lvface-beat-x-ray) below). This is a
ground-truth gap, not a model error.
![Archie Yates, second out-of-cast name in Many Saints](assets/images/many_saints_fpi_yates.jpg)
Archie Yates, t=2521s, 78% confidence. A real detected face, a genuine
lookalike confusion.
![Zooey Deschanel, third out-of-cast name in Many Saints](assets/images/many_saints_fpi_deschanel.jpg)
Zooey Deschanel, t=2819s, 99% confidence. A real detected face at a dinner
table, high-confidence lookalike confusion.
![Talia Balsam, fourth out-of-cast name in Many Saints](assets/images/many_saints_fpi_balsam.jpg)
Talia Balsam, t=4551s, 93% confidence. A real detected face. Talia Balsam
plays Mrs. Jarecki, a guidance counselor, in this film; she is confirmed
on screen by direct inspection of the frame. She does not appear in X-Ray's
`people.csv` for this title. This is a second ground-truth gap in the same
film, not a model error.
Two of these four names are ground-truth gaps (Gardner, Balsam), not
misidentifications. The other two (Yates, Deschanel) are genuine embedding
errors on real faces.
### Lord of War: 3 names
![David Shumbris, first out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_shumbris.jpg)
David Shumbris, t=418s, 81% confidence. A real face in a dim, low-detail
shot under a train track. A genuine lookalike confusion in poor lighting.
![Ronald Reagan, second out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reagan_photo.jpg)
Ronald Reagan, t=1003s, 100% confidence. This is not a lookalike confusion.
The detected face is a photograph of Reagan appearing within the shot, not a
living actor. The detector and matcher both did their job correctly on the
image content in front of them; the error is that a photograph inside the
scene is not the same thing as an actor present in the scene, and the
pipeline has no way to draw that distinction from a face crop alone.
![Lance Reddick, third out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reddick.jpg)
Lance Reddick, t=6424s, 78% confidence. A small, distant, low-detail face at
the edge of frame. A marginal, low-confidence lookalike confusion.
### Lovelace: 1 name
![Chloë Sevigny, out-of-cast name in Lovelace](assets/images/lovelace_fpi_sevigny.jpg)
Chloë Sevigny, t=2451s, 100% confidence. Amanda Seyfried's track is real and
well-tracked through most of this shot, but her bbox is frozen at the exact
same coordinates for t=2450 and t=2451, one second where her box stopped
updating from a fresh detection. Only one real face is detected at t=2451
(confirmed against the dump's own per-frame detections), and it is a tight
IoU-1.0 fit under the Chloë Sevigny box, not the Seyfried one. So the green
Seyfried box in this frame is a ghost, re-emitting her last known position
for that one second, and the fresh, wrong detection is Sevigny, landing on
top of it. Not two competing fresh identities on one crop: one ghost and
one fresh misidentification happening to overlap.
### Scarface: 1 name
![Kirstie Alley, out-of-cast name in Scarface](assets/images/scarface_fpi_alley.jpg)
Kirstie Alley, t=2451s, 89% confidence. Al Pacino is correctly identified in
the foreground at 100%; a background face in the same shot is wrongly
labeled Kirstie Alley. (The t=2451s here and the Lovelace Chloë Sevigny case
above landing on the identical second is a genuine coincidence, verified from
each film's raw stream by [`first_fpi_frames.py`](https://REPOLINK/scripts/docs/first_fpi_frames.py),
not a transcription slip, two unrelated films whose *first* out-of-cast name
happens to fall at the same timestamp.)
### Summary of the nine
| film | name | t (s) | confidence | classification |
|---|---|---|---|---|
| Many Saints of Newark | Germar Terrell Gardner | 848 | 78% | ground-truth gap |
| Many Saints of Newark | Archie Yates | 2521 | 78% | lookalike confusion |
| Many Saints of Newark | Zooey Deschanel | 2819 | 99% | lookalike confusion |
| Many Saints of Newark | Talia Balsam | 4551 | 93% | ground-truth gap |
| Lord of War | David Shumbris | 418 | 81% | lookalike confusion |
| Lord of War | Ronald Reagan | 1003 | 100% | photo-in-frame |
| Lord of War | Lance Reddick | 6424 | 78% | lookalike confusion, marginal |
| Lovelace | Chloë Sevigny | 2451 | 100% | lookalike confusion |
| Scarface | Kirstie Alley | 2451 | 89% | lookalike confusion |
Of nine distinct out-of-cast names across four films, two are ground-truth
gaps, one is a photograph misread as a person, and six are genuine
embedding-space confusions on real detected faces. None trace to extinction
bridging: every one of these nine is a fresh detection on a real face crop
at the second it first appears.
## Where LVFace beat X-Ray
Not every name marked wrong is actually wrong.
[`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)
scores strictly against X-Ray, and X-Ray has gaps of its own.
![LVFace correctly identifies Germar Terrell Gardner, uncredited by X-Ray](assets/images/germar_beats_xray.jpg)
Germar Terrell Gardner, the same name from the table above, does not appear
in X-Ray's `people.csv` for The Many Saints of Newark. Jellyfin's
independent cast metadata does credit him for this film (cross-checked
against `experiments/manifests/jellyfin_casts.json` from the
`experiment-data` artifact package, a data source entirely separate from
X-Ray). Talia Balsam is the same case: confirmed on screen, absent from
X-Ray's cast list for this title.
![Robert Patrick, clearly on screen, scored wrong by a ground-truth gap](assets/images/lovelace_robert_patrick_fpi.jpg)
This extends past uncredited background actors. This is Robert Patrick,
top-billed in Lovelace, clearly on screen reading a newspaper, identified at
100%. The frame is scored wrong because X-Ray's people-in-scene list for
this specific scene omits him, despite crediting him elsewhere in the film.
The identification is correct; the ground truth is missing an entry.
X-Ray is a large, convenient ground truth. It is not a complete one. The
misID and FPI counts reported throughout this document include some fixed
amount of noise from gaps in X-Ray itself, in both directions.
## Summary
LVFace wins the model comparison on every held-out film. It correctly names
19 of 20 people in a crowded funeral scene and correctly identifies a face
displayed on a screen inside the film. Its errors resolve into two
mechanisms: extinction bridging, which is correct on most footage and fails
specifically at hard cuts into long faceless stretches, and the
face-versus-presence ceiling, where X-Ray credits scene membership for
people whose faces never appear on screen. Of the nine distinct
out-of-cast identifications found across the benchmark, two trace to gaps in
X-Ray's own cast data, one is a photograph misread as a person, and six are
genuine lookalike confusions on real faces. The held-out generalization gap,
75.3% training to 67.4% held-out, is real and should be treated as the
expected operating point, not the training-set figure.
-134
View File
@@ -1,134 +0,0 @@
# How we score against X-Ray
Every number in this report, every F1 and misID count, comes from one
comparison. The comparison has a mismatch at its core that shapes nearly
every finding in this report: the ground truth is scene-level, the
pipeline's output is per-second, and the two do not mean the same thing.
This page documents that comparison once, so the findings pages can rely on
it without re-explaining it.
## What Amazon X-Ray records
X-Ray ships three tables per film: `scenes.csv` (a list of `[start, end]`
timespans), `people_in_scenes.csv` (which actors are credited in each
scene), and `people.csv` (actor identities). There is no per-frame or
per-second annotation anywhere in X-Ray. A scene might run 45 seconds, and
X-Ray records one cast list for the entire span, not "on screen from
second 12 to second 30."
To compare this against per-second predictions, `second_score.py` expands
every scene into per-second ground truth by copying the whole scene's cast
list onto every second inside it:
```python
for sn, (t0, t1) in spans.items():
cast = scene_cast.get(sn, [])
for t in range(int(t0), int(t1)):
timeline[t] = cast
```
That is the entire mechanism. If X-Ray credits five actors to a 30-second
scene, all five count as ground truth present for all 30 seconds, including
seconds where only one of them is on screen. This is not a simplification
introduced by the pipeline; it is the only reading of X-Ray's data that is
possible, because X-Ray itself does not record anything finer-grained.
## Why an offscreen name can be scored correct
A name listed under Offscreen with a correct (green) label is not the
pipeline guessing or padding its score. It is the pipeline correctly
answering the question X-Ray actually asks: is this actor part of this
scene. It answers that question using a presence window (`[start, end]`,
held open across cuts by `anneal_sec` and `extinction_sec`), which matches
X-Ray's scene-level semantics more closely than a raw per-frame detection
would.
A system that only reported "this actor is visible in this exact frame"
would score worse against X-Ray's scene-level ground truth, producing a
false negative every time the camera cuts away from a character who is
still present in the scene. Not because it is wrong about the world, but
because it would be answering a stricter, different question than the one
X-Ray's data supports. The presence-window design exists specifically to
answer X-Ray's actual question.
## What this resolves and what it does not
This resolves the semantic mismatch between a scene and an instant. It does
not resolve two other limitations, both discussed in the
[LVFace deep dive](lvface-deep-dive.md).
**The face-vs-presence ceiling.** X-Ray credits scene membership regardless
of whether a face is ever visible: background crew, characters shot from
behind, voice-only presence. No amount of bridging recovers a face that
never appears on screen. This is a hard ceiling on recall, not a defect.
**Extinction bridging can overshoot.** The same presence-window mechanism
that correctly answers "still in this scene" during a normal cut can also
bridge across a scene boundary it has no way to detect. A hard cut into a
different scene with no faces, such as closing credits, carries the
previous scene's identities forward until the window expires. This is the
mechanism behind Downton Abbey's recall collapse, documented in the deep
dive.
## Precision, recall, and the misID weighting
Per sampled second `t`:
**TPI** (true positive instances): actors both X-Ray and the pipeline agree
are present.
**FPI** (false positive instances): actors the pipeline reports that are
not in X-Ray's cast for this second. Split into two categories:
- **FPI_incast**: the actor is in the film's cast, just not credited to
this particular scene. A timing or boundary slip.
- **FPI_misid**: the actor is not in the film's cast at all. A genuine
wrong-identity error, weighted 10x in the precision objective, because
naming someone who is not even in the film is a categorically worse
error than a few seconds of scene-boundary slop.
!!! note "Every headline `P` and `F1` is misID-weighted"
The precision reported throughout this report, and therefore the F1
derived from it, puts each `FPI_misid` into the denominator **10 times**
(`precision = TPI / (TPI + FPI_incast + 10·FPI_misid)`,
[`second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
This is deliberate: the whole point is to punish naming an out-of-film
actor far harder than a scene-boundary slip. But it means the `P` column
is not raw precision, and a misID-heavy film's `P` is depressed
super-linearly. `second_score.py` also emits an unweighted `precision_raw`
(always ≥ the weighted `P`); where the gap matters, The Many Saints of
Newark, weighted `P` 54.7% vs. raw 78.4%, the [LVFace deep dive](lvface-deep-dive.md)
reports both. When comparing `P` across films, remember you are comparing a
quantity that penalizes misIDs, not just a hit rate.
**FN** (false negatives): actors X-Ray lists that the pipeline never
reports, counted only for actors who have a gallery reference embedding.
Across the 9-film benchmark, coverage of X-Ray's credited cast ranges from
20% to 79% by film (see
[the full experiment log](model-bakeoff.md#gallery-coverage-per-film)); an
actor with no reference photo can never be recognized regardless of model
quality, and counting them as a miss would penalize gallery coverage, not
recognition accuracy.
Two further numbers are reported alongside F1:
**agreement_rate**: mean per-second Jaccard overlap
(`|Pred ∩ GT| / |Pred GT|`), partial credit. Naming 2 of 3 present actors
scores 2/3, not 0.
**exact_match_rate**: the fraction of sampled seconds where the pipeline's
named set exactly equals X-Ray's, no partial credit. Far harsher, and
dominated by recall, since any single missed actor zeroes that second.
## Reproduce
```bash
python3 scripts/optimizer/second_score.py \
--pred pred.json --xray experiments/xray/.../<xray_dir> \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5
```
See also [the full experiment log](model-bakeoff.md) for how `pred.json` is
produced, and the [LVFace deep dive](lvface-deep-dive.md) for what these
mechanisms look like frame by frame.
-346
View File
@@ -1,346 +0,0 @@
# Full experiment log
This page reports how the pipeline performs across three questions: which
embedding model is best, whether restricting the gallery to a film's
credited cast helps, and whether promoting confidently identified poses into
a per-film gallery annex helps. It also documents the replay architecture
that made testing all three questions in one pass practical, and every
caveat needed to trust the numbers.
Read [How we score against X-Ray](methodology.md) first for what F1,
precision, recall, and misID mean in this report. All numbers below use the
per-second metric
([`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
r50 (ArcFace w600k-R50) is excluded from the detailed comparison below. Its
gallery was built with roughly 30% fewer reference images per actor than the
other three models on the identical source photos (10808 vs 15055 total
embeddings across the same 2418 actors), which confounds any direct
comparison of its scores against the others. It remains in the
[calibration curve comparison](best-model.md#first-signal-calibration-curves),
which does not depend on the training benchmark.
## Why replay makes this affordable
Decoding video and running face detection, alignment, and embedding is the
expensive part of this pipeline. Everything downstream of that (tracking,
identity matching, scene aggregation) is cheap. KPN++'s node/network
structure means those two stages are separate components connected by
typed channels, so the expensive stage can run once per film, cache its
output, and the cheap stage can be re-run against that cache as many times
as needed with different Config values.
`scene_analyze --dump-embeddings out.h5` runs the expensive half once per
film and writes per-frame face detections and embeddings to HDF5
([`scripts/optimizer/SCHEMA.md`](https://REPOLINK/scripts/optimizer/SCHEMA.md)).
[`scripts/optimizer/replay.py`](https://REPOLINK/scripts/optimizer/replay.py)
then re-assembles the real C++ `face_tracker`, `identity_matcher`, and
`scene_tracker` nodes into a Python-driven KPN network and replays a
film's cached embeddings through them, varying `prob_threshold`,
`anneal_sec`, `extinction_sec`, and `expand_gallery` freely. No GPU
inference and no video decode happen during a replay; each one completes
in seconds. This is what makes a 512-evaluation differential-evolution
search per model, per gallery mode, per expansion setting, tractable, and
what made the full held-out validation across three models in this report
possible in one session rather than requiring three full re-encodes of the
benchmark set.
`optimize.py` runs `differential_evolution` over this replay function as its
objective, with DE-level parallelism (multiple candidate configs evaluated
concurrently, each spawning its own replay subprocesses) on top of it. The
practical ceiling on this machine's GPU was 8 concurrent replay processes;
9 silently degraded every score to 0.0% (well-formed output, wrong numbers,
not a crash), so `optimize.py` was run at `REPLAY_WORKERS=4 DE_WORKERS=2`.
## Search space
`popsize=10, maxiter=15` per combo (3 parameters, up to 512 evaluations,
usually stopping earlier on DE's convergence tolerance).
`anneal_sec`/`extinction_sec` bounds were widened from 1-30/1-15 to 1-60/1-60
partway through the sweep. r50's 4 combos finished before the widening and
used the old, narrower bounds; this is one more reason r50 is excluded from
direct comparison here.
## Training films and held-out films
9 films have dumped embeddings across all 4 models. 4 were used for
optimization:
- Café Society (62-cast)
- Lord of War (64-cast)
- Scarface (67-cast)
- Sound of Metal (14-cast)
5 were held out, never seen by any optimizer run:
- Benny & Joon
- Downton Abbey: A New Era
- Lovelace
- The Many Saints of Newark
- Valerian and the City of a Thousand Planets
## Gallery coverage per film
The gallery has reference embeddings for 2418 actors, but coverage of any
given film's credited cast varies widely. This was previously reported as
one flat number (67% of X-Ray cast lacking a reference embedding, averaged
across the whole benchmark); the per-film breakdown is:
| film | cast credited | in gallery | coverage |
|---|---|---|---|
| Lord of War | 64 | 13 | 20.3% |
| Scarface | 67 | 15 | 22.4% |
| The Many Saints of Newark | 48 | 13 | 27.1% |
| Café Society | 62 | 17 | 27.4% |
| Lovelace | 42 | 15 | 35.7% |
| Valerian and the City of a Thousand Planets | 36 | 13 | 36.1% |
| Benny & Joon | 23 | 12 | 52.2% |
| Downton Abbey: A New Era | 36 | 22 | 61.1% |
| Sound of Metal | 14 | 11 | 78.6% |
Two training films (Lord of War, Scarface) have the worst coverage in the
set, 20-22%. Their training-set F1 numbers below are partly capped by
missing references, not purely by model quality. Downton Abbey has 61%
coverage, the second-best in the benchmark, yet the worst held-out recall
of any film (39.4%, LVFace). Its recall problem is not primarily a coverage
problem; it is the extinction-bridging failure documented in the
[LVFace deep dive](lvface-deep-dive.md#mechanism-1-extinction-bridging).
Reproduce with `scripts/docs/gallery_coverage_per_film.py`.
## Training results, 3 models × 2 gallery modes × 2 expansion settings
Ranked by F1. misid = FPI_misid, the count of true wrong-actor
identifications (naming someone not in the film's cast at all), distinct
from FPI, which also includes in-cast timing slips.
Each combo's row is its best **full-coverage** evaluation: the highest-F1 DE
evaluation in which all 4 training films replayed without a timeout (see
[Dropped-film scoring](#a-scoring-bug-worth-recording-dropped-film-evaluations)
below for why this qualifier is load-bearing and not the same as `argmax F1`
over the raw sweep).
| combo | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| LVFace-B_Glint360K_restricted_exp | 78.3% | 91.0% | 68.9% | 42830 | 3782 | 60 | 19492 |
| LVFace-B_Glint360K_restricted_noexp | 76.7% | 91.5% | 66.2% | 41149 | 3400 | 59 | 21173 |
| arcface_w600k_mbf_restricted_exp | 76.2% | 90.0% | 66.2% | 64328 | 7480 | 0 | 33234 |
| arcface_r18_restricted_exp | 75.5% | 87.6% | 66.5% | 41399 | 5666 | 60 | 20923 |
| LVFace-B_Glint360K_full_exp | 75.3% | 89.7% | 65.4% | 47757 | 3407 | 232 | 26966 |
| arcface_w600k_mbf_restricted_noexp | 75.0% | 91.1% | 63.9% | 39752 | 3465 | 60 | 22570 |
| arcface_r18_restricted_noexp | 73.5% | 91.3% | 61.7% | 38299 | 3220 | 60 | 24023 |
| LVFace-B_Glint360K_full_noexp | 72.3% | 88.3% | 61.8% | 40363 | 3503 | 244 | 25850 |
| arcface_w600k_mbf_full_exp | 72.0% | 87.7% | 61.4% | 39875 | 3729 | 240 | 26338 |
| arcface_w600k_mbf_full_noexp | 71.0% | 93.2% | 57.9% | 41699 | 2472 | 56 | 33024 |
| arcface_r18_full_exp | 69.1% | 87.6% | 57.7% | 37342 | 3119 | 242 | 28871 |
| arcface_r18_full_noexp | 66.6% | 91.3% | 53.1% | 34314 | 2362 | 107 | 31899 |
![All combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
The two clearest patterns: every model's best-scoring combo uses the
restricted gallery, and LVFace leads within both gallery modes. `full_exp`
(the shipped combination) is the best-scoring option that uses only
features the running application currently supports; restriction is not
wired into the application yet (see
[Whole vs. cast-restricted gallery](gallery-scope.md)).
### A scoring bug worth recording: dropped-film evaluations
The numbers above are corrected ones. The raw `rep4_best_*.json` files, and an
earlier version of this table, reported a different `arcface_w600k_mbf_full_noexp`
row: **74.2% F1 at TPI 12645**, a third the TPI of every sibling combo. That was
not a better config; it was an artifact of how the optimizer aggregates.
`optimize.py` builds each candidate's score from only the films whose replay
subprocess returned (`per_film = [m for m in ex.map(_one, films) if m is not
None]`), then **averages** F1/precision/recall and **sums** TPI/FPI/misID over
just those survivors. When a film's replay times out (the sweep ran near the
8-process concurrency ceiling, so this happened intermittently), that film
silently drops from both. A candidate whose hardest film timed out is therefore
scored on an easier subset, and differential evolution, maximizing that score,
will happily converge onto exactly such a candidate. For `mbf_full_noexp` the
reported winner was one of 7 evaluations (out of 512) whose TPI had collapsed to
a partial-film subset; its median-coverage evaluations sit around 51686 TPI.
The fix here was to re-derive each combo's best row from its DE trajectory
(`experiments/trajectories/rep4_*.jsonl`), keeping only evaluations within 30% of
that combo's median TPI (full 4-film coverage) before taking the best F1. This
needs no re-running, the honest best configuration was already in the sweep,
just not the one `argmax F1` selected. Three combos moved: `mbf_full_noexp`
74.2% → **71.0%**, `LVFace_full_noexp` 72.4% → **72.3%** (and its misID, 0 → 244,
was itself a dropped-film artifact), `mbf_restricted_exp` 76.5% → **76.2%**. The
shipped LVFace `full_exp` winner was unaffected, its reported evaluation already
had full coverage (TPI 47757 ≈ median). `experiment_charts.py` applies the same
`clean_best` filter, so every figure on this page matches the corrected table.
The underlying `optimize.py` aggregation is also being fixed so a dropped-film
evaluation can never be selected as a winner again.
### Per-film training breakdown
The 75.3% LVFace training figure is a macro average across 4 films, not a
uniform result:
| film | LVFace F1 | mbf F1 | r18 F1 | best model |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
LVFace does not win every training film. mbf scores higher on Lord of War
(77.2% vs 75.6%). LVFace's own training-film range is 68.1% to 78.8%, a
10.7pp spread, smaller than the 37pp spread seen on held-out films but real.
Reproduce with `scripts/docs/run_holdout_all_models.py --films training`.
## Held-out validation, all 3 models
The training matrix above is training-set fit. Each model's own tuned
`full_exp` config was replayed against the 5 held-out films, scored the
same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on every one of the 5 held-out films; the ranking
never flips. Total misIDs across the 5 films: LVFace 1032, mbf 2197, r18
1224. LVFace has less than half mbf's misID count while also scoring
higher on every film. This directly confirms the model choice out of
sample; it is not inferred from the training numbers alone. See the
[LVFace deep dive](lvface-deep-dive.md) for frame-level detail on where and
why LVFace still fails on the two worst films. Reproduce with
`scripts/docs/run_holdout_all_models.py`.
## Two effects in isolation: gallery scope and pose expansion
Averaging across the 3 compared models (r50 excluded) isolates each variable
from model choice.
**Gallery scope**, averaged over both expansion settings and all 3 models
(6 evaluations per row):
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once. This is not a precision/recall
trade: +4.8pp F1, +6.0pp recall, and roughly a quarter the misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections. Restriction is currently an offline optimizer technique, not a
runtime feature of the application; see
[Whole vs. cast-restricted gallery](gallery-scope.md) for what building it
into the application would require.
**Pose expansion** (promoting a confidently identified track's novel-pose
views into a per-film gallery annex,
[`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)):
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion is a clean win: +1.6pp F1, +3.3pp recall,
misID drops. The annex only competes against the film's own roughly 15-actor
cast, so a new pose of a known actor is unlikely to be confused with someone
else. In full mode, expansion buys +2.1pp F1 and +3.9pp recall but at a real
cost: misID rises from 407 to 714 as the same new-pose view now competes
against the full 2418-actor gallery, where a confidently learned pose is more
likely to match the wrong person. On the full gallery it is a recall-vs-misID
trade, not a free gain. This training-set effect
did not reproduce on held-out data; see
[Does pose expansion help?](pose-expansion.md) for the full held-out test
and the two methodology bugs caught while checking it.
## Calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | sim) = σ(a·sim + b)`,
stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This measures discriminative power independent of whatever
`prob_threshold` a given run used:
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
LVFace has the steepest curve (`a=17.7` vs 15.3-16.2 for the ArcFace
variants) and the lowest P=0.5 decision boundary (similarity 0.23 vs
0.27-0.31), separating same-actor from different-actor pairs more
confidently at a lower similarity than any ArcFace variant tested,
including r50. Generated by
[`scripts/docs/calibration_chart.py`](https://REPOLINK/scripts/docs/calibration_chart.py).
## Extinction and anneal window search
Every one of the 512 DE evaluations for the winning LVFace `full_exp`
combo, plotted over the `prob_threshold` × `extinction_sec` plane:
![DE search landscape: 512 evaluations over prob_threshold × extinction_sec](assets/images/de_search_landscape.png)
Nearly everything scoring well sits at `extinction_sec` above 50, across a
wide range of thresholds. Short extinction windows are uniformly weaker:
under a strict threshold, there is no good configuration in that region of
the search space. The optimizer converged with `anneal_sec=59.2,
extinction_sec=59.2`, about 99% of the widened 60s bound, which raises an
open question not resolved in this round: does performance keep improving
past 60s, or does it plateau there. Not chased further this pass.
## Caveats
- r50's 4 combos used the older, narrower search bounds (1-30/1-15 instead
of 1-60/1-60) and are further confounded by its thinner gallery. Excluded
from all comparisons above except calibration.
- The shipped defaults use `full_exp` (75.3% training F1), not the
higher-scoring `restricted_exp` (78.3%), because cast restriction is not
a runtime feature of the application yet.
- `expand_gallery` is mode-dependent, not a free win. Averaged across models
on the full gallery it trades misIDs for recall (see the pose-expansion
table). For LVFace specifically, though, `full_exp` beats `full_noexp` on
every axis at once (F1 75.3 vs 72.3, precision 89.7 vs 88.3, recall 65.4 vs
61.8, misID 232 vs 244), so the shipped `full_exp` is a clean choice for
this model, not an F1-vs-safety trade. (An earlier version of this page
reported `full_noexp` at 72.4% with zero misIDs and higher precision, which
made it look like the safer option; that was the dropped-film artifact
described above, not a real property of the config.)
- Switching the default model is an operational change: any gallery built
from a different model's embeddings must be rebuilt before the new
default takes effect.
## Reproduce
```bash
# 4-film training matrix, all 4 models × 2 gallery modes × 2 expansion settings
bash experiments/run_rep4_subprocess.sh
# single combo
SAE_EXPAND=1 REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \
--manifest experiments/manifests/rep4_LVFace-B_Glint360K_full.json \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \
--params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \
--popsize 10 --maxiter 15 --trajectory traj.jsonl --out best.json
# held-out validation, all 3 models, 5 films
python3 scripts/docs/run_holdout_all_models.py --out docs_data/holdout_all_models.json
# per-film training breakdown, all 3 models, 4 films
python3 scripts/docs/run_holdout_all_models.py --films training --out docs_data/training_per_film.json
# gallery coverage per film
python3 scripts/docs/gallery_coverage_per_film.py --out docs_data/gallery_coverage_per_film.json
# regenerate this page's charts from experiments/ artifacts
python3 scripts/docs/experiment_charts.py --out-dir docs/assets/images
# one frame per distinct out-of-cast name across all 9 films (used in the deep dive)
python3 scripts/docs/first_fpi_frames.py
```
See also the session log
[`experiments/SESSION_STATE.md`](https://REPOLINK/experiments/SESSION_STATE.md).
-410
View File
@@ -1,410 +0,0 @@
# Implementation plan — per requirement
One entry per requirement that needs work. Requirements marked `Done` in
[`requirements.md`](requirements.md) are omitted.
**Ordering is derived from dependencies, not assigned to phases.** Each entry
lists what it depends on; anything with no unmet dependency is startable. This
replaces the earlier phase-based plan, which encoded ordering assumptions that
stopped being true as the design changed.
Verification for each requirement is specified in
[`requirements.md`](requirements.md) — this document covers *how to build it*,
not how to prove it.
---
## Startable now (no unmet dependencies)
`GR-004` · `IR-004` · `IR-005` · `IR-007` · `IR-008` · `VR-005` · `AR-011` ·
`AR-023` extension · tooling port
These touch disjoint files and can proceed concurrently.
## Blocked on the registry
Everything in `AR-007``AR-022` depends on `AR-012`/`AR-013` landing first,
because they all read or write track state. **This group is one coherent
refactor, not parallel work** — splitting it across concurrent efforts produces
incompatible designs in the same files.
---
# Algorithm
## AR-012, AR-013 — TrackRegistry (the spine)
**Depends on:** nothing. **Blocks:** AR-007, AR-008, AR-014 … AR-022.
Everything else in Part A waits on this, so it goes first.
### Ownership: a shared resource, not a node
The registry is **external to the dataflow network**, created in `main` and
handed to each node that needs it as `std::shared_ptr<TrackRegistry>`. Lifetime
is guaranteed by refcount rather than by the "object must outlive the node"
convention, so no ordering assumption exists between network teardown and
registry destruction.
This is idiomatic here: node functors are already constructed outside the network
and passed by reference (`main.cpp:186-207`), and KPN provides `SharedResource<T>`
for state shared across nodes (KPN SPEC §163, §445).
Not a node, because ownership is not a stage in the stream — it is state several
stages read and write, whose final answer is only known when a track dies.
Not inside `TrackGallery`, because that would couple presence to `expand_gallery`,
a switchable feature.
**The registry *is* the tracker's state.** `FaceTrackerFunc` does not keep its own
`tracks_`/`inactive_` maps and mirror them in — it operates on the registry
directly. Two parallel copies could disagree, and every divergence would surface
as wrong presence windows, silently.
### Per-track state
```
Track
first_seen : double set once, at creation
last_seen : optional<double> UNSET while on screen; set to the last
on-screen timestamp when the face is lost
actor : optional<int> set when a posterior crosses the threshold
belief : {actor_idx -> accumulated_logodds} Bayesian, not a tally
embedding : Embedding running directional mean, for association
```
`last_seen` carries the entire liveness state. Unset = on screen; set = went off
at T. No separate missing-frames counter, no expired flag — the optional *is* the
state machine, and it subsumes the current two-pool split (`tracks_` = unset,
`inactive_` = set).
### Lifecycle
```
face detected, no match → new track, first_seen = t, last_seen = unset
actor identified → update belief; set actor when threshold crossed
face lost → last_seen = t_last_on_screen (stays revivable)
face seen again, embedding match → last_seen = unset (same track continues)
tick(t), t - last_seen > timeout → emit to aggregator, DELETE the entry
```
A presence window is `[first_seen, last_seen]`. Nothing else.
**Interior gaps are claimed; the trailing cool-down is not.** A face lost at t₁
and re-acquired at t₂ within the timeout never closed its track, so the actor is
present across `[t₁, t₂]` — correct, since someone briefly occluded or off-camera
has not left the scene. But a track that dies ends at `last_seen`, not at the
moment of death. That asymmetry is what removes the old `extinction_sec`
over-claim.
**Reaping is a handoff, not a deletion into a holding pen.** The dead track goes
to the result aggregator immediately and the registry drops it, so the registry
holds only live tracks and its size is bounded by concurrent on-screen faces.
### Interface
```
TrackRegistry
tick(timestamp) ← FaceTrackerFunc, every frame
candidates() -> span<Track&> → all live tracks
create(timestamp, embedding) -> track_id
mark_seen(track_id, timestamp, embedding) → updates mean, clears last_seen
mark_lost(track_id, last_on_screen_timestamp)
on_vote(track_id, actor_idx, posterior) ← IdentityMatcherFunc
owner(track_id) -> optional<actor_idx> → TrackGallery
on_track_dead : callback(DeadTrack) → ResultSinkFunc
flush() ← at EOF
```
`candidates()` returns **one pool**; `last_seen` tells the caller whether IoU
applies. There is no separate revival path — matching a dormant track is ordinary
inter-frame association.
`tick()` advances the clock so dead tracks are reaped independently of detection
activity; without it a track only dies when some *other* face happens to appear.
### Locking
The tracker mutates registry state across a frame's association pass, so that
pass holds the lock for its duration (a `frame_scope()` handle). Every other
caller's operations must be individually atomic. A single `std::mutex` over the
whole registry is the right start — contention is a few small updates per frame
against per-frame work measured in GPU milliseconds.
Two cases constrain the API:
- `owner()` is a **read-modify-read** in disguise: `TrackGallery` calls it while
`IdentityMatcher` may be voting on the same track. Tally and verdict must be
read under one lock as a snapshot, or a track can be both unowned and owned
within a single promotion decision.
- `on_vote()` arrives downstream of the tracker's `tick()` for the same frame, so
a vote may land after the clock moved on. **Rule: a vote for a known track
always lands on its tally, regardless of clock.** Only reaping is clock-driven.
A vote for an already-reaped track is dropped and **counted** — a nonzero count
means the timeout is shorter than the matcher's lag.
`on_track_dead` fires from inside `tick()` while the frame lock is held, so the
callback must not re-enter the registry. Keep it to a push onto the aggregator's
storage.
## AR-016 — EOF flush
**Depends on:** AR-012.
`flush()` emits every still-live track through the same callback, closing at
`last_seen` if set and the final tick timestamp otherwise. Idempotent, leaving the
registry empty; the sink's `written_.exchange(true)` guard
(`result_sink_node.hpp:66`) shows the shape.
Must run on **every** termination path that produces output. Not SIGTERM during
opportunistic runs (DP-004) — those push no partial result, so there is nothing
to flush.
Without it a film ending mid-shot silently drops its closing cast, which looks
like a recognition miss rather than a bookkeeping bug.
## AR-014, AR-015 — Contradiction rules
**Depends on:** AR-012, AR-025.
| Condition | Meaning | Action |
|---|---|---|
| Belief on one track swaps A → B | `track_id` carried across a viewpoint change onto a different person | Close at `last_seen`, open a new track for B at the swap frame |
| Two **live** tracks owned by one actor | One person split in two, or an identity attached to the wrong track | Treat as a detected cut: reset affected state, re-associate on embedding |
The second makes identity a **third cut detector**, independent of histogram and
TransNetV2, firing where those failed. Detect it via a reverse index
`actor_idx → live track_ids`, so the condition is caught on the update that
causes it rather than by scanning.
Both counted and reported — the rates measure how often tracking is silently
wrong, which nothing currently reveals.
## AR-007, AR-008 — Tracker on one pool
**Depends on:** AR-012, AR-024.
`FaceTrackerFunc` is constructed with the registry and uses it as state; its
`tracks_`/`inactive_` maps and the cross-cut revival branch collapse into one
pool keyed on `last_seen`. Per frame: `tick()`, association over `candidates()`,
then `create`/`mark_seen`/`mark_lost`.
`track_alpha` becomes **frame-dependent** — normal frames use the tuned blend,
frames flagged `is_cut`/`is_scene_boundary` drop toward embedding-only.
## AR-024 — Probability space everywhere
**Depends on:** AR-023. **Blocks:** AR-007, AR-018, AR-021, AR-025.
Cuts across tracker, matcher and expansion, so it lands with the registry work
rather than after it. Retires `track_max_embed_dist`, `cut_revive_sim`,
`expand_novelty_sim`, `expand_track_spread_max`.
Enforcement is a **static grep check** for bare cosine outside a tagged
`EXCEPTION` — a unit test cannot prove absence across a codebase.
## AR-025 — Bayesian accumulation
**Depends on:** AR-023, AR-024.
Log-odds per candidate actor, added per frame. `on_vote()` is an *update*, not an
increment.
**The independence problem must be handled explicitly.** Consecutive frames are
highly correlated; naive accumulation drives the posterior to certainty on what is
effectively one observation. Preferred mitigation: update only on sufficiently
novel observations, reusing the diversity buffer's existing judgement rather than
inventing a second one. The registry should receive already-discounted evidence.
## AR-017 — Claims carry belief and route
**Depends on:** AR-012, AR-025. `DeadTrack` carries posterior plus how it was
identified (live / deferred / pooled).
## AR-018 … AR-021 — Expansion, deferred pass, clustering
**Depends on:** AR-012, AR-024, AR-026.
Ordering within the group: AR-018 (banded store) → AR-019 (annex) → AR-020 (TBI
queue + deferred pass) → AR-021 (clustering).
AR-021 needs the temporal cannot-link constraint from track extents, so it cannot
start before AR-012. The annex must be a **contiguous matrix** with promotions
appended (AR-026), not a list.
**Output timing changes:** the sink can no longer finalise at EOF — the deferred
pass runs after and may add windows (IR-003).
## AR-022 — Unidentified capture
**Depends on:** AR-020. Unidentified = TBI entries surviving the deferred pass.
Context crops opt-in behind `--dump-unidentified-crops`.
## AR-001 … AR-004 — Detection and backpressure
**Depends on:** nothing (AR-002, AR-011); AR-004 blocks AR-003.
- **AR-002**`min_face_px` stays **40** (VR-013 measured it end to end) but must
be expressed in original resolution rather than decoded-frame space. The value
is already right in `config.hpp`; the change is the coordinate space.
- **AR-011** — feed TransNetV2 at native rate; derive the dedup window from
source fps rather than the hardcoded `0.04 s`.
- **AR-004** — backpressure. `kMaxFaces` (`identity_matcher_node.hpp:133`)
currently **throws**; channel capacities of 16 (`main.cpp:204-207`) were sized
against ≤10 faces/frame. Must block on bytes in flight, not item counts.
- **AR-003** — remove `max_faces`. **Gated on AR-004**, not a follow-up to it.
## AR-026, AR-027 — GEMM and scale
**Depends on:** nothing to start. The annex CPU loop
(`identity_matcher_node.hpp:159-162`) moves into the GEMM path.
---
# Gallery
## GR-004 — Model binding — **DONE**
**Depended on:** nothing. Landed before any measurement work, as intended.
Stamp = model basename + SHA-256 of the ONNX, written as the `/embedder` group at
build time (`gallery_builder.cpp`, `sae_gallery.save_gallery_hdf5`) and verified
at load in `scene_analyze`, `scene_preview`, the `sae_kpn` matcher binding,
`replay.py`, `optimize.py` and `movienet_eval.py`. Mismatch is a hard error naming
both sides, with no bypass. Embedding dumps carry the same stamp, since a replay
has no live embedder to check against.
Unstamped legacy galleries **warn loudly and proceed** rather than failing:
unknown is not known-bad, and hard-failing every pre-existing gallery would turn
the check into something people disable. `--require-gallery-stamp` /
`SAE_REQUIRE_GALLERY_STAMP=1` promotes that to a hard error — measurement runs
should set it. `scripts/stamp_gallery.py` re-binds an existing gallery without
re-embedding, so the warning state is cheap to leave.
Cross-model similarities are meaningless but *look* plausible — this fails
silently and expensively, and it would corrupt every measurement taken during the
rest of this work.
## GR-003 — Coverage reporting
**Depends on:** nothing. Surface what calibration already computes and discards
(`kHistBins = 200`): zero-image actors, under-referenced actors, dedup counts,
and the intra/inter PDFs.
## GR-006 … GR-008 — Provenance tiers
**Depends on:** AR-019. Tier per embedding (baked / harvested / confirmed);
harvested persisted but flagged; bell-curve outlier check
(`EXCEPTION: AR-024`).
---
# Integration
## IR-004, IR-005, IR-007, IR-008 — Audio signature
**Depends on:** nothing. **Fully independent — no existing pipeline file is
touched.** Best candidate for concurrent work.
Implement server spec §3 exactly. Audio decode is a second stream from the
already-linked FFmpeg. Media < 120 s: no signature, no offset. Emit and honour
the `v1:` prefix.
The golden-vector fixture is shared with the plugin repo and runs on CPU, so the
one place two implementations must agree bit-for-bit is verifiable in CI.
## IR-001 … IR-003 — Truth file
**Depends on:** AR-017 (belief), AR-020 (output timing).
Windows carry belief and route; `extraction.*` gains `extinction_sec` and
`gallery_scope`; `anneal_sec` removed. All breaking → **one** coordinated
`schema_version` bump with IR-004 (SR-003).
---
# Validation
## VR-005 — Minimum face size study
**Depends on:** nothing. Standalone Python, no C++ contact. **Done** — knee at
2432 px. It measures the embedder with alignment held perfect, so it bounds the
answer from below rather than setting it; AR-002's floor comes from **VR-013**,
which sweeps input resolution end to end and lands at 40 px.
## VR-013 — Cross-source identification probe
**Depends on:** `sae_embed` exposing `detect()`, `align_face()`, `embed_crop()`
and the gallery calibration — it drives the shipped C++ rather than reimplementing
it, which is what VR-005 could not do.
Gallery from one recording, probes from another, sweeping the probe's **input
resolution before the detector**, so detection and landmark regression degrade
with the frame. `experiments/xsource/`.
**Findings.** Holding 90% of the plateau needs ~50 px end to end against VR-005's
~22 px; `min_face_px` 40 is right and 32 would admit faces in the falling region.
FPI is 0.0% at every scale — resolution loss goes entirely to TBI, never to a
wrong name. The ceiling is **cross-view, not resolution**: everyone matches
themselves within a recording (0.550.85) and collapses across two (0.140.45),
and only the subject with frontal *gallery* references identified reliably — so
the lever is gallery pose coverage (`docs/pose-expansion.md`), not a better
landmark model. Averaging SCRFD's NMS-discarded landmark estimates lifts
cross-clip TPI 41% → 49% for one forward pass.
**Open.** Four identities and one shoot, so the shape is the result and the
absolute rates are not. Both clips hold all four people, so there is no
out-of-gallery class and the 10×-weighted out-of-cast misID is untested — holding
one identity out of the gallery would fix that.
## VR-014 — Audio-signature offset recovery
**Depends on:** `sae_audio` exposing `compute_signature()` and
`signature_from_mono()` — it drives the shipped C++, as VR-013 does, so the
thing measured is the thing that ships.
`scripts/validation/test_audio_offset.py` over
`tests/fixtures/audio/bali_offset_200s.flac`: 200 s of public-domain film audio
(the same Road to Bali clips the replay fixtures use), long enough for a 120 s
window to slide past the ±600-frame search cap. The slide itself is numpy here
on purpose — matching belongs to the consumer, so writing it out keeps this a
test of the signature rather than of somebody's matcher.
**Findings.** Alignment is a solved problem here: the offset is the nearest frame
in every in-cap trial, worst error **46 ms against a 500 ms budget**, and 46 ms is
the quantisation floor — offsets are whole 92.88 ms frames, so no correct answer
can be worse. The `runtime/2` anchor's factor of two holds through real trimmed
files, and out-of-cap offsets and unrelated content are both declined.
**The score is where the slack is, and it costs a tier rather than accuracy.** It
tracks sub-frame misalignment — 0.940.99 near a frame boundary, 0.690.73 at
half a frame — so two thirds of correct alignments miss the server's 0.85 `audio`
threshold and land in `loose`. UT-108 measures the fix rather than proposing one:
±1 frame of slack in the score returns all 40 to `audio` (min 0.906) with false
matches unmoved at 0.120.16, costing 81 ms of the budget. See
[`SPEC.md`](SPEC.md) IR-004 — the score is normative in the server spec, so the
change is theirs to make.
**Open.** One source, one language, one era of recording. The shape (offset exact,
score set by sub-frame phase) should hold generally, but the absolute scores are
this fixture's.
## VR-001 — Dump audit
**Depends on:** nothing. Read-only investigation: confirm the HDF5 dump preserves
everything needed to reconstruct tracks deterministically, including the
park/revive path. **Prerequisite for the CI strategy**, since T2 replay is how
most of AR-007 … AR-022 is verified.
## VR-006 … VR-009
**Depends on:** their subjects landing. VR-009 (posterior calibration holds)
depends on AR-025 and is what stops the Bayesian accumulation being decoration.
---
# Withdrawn from the old plan
The phase structure, the `--presence-mode {frame,track}` flag, and "Phase 2 —
retune `anneal_sec`/`extinction_sec`". Those constants are withdrawn rather than
retuned; comparison against old behaviour uses recorded reference output instead
of a second live code path.
-110
View File
@@ -1,110 +0,0 @@
# Pose expansion: does promoting new poses mid-film help?
`expand_gallery`
([`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp))
promotes a confidently identified track's novel-pose reference views into a
per-film, in-memory gallery annex. The idea: once the pipeline is confident
about an identity, a pose it has not seen before (turned head, different
lighting) becomes an extra reference for recognizing that actor again later
in the same film, without touching the baked gallery.
## Training-set signal
Averaged across the 3 compared models (r50 excluded), on the 4 films used
for optimization. These are the corrected, full-coverage figures, see the
[dropped-film note](model-bakeoff.md#a-scoring-bug-worth-recording-dropped-film-evaluations)
in the experiment log for why an earlier version of this table overstated the
full-mode misID jump (209 → 864) that was itself partly a truncation artifact:
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion looks like a clean win: +1.6pp F1, +3.3pp
recall, lower misID. In full mode it looks like a recall-for-misID trade:
+2.1pp F1, +3.9pp recall, but misID rises from 407 to 714. See
[the full experiment log](model-bakeoff.md) for the per-model breakdown.
This asymmetry motivated the question below: does turning expansion on
change what gets recognized frame by frame, or is the aggregate F1 shift
coming from something else.
## Held-out test
Same model, same tuned config, `expand_gallery` toggled on vs. off, nothing
else changed, full gallery mode, per-second scoring against X-Ray. This
isolates expansion from every other variable that differs between the
training-set rows above.
LVFace-B Glint360K, all 5 held-out films:
| film | F1 (exp) | F1 (noexp) | TPI delta | FN delta |
|---|---|---|---|---|
| Benny & Joon | 83.0% | 83.0% | -2 | +2 |
| Downton Abbey: A New Era | 56.1% | 56.2% | -7 | +7 |
| Lovelace | 77.5% | 77.4% | +33 | -33 |
| The Many Saints of Newark | 46.3% | 46.3% | +2 | -2 |
| Valerian and the City of a Thousand Planets | 74.1% | 74.1% | +2 | -2 |
ArcFace R18, Benny & Joon, r18's own tuned config: F1 77.1% for both, TPI
and FN identical, FPI differs by 2.
Every film, both models tested: F1 differs by 0.1-0.2pp, TPI/FN swings are
in the tens out of tens of thousands. This is noise, not a signal.
Expansion made no measurable difference to per-second on-screen
identification on any held-out film tested.
## Two methodology bugs caught during this check
Getting to the table above required catching two wrong turns, both worth
recording because they are exactly the kind of error that produces a false
positive "expansion helped" finding.
1. **Timeout truncation.** The first Downton Abbey `exp` replay was cut off
by a 60-second subprocess timeout at about 76% through the film (5589 of
7368 expected seconds). This silent data loss produced a large,
convincing-looking TPI gap (47938 vs 52032) purely because one run was
missing a quarter of the film. Caught by comparing `n_seconds` between
runs before trusting any score delta; fixed by re-running with a longer
timeout.
2. **Bbox-matching bug.** An early per-second raw-annotation diff matched
each `exp` detection to the first `noexp` detection with IoU above 0.5,
not the best-overlapping one. With 3 faces close together in frame, this
produced spurious disagreements (for example "exp says Aidan Quinn,
noexp says Johnny Depp" at the same second) that vanished once the match
used the best-IoU candidate instead of the first one. Both configs had
actually output the same three names at the same three boxes.
Both bugs independently pointed toward "expansion is doing something," and
both were artifacts of the comparison harness, not the pipeline. Before
trusting a dramatic before/after diff, check that both runs cover the same
seconds and that entities are matched by best overlap, not first found.
## Conclusion
The training-set aggregate effect, particularly the full-mode misID
increase, does not reproduce on held-out data. At minimum it
is far smaller than the training-set numbers suggested; it may be sampling
variation from only 4 training films rather than a generalizable
mechanism. Note the same *class* of harness bug appears twice in this
investigation, the timeout truncation in bug #1 above, and the dropped-film
aggregation that inflated the raw training-set misID figures. Both make an
inert config look consequential; both are reasons to distrust a dramatic
training-set delta until it survives on held-out films, which this one did
not. This does not mean `expand_gallery` never does anything: the
mechanism is real, and
[`track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)'s
promotion logging confirms tracks get confirmed and views get promoted
into the annex on every film tested. It means whatever effect expansion
has on final per-second identification was too small to detect against 5
held-out films with this scoring method. A cleaner test would need either
more held-out films or a metric that can see the annex's direct
contribution, such as tagging which reference embedding won each match;
neither was in scope for this pass.
Do not treat the training-set exp/noexp numbers in
[the full experiment log](model-bakeoff.md) as proof that expansion changes
real-world behavior in either direction. On the evidence gathered so far,
it does not move the needle enough to see.
-308
View File
@@ -1,308 +0,0 @@
# Quality knee: what does a blurred or small face cost, and can a measure predict it?
VR-012. Companion to the minimum-face-size studies VR-005 and VR-013 (see the
[requirement register](requirements.md)), which located the size floor at 40 px;
this asks the same question for **sharpness**, and asks whether any cheap
measure taken on the aligned crop can be acted on at inference.
Run by
[`scripts/validation/quality_knee.py`](https://REPOLINK/scripts/validation/quality_knee.py)
through the `sae_embed` bindings — detection, the ArcFace warp, the embedder,
the five candidate measures and the Platt calibration are all the shipped C++.
## Protocol
1670 gallery actors with 3 or more mugshots (of 2456 total), one image held out
per actor as a probe, the remaining 10326 embeddings staying in the gallery at
native resolution. Only the probe degrades — reference mugshots are clean and
the face coming out of the video is not.
Each probe passes through a **joint grid**: downscale to *S*×*S* and back to
112 (the sampling loss), then blur at level *L* in canonical pixels. Three blur
families, 36 cells each, 60120 probe-cell records per family:
| family | models | parameter |
|---|---|---|
| Gaussian | soft focus, a generic stand-in | sigma 0 … 3 |
| **Disc** | **real optical defocus** — the circle of confusion | radius 0 … 6 |
| Motion | camera pan or moving subject | length 0 … 21 px |
The three are not interchangeable, and sweeping only the first was the original
design error — one that would have produced a wrong answer, not merely an
incomplete one (Result 3). A defocused lens spreads a point into a **uniform
disc**, whose transfer function is a jinc — `2·J1(x)/x` — that crosses zero and
goes negative, annihilating whole frequency bands and returning the ones beyond
each zero phase-reversed. A Gaussian MTF is strictly positive and monotone and
does neither. More practically: defocus and motion are how a face ends up
**large and useless**, while Gaussian blur as swept here mostly co-occurs with
small faces. That difference decides whether sharpness carries anything the size
filter does not.
Families are compared at matched **per-axis PSF standard deviation** (σ for a
Gaussian, R/2 for a disc, L/√12 for a linear smear), never at equal raw
parameter, which would compare different amounts of damage.
Identification is the pipeline's own decision: per-actor best-of-N cosine →
Platt sigmoid → accept above `prob_threshold` 0.754. Never a raw cosine
(AR-024).
## Result 1 — sharpness is not a sufficient statistic
Sorting the 36 Gaussian cells by `hf_energy_ratio`, the six sigma-3 cells land
at effectively identical measured sharpness:
| size | sigma | hf_energy_ratio | TPI |
|---|---|---|---|
| 16 | 3 | 0.0003 | **15.3%** |
| 24 | 3 | 0.0003 | 63.2% |
| 32 | 3 | 0.0003 | 79.4% |
| 48 | 3 | 0.0003 | 86.6% |
| 64 | 3 | 0.0004 | 88.4% |
| 112 | 3 | 0.0005 | **91.0%** |
Same measured sharpness, a **76-point spread in identification**. It inverts
too: 16 px unblurred measures 0.0033 and scores 23.5%, while 48 px at sigma 2
measures *lower* at 0.0021 and scores 96.6%.
A canonical-frame sharpness scalar cannot separate *attenuated* high
frequencies from *destroyed* spatial sampling. Blur suppresses the high band
while preserving mid-frequency facial geometry exactly; downsampling to 16 px
destroys that geometry outright. Both look alike to any measure keyed on
high-frequency energy.
This is the measured basis for AR-028's rule that the axes are **kept separate
and not collapsed into one scalar**, and it settles the double-counting
question: size and sharpness are not redundant, and neither substitutes for the
other.
## Result 2 — blur is a cliff, and it breaks confidence, not identity
TPI % by size (rows) against Gaussian sigma (columns):
| size | 0 | 0.5 | 1 | 1.5 | 2 | 3 |
|---|---|---|---|---|---|---|
| 16 | 23.5 | 24.0 | 25.0 | 24.6 | 23.9 | 15.3 |
| 24 | 85.7 | 85.1 | 85.6 | 85.9 | 82.6 | 63.2 |
| 32 | 95.9 | 95.9 | 96.0 | 95.5 | 93.7 | 79.4 |
| 48 | 98.7 | 98.7 | 98.4 | 98.1 | 96.6 | 86.6 |
| 64 | 98.6 | 98.6 | 98.8 | 98.4 | 97.5 | 88.4 |
| 112 | 98.9 | 98.9 | 98.8 | 98.6 | 98.0 | 91.0 |
Three regimes: **sigma ≤ 1.5 is free** (every cell moves under 1.5 points, sign
flipping at random — at 16 px it slightly *improves*, smoothing upscale
artifacts); sigma 2 costs 13 points; the 2→3 step costs 719. A smooth
discount curve is therefore the wrong shape — the response is flat, then falls
off a cliff.
**The cost peaks at the size knee, not at full resolution.** Sigma 3 costs
22.5 points at 24 px but only 7.9 at 112 px and 8.3 at 16 px. Blur has no
intrinsic cost; it costs in proportion to how close the observation already sits
to the decision boundary. At 112 px there is margin to spare, at 16 px the probe
is already below threshold, and at 24 px it sits exactly on the knee.
**What blur destroys is confidence, not ranking.** Rank-1 barely moves: 99.3% →
99.2% at 112 px across the whole sigma range. The extreme case is 16 px at sigma
3, where rank-1 is **80.2%** while TPI is **15.3%** — 65 points of probes have
the correct actor ranked first and are rejected anyway for falling under the
probability threshold.
That is why **FPI never left 0.1% in any of the 108 cells across all three
families**. Degradation produces TBI, never a wrong name. The calibration
degrades gracefully, which is what SR-002 needs.
## Result 3 — the blur *family* matters more than the blur *amount*
Comparing families by their raw parameter is meaningless — sigma, radius and
length are different units. They are matched here by the **per-axis standard
deviation of the PSF**, which puts them on one scale:
| family | per-axis σ | level giving σ = 3 px |
|---|---|---|
| Gaussian σ | σ | 3 |
| Disc radius R | R/2 | 6 |
| Motion length L | L/√12 | 10.4 |
For reference the ArcFace template places the eyes 35.2 canonical px apart, so
σ = 3 px is 9% of the inter-ocular distance.
TPI at matched severity, interpolated within each family:
| size | σ=3 Gaussian | σ=3 Motion | σ=3 **Defocus** | defocus penalty |
|---|---|---|---|---|
| 16 | 15.3 | 15.1 | 11.0 | +4.3 |
| 24 | 63.2 | 61.1 | 41.4 | +21.9 |
| 32 | 79.4 | 76.1 | 50.4 | +29.0 |
| 48 | 86.6 | 80.9 | 52.6 | +34.0 |
| 64 | 88.4 | 81.7 | 51.0 | +37.4 |
| 112 | 91.0 | 82.1 | **46.9** | **+44.0** |
**Optical defocus is up to 44 points more destructive than a Gaussian of
identical spread**, and the ordering is defocus ≫ motion > Gaussian throughout.
At σ=1 the three families are indistinguishable, and at σ=2 they differ by under
5 points; the divergence appears only when both the blur is severe *and* the face
is large.
That pattern is physically consistent. At 16 px the resampling has already
removed the high frequencies, so the PSF's shape has nothing left to act on and
all three agree. At 112 px the full spectrum is present and shape decides: a
Gaussian MTF rolls off gently and always leaves *some* energy at every
frequency, so the embedder receives a merely attenuated signal, while a disc MTF
is a jinc that **hits exact zeros** — whole frequency bands annihilated rather
than attenuated, with the bands beyond each zero returning phase-reversed.
Motion sits between them because it ruins one axis and leaves the perpendicular
one untouched.
**The methodological consequence is the important one.** This study originally
swept Gaussian blur alone and concluded blur was a minor effect. On the family
that actually occurs in film, the same nominal severity costs **53% error
instead of 9%** at full resolution. A threshold set from the Gaussian arm would
have been wrong by a factor of five in error rate, and the axis would probably
have been dropped as not worth its cost.
**Defocus is also the case a size gate cannot catch.** Every one of those 112 px
faces is large and confidently detected, and sails through AR-002 untouched.
That, not the Gaussian result, is what justifies a sharpness axis existing at
all.
## Result 4 — variance of Laplacian is anti-predictive at fixed degradation
Pooled across all cells, every candidate scores AUC 0.760.80 for predicting
correct identification, with textbook `var_laplacian` top. That number is close
to worthless: it rewards a measure for detecting *how degraded the crop is*,
which all five do. The question a per-observation discount needs is whether, at
a **fixed** degradation, the measure predicts which faces fail:
| measure | Gaussian | Defocus | Motion |
|---|---|---|---|
| `hf_energy_ratio` | **0.530** | **0.521** | **0.557** |
| `norm_var_laplacian` | 0.520 | 0.507 | 0.539 |
| `dir_min_tenengrad` | 0.524 | 0.512 | 0.506 |
| `tenengrad` | 0.433 | 0.437 | 0.457 |
| `var_laplacian` | 0.423 | 0.422 | 0.473 |
Best is 0.557 — barely above chance, and `hf_energy_ratio` wins on all three
families. `var_laplacian` is anti-predictive on all three too, so that finding
does not depend on the blur model.
**The two metrics measure different jobs, and the candidates split along that
line.** On the motion arm `dir_min_tenengrad` has the best *pooled* AUC by a
wide margin — **0.854** against 0.792 for the next — exactly as its synthetic
directional-blur ladder predicted, yet its within-cell AUC there is 0.506. It is
an excellent detector of *how badly smeared a crop is* and no guide at all to
*which face will be recognised*. Pooled AUC is the right metric for a
gross-degradation flag; within-cell AUC is the right one for a per-observation
discount; a measure can be strong at one and useless at the other.
Deciles within the 16 px Gaussian cell, where 1277 failures give the test real
power:
| `var_laplacian` decile | TPI |
|---|---|
| 0.000710.00192 (blurriest) | **37.1%** |
| 0.002420.00278 | 22.8% |
| 0.003970.00447 | 25.7% |
| 0.006250.01445 (sharpest) | **14.4%** |
The faces the measure calls sharpest are **2.6x less identifiable** than those
it calls blurriest, monotone across ten bins of 167. Within a cell every crop
received identical degradation, so the residual variance is *native contrast*,
not native detail — and hard shadows, high-contrast lighting, sharpening halos
and JPEG ringing all raise Laplacian variance while making a face harder to
match. The measure reads photographic style and encoding artifacts and calls
them sharpness.
`hf_energy_ratio` is the only candidate with a correctly-signed within-cell
trend (16.2% → 35.3% across the same deciles), being a pure ratio in which the
contrast factor cancels.
**Consequence:** a per-face quality *discount* keyed on variance of Laplacian —
the most widely used blur metric in production vision pipelines — would
systematically down-weight the *more* identifiable faces. It is worse than no
discount.
## Result 5 — as a compute gate, sharpness loses to the size filter
Skipping the embed for crops below a threshold, measured as compute saved
against true identifications lost:
| gate | skipped | true IDs lost | of skipped, doomed anyway |
|---|---|---|---|
| `hf_energy_ratio` < 0.00023 | 10.0% | 7.6% | 37.9% |
| `hf_energy_ratio` < 0.00051 | 20.0% | 15.1% | 38.7% |
| **source size < 24 px** | **16.7%** | **4.7%** | **77.3%** |
At a comparable skip rate the size filter loses **4.7% against sharpness's
15.1%** — three times less damage — and it is free, being a bbox dimension
available before alignment or embedding, where sharpness needs the warped crop
plus a colour convert, three convolutions and a 64×64 DFT.
Restricting to large faces (≥64 px) on the **defocus** arm, where the size
filter is blind, improves the gate's precision 3.5x (37% of skipped crops doomed
versus 10.7% on the Gaussian arm) but not its trade: skip 10%, lose 7.0%.
A hard ceiling explains why. **At 112 px with defocus radius 6 — visually
destroyed — 46.9% of faces still identify correctly and rank-1 is still 94.8%.**
Blur does not determine the outcome, so any gate keyed on apparent blur is
predicting a coin flip. The size filter wins not because size is better
measured, but because *smallness destroys identity more completely than blur
does*: 16 px faces succeed only 23.5% of the time, so discarding them is cheap.
## What this means for the requirements
**Do not gate on sharpness; discount on it.** Heavily defocused faces remain
~47% identifiable, so a gate destroys recoverable evidence. This is the first
hard evidence that AR-028's "**discounts the observation, never deletes the
detection**" is right on the merits rather than merely cautious. Since ranking
survives where confidence does not, the per-track accumulation (AR-025) should
recover much of what a single-frame threshold rejects — which is also the
argument for the discount living in `EvidenceDiscounter` rather than in a filter.
**`var_laplacian` and `tenengrad` are disqualified as discounts** by Result 4,
on all three blur families. They remain usable as coarse *gross-degradation*
detectors, the role in which their pooled AUC is real — the same role the size
filter plays — but they must never weight a per-observation belief.
**`hf_energy_ratio` is the only surviving discount candidate**, best on all
three families, and its within-cell signal (0.520.56) is weak enough that
shipping a discount on it needs justification beyond this study.
**`dir_min_tenengrad` earns a different job.** Its pooled 0.854 on the motion arm
makes it the best available detector of gross directional smear — useful as a
per-frame "this shot is unusable" flag, which is a decision about a *frame*, not
a weighting of an *observation*. If AR-029 ships two measures for two roles, this
is the second one, and it must not be confused with the first.
**Model the blur family, not just its amount.** Result 3 makes the choice of
degradation model a first-order design decision rather than a detail: the same
matched severity costs 9% or 53% error depending on the PSF. Any future study
that sweeps blur must state which family it used and why.
**Any discount curve must be flat then steep**, not linear or sigmoid over the
measure. Blur costs nothing until it costs a great deal.
## Limitations
- **Cooperative population.** Gallery mugshots are frontal and well-lit;
within-cell failures are likely dominated by cross-view mismatch, which no
sharpness measure can predict. Read the ~chance within-cell AUCs as "sharpness
does not predict the dominant failure mode *here*", not as "sharpness is
meaningless".
- **Uniform grid, not a natural distribution.** Sizes and blur levels are
sampled evenly, so "skip 16.7%" is exactly the 16 px row. The gate comparisons
are like-for-like on identical records, but the absolute savings are not what
a film would show.
- **TensorRT fp16.** A different realisation of the embedder from the fp32 ONNX
reference — VR-005 measured ~0.85 cosine agreement with separation intact.
Gallery and probes share one session so the study is internally consistent,
but the absolute knee belongs to the fp16 space.
- **Blur is applied in the canonical frame**, after resampling, so its width is
independent of the cell's size. Real optics blur before sampling.
- **The top motion rung is an anchor, not an operating point.** Length 21 is a
per-axis σ of 6.1 — 17% of the inter-ocular distance, a streak rather than a
face — and it is swept to bound the curve, not because a frame like that is
worth reasoning about. Its 3.4% TPI at 112 px should not be quoted as a
headline. The same caution applies less severely to defocus radius 6 (σ = 3).
- **Per-axis σ equates spread, not perceptual damage.** It is the fairest single
scalar for comparing PSFs, but Result 3 is precisely the finding that equal
spread does *not* mean equal harm, so the matched-severity tables compare
like-for-like inputs, not like-for-like severity as a face would experience it.
-383
View File
@@ -1,383 +0,0 @@
# scene-actor-extraction — requirements register
Stable IDs for every requirement in [`SPEC.md`](SPEC.md), which holds the prose.
This file is the **authoritative list**; the CI gate reads its denominators from
here (see [`../../SPEC.md`](../../SPEC.md) §6).
**IDs are permanent.** A withdrawn requirement is marked `Withdrawn` and its
number is never reused — renumbering is what produces orphan TRACES tags. This
register replaces the earlier thematic `A1…E8` scheme, which had already produced
an `A1a` and an out-of-order `E6`.
Tag code with `// TRACES: AR-012 | SR-002`.
| Type | Scope |
|---|---|
| `AR` | Algorithm — the extraction pipeline itself |
| `DP` | Deployment — how it runs |
| `IR` | Integration — contracts with other components |
| `GR` | Gallery — building and maintaining actor references |
| `VR` | Validation — parameter studies and benchmarks |
| `UT` / `IT` | Unit / integration tests |
Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
---
## Algorithm (AR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | Planned |
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done**`max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing |
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Done** — KPN node outputs use `push_blocking`; sentinels stay out-of-band. Verified: 385/385 frames, 0 drops, byte-identical across runs |
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done**`umeyama_similarity()`. The RANSAC fit it replaces disagreed by a median 17 source px on 400 headshots, 83.5% of crops embedding below cos 0.99, and was unstable and RNG-driven: rebuilding caught 1614 near-duplicates against the original build's ~100. **All galleries rebuilt** (2456 actors, 10254 embeddings); measured separation gain is small (0.583 → 0.590), so recorded accuracy figures should be re-run but are not expected to move far |
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done**`track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
| AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted |
| AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done |
| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Done** — decode butterfly joined via `SceneBoundaries`; the sampled branch waits for the detector's watermark. Frames past its last scored window are counted as unverified, never assumed boundary-free |
| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | Planned |
| AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done**`src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track |
| AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done**`last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed |
| AR-014 | Belief swap A→B terminates the track and starts a new one | SR-002 | Medium | **Done** — swap closes at `last_seen` and opens a successor at the swap frame; counted |
| AR-015 | Two live tracks owned by one actor ⇒ treat as a detected cut, re-associate | SR-002 | Medium | **Done** — reverse index detects it on the causing update; counted |
| AR-016 | All tracks closed at EOF — a film ends with faces on screen | SR-002 | High | **Done**`flush()`, idempotent, closes at last sighting or final tick |
| AR-017 | Every presence claim carries its belief and identification route | SR-002 | High | **Done**`DeadTrack` carries belief and observation count |
| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space; replaces `expand_novelty_sim`. Rejections counted |
| AR-019 | Per-film gallery annex from owned tracks; acquires the non-frontal views TMDB lacks | SR-005 | Medium | **Done** — all three discontinuity signals clear the buffers; ownership comes from the registry, not a second local tally |
| AR-020 | Deferred re-identification of unknown tracks against the final expanded gallery | SR-005 | High | Planned |
| AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned |
| AR-022 | Capture still-unidentified tracks: embeddings, metadata, **context crops** for human review | §4 | Medium | Planned |
| AR-023 | Fit sigmoid calibration from intra/inter similarity distributions | SR-002 | High | Done |
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association and accumulation both in probability space; `track_max_embed_dist`, `cut_revive_sim` retired |
| AR-025 | Per-track Bayesian accumulation in log-odds, with correlated-observation discounting | SR-002 | High | **Done** — log-odds accumulation with correlation discounting owned by the registry, `src/evidence_discount.hpp` |
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | In Progress |
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | Planned |
| AR-029 | Sharpness measure on the **aligned crop**, consumed as a discount and **never as a gate** | SR-002 | Medium | **In Progress** — five candidates implemented (`src/quality.hpp`) and ranked by VR-012 over three blur families. `var_laplacian` and `tenengrad` are **disqualified as discounts**: within a fixed degradation they are anti-predictive on *all three* families (AUC 0.420.47; the decile the measure calls sharpest is 2.6× *less* identifiable), since their residual variance is native contrast, not detail. `hf_energy_ratio` is the only correctly-signed survivor, best on all three, and weak (0.520.56). `dir_min_tenengrad` is the best *gross-smear detector* (pooled AUC 0.854 on motion) but ~chance within-cell, so it serves a per-frame flag, not a per-observation weight. The parenthetical this row used to carry — "scale-normalised, so it cannot re-measure size" — was wrong: every candidate responds to source size, and the axes are separable for a different reason (see AR-028) |
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
## Deployment (DP)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| DP-001 | One analysis core; modes are front-ends and must not fork pipeline logic | PR-004 | High | Done |
| DP-002 | Batch CLI over one title | PR-004 | High | Done |
| DP-003 | On-demand resident service with bounded, observable queue | PR-004 | Medium | Planned |
| DP-004 | Opportunistic/idle mode: external trigger, hard stop, implicit re-queue | PR-004 | Medium | Planned |
| DP-005 | Native installer, no Docker; Fedora + Arch | PR-004 | Medium | Planned |
| DP-006 | Background incremental gallery refresh on a timer | PR-003 | Medium | Planned |
| DP-007 | CI builder image, CPU-only, pinned by tag in the Gitea container registry | PR-004 | High | Planned |
| DP-008 | Builder images + release jobs per backend (cpu / cuda / rocm); ship binaries, not engines | PR-004 | Medium | Planned |
## Integration (IR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| IR-001 | Emit the JRay truth format as sibling `.jray.json` | SR-003 | High | Done |
| IR-002 | Windows carry belief + route; `extraction.*` carries `extinction_sec`, `gallery_scope` | SR-003 | High | **Done**`schema_version: 2`; windows are objects with `belief` + `route`; `extraction.*` carries `extinction_sec` and `gallery_scope`; `anneal_sec` removed |
| IR-003 | Output written **after** the deferred pass, not at EOF | SR-003 | High | **In Progress** — sink builds windows from registry claims and flushes at EOF; the deferred pass (AR-020) does not exist yet, so output is still final at EOF |
| IR-004 | Compute the audio signature exactly per server spec §3 | SR-003 | Medium | **Done**`src/audio_signature.*`; not yet emitted into the truth file (IR-002) |
| IR-005 | Golden-vector fixture shared with the plugin repo to prove bit-exactness | SR-003 | High | **Done**`tests/fixtures/audio/`; v1 parameters now normative in server spec §3 |
| IR-007 | Media < 120 s: emit no signature, apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** |
| IR-008 | Emit and honour the signature's own `v1:` version prefix | SR-003 | Low | **Done** |
| IR-006 | Jellyfin round-trip: pull pending queue, push complete results only | SR-001 | High | Done |
## Gallery (GR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| GR-001 | Build gallery from Jellyfin library cast, TMDB profile fallback | SR-001, SR-005 | High | Done |
| GR-002 | Incremental `--merge` refresh without re-embedding known actors | PR-003 | High | Done |
| GR-003 | Report coverage: zero-image actors, under-referenced actors, dedup, calibration PDFs | SR-001 | Medium | Planned |
| GR-004 | Stamp embedder identity into the gallery; **hard startup error** on mismatch | SR-001 | High | **Done** — basename + SHA-256 + `embed_dim`; mismatch fatal with no bypass, unstamped warns unless `--require-gallery-stamp`; `scripts/stamp_gallery.py` migrates in place |
| GR-005 | Gallery data never leaves the instance | **SR-005** | High | Done |
| GR-006 | Provenance tiers: baked / harvested / confirmed, distinguishable per embedding | SR-005 | High | Planned |
| GR-007 | Persist harvested embeddings **flagged and reviewable**, never silently equal to baked | SR-005 | Medium | Planned |
| GR-008 | Flag distributional outliers among an actor's references (poisoning guard) — `EXCEPTION: AR-024` | SR-005 | Medium | Planned |
| GR-009 | Human-confirmed associations persist and improve future extractions | §4 | Medium | TBD |
## Validation (VR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| VR-001 | HDF5 post-inference dump at the embedded-frame boundary | PR-002 | High | Done |
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | **Done** — replay driven from committed fixtures in `tests/test_replay_fixtures.cpp`; determinism asserted |
| VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done |
| VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done |
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 2432 px; 32 px gives 98.1% TPI, 0.0 FPI at every size. Degrades an already-aligned 112×112 crop, so it isolates the embedder and is an **upper bound**; VR-013 measures the same question end to end and AR-002 takes its number, not this one |
| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned |
| VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned |
| VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned |
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | Planned |
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | Planned |
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | **In Progress** — sharpness half done ([`docs/quality-knee.md`](quality-knee.md)): 1670 actors, joint size×blur grid over three blur families (Gaussian, disc defocus, linear motion), 60120 probe-cell records each. Sharpness is **not a sufficient statistic** (equal measured sharpness spans 15.391.0% TPI, ordered by source size); **the blur family matters more than its amount** — at matched per-axis σ=3 on a 112 px face, Gaussian/motion/defocus cost 9/18/**53**% error, so a Gaussian-only sweep understates real lens blur fivefold; blur breaks **confidence, not ranking** (rank-1 80.2% where TPI is 15.3%), so FPI never left 0.1% in any of the 108 cells; a sharpness **gate** loses 3× more true presence than the free size filter at equal saving, because even destroyed faces stay 46.9% identifiable. **Pose half not started** — the AR-030 residual is exposed via `sae_embed.alignment_residual` but no pose arm has been run, so the dedicated-landmark-model question is still open |
| VR-014 | Audio-signature **offset recovery on real content** — a known trim recovered from film audio, not from the synthetic golden tone | PR-002 | Medium | **Done** — 40 random in-cap offsets, every one recovered to the nearest frame: **worst error 46 ms against a 500 ms budget**, and 46 ms is the floor rather than a result, since the offset is quantised to whole 92.88 ms frames. The `runtime/2` anchor confirmed through real head-trimmed files (a `delta` trim moves the window by `delta/2`). The one soft spot is **tier labelling, not accuracy**: the score falls with sub-frame misalignment (0.940.99 near a frame boundary, 0.690.73 at half a frame), so 27/40 correct alignments were demoted to `loose`. ±1 frame of slack in the *score* fixes it — measured, all 40 back to `audio` (min 0.906), false matches unmoved at 0.120.16, costing 81 ms of the budget |
| VR-013 | Cross-source identification probe — gallery from one recording, probes from another, swept over input resolution end to end | PR-002 | Medium | **In Progress** — holding 90% of the plateau needs ~50 px end to end against VR-005's ~22 px, the gap being detection and landmark error; **`min_face_px` 40, since 32 admits faces in the falling region** (AR-002). FPI 0.0% at every scale. Ceiling is cross-view, not resolution |
---
## Verification strategy
**CI runs on an Intel N100 with no discrete GPU.** That is a hard constraint on
how each requirement can be verified, and it shapes the test design rather than
merely limiting it.
Four tiers, in decreasing order of preference:
| Tier | Runs in CI | What it covers |
|---|---|---|
| **T1 — Functor unit** | Yes | A KPN node's `operator()` driven directly with hand-built inputs |
| **T2 — Replay** | Yes | The composed pipeline driven from an HDF5 fixture — no GPU, no video |
| **T3 — CPU inference** | Yes, slowly | ORT CPU provider over a handful of frames; smoke tests only |
| **T4 — GPU** | **No** | Throughput, TRT engines, large-gallery GEMM |
### T1 is the primary tier, and KPN is why
**Node functors are plain callable structs, constructed independently of the
network that wraps them** (`main.cpp:186-207` builds them as stack objects;
`ObjectNode` merely adapts them). So a node is testable by constructing it and
calling `operator()` — no channels, no threads, no network, no fixture.
This is already the established pattern, not a proposal:
`tests/test_face_tracker.cpp` "drives the node's `operator()` with hand-built
`EmbeddedSceneFrame`s and inspects the emitted `track_ids`", and does so
"pure, GPU-free, model-free".
The consequence is that most of the redesign is verifiable **without any
fixture at all**: construct exactly the awkward state — a belief swap, two live
tracks converging on one actor, a film ending mid-track, a gap one frame under
the timeout — rather than hunting for a clip that happens to exhibit it.
Four hazards this removes outright:
- **No fixture-provenance risk** for these tests — the inputs are synthetic and
explicit.
- **No "fixture must be replayed from frame 0"** concern — state is constructed
directly.
- **No cross-test state leakage** (e.g. a tracker's `next_id_` persisting) — each
test constructs a fresh functor.
- **No replay-harness nondeterminism** — no channels, so no EOF-tail heuristics
or silent drops.
It also means **a dead upstream producer does not block testing a downstream
consumer.** `is_scene_boundary` currently has no producer (see AR-010), which
would make a *replay* test of the frame-dependent `track_alpha` pass vacuously —
but a T1 test simply constructs a frame with `is_scene_boundary = true` and
asserts the weighting changes. The producer gap is a pipeline defect to fix, not
a verification blocker.
### T2 covers what T1 cannot
Replay remains necessary for **composition** — that the nodes wired together
behave as the sum of their parts — and for realistic data at scale, which
synthetic inputs cannot honestly imitate. It is the tier that would catch a
wiring error, a channel-capacity problem, or an ordering assumption that only
appears under concurrency.
The HDF5 dump (VR-001) captures state after decode → detect → align → embed, so
replay needs no GPU and no video. That was built for the optimizer; it doubles as
CI, which is a strong argument for keeping the schema honest and for replay
driving the *real* nodes rather than a reimplementation (VR-002).
**Fixtures and studies are generated locally**, on the development machine where
the models, galleries and media already exist. CI consumes them; it never
produces them.
**Small committed fixtures are required.** A few HDF5 dumps covering the awkward
cases — a cut, a belief swap, two live tracks converging, a film ending
mid-track, an unknown track that only resolves after expansion — are worth more
than a large corpus, and they are small enough to commit.
**T4 requirements cannot pass in CI, and the gate must not pretend otherwise.**
For these, CI verifies that a test *exists and is tagged*, not that it passes;
the run happens on a GPU host, nightly or manually, and reports separately. A
requirement whose only evidence is a test that never executes should be visible
as such rather than counted as covered.
| Requirement | Tier | Note |
|---|---|---|
| AR-001, AR-005, AR-006 | T3 | Smoke only — correctness of detection/embedding is a model property, not ours |
| AR-002 | T2 | Size filtering is arithmetic on dumped bboxes |
| AR-003, AR-004 | T1 + T4 | Backpressure logic is unit-testable; saturation behaviour needs real load |
| AR-007 … AR-017 | **T2** | The core of the redesign — fully replayable |
| AR-018 … AR-022 | **T2** | Expansion, deferred pass, clustering: all post-embedding |
| AR-023 … AR-025 | T1 | Calibration fit and log-odds accumulation are pure maths |
| AR-026, AR-027 | T4 | GEMM throughput and scaling — GPU host only |
| DP-* | T1 + manual | Lifecycle logic unit-tested; install paths are manual |
| IR-001 … IR-003 | T1 | Serialisation against a golden truth file |
| IR-004, IR-005 | **T1** | Audio signature is CPU DSP — the golden-vector fixture runs anywhere, which is precisely why it is the right cross-repo check |
| GR-001 … GR-005 | T1 + T3 | Gallery assembly is I/O and bookkeeping; embedding is T3 smoke |
| GR-006 … GR-008 | T1 | Tiering and outlier detection operate on stored embeddings |
| VR-* | Out of CI | Studies are run deliberately and their results committed as documents |
**One consequence worth stating:** AR-027 (arbitrary gallery scale) is
structurally unverifiable on the CI host. It needs a GPU host and a synthetic
large gallery, so it is the requirement most likely to silently regress. Its
benchmark (VR-008) should run on a schedule rather than on demand.
### CI never calls a model
**Not "should not" — cannot.** The N100 has no GPU, and even the ONNX Runtime CPU
provider is impractical: a measured run of the embedder on this hardware sits at
~930 ms per frame, so a 77 s clip at 5 fps would take roughly six minutes of
inference alone. Every model invocation therefore happens **locally, ahead of
time**, and CI consumes the result as data.
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
T2 are the only tiers that can exist in CI at all.
### Fixture corpus — `bali/`
Five clips of **Road to Bali (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Public domain, and that is the reason to use it rather than a convenience:
**derived fixtures — dumps, crops, golden outputs — can be committed without the
rights question that rules out sharing gallery data (SR-005).** A fixture cut
from a copyrighted title could not live in the repository at all.
Two properties to design around rather than discover:
- **480×360 means small faces.** At this resolution a face is often 4080 px, so
the AR-002 minimum of 40 px (original resolution) sits at the very bottom of
that range: the filter is close to binding, and anything shot wider is lost.
Fixture generation must set `--min-face-px` explicitly and record it, or the
dumps will be sparse for reasons unrelated to what is being tested.
- **77 s is short.** At 1 fps that is 77 frames — too thin to exercise an
extinction window measured in tens of seconds. Generate at 5 fps (≈385 frames,
~1 MB) and record the rate in provenance, since the behaviour under test
changes with it.
> **AR-004 blocks reproducible fixture generation.** A trial run of one clip
> produced 49 frames of an expected ~385, ending at 51 s of 77 s, with the
> diagnostics reporting 285 frames dropped at `camera_pos` and 51 at
> `face_aligner`. Channels overflow and **drop** rather than blocking, and what
> gets dropped depends on timing — so the same command run twice can produce
> different dumps. Golden fixtures cannot be built on that. AR-004 is therefore
> a prerequisite for VR-001 fixtures, not merely a throughput concern for crowd
> scenes.
### Fixtures — precomputed inference, pulled by CI
The N100 cannot run inference at any useful rate, so **inference output is
precomputed on a GPU host and consumed by CI as data.** This converts most of
what looks like GPU work into pure CPU replay.
| Fixture | Contents | Size | Storage |
|---|---|---|---|
| **Edge-case dumps** | ~6 short clips (3060 s), one per awkward behaviour | ~0.11 MB each | **Committed in-repo** |
| **Corpus dumps** | Full-length titles from the validation corpus | ~2138 MB each | **Gitea package registry**, pinned by version + checksum |
| **Synthetic gallery** | Random unit-norm embeddings, fixed seed | small | Generated at test time |
| **Golden truth files** | Expected output for each edge-case dump | KB | Committed |
| **Audio golden vectors** | FLAC + expected signature + parameter contract | ~600 KB | Committed, **shared with the plugin repo** |
Edge-case dumps are small enough to commit, and being in-repo means they version
with the code that reads them.
**Corpus dumps go to the Gitea package registry, not Git LFS.** Both are
available — the models already use LFS — but their fetch semantics differ in a
way that matters here. LFS objects are pulled on clone unless a developer
explicitly skips them, so ~38 MB per title behind LFS taxes everyone who clones,
forever, for data that only CI and the optimizer ever read. Registry artifacts
are fetched on demand by the job that needs them.
Rule of thumb: **LFS for what the build needs; the package registry for what a
particular job needs.** Models are the former; corpus dumps and the CI image
(DP-007) are the latter.
Pin by version and verify by checksum on fetch. A fixture that changes silently
under CI is worse than a missing one, because the failure presents as a code
regression.
**Generation must be reproducible and versioned.** A script, run on a GPU host,
regenerates every fixture from source clips; it is re-run when the VR-001 schema
version bumps. A fixture whose provenance is unknown is worse than no fixture,
because it will be trusted.
> **The limitation that must stay visible:** replay fixtures freeze upstream
> behaviour. A test driven from a dump verifies AR-007 onward *given those
> embeddings* — it cannot detect a regression in detection, alignment or
> embedding, because those produced the fixture. Nothing in CI can. That gap is
> covered only by the T3 smoke test and the scheduled GPU run, and it should not
> be papered over by a high replay-coverage number.
### Per-requirement verification plan
| ID | Tier | Test asserts | Edge cases to cover |
|---|---|---|---|
| AR-001 | T3 | Detector returns plausible boxes on a known frame | — smoke only |
| AR-002 | T2 | Faces below 40 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement |
| AR-003 | T2 | No cap applied; a 40-face frame yields 40 | Crowd frame |
| AR-004 | T1 | Saturated input blocks rather than drops or throws | Bounded queue at capacity; **byte-based** limit with large crops; SIGTERM mid-block |
| AR-005 | T1 | Known landmarks → expected 112×112 warp; the fit never mirrors | Landmarks near frame edge; degenerate/collinear points; a mirrored set — SVD returns a reflection unless the determinant guard rejects it |
| AR-006 | T3 | Embeddings are unit-norm | Batch smaller than, equal to, larger than `embed_batch_size` |
| AR-007 | T2 | Association picks the right track | Two faces crossing paths; one leaving frame as another enters |
| AR-008 | T2 | One pool; dormant tracks match on embedding, not IoU | Dormant track whose old bbox overlaps a *different* new face — must not match on position |
| AR-009/010 | T2 | Cut/boundary shifts weighting toward embedding | Cut with same people; cut with all-new people |
| AR-011 | T1 | TransNetV2 receives native-rate frames | Source at 24/25/30 fps — dedup window derived, not assumed |
| AR-012 | **T2** | Window spans full track extent, not first recognition | Actor recognised only at track end — window must still start at `first_seen` |
| AR-013 | **T2** | `last_seen` set/unset; window ends at last sighting | Gap just under vs just over timeout; reappearance after timeout → two windows |
| AR-014 | T2 | Belief swap closes one window, opens another | No blended window; no overlap at the swap frame |
| AR-015 | T2 | Two live tracks on one actor trigger re-association | Counter increments |
| AR-016 | **T2** | Every track closed at EOF | Film ending mid-shot — window ends at final frame, not dropped |
| AR-017 | T1 | Claim carries posterior and route | Deferred and pooled routes distinguishable |
| AR-018 | T1 | Band admits only within bounds | At each bound exactly; store never admits below lower bound |
| AR-019 | T2 | Promotion only when all three signals quiet | Cut mid-track blocks promotion |
| AR-020 | **T2** | Unknown resolved after expansion | Track failing at minute 12, resolved at EOF — the ordering-independence claim |
| AR-021 | T2 | Clustering merges same person, respects cannot-link | **Temporally overlapping tracks never merge**; measure how many merges the constraint rejects |
| AR-022 | T1 | Context crops retained, bounded per track | Track running for minutes |
| AR-023 | T1 | Sigmoid fit on synthetic separable data | Too few positive pairs → `valid=false`, fallback engages |
| AR-024 | **Static check** | No bare cosine outside a tagged `EXCEPTION` | Grep-based; this is the invariant's enforcement |
| AR-025 | T1 | Log-odds accumulate; correlated frames discounted | 30 identical frames must **not** reach the certainty of 30 diverse ones |
| AR-026 | T1 + T4 | GEMM path produces same result as reference loop | Equivalence on small input in CI; throughput on GPU host |
| AR-027 | **T4** | Throughput at 10²…10⁵ actors | Scheduled, not on-demand |
| AR-028 | **T2** | No embedding reaches the matcher unscored; the vector survives into the dump | Face failing exactly one axis; all three healthy; a face whose landmarks are degenerate — scored, not silently vanished |
| AR-029 | T1 | Synthetic blur ladder → monotonically falling sharpness | Gaussian vs motion blur; **small sharp face vs large soft one** — size must not leak into this axis |
| AR-030 | T1 | Alignment residual rises monotonically with foreshortening | **In-plane roll, scale and translation must leave it at zero** — the property that makes it a pose measure rather than a pose-and-everything-else measure; face size must not shift it; degenerate landmarks report not-ok rather than a number |
| VR-012 | **T4** | Knee located per axis on held-out films | Report each candidate threshold's cost in **lost true presence**, not only its gain in precision — a gate that improves misID by discarding half the cast has not helped |
| VR-013 | **T4** | Identification holds across two recordings of the same people, and degrades to TBI rather than to a wrong name as input resolution falls | Gallery and probes must come from *different* recordings — a hold-one-out over one recording measures a much easier problem and will not surface the cross-view failure. Ground truth is hand-sorted; labels propagated by embedding similarity would keep only the faces the embedder already gets right |
| IR-001/002 | T1 | Serialised output matches golden file | Zero-length window; actor with many windows |
| IR-003 | T1 | Output written after deferred pass | Not at EOF |
| IR-004/005 | **T1** | Signature matches golden vector bit-for-bit | Identical result in both producer repos |
| VR-014 | **T2** | A known trim offset is recovered from **real film audio**, to the nearest frame | An offset past the ±600-frame cap and unrelated content must both be *declined*, never given a best-effort alignment. Fixture and signature are both CPU-only, so unlike the other VR rows this one is CI-*executable* — though the repo's only workflow today is the traceability gate, so nothing runs it there yet. The signature comes from the shipped C++ through `sae_audio`; a numpy port would be a third implementation nobody checks against the golden vector |
| IR-006 | T1 + manual | Queue pull and result push against a stubbed Jellyfin API | Partial result never pushed; push only after the deferred pass |
| IR-007 | **T1** | Media < 120 s emits no signature at all | Exactly 120 s; just under; zero-length audio. Must match the plugin's cutoff exactly — a caller-varying window length is what SR-004 forbids |
| IR-008 | T1 | `v1:` prefix emitted and honoured on read | Unknown prefix rejected, not guessed |
| GR-009 | T1 | Human-confirmed associations persist and are tier-tagged | Survives a gallery rebuild; distinguishable from baked and harvested |
| GR-004 | T1 | Mismatched embedder → hard startup error | Error names both sides; **unstamped warns, and errors under `SAE_REQUIRE_GALLERY_STAMP`**; same filename + different SHA-256 must still be a mismatch |
| GR-008 | T1 | Outlier flagged among an actor's references | Injected poisoned embedding detected |
| VR-009 | T1 | Posterior calibration holds | A 0.99 posterior is wrong ~1% of the time on held-out tracks |
Three of these are worth singling out because they verify claims that would
otherwise be assertions: **AR-012** (window starts at `first_seen` even when
recognition comes late) is the entire point of the redesign; **AR-020** (a track
failing mid-film resolves at EOF) is the claim that ordering stops mattering; and
**AR-025** (30 identical frames ≠ 30 diverse ones) is what stops the Bayesian
accumulation from being decoration.
---
## Withdrawn
| ID | Requirement | Reason |
|---|---|---|
| — | `anneal_sec` window merging | Superseded by AR-012/AR-013: a track survives its own gaps, so there is nothing to anneal |
| — | `extinction_sec` actor keep-alive | Superseded by AR-013: windows end at last sighting, which is what this over-claimed |
Both were deleted rather than retained at zero — a field naming a mechanism the
pipeline no longer has is actively misleading (see `SPEC.md` A6.6).
---
## Notes on coverage
- **VR-*** traces to PR-002 (scene-granularity answers) rather than to a system
requirement: parameter studies are single-repo work serving accuracy, and this
is correct rather than a gap.
- **PR-005** (leak nothing) has no `AR`/`DP` row. It is satisfied *structurally*
by SR-004 and GR-005 — the server holds no binary, the gallery never leaves the
instance — not by any component doing something. It cannot be verified by
pointing at code, and it dies the moment either prohibition is relaxed.
-215
View File
@@ -1,215 +0,0 @@
# Conversion to service — a native idle-GPU worker
Status: **design / proposal**. Nothing here is built yet.
## The idea
Turn the CLI tools into a **turnkey batch worker that uses the machine's idle
GPU**: it analyses newly-added Jellyfin media when you're not using the computer
(screen locked), and stops the instant you come back. It's an overnight job on
your own Linux box.
**No Docker.** This runs on your own machine with your own drivers, so a container
buys little and costs a lot: GPU passthrough (nvidia-container-toolkit, or
`/dev/kfd`+`/dev/dri`+`video` group for ROCm) is the single most fragile part of a
containerised setup, and it exists *only* because of the container. Natively, the
GPU just works with the drivers you already have, and the media paths Jellyfin
reports are just real paths — no re-mounting. So we ship a **native installer**
instead of an image builder.
Two deliverables:
1. **An installer**`scripts/build_install.py`. Detects your distro, ensures the
GPU/build dependencies are present (via `dnf`/`pacman`), compiles `scene_analyze`
for your GPU, and installs the binary + Python glue + two systemd **user**
units under `~/.local`.
2. **A screen-lock gate** — one of those systemd units watches logind lock/unlock
and starts/stops the worker. Lock → analyse. Unlock → stop.
## What already exists (reuse, don't rebuild)
The processing loop is already implemented — this is packaging, building, and
lock-gating, not new pipeline logic.
| Piece | Where | What it does |
|---|---|---|
| Analysis engine | `build/scene_analyze` | Video → face detect/align/embed → gallery match → result JSON |
| Backend selection | [`CMakeLists.txt`](https://REPOLINK/CMakeLists.txt) (`SAE_INFERENCE_BACKEND`, `SAE_GEMM_BACKEND`) | ORT/TRT + ROCm/CUDA, chosen **at build time** |
| New-media queue | JRay plugin → `GET /Plugins/JRay/Tasks/Pending` | Backlog of items with no results yet |
| Worker loop | [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)` --worker` | Poll Pending → run `scene_analyze` → push results |
| Result push | `PUT /Plugins/JRay/Items/{id}/Truth` | Stores per-actor scene windows back in Jellyfin |
| Incremental gallery | [`scripts/make_jellyfin_gallery.py`](https://REPOLINK/scripts/make_jellyfin_gallery.py)` --merge` | Embeds only cast not already in the gallery |
| Secrets loader | `.env` via [`scripts/sae_env.py`](https://REPOLINK/scripts/sae_env.py) | `JELLYFIN_URL`, `JELLYFIN_API_KEY`, `TMDB_API_KEY` |
## Installer config
One file. Build-time settings (fixed when we compile) vs. run-time settings (in the
worker's `.env`, editable without recompiling).
```yaml
# install.yaml — consumed by scripts/build_install.py
platform: nvidia # nvidia | amd | cpu → picks the cmake backend
model:
arcface: LVFace-B_Glint360K.onnx # embedder compiled against; gallery MUST match
schedule:
gallery_scan_interval: 24h # incremental --merge cadence; 0 disables the scanner
prefix: ~/.local # install root (bin, share, systemd user units)
# runtime (written to the worker .env, not compiled in):
runtime:
jellyfin_url: http://localhost:8096
# JELLYFIN_API_KEY / TMDB_API_KEY are filled into .env by hand after install
```
**Secrets never go in the repo or a build artifact** — the installer writes a
`.env` under the install prefix with blanks for the keys, and you fill them in
once. `sae_env.py` already loads it.
**Model ⇄ gallery coupling (guard, don't just document):** embeddings from
different recognition models aren't interchangeable. We compile against one
embedder; the gallery must be built with the same one. Stamp the embedder name
into `gallery.json`, and have the worker **refuse to start** if the gallery's
embedder ≠ the configured `model.arcface`, rather than silently mismatching.
## Dependencies via the system package manager
The heavy build/runtime deps (OpenCV, ffmpeg, the GPU stack) are best provided by
the distro, not vendored. The installer ships a per-distro dependency list and
either installs them or prints the exact command. Targets: **Fedora (dnf)** and
**Arch (pacman)** first.
| Dependency | Fedora (dnf) | Arch (pacman) |
|---|---|---|
| OpenCV | `opencv-devel` | `opencv` |
| ffmpeg | `ffmpeg-free`/`ffmpeg` (RPM Fusion) | `ffmpeg` |
| CMake / toolchain | `cmake gcc-c++` | `cmake gcc` |
| CUDA + TensorRT (nvidia) | NVIDIA CUDA repo + `libnvinfer-*` | `cuda`, `tensorrt` |
| ROCm (amd) | `rocm-hip-sdk` / `rocblas-devel` | `rocm-hip-sdk`, `rocblas` |
| ONNX Runtime | **not packaged** — installer fetches a pinned release tarball into the prefix | AUR `onnxruntime` (or same pinned-tarball fallback) |
So the flow is: **detect distro → check each package → install via the native
manager (or print `sudo dnf install …` / `sudo pacman -S …`)**, with ONNX Runtime
as the one known gap the installer fills itself (a pinned upstream release
extracted under the install prefix, so it doesn't depend on a system package that
may not exist). CUDA/ROCm being present is *assumed* — you already run a GPU
desktop; the installer verifies and points you at the vendor repo if not.
## What `build_install.py` does
```
build_install.py install.yaml
├─ detect distro (dnf vs pacman) and platform from config
├─ ensure deps: install via manager, or print the exact command; fetch ONNX Runtime if needed
├─ cmake + build scene_analyze with the platform's backend flags:
│ nvidia → -DSAE_INFERENCE_BACKEND=TRT -DSAE_GEMM_BACKEND=CUDA
│ amd → -DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=ROCM
│ cpu → -DSAE_INFERENCE_BACKEND=ORT (CPU EP; slow, for smoke tests)
├─ install into <prefix>:
│ bin/sae-scene-analyze the compiled binary
│ share/sae-worker/ Python glue + a venv (requests, etc.), models/
│ share/sae-worker/.env runtime config (keys blank, url from config)
├─ install systemd --user units:
│ sae-worker.service runs the worker + gallery-scan supervisor
│ sae-lock-gate.service watches logind lock/unlock, start/stops the worker
└─ print next steps (edit .env, `systemctl --user enable --now sae-lock-gate`)
```
## The worker service (supervisor)
`sae-worker.service` runs a small Python supervisor as its main process:
- starts the **worker loop** (`run_from_jellyfin.py --worker`) — the hot path,
- starts a **gallery-scan timer** — sleeps `gallery_scan_interval`, runs
`make_jellyfin_gallery.py --merge`, repeats,
- exits cleanly on SIGTERM (see re-queue below).
## The lock gate
`sae-lock-gate.service` runs a tiny watcher that subscribes to logind
lock/unlock signals and drives the worker service:
```
screen locks → systemctl --user start sae-worker.service
screen unlocks → systemctl --user stop sae-worker.service (SIGTERM)
```
**Screen-lock is the only signal — deliberately.** We don't also gate on GPU/CPU
load, because our own worker *is* the load: a load threshold would form a feedback
loop (worker starts → GPU spikes → threshold trips → worker stops → load drops →
restart → …). Lock state is external to what the worker does, so it can't
oscillate.
Signal source is desktop-dependent: logind `Lock`/`Unlock` (GNOME/KDE via
`loginctl`/D-Bus) covers most setups; a `swayidle`/`xss-lock` hook is the fallback
for wlroots/X-only compositors. The installer picks based on what's present.
## On resume: hard stop + re-queue (it's free)
Stopping the worker mid-analysis costs nothing to reschedule, because of how the
JRay queue works: **an item only leaves `/Tasks/Pending` once its results are
pushed** (`push_truth`). A worker stopped mid-`scene_analyze` simply leaves that
item Pending — next lock picks it up again. No re-queue bookkeeping.
Two small correctness requirements (the only worker changes needed):
1. **Never push a partial result.** Already true — `push_truth` runs only after
`scene_analyze` returns; a killed run pushes nothing. ✓ (keep it that way).
2. **Clean up on signal.** `process_item` writes a temp filtered-gallery file and
unlinks it in a `finally`; a SIGKILL skips `finally`. Fix: write temps under a
dir the worker wipes on start, and/or a SIGTERM handler that unlinks before
exit. Minor.
Accepted trade-off: a partially-analysed title restarts from scratch next lock.
Fine for an overnight/idle workload; no mid-video checkpointing.
## The end-to-end UX
```bash
# once: build + install for your GPU + model
./scripts/build_install.py install.yaml
# detects Fedora/Arch, ensures deps, compiles, installs units under ~/.local
# once: set your keys, enable the gate
$EDITOR ~/.local/share/sae-worker/.env # JELLYFIN_API_KEY, TMDB_API_KEY
systemctl --user enable --now sae-lock-gate.service
# from then on: nothing. Lock your screen → it analyses. Unlock → it stops.
```
No Docker, no GPU passthrough config, no media re-mounting — the worker sees the
same filesystem and GPU as everything else on the box.
## Implementation plan (follow-up commits)
Ordered so each step stands alone:
1. **installer skeleton**`scripts/build_install.py`: parse `install.yaml`,
distro detect, dependency check/print (start with cpu platform so it builds
without a GPU), cmake+build, copy into prefix.
2. **supervisor + cleanup**`scripts/service.py` (worker loop + gallery-scan
timer + SIGTERM); temp-file cleanup fix in `run_from_jellyfin.py`.
3. **systemd units + lock gate** — generate/install `sae-worker.service`,
`sae-lock-gate.service`, and the logind lock watcher.
4. **gallery/model guard** — stamp embedder into `gallery.json`; startup mismatch
check.
5. **platform + distro matrix** — nvidia/amd backends; dnf/pacman dep lists; ONNX
Runtime fetch fallback.
6. **docs** — README "Run on your idle GPU" section.
## Settled decisions
- **ONNX Runtime build** — the installer fetches the **ROCm ORT** release. It
serves the `amd` platform, and its CPU execution provider covers the `cpu`
smoke-test fallback too, so one download handles both. (nvidia uses raw TRT and
doesn't need ORT.)
- **`dnf`/`pacman` invocation** — **auto-install.** The installer runs `sudo dnf
install …` / `sudo pacman -S …` itself (prompting for sudo), rather than only
printing the command. It still prints what it's about to install first.
- **Distro coverage****Fedora + Arch only** for now. Debian/Ubuntu (`apt`) is
out of scope.
## Open questions
*(none blocking — the spec above is buildable as-is.)*
-777
View File
@@ -1,777 +0,0 @@
# Requirements traceability matrix
<!-- GENERATED FILE - do not edit by hand. -->
<!-- Regenerate: scripts/traceability/traceability-gate.sh -->
**Generated:** 2026-07-31T14:44:58+00:00
Denominators are read from [`requirements.md`](requirements.md) at run time, never hardcoded. Coverage counts a requirement only when it is tagged in source **and** has a verification tier this repo's CI host can execute (`T1, T2, T3, static`).
## Summary
| Metric | Value |
|---|---|
| Source files scanned | 111 |
| TRACES tags found | 132 |
| EXCEPTION tags found | 0 |
| Requirements defined | 69 |
| Requirements covered | 38 |
| **Coverage** | **55.1%** (38/69) |
| Coverage of CI-executable scope | 67.9% (38/56) |
| Tagged but unexecuted in CI | 5 |
| Orphan tags | 0 |
### By type
| Type | Covered | Tagged but unexecuted | Defined |
|---|---|---|---|
| AR | 22 | 1 | 30 |
| DP | 2 | 0 | 8 |
| IR | 8 | 0 | 8 |
| GR | 5 | 0 | 9 |
| VR | 1 | 4 | 14 |
- **UT** tags present (separate taxonomy, not counted in coverage): UT-001, UT-101, UT-102, UT-103, UT-104, UT-105, UT-106, UT-107, UT-108
- **IT** tags present (separate taxonomy, not counted in coverage): IT-001
- **PR** tags present (separate taxonomy, not counted in coverage): PR-002, PR-004
- **SR** tags present (separate taxonomy, not counted in coverage): SR-001, SR-002, SR-003, SR-005
## Not executable in CI
These requirements have no verification tier this repo's CI host can run, so a tag on them is evidence of *intent*, not of verification. They are never counted as covered.
| ID | Tiers | Tagged in source | Requirement |
|---|---|---|---|
| AR-027 | T4 | yes | Throughput acceptable for **arbitrary** gallery size |
| VR-001 | out-of-ci | yes | HDF5 post-inference dump at the embedded-frame boundary |
| VR-002 | out-of-ci | yes | Replay drives the **real** KPN nodes, not a reimplementation |
| VR-003 | out-of-ci | yes | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
| VR-004 | out-of-ci | yes | Reproducible validation corpus with ground truth |
| VR-005 | out-of-ci | no | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-006 | out-of-ci | no | Re-tune `scene_threshold` once native-rate decode lands |
| VR-007 | out-of-ci | no | Expansion band, clustering threshold, and deferred-pass ablation |
| VR-008 | out-of-ci | no | Gallery scaling benchmark — throughput vs gallery size |
| VR-010 | out-of-ci | no | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-011 | out-of-ci | no | Rewrite the replay harness for the post-AR-012 output contract |
| VR-012 | T4, out-of-ci | no | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did … |
| VR-013 | T4, out-of-ci | no | Cross-source identification probe — gallery from one recording, probe… |
**Tagged but unexecuted:** AR-027, VR-001, VR-002, VR-003, VR-004 — a test exists and is tagged, but this CI host cannot run it. Report those runs separately.
## Orphan tags
A tag naming an ID `requirements.md` does not define. This is what renumbering produces, and what a typo produces.
_None._
## Requirements tracing up to nothing
A register row whose `Traces to` cell names no parent. Work serving no stated goal is how scope creeps in, and it is invisible unless something looks.
_None._
## Recorded exceptions
Deliberate, documented departures from an invariant (`EXCEPTION: XX-nnn <reason>`). Reported separately and never counted as coverage — an exception is a decision to be reviewed, not evidence a requirement is met.
_None._
## Register
| ID | Status | Tier | Traces to | Trace state | Tagged in | Requirement |
|---|---|---|---|---|---|---|
| AR-001 | Done | T3 | SR-002 | covered | `src/nodes/face_detector_node.hpp` | Detect faces in sampled frames; emit bbox, confidence, 5-point landma… |
| AR-002 | Planned | T2 | SR-002 | untagged | - | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's … |
| AR-003 | **Done**`max_fac… | T1, T2, T4 | SR-002 | covered | `src/config.hpp`, `src/nodes/face_detector_node.hpp`, `src/nodes/identity_matcher_node.hpp` | No fixed per-frame face cap — crowd scenes must not lose background c… |
| AR-004 | **Done** — KPN node… | T1, T4 | SR-002 | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `tests/test_replay_fixtures.cpp` | Backpressure: unbounded faces/frame absorbed by slowing, never by dro… |
| AR-005 | **Done**`umeyama… | T1, T3 | SR-002 | covered | `src/face_utils.hpp`, `tests/test_face_utils.cpp` | Align to 112×112 via ArcFace 5-point similarity transform, fitted by … |
| AR-006 | Done | T3 | SR-002 | covered | `src/nodes/embedder_node.hpp` | 512-d L2-normalised embeddings, batched |
| AR-007 | **Done**`track_a… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | Associate detections by IoU + embedding, with **frame-dependent** wei… |
| AR-008 | **Done** — one pool… | T2 | SR-002 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp` | One track pool keyed on `last_seen`; no separate revival path |
| AR-009 | Done | T2 | SR-002 | covered | `src/nodes/camera_position_change_detector_node.hpp` | Camera-cut detection (histogram) as an association hint |
| AR-010 | **Done** — decode b… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/scene_boundary_annotator_node.hpp`, `src/nodes/scene_detector_node.hpp`, `src/scene_boundaries.hpp` | Scene-boundary detection (TransNetV2) as an association hint |
| AR-011 | Planned | T1, T2 | SR-002 | untagged | - | **Every model is fed the input it was trained for** — cost reduced by… |
| AR-012 | **Done**`src/tra… | T2 | **SR-002** | covered | `src/main.cpp`, `src/nodes/identity_matcher_node.hpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_replay_fixtures.cpp`, `tests/test_track_registry.cpp` | Presence follows **track extent**, not per-frame recognition |
| AR-013 | **Done**`last_se… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_replay_fixtures.cpp`, `tests/test_track_registry.cpp` | `last_seen` optional state machine; window ends at last sighting, nev… |
| AR-014 | **Done** — swap clo… | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Belief swap A→B terminates the track and starts a new one |
| AR-015 | **Done** — reverse … | T2 | SR-002 | covered | `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Two live tracks owned by one actor ⇒ treat as a detected cut, re-asso… |
| AR-016 | **Done**`flush()… | T2 | SR-002 | covered | `src/main.cpp`, `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | All tracks closed at EOF — a film ends with faces on screen |
| AR-017 | **Done**`DeadTra… | T1, T2 | SR-002 | covered | `src/nodes/result_sink_node.hpp`, `src/track_registry.hpp`, `tests/test_track_registry.cpp` | Every presence claim carries its belief and identification route |
| AR-018 | **Done** — banded a… | T1, T2 | SR-005 | covered | `src/config.hpp`, `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-subject embedding store with banded admission (novel enough, safe… |
| AR-019 | **Done** — all thre… | T2 | SR-005 | covered | `src/gallery/track_gallery.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-film gallery annex from owned tracks; acquires the non-frontal vi… |
| AR-020 | Planned | T2 | SR-005 | untagged | - | Deferred re-identification of unknown tracks against the final expand… |
| AR-021 | Planned | T2 | SR-005 | untagged | - | Cluster unknown tracks into one entity per person, under temporal can… |
| AR-022 | Planned | T1, T2 | §4 | untagged | - | Capture still-unidentified tracks: embeddings, metadata, **context cr… |
| AR-023 | Done | T1 | SR-002 | covered | `src/gallery/gallery_calibration.hpp`, `src/nodes/identity_matcher_node.hpp` | Fit sigmoid calibration from intra/inter similarity distributions |
| AR-024 | **Done** — associat… | T1, static | SR-002 | covered | `src/config.hpp`, `src/evidence_discount.hpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/track_gallery.hpp`, `src/main.cpp`, `src/nodes/face_tracker_node.hpp`, `src/nodes/identity_matcher_node.hpp` | **Always the calibrated probability, never a raw cosine** — exception… |
| AR-025 | **Done** — log-odds… | T1 | SR-002 | covered | `src/evidence_discount.hpp`, `src/nodes/identity_matcher_node.hpp` | Per-track Bayesian accumulation in log-odds, with correlated-observat… |
| AR-026 | In Progress | T1, T4 | SR-001 | covered | `src/backends/gemm_backend.cpp` | All similarity computed as GEMM, including annex and deferred pass |
| AR-027 | Planned | T4 | SR-001 | tagged, unexecuted | `src/backends/gemm_backend.cpp` | Throughput acceptable for **arbitrary** gallery size |
| AR-028 | Planned | T2 | SR-002 | untagged | - | **Embedding input quality assessed and carried** — every face scored … |
| AR-029 | Planned | T1 | SR-002 | untagged | - | Sharpness measure on the **aligned crop** (scale-normalised, so it ca… |
| AR-030 | **In Progress** — m… | T1 | SR-002 | covered | `src/face_utils.hpp`, `tests/test_face_utils.cpp` | Visibility measure from the AR-001 5-point landmarks — extreme pose o… |
| DP-001 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | One analysis core; modes are front-ends and must not fork pipeline lo… |
| DP-002 | Done | T1, manual | PR-004 | covered | `src/main.cpp` | Batch CLI over one title |
| DP-003 | Planned | T1, manual | PR-004 | untagged | - | On-demand resident service with bounded, observable queue |
| DP-004 | Planned | T1, manual | PR-004 | untagged | - | Opportunistic/idle mode: external trigger, hard stop, implicit re-que… |
| DP-005 | Planned | T1, manual | PR-004 | untagged | - | Native installer, no Docker; Fedora + Arch |
| DP-006 | Planned | T1, manual | PR-003 | untagged | - | Background incremental gallery refresh on a timer |
| DP-007 | Planned | T1, manual | PR-004 | untagged | - | CI builder image, CPU-only, pinned by tag in the Gitea container regi… |
| DP-008 | Planned | T1, manual | PR-004 | untagged | - | Builder images + release jobs per backend (cpu / cuda / rocm); ship b… |
| IR-001 | Done | T1 | SR-003 | covered | `src/nodes/result_sink_node.hpp` | Emit the JRay truth format as sibling `.jray.json` |
| IR-002 | **Done**`schema_… | T1 | SR-003 | covered | `src/config.hpp`, `src/main.cpp`, `src/nodes/result_sink_node.hpp` | Windows carry belief + route; `extraction.*` carries `extinction_sec`… |
| IR-003 | **In Progress** — s… | T1 | SR-003 | covered | `src/main.cpp` | Output written **after** the deferred pass, not at EOF |
| IR-004 | **Done**`src/aud… | T1 | SR-003 | covered | `scripts/validation/test_audio_offset.py`, `src/audio_bindings.cpp`, `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Compute the audio signature exactly per server spec §3 |
| IR-005 | **Done**`tests/f… | T1 | SR-003 | covered | `src/audio_bindings.cpp`, `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Golden-vector fixture shared with the plugin repo to prove bit-exactn… |
| IR-006 | Done | T1, manual | SR-001 | covered | `scripts/run_from_jellyfin.py` | Jellyfin round-trip: pull pending queue, push complete results only |
| IR-007 | **Done** | T1 | SR-003 | covered | `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Media < 120 s: emit no signature, apply no sync offset — identical ru… |
| IR-008 | **Done** | T1 | SR-003 | covered | `src/audio_signature.cpp`, `src/audio_signature.hpp`, `tests/test_audio_signature.cpp` | Emit and honour the signature's own `v1:` version prefix |
| GR-001 | Done | T1, T3 | SR-001, SR-005 | covered | `scripts/make_jellyfin_gallery.py` | Build gallery from Jellyfin library cast, TMDB profile fallback |
| GR-002 | Done | T1, T3 | PR-003 | covered | `scripts/make_jellyfin_gallery.py` | Incremental `--merge` refresh without re-embedding known actors |
| GR-003 | Planned | T1, T3 | SR-001 | covered | `src/build_gallery.cpp`, `src/gallery/gallery_calibration.hpp`, `src/gallery/gallery_report.hpp` | Report coverage: zero-image actors, under-referenced actors, dedup, c… |
| GR-004 | **Done** — basename… | T1, T3 | SR-001 | covered | `scripts/filter_gallery.py`, `scripts/make_gallery.py`, `scripts/make_jellyfin_gallery.py`, `scripts/movienet_eval.py`, `scripts/optimizer/fetch_missing_actors.py`, `scripts/optimizer/optimize.py`, `scripts/optimizer/reembed_gallery.py`, `scripts/optimizer/replay.py`, `scripts/sae_embed_loader.py`, `scripts/sae_gallery.py`, `scripts/sae_stamp.py`, `scripts/stamp_gallery.py`, `src/config.hpp`, `src/gallery/embedder_stamp.cpp`, `src/gallery/embedder_stamp.hpp`, `src/gallery/gallery_builder.cpp`, `src/gallery/gallery_store.cpp`, `src/kpn_bindings.cpp`, `src/main.cpp`, `src/nodes/embedding_dump_node.hpp`, `src/scene_preview.cpp`, `src/types.hpp`, `tests/test_gallery_store.cpp` | Stamp embedder identity into the gallery; **hard startup error** on m… |
| GR-005 | Done | T1, T3 | **SR-005** | covered | `src/gallery/gallery_store.hpp` | Gallery data never leaves the instance |
| GR-006 | Planned | T1 | SR-005 | untagged | - | Provenance tiers: baked / harvested / confirmed, distinguishable per … |
| GR-007 | Planned | T1 | SR-005 | untagged | - | Persist harvested embeddings **flagged and reviewable**, never silent… |
| GR-008 | Planned | T1 | SR-005 | untagged | - | Flag distributional outliers among an actor's references (poisoning g… |
| GR-009 | TBD | T1 | §4 | untagged | - | Human-confirmed associations persist and improve future extractions |
| VR-001 | Done | out-of-ci | PR-002 | tagged, unexecuted | `src/nodes/embedding_dump_node.hpp`, `tests/test_replay_fixtures.cpp` | HDF5 post-inference dump at the embedded-frame boundary |
| VR-002 | **Done** — replay d… | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/replay.py`, `tests/test_replay_fixtures.cpp` | Replay drives the **real** KPN nodes, not a reimplementation |
| VR-003 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/optimizer/second_score.py` | Scoring: micro-F1 against X-Ray, precision/recall logged at every eva… |
| VR-004 | Done | out-of-ci | PR-002 | tagged, unexecuted | `scripts/validation/ground_truth.py` | Reproducible validation corpus with ground truth |
| VR-005 | **Done** — knee at … | out-of-ci | PR-002 | untagged | - | Minimum face size study — TPI/FPI vs probe size, gallery held at nati… |
| VR-006 | Planned | out-of-ci | PR-002 | untagged | - | Re-tune `scene_threshold` once native-rate decode lands |
| VR-007 | Planned | out-of-ci | PR-002 | untagged | - | Expansion band, clustering threshold, and deferred-pass ablation |
| VR-008 | Planned | out-of-ci | PR-002 | untagged | - | Gallery scaling benchmark — throughput vs gallery size |
| VR-009 | Planned | T1, out-of-ci | PR-002 | untagged | - | Verify accumulated posteriors are calibrated against held-out tracks |
| VR-010 | Planned | out-of-ci | PR-002 | untagged | - | Dump provenance attributes — embedder model, detector settings, `dens… |
| VR-011 | Planned | out-of-ci | PR-002 | untagged | - | Rewrite the replay harness for the post-AR-012 output contract |
| VR-012 | Planned | T4, out-of-ci | PR-002 | untagged | - | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did … |
| VR-013 | **In Progress** — h… | T4, out-of-ci | PR-002 | untagged | - | Cross-source identification probe — gallery from one recording, probe… |
| VR-014 | **Done** — 40 rando… | T2, out-of-ci | PR-002 | covered | `scripts/validation/test_audio_offset.py` | Audio-signature **offset recovery on real content** — a known trim re… |
## Detailed mapping
### AR-001
**Locations:** 1
- [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown`
### AR-003
**Locations:** 3
- [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown`
- [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:`
- [`src/nodes/identity_matcher_node.hpp:166`](../src/nodes/identity_matcher_node.hpp#L166) — `std::vector<float> host_query(static_cast<size_t>(kMaxFaces) * 512);`
### AR-004
**Locations:** 4
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:166`](../src/nodes/identity_matcher_node.hpp#L166) — `std::vector<float> host_query(static_cast<size_t>(kMaxFaces) * 512);`
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
### AR-005
**Locations:** 2
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### AR-006
**Locations:** 1
- [`src/nodes/embedder_node.hpp:21`](../src/nodes/embedder_node.hpp#L21) — `struct EmbedderFunc`
### AR-007
**Locations:** 3
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
### AR-008
**Locations:** 3
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
### AR-009
**Locations:** 1
- [`src/nodes/camera_position_change_detector_node.hpp:30`](../src/nodes/camera_position_change_detector_node.hpp#L30) — `struct CameraPositionChangeDetectorFunc`
### AR-010
**Locations:** 9
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown`
- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));`
- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown`
- [`src/nodes/scene_boundary_annotator_node.hpp:2`](../src/nodes/scene_boundary_annotator_node.hpp#L2) — `Unknown`
- [`src/nodes/scene_detector_node.hpp:36`](../src/nodes/scene_detector_node.hpp#L36) — `static constexpr std::string_view label() { return "scene_detector"; }`
- [`src/nodes/scene_detector_node.hpp:110`](../src/nodes/scene_detector_node.hpp#L110) — `void flush_remaining()`
- [`src/nodes/scene_detector_node.hpp:144`](../src/nodes/scene_detector_node.hpp#L144) — `void write_output()`
- [`src/scene_boundaries.hpp:2`](../src/scene_boundaries.hpp#L2) — `Unknown`
### AR-012
**Locations:** 9
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/identity_matcher_node.hpp:125`](../src/nodes/identity_matcher_node.hpp#L125) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`src/nodes/identity_matcher_node.hpp:272`](../src/nodes/identity_matcher_node.hpp#L272) — `Unknown`
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-013
**Locations:** 3
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-014
**Locations:** 2
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-015
**Locations:** 2
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-016
**Locations:** 4
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }`
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-017
**Locations:** 3
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### AR-018
**Locations:** 3
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
### AR-019
**Locations:** 3
- [`src/gallery/track_gallery.hpp:122`](../src/gallery/track_gallery.hpp#L122) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/nodes/identity_matcher_node.hpp:147`](../src/nodes/identity_matcher_node.hpp#L147) — `MatchedSceneFrame operator()(TrackedSceneFrame tf)`
- [`src/nodes/identity_matcher_node.hpp:285`](../src/nodes/identity_matcher_node.hpp#L285) — `Unknown`
### AR-023
**Locations:** 3
- [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
- [`src/nodes/identity_matcher_node.hpp:117`](../src/nodes/identity_matcher_node.hpp#L117) — `const GalleryCalibration& calibration() const { return cal_; }`
### AR-024
**Locations:** 10
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
- [`src/gallery/track_gallery.hpp:132`](../src/gallery/track_gallery.hpp#L132) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
- [`src/nodes/identity_matcher_node.hpp:117`](../src/nodes/identity_matcher_node.hpp#L117) — `const GalleryCalibration& calibration() const { return cal_; }`
### AR-025
**Locations:** 3
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:125`](../src/nodes/identity_matcher_node.hpp#L125) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`src/nodes/identity_matcher_node.hpp:272`](../src/nodes/identity_matcher_node.hpp#L272) — `Unknown`
### AR-026
**Locations:** 1
- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;`
### AR-027
**Locations:** 1
- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;`
### AR-030
**Locations:** 2
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### DP-001
**Locations:** 1
- [`src/main.cpp:3`](../src/main.cpp#L3) — `Unknown`
### DP-002
**Locations:** 1
- [`src/main.cpp:3`](../src/main.cpp#L3) — `Unknown`
### GR-001
**Locations:** 1
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
### GR-002
**Locations:** 1
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
### GR-003
**Locations:** 13
- [`src/build_gallery.cpp:83`](../src/build_gallery.cpp#L83) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:80`](../src/gallery/gallery_calibration.hpp#L80) — `struct GalleryCalibrationStats`
- [`src/gallery/gallery_calibration.hpp:145`](../src/gallery/gallery_calibration.hpp#L145) — `std::vector<bool> actor_eligible(n_actors, false);`
- [`src/gallery/gallery_calibration.hpp:294`](../src/gallery/gallery_calibration.hpp#L294) — `Unknown`
- [`src/gallery/gallery_report.hpp:2`](../src/gallery/gallery_report.hpp#L2) — `Unknown`
- [`src/gallery/gallery_report.hpp:52`](../src/gallery/gallery_report.hpp#L52) — `struct GalleryBuildAudit`
- [`src/gallery/gallery_report.hpp:73`](../src/gallery/gallery_report.hpp#L73) — `struct GalleryReport`
- [`src/gallery/gallery_report.hpp:154`](../src/gallery/gallery_report.hpp#L154) — `inline GalleryReport build_gallery_report(const ActorGallery& gallery,`
- [`src/gallery/gallery_report.hpp:295`](../src/gallery/gallery_report.hpp#L295) — `inline nlohmann::json gallery_report_to_json(const GalleryReport& r)`
- [`src/gallery/gallery_report.hpp:361`](../src/gallery/gallery_report.hpp#L361) — `inline GalleryReport gallery_report_from_json(const nlohmann::json& j)`
- [`src/gallery/gallery_report.hpp:446`](../src/gallery/gallery_report.hpp#L446) — `inline void save_gallery_report(const std::string& path, const GalleryReport& r)`
- [`src/gallery/gallery_report.hpp:454`](../src/gallery/gallery_report.hpp#L454) — `inline GalleryReport load_gallery_report(const std::string& path)`
- [`src/gallery/gallery_report.hpp:464`](../src/gallery/gallery_report.hpp#L464) — `return gallery_report_from_json(j);`
### GR-004
**Locations:** 44
- [`src/config.hpp:54`](../src/config.hpp#L54) — `Unknown`
- [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown`
- [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown`
- [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)`
- [`src/gallery/gallery_store.cpp:82`](../src/gallery/gallery_store.cpp#L82) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/gallery/gallery_store.cpp:167`](../src/gallery/gallery_store.cpp#L167) — `H5::DataSpace scalar(H5S_SCALAR);`
- [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown`
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor`
- [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:201`](../tests/test_gallery_store.cpp#L201) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:219`](../tests/test_gallery_store.cpp#L219) — `TempFile tf("gallery_unstamped.h5");`
- [`tests/test_gallery_store.cpp:240`](../tests/test_gallery_store.cpp#L240) — `TempFile tf("gallery_json_stamp.json");`
- [`tests/test_gallery_store.cpp:252`](../tests/test_gallery_store.cpp#L252) — `Unknown`
- [`tests/test_gallery_store.cpp:268`](../tests/test_gallery_store.cpp#L268) — `Unknown`
- [`tests/test_gallery_store.cpp:300`](../tests/test_gallery_store.cpp#L300) — `Unknown`
- [`tests/test_gallery_store.cpp:309`](../tests/test_gallery_store.cpp#L309) — `Unknown`
- [`tests/test_gallery_store.cpp:333`](../tests/test_gallery_store.cpp#L333) — `Unknown`
- [`tests/test_gallery_store.cpp:348`](../tests/test_gallery_store.cpp#L348) — `Unknown`
- [`tests/test_gallery_store.cpp:367`](../tests/test_gallery_store.cpp#L367) — `Unknown`
- [`tests/test_gallery_store.cpp:384`](../tests/test_gallery_store.cpp#L384) — `TempFile tf("fake_model.onnx");`
- [`scripts/filter_gallery.py:80`](../scripts/filter_gallery.py#L80) — `if actor_jellyfin_id(a) in cast_ids]`
- [`scripts/make_gallery.py:181`](../scripts/make_gallery.py#L181) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:448`](../scripts/make_jellyfin_gallery.py#L448) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:456`](../scripts/make_jellyfin_gallery.py#L456) — `Unknown`
- [`scripts/movienet_eval.py:65`](../scripts/movienet_eval.py#L65) — `with open(args.gt) as f:`
- [`scripts/optimizer/fetch_missing_actors.py:62`](../scripts/optimizer/fetch_missing_actors.py#L62) — `def fetch(missing_path, out_path, token, build_dir, models_dir, arcface,`
- [`scripts/optimizer/fetch_missing_actors.py:109`](../scripts/optimizer/fetch_missing_actors.py#L109) — `def merge(base_path, add_path, out_path):`
- [`scripts/optimizer/fetch_missing_actors.py:124`](../scripts/optimizer/fetch_missing_actors.py#L124) — `def merge(base_path, add_path, out_path):`
- [`scripts/optimizer/optimize.py:186`](../scripts/optimizer/optimize.py#L186) — `Unknown`
- [`scripts/optimizer/optimize.py:202`](../scripts/optimizer/optimize.py#L202) — `if not Path(f["dump"]).exists():`
- [`scripts/optimizer/reembed_gallery.py:62`](../scripts/optimizer/reembed_gallery.py#L62) — `for i, a in enumerate(ref["actors"], 1):`
- [`scripts/optimizer/replay.py:113`](../scripts/optimizer/replay.py#L113) — `Unknown`
- [`scripts/optimizer/replay.py:253`](../scripts/optimizer/replay.py#L253) — `Unknown`
- [`scripts/sae_embed_loader.py:23`](../scripts/sae_embed_loader.py#L23) — `def resolve_arcface(models_dir: str, arcface: str \| None = None) -> str:`
- [`scripts/sae_gallery.py:171`](../scripts/sae_gallery.py#L171) — `if not _stamp_empty(embedder):`
- [`scripts/sae_gallery.py:200`](../scripts/sae_gallery.py#L200) — `for a in range(len(offset)):`
- [`scripts/sae_stamp.py:3`](../scripts/sae_stamp.py#L3) — `Unknown`
- [`scripts/stamp_gallery.py:4`](../scripts/stamp_gallery.py#L4) — `Unknown`
### GR-005
**Locations:** 1
- [`src/gallery/gallery_store.hpp:15`](../src/gallery/gallery_store.hpp#L15) — `Unknown`
### IR-001
**Locations:** 1
- [`src/nodes/result_sink_node.hpp:2`](../src/nodes/result_sink_node.hpp#L2) — `Unknown`
### IR-002
**Locations:** 5
- [`src/config.hpp:20`](../src/config.hpp#L20) — `struct Config`
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
- [`src/nodes/result_sink_node.hpp:122`](../src/nodes/result_sink_node.hpp#L122) — `void write_output()`
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
### IR-003
**Locations:** 1
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
### IR-004
**Locations:** 18
- [`src/audio_bindings.cpp:3`](../src/audio_bindings.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:3`](../src/audio_signature.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:265`](../src/audio_signature.cpp#L265) — `std::vector<std::uint8_t> pack_frames(const std::vector<float>& mono)`
- [`src/audio_signature.cpp:320`](../src/audio_signature.cpp#L320) — `std::optional<std::string> signature_from_mono(const std::vector<float>& mono)`
- [`src/audio_signature.cpp:327`](../src/audio_signature.cpp#L327) — `std::optional<std::vector<float>> decode_centre_window(const std::string& path)`
- [`src/audio_signature.cpp:421`](../src/audio_signature.cpp#L421) — `std::optional<std::string> compute_signature(const std::string& path)`
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:129`](../tests/test_audio_signature.cpp#L129) — `Unknown`
- [`tests/test_audio_signature.cpp:151`](../tests/test_audio_signature.cpp#L151) — `Unknown`
- [`tests/test_audio_signature.cpp:167`](../tests/test_audio_signature.cpp#L167) — `Unknown`
- [`tests/test_audio_signature.cpp:236`](../tests/test_audio_signature.cpp#L236) — `Unknown`
- [`tests/test_audio_signature.cpp:263`](../tests/test_audio_signature.cpp#L263) — `TempWav w("centred300");`
- [`tests/test_audio_signature.cpp:301`](../tests/test_audio_signature.cpp#L301) — `Unknown`
- [`tests/test_audio_signature.cpp:314`](../tests/test_audio_signature.cpp#L314) — `kNumBands * std::log10(1000.0 / kBandLoHz) / std::log10(kBandHiHz / kBandLoHz)));`
- [`tests/test_audio_signature.cpp:328`](../tests/test_audio_signature.cpp#L328) — `std::vector<float> a(kWindowSamples / 50);`
- [`tests/test_audio_signature.cpp:344`](../tests/test_audio_signature.cpp#L344) — `return base64_encode(reinterpret_cast<const std::uint8_t*>(s.data()), s.size());`
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
### IR-005
**Locations:** 6
- [`src/audio_bindings.cpp:3`](../src/audio_bindings.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:421`](../src/audio_signature.cpp#L421) — `std::optional<std::string> compute_signature(const std::string& path)`
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:129`](../tests/test_audio_signature.cpp#L129) — `Unknown`
- [`tests/test_audio_signature.cpp:138`](../tests/test_audio_signature.cpp#L138) — `Unknown`
### IR-006
**Locations:** 1
- [`scripts/run_from_jellyfin.py:4`](../scripts/run_from_jellyfin.py#L4) — `Unknown`
### IR-007
**Locations:** 8
- [`src/audio_signature.cpp:3`](../src/audio_signature.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:327`](../src/audio_signature.cpp#L327) — `std::optional<std::vector<float>> decode_centre_window(const std::string& path)`
- [`src/audio_signature.cpp:421`](../src/audio_signature.cpp#L421) — `std::optional<std::string> compute_signature(const std::string& path)`
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:201`](../tests/test_audio_signature.cpp#L201) — `TempWav w("short30");`
- [`tests/test_audio_signature.cpp:219`](../tests/test_audio_signature.cpp#L219) — `TempWav w("exact120");`
- [`tests/test_audio_signature.cpp:229`](../tests/test_audio_signature.cpp#L229) — `TempWav w("exact120");`
### IR-008
**Locations:** 7
- [`src/audio_signature.cpp:3`](../src/audio_signature.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:320`](../src/audio_signature.cpp#L320) — `std::optional<std::string> signature_from_mono(const std::vector<float>& mono)`
- [`src/audio_signature.cpp:421`](../src/audio_signature.cpp#L421) — `std::optional<std::string> compute_signature(const std::string& path)`
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:129`](../tests/test_audio_signature.cpp#L129) — `Unknown`
- [`tests/test_audio_signature.cpp:167`](../tests/test_audio_signature.cpp#L167) — `Unknown`
### IT-001
**Locations:** 1
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
### PR-002
**Locations:** 4
- [`src/nodes/embedding_dump_node.hpp:2`](../src/nodes/embedding_dump_node.hpp#L2) — `Unknown`
- [`scripts/optimizer/replay.py:5`](../scripts/optimizer/replay.py#L5) — `Reads an embedding dump (scripts/optimizer/SCHEMA.md), feeds each frame as an`
- [`scripts/optimizer/second_score.py:5`](../scripts/optimizer/second_score.py#L5) — `Unknown`
- [`scripts/validation/ground_truth.py:24`](../scripts/validation/ground_truth.py#L24) — `Unknown`
### PR-004
**Locations:** 1
- [`src/main.cpp:3`](../src/main.cpp#L3) — `Unknown`
### SR-001
**Locations:** 60
- [`src/backends/gemm_backend.cpp:44`](../src/backends/gemm_backend.cpp#L44) — `constexpr int kDim = 512;`
- [`src/build_gallery.cpp:83`](../src/build_gallery.cpp#L83) — `Unknown`
- [`src/config.hpp:54`](../src/config.hpp#L54) — `Unknown`
- [`src/gallery/embedder_stamp.cpp:1`](../src/gallery/embedder_stamp.cpp#L1) — `Unknown`
- [`src/gallery/embedder_stamp.hpp:2`](../src/gallery/embedder_stamp.hpp#L2) — `Unknown`
- [`src/gallery/gallery_builder.cpp:45`](../src/gallery/gallery_builder.cpp#L45) — `ActorGallery build_gallery(const BuildConfig& cfg)`
- [`src/gallery/gallery_calibration.hpp:80`](../src/gallery/gallery_calibration.hpp#L80) — `struct GalleryCalibrationStats`
- [`src/gallery/gallery_calibration.hpp:145`](../src/gallery/gallery_calibration.hpp#L145) — `std::vector<bool> actor_eligible(n_actors, false);`
- [`src/gallery/gallery_calibration.hpp:294`](../src/gallery/gallery_calibration.hpp#L294) — `Unknown`
- [`src/gallery/gallery_report.hpp:2`](../src/gallery/gallery_report.hpp#L2) — `Unknown`
- [`src/gallery/gallery_report.hpp:52`](../src/gallery/gallery_report.hpp#L52) — `struct GalleryBuildAudit`
- [`src/gallery/gallery_report.hpp:73`](../src/gallery/gallery_report.hpp#L73) — `struct GalleryReport`
- [`src/gallery/gallery_report.hpp:154`](../src/gallery/gallery_report.hpp#L154) — `inline GalleryReport build_gallery_report(const ActorGallery& gallery,`
- [`src/gallery/gallery_report.hpp:295`](../src/gallery/gallery_report.hpp#L295) — `inline nlohmann::json gallery_report_to_json(const GalleryReport& r)`
- [`src/gallery/gallery_report.hpp:361`](../src/gallery/gallery_report.hpp#L361) — `inline GalleryReport gallery_report_from_json(const nlohmann::json& j)`
- [`src/gallery/gallery_report.hpp:446`](../src/gallery/gallery_report.hpp#L446) — `inline void save_gallery_report(const std::string& path, const GalleryReport& r)`
- [`src/gallery/gallery_report.hpp:454`](../src/gallery/gallery_report.hpp#L454) — `inline GalleryReport load_gallery_report(const std::string& path)`
- [`src/gallery/gallery_report.hpp:464`](../src/gallery/gallery_report.hpp#L464) — `return gallery_report_from_json(j);`
- [`src/gallery/gallery_store.cpp:82`](../src/gallery/gallery_store.cpp#L82) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/gallery/gallery_store.cpp:167`](../src/gallery/gallery_store.cpp#L167) — `H5::DataSpace scalar(H5S_SCALAR);`
- [`src/gallery/gallery_store.cpp:219`](../src/gallery/gallery_store.cpp#L219) — `Unknown`
- [`src/kpn_bindings.cpp:167`](../src/kpn_bindings.cpp#L167) — `Unknown`
- [`src/kpn_bindings.cpp:217`](../src/kpn_bindings.cpp#L217) — `Unknown`
- [`src/main.cpp:188`](../src/main.cpp#L188) — `Unknown`
- [`src/nodes/embedding_dump_node.hpp:30`](../src/nodes/embedding_dump_node.hpp#L30) — `static constexpr std::string_view label() { return "embedding_dump"; }`
- [`src/nodes/embedding_dump_node.hpp:102`](../src/nodes/embedding_dump_node.hpp#L102) — `H5::StrType str(H5::PredType::C_S1, H5T_VARIABLE);`
- [`src/scene_preview.cpp:133`](../src/scene_preview.cpp#L133) — `int main(int argc, char** argv)`
- [`src/types.hpp:148`](../src/types.hpp#L148) — `struct Actor`
- [`tests/test_gallery_store.cpp:182`](../tests/test_gallery_store.cpp#L182) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:201`](../tests/test_gallery_store.cpp#L201) — `TempFile tf("gallery_stamped.h5");`
- [`tests/test_gallery_store.cpp:219`](../tests/test_gallery_store.cpp#L219) — `TempFile tf("gallery_unstamped.h5");`
- [`tests/test_gallery_store.cpp:240`](../tests/test_gallery_store.cpp#L240) — `TempFile tf("gallery_json_stamp.json");`
- [`tests/test_gallery_store.cpp:252`](../tests/test_gallery_store.cpp#L252) — `Unknown`
- [`tests/test_gallery_store.cpp:268`](../tests/test_gallery_store.cpp#L268) — `Unknown`
- [`tests/test_gallery_store.cpp:300`](../tests/test_gallery_store.cpp#L300) — `Unknown`
- [`tests/test_gallery_store.cpp:309`](../tests/test_gallery_store.cpp#L309) — `Unknown`
- [`tests/test_gallery_store.cpp:333`](../tests/test_gallery_store.cpp#L333) — `Unknown`
- [`tests/test_gallery_store.cpp:348`](../tests/test_gallery_store.cpp#L348) — `Unknown`
- [`tests/test_gallery_store.cpp:367`](../tests/test_gallery_store.cpp#L367) — `Unknown`
- [`tests/test_gallery_store.cpp:384`](../tests/test_gallery_store.cpp#L384) — `TempFile tf("fake_model.onnx");`
- [`scripts/filter_gallery.py:80`](../scripts/filter_gallery.py#L80) — `if actor_jellyfin_id(a) in cast_ids]`
- [`scripts/make_gallery.py:181`](../scripts/make_gallery.py#L181) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:448`](../scripts/make_jellyfin_gallery.py#L448) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:456`](../scripts/make_jellyfin_gallery.py#L456) — `Unknown`
- [`scripts/movienet_eval.py:65`](../scripts/movienet_eval.py#L65) — `with open(args.gt) as f:`
- [`scripts/optimizer/fetch_missing_actors.py:62`](../scripts/optimizer/fetch_missing_actors.py#L62) — `def fetch(missing_path, out_path, token, build_dir, models_dir, arcface,`
- [`scripts/optimizer/fetch_missing_actors.py:109`](../scripts/optimizer/fetch_missing_actors.py#L109) — `def merge(base_path, add_path, out_path):`
- [`scripts/optimizer/fetch_missing_actors.py:124`](../scripts/optimizer/fetch_missing_actors.py#L124) — `def merge(base_path, add_path, out_path):`
- [`scripts/optimizer/optimize.py:186`](../scripts/optimizer/optimize.py#L186) — `Unknown`
- [`scripts/optimizer/optimize.py:202`](../scripts/optimizer/optimize.py#L202) — `if not Path(f["dump"]).exists():`
- [`scripts/optimizer/reembed_gallery.py:62`](../scripts/optimizer/reembed_gallery.py#L62) — `for i, a in enumerate(ref["actors"], 1):`
- [`scripts/optimizer/replay.py:113`](../scripts/optimizer/replay.py#L113) — `Unknown`
- [`scripts/optimizer/replay.py:253`](../scripts/optimizer/replay.py#L253) — `Unknown`
- [`scripts/run_from_jellyfin.py:4`](../scripts/run_from_jellyfin.py#L4) — `Unknown`
- [`scripts/sae_embed_loader.py:23`](../scripts/sae_embed_loader.py#L23) — `def resolve_arcface(models_dir: str, arcface: str \| None = None) -> str:`
- [`scripts/sae_gallery.py:171`](../scripts/sae_gallery.py#L171) — `if not _stamp_empty(embedder):`
- [`scripts/sae_gallery.py:200`](../scripts/sae_gallery.py#L200) — `for a in range(len(offset)):`
- [`scripts/sae_stamp.py:3`](../scripts/sae_stamp.py#L3) — `Unknown`
- [`scripts/stamp_gallery.py:4`](../scripts/stamp_gallery.py#L4) — `Unknown`
### SR-002
**Locations:** 32
- [`src/config.hpp:44`](../src/config.hpp#L44) — `Unknown`
- [`src/config.hpp:108`](../src/config.hpp#L108) — `Unknown`
- [`src/evidence_discount.hpp:2`](../src/evidence_discount.hpp#L2) — `Unknown`
- [`src/face_utils.hpp:2`](../src/face_utils.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:2`](../src/gallery/gallery_calibration.hpp#L2) — `Unknown`
- [`src/gallery/gallery_calibration.hpp:53`](../src/gallery/gallery_calibration.hpp#L53) — `float boundary_at(float p = 0.5f, float log_prior_odds = 0.f) const`
- [`src/main.cpp:86`](../src/main.cpp#L86) — `static constexpr std::size_t kSceneJoinDepth = 256;`
- [`src/main.cpp:213`](../src/main.cpp#L213) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/main.cpp:230`](../src/main.cpp#L230) — `reg_cfg, EvidenceDiscounter(same_person));`
- [`src/main.cpp:299`](../src/main.cpp#L299) — `Unknown`
- [`src/main.cpp:309`](../src/main.cpp#L309) — `Unknown`
- [`src/main.cpp:379`](../src/main.cpp#L379) — `return run_net(std::move(net));`
- [`src/main.cpp:411`](../src/main.cpp#L411) — `Unknown`
- [`src/nodes/camera_position_change_detector_node.hpp:30`](../src/nodes/camera_position_change_detector_node.hpp#L30) — `struct CameraPositionChangeDetectorFunc`
- [`src/nodes/embedder_node.hpp:21`](../src/nodes/embedder_node.hpp#L21) — `struct EmbedderFunc`
- [`src/nodes/face_detector_node.hpp:2`](../src/nodes/face_detector_node.hpp#L2) — `Unknown`
- [`src/nodes/face_detector_node.hpp:47`](../src/nodes/face_detector_node.hpp#L47) — `private:`
- [`src/nodes/face_tracker_node.hpp:2`](../src/nodes/face_tracker_node.hpp#L2) — `Unknown`
- [`src/nodes/identity_matcher_node.hpp:117`](../src/nodes/identity_matcher_node.hpp#L117) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`src/nodes/identity_matcher_node.hpp:125`](../src/nodes/identity_matcher_node.hpp#L125) — `const GalleryCalibration& calibration() const { return cal_; }`
- [`src/nodes/identity_matcher_node.hpp:166`](../src/nodes/identity_matcher_node.hpp#L166) — `std::vector<float> host_query(static_cast<size_t>(kMaxFaces) * 512);`
- [`src/nodes/identity_matcher_node.hpp:272`](../src/nodes/identity_matcher_node.hpp#L272) — `Unknown`
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
- [`src/nodes/result_sink_node.hpp:63`](../src/nodes/result_sink_node.hpp#L63) — `void set_pre_write_hook(std::function<void(double)> fn) { pre_write_ = std::move(fn); }`
- [`src/nodes/result_sink_node.hpp:161`](../src/nodes/result_sink_node.hpp#L161) — `struct ActorMeta { std::string name, imdb_id, tmdb_id, jellyfin_id; };`
- [`src/nodes/scene_boundary_annotator_node.hpp:2`](../src/nodes/scene_boundary_annotator_node.hpp#L2) — `Unknown`
- [`src/nodes/scene_detector_node.hpp:36`](../src/nodes/scene_detector_node.hpp#L36) — `static constexpr std::string_view label() { return "scene_detector"; }`
- [`src/nodes/scene_detector_node.hpp:110`](../src/nodes/scene_detector_node.hpp#L110) — `void flush_remaining()`
- [`src/nodes/scene_detector_node.hpp:144`](../src/nodes/scene_detector_node.hpp#L144) — `void write_output()`
- [`src/scene_boundaries.hpp:2`](../src/scene_boundaries.hpp#L2) — `Unknown`
- [`src/track_registry.hpp:2`](../src/track_registry.hpp#L2) — `Unknown`
- [`tests/test_face_utils.cpp:1`](../tests/test_face_utils.cpp#L1) — `Unknown`
### SR-003
**Locations:** 7
- [`src/audio_bindings.cpp:3`](../src/audio_bindings.cpp#L3) — `Unknown`
- [`src/audio_signature.cpp:3`](../src/audio_signature.cpp#L3) — `Unknown`
- [`src/audio_signature.hpp:4`](../src/audio_signature.hpp#L4) — `Unknown`
- [`src/config.hpp:20`](../src/config.hpp#L20) — `struct Config`
- [`src/nodes/result_sink_node.hpp:2`](../src/nodes/result_sink_node.hpp#L2) — `Unknown`
- [`src/nodes/result_sink_node.hpp:49`](../src/nodes/result_sink_node.hpp#L49) — `static constexpr std::string_view label() { return "result_sink"; }`
- [`src/nodes/result_sink_node.hpp:122`](../src/nodes/result_sink_node.hpp#L122) — `void write_output()`
### SR-005
**Locations:** 9
- [`src/config.hpp:152`](../src/config.hpp#L152) — `Unknown`
- [`src/gallery/gallery_store.hpp:15`](../src/gallery/gallery_store.hpp#L15) — `Unknown`
- [`src/gallery/track_gallery.hpp:122`](../src/gallery/track_gallery.hpp#L122) — `void forget(int track_id) { tracks_.erase(track_id); }`
- [`src/gallery/track_gallery.hpp:132`](../src/gallery/track_gallery.hpp#L132) — `void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }`
- [`src/gallery/track_gallery.hpp:161`](../src/gallery/track_gallery.hpp#L161) — `struct TrackState`
- [`src/nodes/identity_matcher_node.hpp:110`](../src/nodes/identity_matcher_node.hpp#L110) — `std::vector<float> host_gallery(static_cast<size_t>(n_gallery_) * 512);`
- [`src/nodes/identity_matcher_node.hpp:147`](../src/nodes/identity_matcher_node.hpp#L147) — `MatchedSceneFrame operator()(TrackedSceneFrame tf)`
- [`src/nodes/identity_matcher_node.hpp:285`](../src/nodes/identity_matcher_node.hpp#L285) — `Unknown`
- [`scripts/make_jellyfin_gallery.py:4`](../scripts/make_jellyfin_gallery.py#L4) — `Unknown`
### UT-001
**Locations:** 1
- [`tests/test_track_registry.cpp:3`](../tests/test_track_registry.cpp#L3) — `Unknown`
### UT-101
**Locations:** 5
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:129`](../tests/test_audio_signature.cpp#L129) — `Unknown`
- [`tests/test_audio_signature.cpp:138`](../tests/test_audio_signature.cpp#L138) — `Unknown`
- [`tests/test_audio_signature.cpp:151`](../tests/test_audio_signature.cpp#L151) — `Unknown`
- [`tests/test_audio_signature.cpp:167`](../tests/test_audio_signature.cpp#L167) — `Unknown`
### UT-102
**Locations:** 5
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:201`](../tests/test_audio_signature.cpp#L201) — `TempWav w("short30");`
- [`tests/test_audio_signature.cpp:219`](../tests/test_audio_signature.cpp#L219) — `TempWav w("exact120");`
- [`tests/test_audio_signature.cpp:229`](../tests/test_audio_signature.cpp#L229) — `TempWav w("exact120");`
- [`tests/test_audio_signature.cpp:236`](../tests/test_audio_signature.cpp#L236) — `Unknown`
### UT-103
**Locations:** 2
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:263`](../tests/test_audio_signature.cpp#L263) — `TempWav w("centred300");`
### UT-104
**Locations:** 5
- [`tests/test_audio_signature.cpp:3`](../tests/test_audio_signature.cpp#L3) — `Unknown`
- [`tests/test_audio_signature.cpp:301`](../tests/test_audio_signature.cpp#L301) — `Unknown`
- [`tests/test_audio_signature.cpp:314`](../tests/test_audio_signature.cpp#L314) — `kNumBands * std::log10(1000.0 / kBandLoHz) / std::log10(kBandHiHz / kBandLoHz)));`
- [`tests/test_audio_signature.cpp:328`](../tests/test_audio_signature.cpp#L328) — `std::vector<float> a(kWindowSamples / 50);`
- [`tests/test_audio_signature.cpp:344`](../tests/test_audio_signature.cpp#L344) — `return base64_encode(reinterpret_cast<const std::uint8_t*>(s.data()), s.size());`
### UT-105
**Locations:** 1
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
### UT-106
**Locations:** 1
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
### UT-107
**Locations:** 1
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
### UT-108
**Locations:** 1
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
### VR-001
**Locations:** 2
- [`src/nodes/embedding_dump_node.hpp:2`](../src/nodes/embedding_dump_node.hpp#L2) — `Unknown`
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
### VR-002
**Locations:** 2
- [`tests/test_replay_fixtures.cpp:3`](../tests/test_replay_fixtures.cpp#L3) — `Unknown`
- [`scripts/optimizer/replay.py:5`](../scripts/optimizer/replay.py#L5) — `Reads an embedding dump (scripts/optimizer/SCHEMA.md), feeds each frame as an`
### VR-003
**Locations:** 1
- [`scripts/optimizer/second_score.py:5`](../scripts/optimizer/second_score.py#L5) — `Unknown`
### VR-004
**Locations:** 1
- [`scripts/validation/ground_truth.py:24`](../scripts/validation/ground_truth.py#L24) — `Unknown`
### VR-014
**Locations:** 1
- [`scripts/validation/test_audio_offset.py:5`](../scripts/validation/test_audio_offset.py#L5) — `The golden vector (IR-005) proves the *arithmetic* is identical in both`
-269
View File
@@ -1,269 +0,0 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15119,
"FPI": 1845,
"FPI_misid": 0,
"FPI_incast": 1845,
"FN": 4343,
"precision": 0.8912402735203961,
"precision_raw": 0.8912402735203961,
"recall": 0.7768471893947179,
"f1": 0.8301213418986437,
"agreement_rate": 0.7239983093829193,
"exact_match_rate": 0.41098901098901097,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 52022,
"FPI": 1160,
"FPI_misid": 0,
"FPI_incast": 1160,
"FN": 80089,
"precision": 0.9781881087586025,
"precision_raw": 0.9781881087586025,
"recall": 0.39377493168623356,
"f1": 0.5615106884771687,
"agreement_rate": 0.4057686401759256,
"exact_match_rate": 0.033084311632870865,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14988,
"FPI": 1086,
"FPI_misid": 58,
"FPI_incast": 1028,
"FN": 7087,
"precision": 0.9031091829356471,
"precision_raw": 0.9324374766703994,
"recall": 0.6789580973952435,
"f1": 0.775154508546456,
"agreement_rate": 0.7204967829586512,
"exact_match_rate": 0.3597703211914588,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15928,
"FPI": 4394,
"FPI_misid": 974,
"FPI_incast": 3420,
"FN": 23785,
"precision": 0.5475797579757976,
"precision_raw": 0.7837811239051274,
"recall": 0.40107773273235464,
"f1": 0.46301652592258835,
"agreement_rate": 0.3705156874642392,
"exact_match_rate": 0.04588936642173853,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18658,
"FPI": 548,
"FPI_misid": 0,
"FPI_incast": 548,
"FN": 12472,
"precision": 0.9714672498177653,
"precision_raw": 0.9714672498177653,
"recall": 0.5993575329264376,
"f1": 0.7413382072472983,
"agreement_rate": 0.5877853464704299,
"exact_match_rate": 0.21980294368081743,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15219,
"FPI": 2463,
"FPI_misid": 180,
"FPI_incast": 2283,
"FN": 4243,
"precision": 0.7884675163195524,
"precision_raw": 0.8607058025110281,
"recall": 0.7819854074606927,
"f1": 0.7852130843050252,
"agreement_rate": 0.7072306082196138,
"exact_match_rate": 0.34911242603550297,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 53043,
"FPI": 2383,
"FPI_misid": 604,
"FPI_incast": 1779,
"FN": 79068,
"precision": 0.8715290328940882,
"precision_raw": 0.9570057373795692,
"recall": 0.4015032813316075,
"f1": 0.5497453011561202,
"agreement_rate": 0.4094114144765671,
"exact_match_rate": 0.032817502668089645,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14606,
"FPI": 1337,
"FPI_misid": 180,
"FPI_incast": 1157,
"FN": 7469,
"precision": 0.8316346865569664,
"precision_raw": 0.916138744276485,
"recall": 0.6616534541336353,
"f1": 0.7369695746505879,
"agreement_rate": 0.6907677036961077,
"exact_match_rate": 0.31742329086667864,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15223,
"FPI": 4574,
"FPI_misid": 994,
"FPI_incast": 3580,
"FN": 24490,
"precision": 0.52962460425147,
"precision_raw": 0.768954892155377,
"recall": 0.383325359454083,
"f1": 0.44475283393712756,
"agreement_rate": 0.3554753116932427,
"exact_match_rate": 0.03715513655899071,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18472,
"FPI": 853,
"FPI_misid": 239,
"FPI_incast": 614,
"FN": 12658,
"precision": 0.860122927919538,
"precision_raw": 0.9558602846054334,
"recall": 0.5933825891423065,
"f1": 0.7022773067710907,
"agreement_rate": 0.5795914643682625,
"exact_match_rate": 0.18817662084904513,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 13580,
"FPI": 1666,
"FPI_misid": 60,
"FPI_incast": 1606,
"FN": 5882,
"precision": 0.86025592296972,
"precision_raw": 0.8907254361799817,
"recall": 0.697770013359367,
"f1": 0.7705401724920563,
"agreement_rate": 0.6547675401521545,
"exact_match_rate": 0.32578191039729504,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 48545,
"FPI": 1066,
"FPI_misid": 180,
"FPI_incast": 886,
"FN": 83566,
"precision": 0.9475708067381078,
"precision_raw": 0.9785128298159682,
"recall": 0.3674561542944948,
"f1": 0.5295567845883649,
"agreement_rate": 0.3815368792000116,
"exact_match_rate": 0.032950907150480255,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 13615,
"FPI": 963,
"FPI_misid": 120,
"FPI_incast": 843,
"FN": 8460,
"precision": 0.8695235662281262,
"precision_raw": 0.933941555768967,
"recall": 0.6167610419026047,
"f1": 0.7216494845360825,
"agreement_rate": 0.6506356469257915,
"exact_match_rate": 0.2894311860757222,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 13489,
"FPI": 3757,
"FPI_misid": 796,
"FPI_incast": 2961,
"FN": 26224,
"precision": 0.5526013928717739,
"precision_raw": 0.7821523831613127,
"recall": 0.3396620753909299,
"f1": 0.42072267361165266,
"agreement_rate": 0.3229817885335633,
"exact_match_rate": 0.04422570359073894,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 17692,
"FPI": 397,
"FPI_misid": 68,
"FPI_incast": 329,
"FN": 13438,
"precision": 0.9460456660071654,
"precision_raw": 0.9780529603626513,
"recall": 0.5683263732733698,
"f1": 0.710080070638759,
"agreement_rate": 0.5633540120828806,
"exact_match_rate": 0.13404695292543486,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
}
}
-221
View File
@@ -1,221 +0,0 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 14499,
"FPI": 1380,
"FPI_misid": 0,
"FPI_incast": 1380,
"FN": 12231,
"precision": 0.9130927640279615,
"precision_raw": 0.9130927640279615,
"recall": 0.5424242424242425,
"f1": 0.6805604449764134,
"agreement_rate": 0.57285804629501,
"exact_match_rate": 0.18947003810183582,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13893,
"FPI": 1654,
"FPI_misid": 174,
"FPI_incast": 1480,
"FN": 5737,
"precision": 0.811838952842868,
"precision_raw": 0.8936129156750499,
"recall": 0.7077432501273561,
"f1": 0.7562256756388972,
"agreement_rate": 0.7005158404089996,
"exact_match_rate": 0.38715420432758146,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 20518,
"FPI": 1078,
"FPI_misid": 58,
"FPI_incast": 1020,
"FN": 14722,
"precision": 0.9276607288181572,
"precision_raw": 0.9500833487682904,
"recall": 0.5822360953461975,
"f1": 0.7154363820216882,
"agreement_rate": 0.6297735703976657,
"exact_match_rate": 0.25910733470065433,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 13349,
"FPI": 677,
"FPI_misid": 0,
"FPI_incast": 677,
"FN": 6504,
"precision": 0.9517324967916726,
"precision_raw": 0.9517324967916726,
"recall": 0.6723920818012391,
"f1": 0.7880397886596416,
"agreement_rate": 0.7112222835587533,
"exact_match_rate": 0.40311896218603366,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 13456,
"FPI": 1434,
"FPI_misid": 180,
"FPI_incast": 1254,
"FN": 13274,
"precision": 0.8150211992731677,
"precision_raw": 0.9036937541974479,
"recall": 0.5034044145155256,
"f1": 0.622386679000925,
"agreement_rate": 0.5463565775524695,
"exact_match_rate": 0.19154832005542086,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13778,
"FPI": 1740,
"FPI_misid": 60,
"FPI_incast": 1680,
"FN": 5852,
"precision": 0.8580146967243741,
"precision_raw": 0.8878721484727413,
"recall": 0.7018848700967907,
"f1": 0.7721362923111411,
"agreement_rate": 0.6881858851455998,
"exact_match_rate": 0.36469460421802247,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 18863,
"FPI": 862,
"FPI_misid": 0,
"FPI_incast": 862,
"FN": 16377,
"precision": 0.956299112801014,
"precision_raw": 0.956299112801014,
"recall": 0.535272417707151,
"f1": 0.6863640498499044,
"agreement_rate": 0.594178111391709,
"exact_match_rate": 0.2357652114464303,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12642,
"FPI": 554,
"FPI_misid": 0,
"FPI_incast": 554,
"FN": 7211,
"precision": 0.9580175810851773,
"precision_raw": 0.9580175810851773,
"recall": 0.6367803354656727,
"f1": 0.7650458410239341,
"agreement_rate": 0.687468948385309,
"exact_match_rate": 0.3789677063207287,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 12207,
"FPI": 1119,
"FPI_misid": 60,
"FPI_incast": 1059,
"FN": 14523,
"precision": 0.88035482475119,
"precision_raw": 0.9160288158487168,
"recall": 0.45667789001122333,
"f1": 0.6013892994383683,
"agreement_rate": 0.5125626845924863,
"exact_match_rate": 0.1674748874263942,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13122,
"FPI": 1409,
"FPI_misid": 60,
"FPI_incast": 1349,
"FN": 6508,
"precision": 0.870678787074514,
"precision_raw": 0.9030348909228546,
"recall": 0.6684666327050433,
"f1": 0.7562894441082388,
"agreement_rate": 0.6698963754222389,
"exact_match_rate": 0.3389482333607231,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 16961,
"FPI": 685,
"FPI_misid": 0,
"FPI_incast": 685,
"FN": 18279,
"precision": 0.961181004193585,
"precision_raw": 0.961181004193585,
"recall": 0.48129965947786607,
"f1": 0.6414173883447416,
"agreement_rate": 0.5440709115628456,
"exact_match_rate": 0.2017775173356773,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12017,
"FPI": 591,
"FPI_misid": 122,
"FPI_incast": 469,
"FN": 7836,
"precision": 0.8767692981176127,
"precision_raw": 0.953125,
"recall": 0.6052989472623784,
"f1": 0.7161715188176049,
"agreement_rate": 0.6594534915815532,
"exact_match_rate": 0.3573005796301408,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
}
}
-362
View File
@@ -1,362 +0,0 @@
[
{
"crop": "eval/probe/nm0001589_0000.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0119_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0001.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0125_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0002.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0128_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0003.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0225_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0004.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0306_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0005.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0308_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0006.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0311_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0007.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0315_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0008.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0317_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0009.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0319_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0000.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0001.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0074_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0002.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0078_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0003.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0079_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0004.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0080_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0005.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0638_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0006.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0641_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0007.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0017_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0008.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0022_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0009.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0024_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0000.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0097_img_2.jpg"
},
{
"crop": "eval/probe/nm0005042_0001.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0100_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0002.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0108_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0003.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0110_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0004.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0121_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0005.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0123_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0006.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0151_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0007.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0158_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0008.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0178_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0009.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0180_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0000.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0154_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0001.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0157_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0002.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0161_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0003.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0163_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0004.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0164_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0005.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0167_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0006.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0199_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0007.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0200_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0008.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0201_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0009.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0202_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0000.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0005_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0001.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0009_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0002.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0010_img_2.jpg"
},
{
"crop": "eval/probe/nm1385871_0003.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0013_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0004.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0014_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0005.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0543_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0006.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0585_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0007.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0628_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0008.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0631_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0009.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0635_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0000.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0019_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0001.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0032_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0002.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0055_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0003.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0057_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0004.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0059_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0005.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0063_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0006.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0007.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0074_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0008.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0075_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0009.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0080_img_0.jpg"
}
]
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-20
View File
@@ -1,20 +0,0 @@
# MovieNet Validation Report
## Summary
| Model | Rank-1 | Det.Fail | Mean-sim | Probes | Size |
| ----- | ------ | -------- | -------- | ------ | ----- |
| R50 | 85.2% | 10.0% | 0.470 | 60 | 167MB |
| R18 | 72.2% | 10.0% | 0.453 | 60 | 46MB |
| MBF | 83.3% | 10.0% | 0.383 | 60 | 13MB |
## Per-Actor Recall
| Actor | R50 | R18 | MBF |
| ------------------ | ------------ | ------------ | ------------ |
| Steve Buscemi | 100% (9/9) | 100% (9/9) | 100% (9/9) |
| Michael Palin | 88% (7/8) | 50% (4/8) | 62% (5/8) |
| Jason Isaacs | 100% (10/10) | 100% (10/10) | 100% (10/10) |
| Paddy Considine | 50% (5/10) | 50% (5/10) | 60% (6/10) |
| Olga Kurylenko | 89% (8/9) | 67% (6/9) | 100% (9/9) |
| Andrea Riseborough | 88% (7/8) | 62% (5/8) | 75% (6/8) |
-32
View File
@@ -1,32 +0,0 @@
# experiments/ in git keeps only scripts, README.md, SESSION_STATE.md, and this
# file. Every data artifact — galleries, embedding dumps, the X-Ray corpus,
# montage/frame images, DE trajectories, film manifests, and result summaries —
# is pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to the Gitea
# generic package registry instead (see docs/rep4-optimizer-results.md).
xray/
dumps/
galleries/
*.h5
manifests/
trajectories/
results/
# Cross-source identification study: source clips and the hand-sorted face
# crops. The sorting is human ground truth and expensive to redo, so it goes to
# the artifact registry rather than being regenerated — push it once sorted.
xsource/clips/
xsource/labelling/
xsource/frames/
xsource/cache/
xsource/results_*.json
xsource/failure_analysis.json
xsource/*.jpg
# Raw run logs and scratch scripts (regenerated by every run).
_scratch/
# Real local media paths (film slug -> path on this machine). Never committed —
# these paths embed the specific source file names, which can include
# scene-release tags. Only file-lut.template.json (placeholders) is tracked;
# copy it to file-lut.json and fill in your own paths.
file-lut.json
-59
View File
@@ -1,59 +0,0 @@
# experiments/ — X-Ray validation & optimizer artifacts
Durable home (in the repo tree, NOT `/tmp` scratch — a scratch wipe once cost an hour)
for the data behind the X-Ray threshold-optimization and embedding-model bake-off.
## Layout
- `xray/` — Amazon X-Ray Zenodo dataset (gitignored, ~140MB; DOI 10.5281/zenodo.17659734).
- `dumps/` — per-model embedding dumps, one HDF5 per (model, film). Gitignored (large).
Naming: `<model>/dump_<Film>.h5`. Regenerate with `scene_analyze --dump-embeddings`.
- `galleries/` — per-model galleries (gitignored JSON). `gallery_<model>.json` +
augmented variants. Regenerate with build_gallery / fetch_missing_actors.
- `manifests/` — film manifests (committed — small, and the Jellyfin ID join is the
authoritative record of which films/paths/X-Ray-dirs were used).
- `trajectories/` — DE trajectories, one JSONL per run (committed — the evidence).
- `results/` — final per-run metrics + the model comparison table (committed).
## Embedding-model bake-off (July 2026)
Question: is LVFace-B (455MB) actually the best vs X-Ray, or just the biggest?
Method: **optimize per model** — each model gets its own dumps + gallery + full DE run,
then compare each model at ITS OWN optimum (fairest — no model penalised by another's
threshold). Scored by the weighted per-scene metric (out-of-cast misID ×10; see
docs/optimizer-experiments.md).
Models:
| model | file | size | MovieNet rank-1 (prior) |
| ----- | ---- | ---- | ----------------------- |
| LVFace-B_Glint360K | models/LVFace-B_Glint360K.onnx | 455 MB | — |
| ArcFace w600k R50 | models/arcface_w600k_r50.onnx | 174 MB | 85.2% |
| ArcFace R18 | models/arcface_r18.onnx | 48 MB | 72.2% |
| ArcFace w600k MBF | models/arcface_w600k_mbf.onnx | 13 MB | 83.3% |
9 genuine X-Ray-overlap films (Jellyfin ID join): Benny & Joon, Café Society,
Downton Abbey: A New Era, Lord of War, Lovelace, The Many Saints of Newark, Scarface,
Sound of Metal, Valerian.
## Gallery-mode bake-off (full vs cast-restricted)
Second axis alongside the model comparison: does restricting the matcher's candidate
set to a title's credited cast reduce cross-film misIDs (e.g. naming Archie Yates in a
film he's not in) vs. matching against the whole 2418-actor gallery?
- **full** — match against the entire model gallery (2418 actors).
- **restricted** — per film, match only against its Jellyfin credited cast, filtered
from the gallery by jellyfin_id. This is what run_from_jellyfin.py does in production.
**LIMITATION — Jellyfin stores only ~15 actors per title.** Jellyfin's People list is
capped at the top-billed cast (~15 Actors), NOT the full IMDb/X-Ray cast (e.g. Scarface:
Jellyfin 15 vs X-Ray 67). This is a hard limit of the metadata Jellyfin imports — not a
query parameter (verified: /Items?Fields=People returns 15 regardless; the single-item
/Items/{id} endpoint 400s on this server). So the "restricted" arm restricts to the ~15
top-billed leads, which caps its achievable recall at whatever fraction of on-screen
actors are top-billed, but should drive out-of-cast misIDs toward zero. A production
deployment wanting fuller cast restriction would need a richer cast source than Jellyfin
(TMDB/IMDb full credits).
Matrix: 4 models × {full, restricted} = 8 DE runs, all reusing the 36 dumps + 4 baseline
galleries (no augmentation — avoids test-set leakage on either arm). Scored by the
duration-weighted per-scene metric with the misID split (report_rates.py).

Some files were not shown because too many files have changed in this diff Show More