dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: dtourolle/scene-actor-extraction:gitea-pages
Branches Tags
dtourolle/scene-actor-extraction:master dtourolle/scene-actor-extraction:feature/opencv5 dtourolle/scene-actor-extraction:feature/quality-knee dtourolle/scene-actor-extraction:feature/traceability-tooling dtourolle/scene-actor-extraction:gitea-pages
dtourolle/scene-actor-extraction:v1.0.0
..
compare: dtourolle/scene-actor-extraction:e1423062e260d6a393affc0c2ffe258dec7ce612
Branches Tags
dtourolle/scene-actor-extraction:master dtourolle/scene-actor-extraction:feature/opencv5 dtourolle/scene-actor-extraction:feature/quality-knee dtourolle/scene-actor-extraction:feature/traceability-tooling dtourolle/scene-actor-extraction:gitea-pages
dtourolle/scene-actor-extraction:v1.0.0
114 Commits
gitea-pages .. e1423062e2
Author SHA1 Message Date
dtourolle e1423062e2 chore(experiments): X-Ray rerun and opencv5 dump runners 
- run_xray_lvface_opencv5.sh: end-to-end X-Ray benchmark (scene_analyze
  per film → sample_eval) on the current build with LVFace-B.
- dump_lvface_opencv5.sh: fresh LVFace-B embedding dumps (plain front-half,
  histogram cuts baked in) for the optimizer replay corpus. No decode-fps
  cap — that only mattered under parallel dumping; serial it just halved
  throughput.
- .gitignore: ignore build-*/ out-of-tree build dirs.
 2026-08-09 10:22:32 +02:00
dtourolle c374c262f5 perf(rocm): enable MIOpen exhaustive search and TunableOp 
The ROCm execution provider was appended with only device_id set, leaving
MIOpen on its no-workspace convolution fallback (the "GemmFwdRest,
provided ptr: 0 size: 0" warnings) and TunableOp off. Enable
miopen_conv_exhaustive_search and tunable_op_enable/tuning so MIOpen picks
the fast conv kernels and the GEMMs autotune; both cache to the MIOpen
user DB (MIOPEN_USER_DB_PATH), so the tuning cost is paid once per shape.
This helps the 2D-conv models that actually run on the ROCm GPU (SCRFD,
ArcFace). Opt out with SAE_ROCM_NOTUNE=1 for a quick no-warmup run.
 2026-08-09 10:22:07 +02:00
dtourolle 84513d3fa7 fix(eval): read schema-v2 scene windows 
The output schema moved to version 2, where each actor's "scenes" is a
list of {start, end, belief, route} objects rather than [t0, t1] pairs.
Both scorers still unpacked pairs and raised "too many values to unpack
(expected 2, got 4)" on current output. Read either form: an object's
start/end, or a two-element list. Fixes second_score.py (the optimizer's
per-second scorer) and sample_eval.py (the X-Ray/MovieNet presence eval).
 2026-08-09 10:21:56 +02:00
dtourolle d113c83189 feat(optimizer): sweep expansion bands + presence mode; tolerate scattered dropped votes 
Make the flood-fill and expansion knobs reachable from the DE sweep:

- kpn_bindings: read expand_band_lo/hi and presence_mode from the replay
  cfg dict (presence_mode accepts "flood"/"track_extent" or a numeric
  >=0.5 toggle), and carry is_scene_boundary onto the replayed frame.
- replay.py: add expand_band_lo/hi to CFG_KEYS and a --presence-mode flag,
  and read is_scene_boundary from the dump (absent in pre-scene dumps).
- optimize.py: map the continuous presence_flood knob (0..1, >=0.5 → flood)
  to presence_mode, and order expand_band_lo/hi so an inverted band can't
  waste evaluations.

Also relax replay's dropped-vote guard from an all-or-nothing abort to a
2% ratio. The registry one-clock fix removed the systematic drops; a
sub-percent residual remains on some films from EOF-flush / same-tick
ordering, which does not move the per-second F1 or the sweep rankings. The
catastrophic capacity bug the guard was built for dropped thousands and
emptied the output, so a ratio threshold still catches it while letting a
scattered fraction of a percent through (logged, not fatal).
 2026-08-09 10:21:45 +02:00
dtourolle 584f23546a feat(presence): flood-fill presence mode 
Add PresenceMode::flood alongside the default track_extent. In flood mode
the result sink snaps each presence claim to the shot it sits in, so an
actor seen once anywhere in a shot is reported for the whole shot
[prev_boundary, next_boundary]. This trades precision for recall against
X-Ray's per-scene cast granularity and is a toggleable knob for the
optimizer to weigh rather than a default.

Boundaries come from the frame stream, now carried through SceneAnnotation
(is_cut and is_scene_boundary). Flood prefers TransNetV2 shot boundaries
when a scene detector populated them, otherwise falls back to the
always-on histogram cuts (camera_position_change_detector); with no
boundaries it degrades to track_extent per claim. The is_scene_boundary
path stays dormant so an out-of-process scene detector can be revived
later without re-wiring.

Selected with --presence-mode flood|track_extent (default track_extent),
so existing output is byte-for-byte unchanged. The dump_embeddings header
note records why TransNetV2 scene detection is not run in that process.
 2026-08-09 10:21:13 +02:00
dtourolle de02e25e6a fix(registry): one clock for association and reaping 
FrameScope::candidates() filtered associable tracks on the tracker clock
(now_) while reap_locked() retired them on the matcher's evidence
watermark (evidence_through_). Because the tracker runs ahead of the
matcher (backpressure turns channel depth into lag, AR-004), a track the
registry had already reaped on the watermark could still be offered for
association on the tracker clock; the matcher's later vote then landed on
an erased id and was counted as a dropped vote, silently under-reporting
presence. Rare live (small lag), routine in replay where the Python
source drives the tracker far ahead of the matcher.

candidates() now filters on the same clock reaping uses
(awaits_evidence_ ? evidence_through_ : now_) with the same
track_extinction_sec horizon, so the offered pool and the live pool are
identical: nothing is offered past its reap horizon, nothing is reaped
while still offerable. This also cannot reintroduce the older zombie-merge
bug (offering on a looser horizon than the reap), because the horizon is
now identical rather than looser.

track_extinction_sec is the tracker's association window, so it must be
evaluated on the horizon the registry retires tracks on.
 2026-08-09 10:19:22 +02:00
dtourolle 7b73bf923a fix(AR-025): only evidence spends the evidence budget 
The correlation discount is an effective-sample correction: with observations
correlated at rho, the n-th is worth n_eff(n+1) - n_eff(n) = 2/((n+1)(n+2))
at rho=0.5, so it decays quadratically and the total converges to 1/rho = 2.
That saturation is deliberate and stays — a long static shot must not
out-argue varied evidence purely by lasting longer.

What was not deliberate is that every scored face spent it. An observation at
p=0.02 contributes log(0.98) = -0.02 of belief, which is nothing, while
consuming the same increment as one at p=0.95. On SuperHero-2, track 3 carried
103 observations for an effective weight of 2.026 and a belief of 0.455
against a 0.881 threshold — with the 51 frames that *did* identify the actor
arriving when each was worth 0.0002. The budget had been spent by the frames
that recognised nobody.

It also made the answer depend on frame rate: deliver more frames, dilute the
budget with more non-matches, and a track that was owned stops being owned.
That is the defect AR-013 already had to fix once for reaping, still present
in accumulation. It is how this was found — SuperHero-2 identified Jeremy
before a KPN throughput fix and not after, from identical input, and bisect
put the change at the KPN bump in d98dc28 rather than anywhere in this repo.

The floor is 0.5, which is where the posterior stops favouring the hypothesis,
not a tuned threshold. Near-misses still count: the identity matcher
deliberately feeds every scored face rather than only accepted ones, and an
observation at 0.7 — under the matcher's 0.754 acceptance — still accumulates,
which the second test pins. Below 0.5 the observation argues *against*, which
noisy-OR cannot represent, so declining to spend a budget on it loses nothing.

Scored against the DVU knowledge-graph ground truth, all ten SuperHero scenes:

                 TP   FP   FN   precision   recall     F1
  before          9    0   15       1.000    0.375   0.545
  after          13    0   14       1.000    0.481   0.650

Four more true positives and no false positives — the recall gain is not
bought with precision. Scene 7's Isabelle and scene 10's Isabelle and Jeremy
are all in the ground truth. Apples-to-apples over scenes 1-9 (scene 10's
before-run timed out): TP 9 to 11, FN 15 to 13, FP 0 either way.

An earlier attempt at this gave each actor its own budget. It was a no-op:
measured on the same track, n_obs_for[best] equalled n_obs at 103, because
every observation votes for the best-matching actor. Reverted rather than
kept.

151/151.

TRACES: AR-025, AR-013 | SR-002
 2026-08-08 14:48:40 +02:00
dtourolle 24d35cbde3 fix(AR-013): reap tracks on the evidence watermark, not the tracker's clock 
The registry closed a track when the *tracker's* timestamp passed
`track_extinction_sec`. But votes arrive from the matcher, which is a separate
KPN node behind a channel, and much the slower of the pair. Backpressure —
working exactly as AR-004 intends — turns that channel's depth into lag, so
the tracker's clock can be far ahead of the last frame anybody has voted on.
Tracks were therefore closed before their evidence arrived: the votes landed
on ids that no longer existed, were counted as dropped, and the track was
emitted unowned or not at all.

The symptom is the part worth remembering: **a deeper channel produced fewer
identifications, from identical input.** On the SuperHero fixture, 5 actors /
16 windows at depth 32 against 3 actors / 5 windows at depth 10322; through
the replay harness, capacity 32 gave 5 actors and 10322 gave 0. A throughput
knob was silently changing the answer, which makes every sweep tuned against
it suspect.

The fix is not to bound the channel against `track_extinction_sec` — that
makes an algorithm constant police a throughput knob and leaves the result a
function of scheduling. It is to reap on an evidence watermark: the matcher
advances it as it folds each frame in, and a track is only finished once
everything up to its extinction point has actually been voted on. Same device
`SceneBoundaries::scored_through()` uses for the AR-010 join — a consumer past
that point is asking about frames nobody has looked at yet, and the honest
answer is to wait rather than guess.

Association keeps the tracker's clock, and separating the two is the other
half. They answer different questions: "may this detection link to that
track?" is asked now, about a box seen `track_extinction_sec` ago; "is that
track finished?" cannot be answered until every vote is in. Deferring
association to the evidence clock — which deferring the erase alone did — left
retired tracks associable for as long as the matcher lagged, so a new face
re-associated onto a long-dead track and two people merged into one window.

The watermark is monotonic and only ever *delays* a reap, so no window is
extended by it: AR-013's "a window ends at the last sighting, never after" is
a property of `emit_locked`, which takes `last_seen` and never `now`.

`dropped_votes` is exposed and reported — by main at shutdown and through the
replay bindings — because this failed silently for as long as it did precisely
because nothing counted it. It warns rather than aborts: a dropped frame means
the output describes footage nobody analysed and is always wrong, while a
dropped vote degrades a claim without falsifying it, and there is no
measurement yet of how often it happens on real content.

replay.py's channel capacity stops being the whole film. It was sized that way
to dodge a PyNode overflow drop that AR-004 has since replaced with parking,
and removing backpressure that way is what made the defect above so extreme.

Tag separators in kpn_bindings.cpp corrected to pipes between requirement
types, which the traceability gate was reporting as diagnostics; the matrix is
regenerated and reports 0 orphan tags.

149/149.

TRACES: AR-004, AR-012, AR-013, AR-025 | VR-011 | SR-002 | PR-002
 2026-08-08 12:08:15 +02:00
dtourolle 0e27339ab6 chore: update KPN — unconditional push wake, and the start() lock it exposed 
Two commits on top of c9aa246.

6802328 fixes a lost wake in the channel itself: push fired push_callback_
only on the empty->non-empty edge, computed from a head_ sampled before the
item was published, so a pop landing in that window left the item in the ring
with the consumer idle and no wake outstanding. Absorbing, because every later
push then sees a non-empty ring and the edge never fires again. Firing
unconditionally is correct by construction — the callback runs after the
publishing store — and the redundant wakes are cheap, since on_input_ready
re-checks the level and the submit gate collapses a wake arriving mid-firing.

771b9f8 is the gap that change exposed. abbb2d4 guarded ThreadPool::submit
against stop(), which clears queues_, but missed that start() rebuilds the
same vector under no lock. A submission genuinely lands there — a network
starts nodes one at a time, and an already-started node fires into the next
one's channel, whose callback submits — and push_back can reallocate under a
reader that has already indexed it. Unobserved before; 4 races in one run
after callbacks became frequent enough during startup.

Verified: KPN 148/148, this repo 149/149, ThreadSanitizer clean across five
unit runs and two stress runs.

Not re-measured on a clip. The GPU has a driver/library version mismatch
(kernel module 610.43.03 against NVML 610.57 — an update without a reboot),
so scene_analyze aborts in cuInit before it decodes anything. The SuperHero-1
measurement recorded in c599e07 — exit 0, outran() 0, scenes.json and presence
byte-identical — stands against c9aa246; both commits added since are
concurrency fixes with no effect on pipeline semantics, but a re-run is worth
doing once the driver is sorted.

TRACES: AR-004 | SR-002
 2026-08-06 22:59:18 +02:00
dtourolle c599e07d4b chore: update KPN — a re-offered sentinel is not data loss 
Picks up c9aa246, which the SuperHero run turned up. 139bfbb made the channel
refuse a sentinel offered while one was still pending — right, and it fixes a
real use-after-free — but it recorded the refusal as a drop and reported it as
an overflow.

frame_source emits EOF once and then returns it forever
(frame_source_node.hpp:76), so the token is re-offered on every firing. This
pipeline's own loss detector therefore fired on a clean run:

    [main] ERROR: frames were dropped (channel overflow):
      frame_source: 2
      scene_annotate: 1
    [main] The output would describe footage that was never analysed.
           Refusing to report success.

exit 2, with nothing dropped and every frame analysed. A loss detector that
cries loss on a clean run is worse than none, because the next real one gets
ignored.

Measured on SuperHero-1 at --fps 5 --scene-detect after the fix: exit 0, no
drops, and outran() 0 — the AR-004 join-depth derivation (59a2927) holds at 95
slots where it was pinned at 256.

TRACES: AR-004 | SR-002
 2026-08-06 20:54:45 +02:00
dtourolle 4dcef8d6c5 fix(AR-004): the TransNetV2 window stores the model's input, not the frame 
The rolling window held frames as decoded — `images_.push_back(f.image)` — and
left the downscale to the backend. TransNetV2's input is 48x27, so the buffer
held roughly 590 MB at 1080p to feed a model that needs about 380 KB. The
config note for `dense_scale` says as much outright: "TransNetV2 downsamples to
48x27 regardless".

This is not a channel capacity, so no amount of tuning channel depths would
ever have found it. It is a `std::deque<cv::Mat>` member, and it is the single
largest allocation in the scene branch.

It is also redundant work. Windows overlap by `kWindow - stride`, so a frame
appears in several of them and was re-downscaled once per window it appeared
in; now it is downscaled once, on arrival.

**The risk here is the invariant, not the memory.** Every model gets the input
it was trained for — a model run off-distribution returns confident, plausible,
wrong output, and for a boundary detector that means fabricated cuts, which are
indistinguishable from real ones in the output. So this reproduces the
backends' preprocessing exactly rather than doing its own: both
ort_backend.cpp and trt_backend.cpp guard mis-sized input with
`convertTo(CV_8UC3)` and then
`cv::resize(..., {kFrameW, kFrameH}, 0, 0, cv::INTER_AREA)`, in that order, and
`to_model_input` performs the same two operations. The backend guard then sees
a correctly-sized frame and does nothing, so the tensor the model receives is
unchanged. The interface has always specified this as the caller's job — "Each
frame must already be kFrameW x kFrameH, BGR, CV_8UC3" — so the node now meets
a contract it was already given.

The tests assert equivalence, not size. They perform the backend's own two
operations independently and compare byte for byte, on a gradient rather than a
flat fill, since INTER_AREA averages and a constant image would compare equal
under almost any resize. Order is pinned too: converting a 4-channel frame
after downscaling averages alpha into the colour channels and gives different
pixels.

Verified in both directions. With INTER_LINEAR substituted for INTER_AREA —
the most plausible way to get this subtly wrong — three assertions fail. With
the backend's own operations, byte-identical at 1920x1080, 640x360 and 720x480.
149/149.

Still unmeasured on real content, as with the previous commit: the equivalence
argument says the model sees the same tensor, but a run comparing scenes.json
before and after on a real clip is what would settle it, and I could not launch
one here.

TRACES: AR-004, AR-010 | SR-002
 2026-08-06 20:34:16 +02:00
dtourolle 59a2927a15 fix(AR-004): derive the scene join depth instead of pinning it at 256 
`kSceneJoinDepth` was a constant 256, used for two different channels. Both
uses were wrong, in different ways.

**It is a span of film, not a count of slots.** The join needs the sampled
branch to trail the dense one by however long TransNetV2 takes to be able to
answer: its input queue (128) plus the window it must fill (100), over native
frame rate — about 9.5 s at 24 fps, which is the conservative floor since a
slower source makes the same frame count span more film. The slots needed to
hold that depend on `sample_fps`, so the constant meant ~256 s of lag at 1 fps
— 27x what the join needs — and nothing recomputed it when `sample_fps`
changed. The one number AR-010's correctness rests on drifted with an
unrelated knob. It is now derived, giving a constant 19 s of lag at any sample
rate: 19 slots at 1 fps, 95 at 5.

**The decimator's buffer was on the wrong side of the decimator.** Its *input*
carries the full-rate stream, so 256 slots held ~256 full decoded frames of
which, at 1 fps against 24 fps native, 23 in every 24 existed only to be
discarded by the predicate a moment later. Holding ~1.5 GB of images for
frames the very next node throws away is the worst available use of the
budget. A filter is a pass-through, not a reservoir: the lag belongs after
decimation, where a slot buys 1/sample_fps seconds instead of 1/native_fps.
Its input is now sized only to keep it fed.

Together, at 1080p and 1 fps, those two channels go from ~3.0 GB to ~206 MB
while the join keeps 2x margin over its requirement. Every message embeds
`Frame source`, so a slot on either branch holds a full decoded image — which
is visible now that the byte counter is honest (5e46f52), and was not before.

**Not verified against a running pipeline.** The check that matters is
`SceneBoundaries::outran()` — sampled frames that arrived before the detector
had scored them, which must stay 0 — and that needs a real clip through
`--scene-detect`. I could not launch it here. The arithmetic is stated above
so it can be checked by inspection, and the margin is deliberately 2x rather
than tight, but a run should confirm outran() is still 0 before this is
trusted on real content.

TRACES: AR-004, AR-010 | SR-002
 2026-08-06 19:44:44 +02:00
dtourolle 5e46f52ad2 fix(AR-004): make the channel byte counter measure the payload 
`kpn::ChannelDataSize<T>` is what a channel reports as bytes pushed, and its
primary template returns `sizeof(T)`. It was never specialised in this repo —
only in a KPN example — so every message type reported its header size. Each
of them is a few vectors and a `cv::Mat` header owning megabytes on the heap,
so a message carrying a full decoded frame was reported at roughly 200 bytes
against 5.9 MB at 1080p. Four orders of magnitude.

That is not a cosmetic stat. It is the one instrument for choosing channel
capacities against a memory ceiling — the open half of AR-004 — and anyone
who read the MB/s column to size a channel was reading fiction. The gap could
not be measured with the tool that exists to measure it.

Every message embeds `Frame source`, so this is not confined to the
crop-carrying channels: the full decoded image rides the whole chain, and the
byte figure now says so.

Two things worth stating about what the number means. `cv::Mat` is
reference-counted, so one frame referenced from several messages is counted
once per reference — an upper bound on distinct bytes, and the right bound for
"what would this channel keep alive if nothing else held it", which is the
question a capacity answers. And an eof sentinel carries no image, so it costs
only its header and is not charged for one.

Declared against a forward declaration of the primary template rather than by
including <kpn/channel.hpp>, so the message definitions keep no dependency on
the framework carrying them, and any translation unit that can see these types
also sees their sizes — which is what stops one channel being instantiated
with the default while another gets the specialisation.

Verified in both directions: with the specialisations removed, three of the
five cases fail, `SceneAnnotation` reporting 40 bytes against the 37,632 its
single 112x112 crop occupies. 145/145 with them.

No behaviour change — this only corrects what is reported. Choosing capacities
against the corrected numbers is the next commit.

TRACES: AR-004 | SR-002
 2026-08-05 20:17:03 +02:00
dtourolle bb7a9ed718 docs(AR-004): record the three holes the wedging audit closed 
The register said "two remaining holes now closed" and the SPEC's Current:
still described `push_blocking`, which parking replaced. Both now match the
code.

Three additions, in the order they surfaced:

  (c) FilterNode and RouterNode were the last data paths still using the
      throwing push() with the exception swallowed, so a full output discarded
      the value — including the EOF sentinel. The decimator passes EOF by
      predicate but its output is reliably full, the embedder being the slowest
      node, so the token went nowhere and nothing downstream shut down. That is
      the wedge the runs were being killed for, and it is worth the register
      saying so plainly.
  (d) The sentinel could be delivered ahead of a value still queued behind it,
      losing the tail to any consumer treating EOF as a hard stop.
  (e) Two firings of one node could overlap, which breaks the one-slot park
      itself: a parked value can be overwritten with no drop recorded.

(e) also corrects (b). The startup lost wake was recorded as a missed
empty->non-empty edge closed by a level-triggered re-check; the actual cause
was the callbacks being written while a running neighbour read them — ten
ThreadSanitizer races — and they are now installed in a prepare() pass before
any node starts. The re-check stays and is still needed, but for a benign
ordering rather than as cover for a race.

Two things the requirement now carries that it did not before. A channel holds
at most one undelivered sentinel: a second is refused and reported rather than
silently overwriting the first, which matters the moment a pipeline is reused
for a second input. And a lossless decimator is a backpressure point rather
than a relief valve, so the source throttles to the face branch instead of
quietly thinning it — what the requirement asks for, but it changes the shape
of a loaded run and is not yet benchmarked.

The Gap is unchanged and still open: capacity is counted in items, not bytes,
so a crowd frame carrying 60 crops occupies one slot exactly as an empty one
does. AR-004 stays **Mostly** for that reason.

Verification plan updated with the cases the KPN suite now pins.

TRACES: AR-004 | SR-002
 2026-08-05 19:47:06 +02:00
dtourolle 48332d2041 feat(replay): the whole replay chain is C++, including the sink 
The sae_kpn module has not compiled since the AR-007/AR-008 tracker redesign,
and was switched off at the build rather than patched because the fix is a
restructuring. Two failures, one cause.

It did not compile: `add_face_tracker` built FaceTrackerFunc from a Config
alone, and the tracker has required a TrackRegistry and a calibration since
association moved into probability space.

And presence was rebuilt in Python. `replay.py::build_minimal` merged
per-frame detections into windows by annealing gaps, which is what the
pipeline did before AR-012. The sink builds a window from a TrackRegistry
claim instead — the extent of a track an actor owned, starting when they
appeared rather than when recognition first succeeded. Those answer different
questions, so every sweep was tuning against a contract the shipped code had
stopped honouring.

Both follow from the seam being a factory per node. The chain has a
construction order — the matcher fits the calibration, the registry needs a
discounter built from it, the tracker needs both, and the sink needs the
registry's claims — and independent factories cannot express it, so the
tracker kept being built against a signature that no longer existed. One
`add_pipeline` mirrors main.cpp exactly and is now the only way to build the
chain, so the ordering cannot be got wrong again from Python. DP-001 is the
requirement behind it: a replay harness is a front-end, and its job is to
supply frames and read the result, not to re-derive presence.

Lifetimes needed a home. ResultSinkFunc holds `const Config&` and
`std::atomic<bool>&`, which under main() are locals in a frame outliving the
pipeline; there is no such frame when the network is built and torn down from
Python. ReplaySession owns both for the network's lifetime, keyed by network
and released explicitly — a sweep builds one network per replay and the sink
retains every annotation, so holding them forever would grow with films x
configs. Getting this wrong presented as an empty output_path: the sink
announced `[result_sink] writing ` and wrote nothing.

test_sae_kpn.py is ported rather than left behind. It called all three removed
factories and asserted on SceneAnnotations read back per frame; neither half
survives, so it now waits on pipeline_done and asserts on the file the sink
writes. Verified against gallery_lvface.h5: three frames through the real
chain, timestamps 0/1/2, truth file written. EOF is a control token the sink
flushes on and does not record, so three inputs give three frames, never four.

SAE_BUILD_KPN_BINDINGS goes back to ON.

TRACES: VR-011, VR-002 | DP-001 | PR-002
 2026-08-05 19:40:25 +02:00
dtourolle 1141172b04 chore: update KPN to the wedging-audit fixes 
Twenty commits, of which the one that matters for this repo is the filter
and router losslessness. FilterNode and RouterNode were the last nodes on a
data path still using the throwing push() and swallowing the exception, so
`decimate` — which passes EOF by predicate — discarded that token whenever
its output was full. Which it reliably is: the embedder is the slowest node
in the chain. Nothing downstream ever received EOF, `done` was never set,
and the run had to be killed. That is the wedge.

The rest, in rough order of how much they affect a run here:

- A sentinel could be delivered ahead of a value still queued behind it,
  losing that value to any consumer treating EOF as a hard stop.
- Two fire_once invocations for one node could overlap, which breaks the
  one-slot park: a parked value can be overwritten with no drop recorded.
- A node's push/space callbacks were written while a running neighbour read
  them — ten ThreadSanitizer races, and the root cause of the startup lost
  wake that AR-004 records as closed by a level-triggered re-check.
- shutdown() polled every channel in the graph with no deadline, and could
  fail to terminate outright on an index underflow in the fill calculation.
- Submitting to a stopped pool indexed a cleared vector: a segfault, which
  reproduced 12 runs in 20 once sources stopped before their consumers.
- stop() returned while a firing was still touching the node's members.
- Idle pool workers burned ~6.5 cores while one task ran (1991 ms of CPU
  against 0.4 ms). Latent here, since every node owns a private one-thread
  pool, but not for anything using a shared one.

Two consequences worth holding onto. `decimate` is now a backpressure point
rather than a relief valve, so the source throttles to the face branch
instead of quietly thinning it — the intended behaviour, but it changes the
shape of a loaded run and has not been benchmarked. And a channel now
carries at most one undelivered sentinel; a second is refused and reported
rather than silently overwriting the first.

Verified: KPN 148/148 with ThreadSanitizer clean across six runs, this repo
138/138 with every target building warning-free against the new headers.

TRACES: AR-004 | SR-002
 2026-08-05 18:26:11 +02:00
dtourolle 90b44e0975 docs(register): make the status column describe the code 
Eleven rows corrected, in both directions.

Overstated: AR-011 (the derived dedup window reached scenes.json only),
AR-017 (route was a literal), AR-019 (the local plurality tally was
still deciding), AR-024 (its "static check" enforcement did not exist),
DP-001 (scene_preview had forked and stopped compiling), VR-002 (the
Python replay bindings have not compiled since the tracker redesign, and
the fixtures it calls committed are gitignored registry artifacts).

Understated: VR-010 was marked Planned while five VR-010 tags sat in the
code implementing it.

Rescoped: VR-007 now names the four AR-025 constants it was already
being deferred to for, and which no sweep could reach until this pass.

The Withdrawn note gets the longest correction, because it asserted a
removal that had not happened and nothing could have caught that: the
gate reads tags, and a withdrawn requirement has no tag to be orphaned.
The general form is now written down there -- a status column is a
claim, and the only claims this project checks automatically are the
ones a test or a static check makes. Four of the rows above are the same
pattern: recorded as done, and done in one place out of two.

New: VR-016, a cadence study for cut_threshold. It is the one always-on
signal with no recorded provenance, and its input rate depends on an
unrelated flag -- with --scene-detect off, camera_pos compares frames a
full second apart at the default sample_fps, and with it on, native-rate
frames. Same constant, two meanings, and is_cut drives track_alpha to 0
and clears every expansion buffer.

Also stops check_raw_cosine.py inflating its own metric: the extractor
scans scripts/, so the tool's prose describing the exception tag was
counted as four recorded exceptions. The count now reads 1, which is the
number of real ones.

TRACES: AR-011, AR-017, AR-019, AR-024, AR-025 | DP-001, DP-007 | IR-004 | VR-002, VR-007, VR-010, VR-016
 2026-08-05 17:51:01 +02:00
dtourolle 06373817a2 refactor(config): give the tuned constants a real provenance, and make them reachable 
Two problems, both of which made a number look more settled than it is.

The provenance was a dead link. config.hpp cited
docs/rep4-optimizer-results.md for prob_threshold, extinction_sec,
anneal_sec and the expansion default. That file was renamed to
model-bakeoff.md and then rewritten; the comments were never repointed,
so the most consequential constant in the pipeline appeared to have no
source at all.

Following it up produced something worse than a broken link.
prob_threshold=0.754 comes from the ORIGINAL rep4 document (still
readable at `git show d340da7:docs/rep4-optimizer-results.md`). The
rewrite that replaced it reports finding "a real scoring bug in
optimize.py: a candidate whose hardest film's replay timed out was
averaged over survivors instead of penalized, silently rewarding partial
coverage. Affected 3 of 16 training combos". So 0.754 was fitted under
scoring that was later found wrong, the corrected sweep converged
elsewhere, and no corrected prob_threshold is recorded anywhere. The
comment now says that, along with the surviving document's own verdict
that the optimum "generalizes unevenly -- strong on 3 of 5 held-out
films, badly broken on 2".

Four constants were unreachable. ownership_logodds lived on
TrackRegistry::Config, and max_views/admit_below/rho_max on
EvidenceDiscounter::Config, which main built with the one-argument
constructor -- so nothing short of a recompile could move any of them.
rho_max's own comment defers to "the sweep (VR-007)" for where it
belongs, and that sweep could not reach it.

They now live in Config with CLI flags and are exposed to the replay
harness. ownership_logodds is worth singling out: below it a track makes
no presence claim at all, so it decides whether an actor is reported
rather than how confidently -- arguably the most consequential constant
after prob_threshold, and until now unswept and unsettable.

No behaviour change: every default is the value that was compiled in.

TRACES: AR-025, AR-017 | SR-002
 2026-08-05 17:43:51 +02:00
dtourolle 88c42573a5 fix(pipeline): finish three changes that had only been half applied 
Each of these was recorded as done and was done in one place out of two.

AR-011 -- the TransNetV2 dedup window. The derived window
(dedup_window_sec, median observed interval halved) reached scenes.json
and nothing else. SceneBoundaries, the path that actually feeds
is_scene_boundary to the tracker, kept the literal 0.04 s under a
comment claiming it "matches the dedup scenes.json applies, so the two
views agree". They did not agree. 0.04 is one frame at 25 fps and wider
than a frame at 30, so two cuts on consecutive frames merged into one
and the loss was invisible: the pipeline simply saw fewer boundaries.
The detector now supplies the window it derived.

AR-019 -- ownership. The register says ownership "comes from the
registry, not a second local tally". Both existed: promotion fired on a
local accepted-frame count and fell back to a local per-actor plurality
when the registry had not yet claimed the track. That fallback was
reachable in the live pipeline, not just in tests -- three accepted
frames arrive well before a posterior crosses the ownership threshold --
so in practice the plurality usually decided, and it could not see the
AR-025 correlation discounting it was meant to defer to. The tally is
gone; promotion now requires the registry's verdict, with the accepted
-frame count demoted to an explicit evidence floor.

AR-017 -- the route. DeadTrack carried belief but no route, and the sink
wrote the literal string "live", so a field the schema publishes could
not distinguish anything. AR-017's own verification asks for "deferred
and pooled routes distinguishable". Route is now an enum on the claim.
Only `live` occurs today; `deferred` exists so AR-020's pass has
somewhere to write instead of a serialisation change to make.

Also: TrackGallery::forget had no callers, under a comment asserting the
matcher called it "on a cut or track disappearance". The cut half was
true by another route; the disappearance half was not, so a track that
died quietly kept its diversity buffer until the next cut cleared
everything. Replaced with prune_dead against the registry's own
liveness, the same shape as the tracker's prune_boxes -- a second
opinion about which tracks exist is a second thing that can be wrong.

Removes dead logistic/logit helpers and fixes five TRACES tags that used
a comma where a pipe separates requirement types, which the gate had
been reporting as diagnostics.

TRACES: AR-011, AR-017, AR-019 | IR-002 | SR-002, SR-005
 2026-08-05 17:33:12 +02:00
dtourolle 7c7d4934ae refactor(presence): execute the extinction_sec/anneal_sec withdrawal 
docs/SPEC.md specified this removal, listed its parts, and ended "grep
for both names and expect no survivors". There were about forty.
docs/requirements.md meanwhile recorded both constants as Withdrawn and
"deleted rather than retained at zero", on the grounds that a field
naming a mechanism the pipeline no longer has is actively misleading.
Neither statement was true of the code: Config still carried
extinction_sec 57.4 and anneal_sec 35.5, --extinction and --anneal still
parsed, and SceneTrackerFunc still ran its keep-alive in both shipped
pipelines, announcing its timeout at every startup.

SceneTrackerFunc is replaced by FrameAnnotationFunc, which is stateless:
same ports, same output type, no keep-alive. Presence belongs to
TrackRegistry (AR-012), where a window is the extent of a track an actor
owned and ends at the last sighting (AR-013). The keep-alive answered
that question a second time and answered it worse, by re-opening exactly
the trailing cool-down AR-013 refuses.

Visible change: --verbosity standard's frames[].identified listed every
actor inside the keep-alive, including ones absent from the frame. It
now lists what was matched in that frame. Minimal and xray output is
untouched -- both were already built from registry claims and never
consulted this node. No schema bump: the published extraction block
reports track_extinction_sec, a different knob that bounds
re-association and never extends a claim.

TrackRegistry::Config::extinction_sec is renamed track_extinction_sec to
match the Config field feeding it, so the grep SPEC.md asks for now
returns nothing rather than one confusing false positive.

Two targets turned out to have been silently dead, both since the
AR-007/AR-008 tracker redesign, and both for the same reason -- they
construct FaceTrackerFunc from a Config alone, a signature that stopped
existing when association moved into probability space:

- scene_preview is fixed here. It now mirrors main.cpp's construction
  order exactly (matcher, then registry, then tracker) and wires the
  registry's claims into the sink, which it was not doing. DP-001 says
  modes are front-ends that must not fork pipeline logic; this one had
  forked it and then rotted.
- sae_kpn is not fixed. Restructuring the seam so the tracker can reach
  a calibration that only exists once the matcher is built is VR-011's
  rewrite, not a patch, and presence claims do not cross the seam at all
  today. It is now behind SAE_BUILD_KPN_BINDINGS=OFF with the reason
  recorded, so `cmake --build` succeeds and the breakage is attributed
  rather than rediscovered.

That second one is worth stating plainly: VR-002 ("replay drives the
real KPN nodes, not a reimplementation") is marked Done, and the module
that makes replay possible has not compiled for some time. The .so in a
stale build/ predates the change.

Python side: the two names are gone from optimize.py, replay.py and
run_holdout_all_models.py as Config keys. anneal_sec survives as
REPLAY_LOCAL_KEYS -- it still configures replay.py's own windowing,
which is a Python reimplementation that no longer matches the sink and
is documented as such. That divergence is VR-011's.

TRACES: AR-012, AR-013 | DP-001 | SR-002
 2026-08-05 16:21:15 +02:00
dtourolle e1de98e783 feat(ar-024): enforce the invariant statically, and delete the fallback it caught 
AR-024's register row gives its verification tier as "Static check -- no
bare cosine outside a tagged EXCEPTION". No such check existed, so the
invariant was enforced by reading, and reading had missed a live
violation.

scripts/ci/check_raw_cosine.py is that check, wired into the
traceability workflow as a blocking step. It is honest about its reach:
it catches direct cosine_similarity() uses not routed through a
calibration, and it cannot follow a cosine through a variable across
statements. That limit is documented in the script rather than left for
someone to discover after trusting a pass.

What it caught, and what this commit removes with it:

The identity matcher's no-calibration fallback thresholded raw cosine
distance (match_threshold) plus a ratio test (match_ratio,
match_ratio_ceil). Worse than the invariant breach: it fed
max(0, cosine) into TrackRegistry::observe, whose contract reads
"posterior is a calibrated probability, never a raw cosine (AR-024) ...
so the accumulation cannot be fed an uncalibrated number by a careless
caller". It could, and did. And it disagreed with the rest of the
pipeline about what "the fit failed" means -- same_person_probability
answers that with the untuned default sigmoid and a loud warning, so
association stayed in probability space while matching alone left it.
One run, two policies, no announcement.

Now one rule: cal_.probability() always, with a warning when the fit is
not real. A worse answer than a fitted calibration, a better one than a
number whose units nothing else shares.

TrackGallery::set_calibration is mandatory for the same reason. Its
default was max(0, cosine), which made expand_band_lo = 0.90 mean
"cosine > 0.9" in a test and "P(same person) > 0.9" in production.
FaceTrackerFunc already threw without one; the expansion store now
matches.

One exception is recorded, in the calibration's own dedup. It is not a
close call: at 1 - 1e-7 it asks whether two vectors are the same vector,
and it runs on the fit's input, so a calibrated comparison there would
have to be calibrated by the fit it is feeding.

Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys
are read with a contains() check, so each one had been silently inert
since the field behind it was deleted -- a sweep varying one of them
measured nothing and reported an ordinary-looking F1.

TRACES: AR-024, AR-023 | SR-002
 2026-08-05 15:46:33 +02:00
dtourolle fd078c399f ci(tests): actually run the tier the verification strategy is built on 
docs/requirements.md describes four verification tiers and argues that
T1 (functor unit) and T2 (replay) are "the only tiers that can exist in
CI at all". The traceability gate then reports a CI-scope coverage
fraction over exactly those tiers. Nothing ran them: the only workflow
was the gate itself, which reads source comments, and SAE_BUILD_TESTS
defaults to OFF. "Covered" meant a TRACES tag existed in a file.

That is the same failure the gate's own config warns about one level up
-- counting a test that cannot run -- and the gate cannot see it,
because a tag is all a static reader has.

Runs in the pinned DP-007 CPU builder image, which the image script
already expected this workflow to exist (it names unit-tests.yml and
asserts its tag). Nothing here calls a model: T1 constructs node
functors directly and T2 replays a precomputed dump, so the GPU-free
N100 runner is sufficient by construction rather than by concession.

Two deliberate hard failures. A missing replay fixture fails the job
instead of skipping, because pull_artifacts.sh warns-and-continues and a
T2 test whose input never arrived must not look like a pass. And the
image reporting a tag other than the pinned one fails rather than
building against an unknown toolchain.

It found a real bug on its first run: see the preceding commit. ctest
runs each case in its own process, which turned a 1-in-4 heap corruption
from noise in the aggregate binary into a reproducible failure.

TRACES: DP-007 | PR-004
 2026-08-05 15:17:17 +02:00
dtourolle bedc859d3c fix(audio): zero-initialise the channel layouts before the resampler copy 
av_channel_layout_copy() documents that it always uninitialises the
destination first, and av_channel_layout_uninit() calls av_freep() on
u.map. Declaring the layouts without {} therefore handed free() whatever
pointer-shaped garbage occupied that stack slot.

Not theoretical: UT-103 aborted with "free(): invalid pointer" in about
1 run in 4. Zero failures in 40 runs after the fix, against 10 in 40
before it.

Two things kept it hidden, both worth remembering:

- It is stack-dependent, so it disappears under an AddressSanitizer
  build and reads as a flake in the aggregate test binary, where the
  case usually passes. ctest, which runs each case in its own process,
  is what made it a reproducible failure rather than noise.
- UT-103 is the only test that reaches this branch, because it is the
  only one whose input is stereo. The golden-vector tests use a mono
  11025 Hz fixture chosen so the vector cannot depend on libswresample
  — which is right, and means bit-exactness against the golden vector
  is not evidence about the downmix path.

TRACES: IR-004 | SR-003
 2026-08-05 15:16:46 +02:00
dtourolle 718dad688d docs(register): record the quality vector, the benchmark, and what lossless fanout costs 
Status for the two changes just landed, plus the consequence AR-004's
fix has for the scene join.

The annotator's old comment said blocking there was safe because the
branches are independent. That stopped being true when the fanout became
lossless: it now stops popping once one branch stops taking, so a
starved detector and a waiting annotator would wedge. What actually
makes it safe is join depth -- the fanout can run the dense branch ahead
by the whole of the sampled branch's buffering, which at kSceneJoinDepth
256 and sample_fps 5 against a 25 fps source is ~1200 dense frames
against TransNetV2's 100-frame window. Cutting kSceneJoinDepth below the
window would reintroduce the wedge, so it is now a correctness
precondition rather than a tuning knob.

TRACES: AR-004, AR-010, AR-028, AR-029 | VR-015 | SR-002
 2026-08-05 14:38:42 +02:00
dtourolle a5ee3c05ce feat(benchmark): per-node cost and bottleneck attribution for a run 
--benchmark <path> reports cumulative CPU and wall time per node and
names the node pacing the run. The pacing node is located from sampled
channel occupancy, not from time-in-node: backpressure inflates
time-in-node for everything downstream of the real bottleneck, so the
obvious measure names the victim rather than the cause.

Sampling starts with the network and stops before it is destroyed.
Channel fill is instantaneous and everything has drained by shutdown, so
a single read at the end reports an idle pipeline however congested it
was.

kill -USR1 dumps the table from a running or wedged process. Channel
occupancy identifies a stalled node -- full input, empty output --
without a debug build or a debugger, which is the difference between
diagnosing the AR-004 hang in seconds and reproducing it under gdb.

Two knobs this exposes for measurement rather than sets: SAE_CV_THREADS,
because OpenCV's TBB arena and KPN's thread-per-node are two schedulers
unaware of each other on the same cores; and SAE_CUDA_BLOCKING_SYNC,
because the default spin-wait held the embedder thread at 99.7% user
time while nvidia-powerd cut the GPU's clock from 1005 to 210 MHz.
Neither default changes until a measurement says it should.

TRACES: VR-015 | PR-004
 2026-08-05 14:38:15 +02:00
dtourolle 777c98cb33 feat(quality): score every face on sharpness and alignment before it is evidence 
Every embedding now carries the quality of the input it came from. Both
axes fall out of the AR-005 warp for free: crop_sharpness() is the
normalised Laplacian variance over the aligned 112x112, so contrast and
size cannot leak into it, and the alignment residual is the part of the
landmark deformation a similarity transform cannot explain, so in-plane
roll reads as zero and foreshortening does not.

Carried, not consumed. Nothing discounts or thresholds on either number
yet -- that is AR-030 and VR-012, and the knee has to be located against
recorded data before a gate is chosen. What this change buys is that the
data exists to locate it with.

No face is admitted unscored: the -1 sentinel is preserved rather than
clamped, and a degenerate landmark fit is counted rather than silently
dropped.

Takes the VR-001 dump to schema_version 2. The bump is not for readers,
which check for the datasets by name and replay a v1 dump unchanged; it
is so a consumer can tell "never scored" from "scored zero", which is
not recoverable from the arrays afterwards.

TRACES: AR-028, AR-029, AR-030 | VR-001 | SR-002
 2026-08-05 14:37:30 +02:00
dtourolleandClaude Opus 5 c1155cb607 feat(gemm): the annex is a matrix, not a list — scored by the same GEMM 
The per-film annex was folded in after the gallery multiply by a host-side
cosine loop over a vector of {embedding, actor} structs, justified in-comment
by "tens of embeddings". AR-018/AR-019 retired that assumption: every owned
track promotes, so the annex grows with cast size and film length.

TrackGallery now holds it as a contiguous row-major matrix with a parallel
actor index — the flat_emb_/flat_actor_ shape the baked gallery already uses —
and hands newly promoted rows to the matcher once per frame. The matcher pushes
them into the similarity engine's resident matrix through a new
ISimilarityEngine::append_rows, so one SGEMM covers baked and promoted
references alike and best-of-N is a single pass over one similarity column.
Capacity doubles on overflow, and the GPU backends grow device-to-device, so a
promotion never re-uploads the gallery across the bus.

Absorbing promotions runs once per frame, after every face has been scored.
Appending mid-frame would invalidate the similarity pointer the chunk loop is
still reading, and it also removes an incidental dependence on face order
within a frame — a promotion helps subsequent frames, never the one that
produced it, which is the semantics the expansion store already documented.

OpenBLAS becomes a requirement of the CPU GEMM backend rather than an
opportunistic upgrade. That path is what CI and the cpu builder image run, so
falling back to the scalar loop in silence meant AR-027 could be measured — or
believed — on a kernel no release uses. The loop survives as the correctness
oracle the BLAS backends are diffed against, behind SAE_ALLOW_SCALAR_GEMM.

Call site 3, the deferred TBI pass, is untouched: it does not exist until
AR-020, so AR-026 stays In Progress.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-026 | UT-004, UT-005 | SR-001
 2026-08-04 21:20:31 +02:00
dtourolleandClaude Opus 5 f33403fff8 feat(scene): feed TransNetV2 at native rate, derive the dedup window from it 
Closes both violations SPEC.md named under "Every model gets the input it
was trained for". They are one bug, not two.

The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window
spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over
twice its temporal context. Boundary timestamps stayed correct throughout,
which is exactly why the degradation was invisible and why the compressed
separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a
property of the ONNX export rather than of the input.

Dedup then merged boundaries closer than a literal 0.04 s — one frame at
25 fps, and wider than a frame at 30, so two cuts on consecutive frames
became one. Nothing in scenes.json showed it; the file simply had fewer
boundaries. Native rate is where that constant did the most damage, which
is why fixing the decode rate without fixing the dedup would have made
things worse.

dedup_window_sec() now takes the median interval the detector was actually
fed and halves it. Half a frame rather than a whole one: the only thing
being merged is one frame scored by two overlapping windows, and two
distinct frames are a full interval apart.

Cost is real — dense decode is the pipeline's cost driver. It is accepted;
dense_scale and scene_stride remain the reductions that do not run the
model off-distribution. scene_threshold 0.60 was fitted against the 12 fps
input and is now stale, so VR-006 goes from Low to Medium: it is no longer
a refinement, it is a constant that no longer describes the input.

AR-002 rides along because it was already implemented, just untagged and
unverified — the register said Planned while the code was correct. The size
filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold
and at dense_scale 0.5, and checked end to end against the superhero dump,
whose smallest face is exactly its recorded 32 px minimum, so the fixture
check cannot pass vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001
 2026-08-04 21:17:57 +02:00
dtourolle 079b490ede Merge branch 'feature/ci-images' into feature/opencv5 2026-08-04 14:53:24 +02:00
dtourolle a3827646b9 feat(ci): CPU builder image for the unit-test workflow 
TRACES: DP-007 | PR-004

Pinned by tag rather than :latest, so a workflow run is reproducible against
the image it was written for.
 2026-08-04 14:53:12 +02:00
dtourolle 22758da118 docs: SuperHero benchmark — how to reproduce it and what it scores 
Records the reference film end to end: fetching the annotations and mugshots,
fusing the scene clips into one stream, building the gallery at the 66 px face
floor, and the measured result (precision 1.00, recall 0.65, F1 0.79).

Three things are written down because each cost time to discover:

- Why Bali was withdrawn. Its reference crops have a median detected face of
  27 px against a 69 px maximum, so every reference was upscaled past what the
  embedder was trained for (AR-011). No threshold fixed it — at 66 px, 2 of 69
  references survived. Any accuracy figure recorded against Bali measures
  upscaling artifacts as much as the pipeline.
- Run it as one film. Per-scene clips defeat per-film gallery expansion
  (AR-019) and pay model load ten times over.
- Check you are on the GPU. ORT's CUDA provider fails to load here and falls
  back to CPU silently, so a build-ort timing is a CPU number wearing a GPU
  label — a 15x error whose only symptom is a number with no baseline.

TRACES: AR-011, AR-019 | VR-001, VR-005 | SR-002
 2026-08-04 14:32:33 +02:00
dtourolle 960a7c4eed chore: bump KPN to 454f72c (ignore generated ORT cache) 2026-08-04 14:08:13 +02:00
dtourolle ae60ac7657 chore(models): track 2d106det and the larger SCRFD variants via LFS 
Detector variants used by the resolution and min-face studies. LFS per
.gitattributes, so the repo carries pointers rather than 20 MB of weights.
 2026-08-04 14:04:50 +02:00
dtourolle f891e579c5 chore(traces): put TRACES tags on their own line; regenerate the report 
The parser reads a tag up to end of line, so `# TRACES: GR-004 | SR-001 —
prose` swallowed the prose into the tag and the row went unmatched. Splitting
the comment leaves the tag greppable by the same pattern as the code tags and
the commit trailers, which is the point of the house format.

Mechanical throughout; no logic touched. The regenerated report reflects this
session's new tags: 137 -> 148 found, and one more tagged-but-unexecuted, which
is the SuperHero accuracy assertion that is documented but not yet a test.
 2026-08-04 14:04:21 +02:00
dtourolle d98dc2855a refactor(bench): SuperHero replaces Road to Bali as the reference film 
Bali was chosen because the TRECVID DVU set ships character mugshots, but
its reference crops are unusable at scale: median detected face 27 px
against a 69 px maximum, so every reference was upscaled 4x or more past
what the embedder was trained for (AR-011). A 66 px floor left 2 of 69
references; no threshold exists that both keeps the faces in distribution
and leaves enough of them to calibrate.

SuperHero is 69 px median and 241 px max. Its gallery builds at a 66 px
floor with 14 references over 5 characters, and calibrates on its own
(a=15.2867 b=-4.98633, 100% train accuracy) instead of borrowing constants.

Measured on the fused 17-minute film, one stream rather than per-scene
clips so presence windows cross real scene boundaries as SR-002 intends:
precision 1.00, recall 0.65, F1 0.79 — 13 true positives, 0 false
positives, 7 misses. Every out-of-gallery character was declined rather
than forced onto a nearest match. The misses are the short scenes (14 s,
38 s, 27 s), consistent with per-track accumulation needing sightings.

- build_gallery gains --min-face-px, filtering the *detected face* rather
  than the crop. The DVU images are scene crops, not mugshots, so crop
  dimensions say nothing about face scale. A poisoned reference is
  permanent in a way a bad frame is not: it corrupts every future match
  against that identity.
- scripts/fetch_dvu.sh fetches mugshots, scene graphs and segmentation for
  any DVU film. NIST names the same film three different ways, so KG_DIR
  and KG_FILE are overridable rather than derived. This exists as a script
  because the first copy of this data was assembled ad hoc in /tmp and was
  lost with it, taking the working gallery along.
- Replay fixtures move to the artifact registry: push/pull_artifacts.sh
  gain a replay-fixtures target, and tests/fixtures/dumps/.gitignore keeps
  them out of git. superhero.h5 is ~9 MB and regenerating it needs the
  film, the models and a GPU — none of which CI has. The gallery ships
  with the dumps, since a dump only replays against the gallery it was
  produced with.
- AR-012 and AR-013 coverage is ported onto the new fixture rather than
  dropped with the Bali cases: 12369 assertions, up from 7991, since the
  film is an order of magnitude larger than the clips.

Suite: 15679 assertions, 101 test cases.

TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
 2026-08-04 13:49:11 +02:00
dtourolleandClaude Opus 5 eff696b49a fix(expansion): finish AR-018, retiring the last two expansion cosines 
AR-018 was marked Done while the promotion path still ran on the
constants it was meant to replace. track_gallery.hpp rejected a track
when buffer_spread (1 minus the minimum pairwise cosine) exceeded
expand_track_spread_max, and skipped a view when its raw gal_sim cleared
expand_novelty_sim. Both were bare cosines with no recorded EXCEPTION,
so both were defects under the AR-024 invariant rather than tagging gaps.

The calibrated band was real but unreachable. expand_band_lo/hi were
declared in Config and read nowhere, and set_band() had no callers, so
the gate always ran at the hardcoded 0.90/0.95 while --expand-novelty-sim
and --expand-spread-max stayed live flags.

The spread gate becomes store_coherence: the band's lower bound asked of
every pair in the store, in probability space, rather than a second
constant. admit() compares a newcomer only against its nearest existing
member, so a gradually drifting track chains A to B to C with every step
inside the band while A and C are strangers — the shape a track-ID
collision takes over a slow pan. The bound is re-asked pairwise before
anything reaches an actor's annex.

The novelty gate is deleted rather than converted. SPEC section AR-018
contrasts the band with expand_novelty_sim as the thing it replaces, and
AR-019 requires only that the band is satisfied. Novelty-seeking now
lives entirely in the eviction ordering, which ranks by similarity to the
actor's references instead of cutting at a constant, so there is nothing
left to tune but the two bounds.

BufEntry stored a raw cosine and the eviction loop compared two of them.
The map is monotonic so the ranking was never wrong, but it left a bare
cosine as a decision variable; it now stores the calibrated probability.

The [AR-018] Catch2 tag previously sat on the spread gate, reporting the
replaced mechanism as verification of its replacement. It now sits on the
band: both bounds asserted exactly, since they are inclusive and an
off-by-one there is invisible anywhere else; refusal counted on each
side; and the config bounds driven away from the shipped defaults so a
hardcoded fallback fails. The case that carries the invariant is "band
thresholds probability, not cosine" — under a calibration shifted by
0.10, cosine 0.84 is admitted and cosine 0.92 refused, the opposite of
their raw verdicts. A raw-cosine gate passes an identity-calibrated test
by accident and cannot pass that one. 15 cases, 38 assertions, passing.

scene_preview.cpp takes the flag rename because it would otherwise
reference deleted Config fields. It still does not compile, for reasons
predating this change: it also reads track_max_embed_dist and
track_max_frames_missing, retired by the earlier AR-024 tracker work, and
constructs FaceTrackerFunc with one argument where the registry and
calibration are now required.

Two notes for anyone reading the chain. The main.cpp flag rename and the
AR-018/AR-024 register rows landed in 35e7033, whose trailer names AR-004
only, so git log --grep=AR-018 will not surface them. And
docs/traceability.md is left uncommitted on purpose: regenerating it now
would bake in VR-013 rows for two experiment scripts that are not yet
committed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-018, AR-024 | SR-005
 2026-07-31 22:48:07 +02:00
dtourolleandClaude Opus 5 ffdad9873d test: tag the untagged suites; correct two stale headers 
Four test files and one node header carried no TRACES tag, so the
requirements they verify read as implemented-but-unverified. Tagging a
test is what distinguishes the two.

test_calibration.cpp is AR-023; its three [report] cases verify GR-003
and are tagged separately, since the report is fitted from the same
distributions but is its own requirement. test_similarity.cpp is the CI
half of AR-026 — equivalence against hand-computed dot products, where
throughput at scale is AR-027 and cannot run on this host.
test_face_tracker.cpp is AR-007 and AR-008.

Two headers described code that no longer exists. face_aligner_node.hpp
still documented the RANSAC fit AR-005 replaced with an Umeyama
least-squares fit over all five points — not merely out of date but the
opposite of what the file does, and it reads as a rationale for
discarding the landmarks AR-030 measures. test_face_tracker.cpp still
described the park/revive branch AR-008 deleted, and the raw-cosine
cut_revive_sim that guarded it, which AR-024 retired.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-005, AR-007, AR-008, AR-023, AR-026, AR-030 | GR-003 | SR-001, SR-002
 2026-07-31 22:47:59 +02:00
dtourolle 5c6603e63b fix(kpn): park on full outputs; surface node exceptions 
Adopts the KPN backpressure fix (28e0667) and registers the application
error listener it exposes.

`push_blocking` parked a scheduler worker inside the push. Each ObjectNode
owns a private single-thread pool, so the parked thread was the only one
that could drain that node's own input: under sustained backpressure
frame_source, camera_pos, face_detector and face_aligner all slept in
nanosleep at once and the pipeline stopped. Nodes now hold the value,
release the worker, and resume on a channel space-callback.

main.cpp registers set_error_handler so a node that throws names itself
and its exception. Previously the exception was discarded at the node
boundary and survived only as "node 'x' stopped unexpectedly", which says
that a node died but not why — the missing detail that made this slow to
diagnose.

AR-004 drops from Done to Mostly. Two gaps are recorded rather than
claimed fixed: a hang surviving at roughly 1 run in 20 against a 300 s
timeout (down from every run failing), and FanoutNode still dropping on
overflow instead of parking, which sheds frames on the AR-010 scene join
precisely when the dense branch falls behind.

TRACES: AR-004 | SR-002
 2026-07-31 22:42:19 +02:00
dtourolleandClaude Opus 5 3bf4d60a6f docs: regenerate the traceability matrix for VR-014 
The committed matrix predated the audio-signature binding, so VR-014 and
the four UT tags in `test_audio_offset.py` were absent from it while being
present in the register — the one inconsistency a generated file is
supposed to make impossible.

VR-014 also needed an explicit tier row. The blanket `VR-* | Out of CI`
line is right about every other study and wrong about this one: its
fixture is committed and its signature is CPU-only DSP, so it is a test a
CI host can run rather than a measurement someone has to remember to
repeat. Left as an exception under the blanket rather than rewriting the
rule, because the rule still describes the other thirteen.

Coverage unchanged at 38/69; the gate reports no orphan tags.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-014
 2026-07-31 17:02:45 +02:00
dtourolle 5f6daefc40 Merge branch 'feature/quality-knee' into feature/opencv5 
# Conflicts:
#	docs/requirements.md
 2026-07-31 16:54:09 +02:00
dtourolle 629d698ad9 Merge branch 'feature/dump-provenance' into feature/opencv5 2026-07-31 16:53:58 +02:00
dtourolleandClaude Opus 5 71354e862a docs: VR-013 and VR-014 results; AR-002 raised to 40px 
Records study results and the requirement change that follows from them.

VR-013 measures minimum face size end to end — gallery from one recording,
probes from another — rather than by degrading an already-aligned crop. Holding
90% of the plateau needs ~50 px that way against VR-005's ~22 px, the gap being
detection and landmark error rather than the embedder. AR-002 therefore takes
40 px, not 32: VR-005 isolates the embedder and is an upper bound, and 32 admits
faces in the falling region. FPI stayed 0.0% at every scale, and the ceiling is
cross-view rather than resolution.

VR-014 exercises audio-signature offset recovery on real film audio instead of
the synthetic golden tone. Forty random in-cap offsets, every one recovered to
the nearest frame, worst error 46 ms against a 500 ms budget — and 46 ms is the
quantisation floor rather than a result, since offsets land on whole 92.88 ms
frames. The runtime/2 anchor is confirmed through head-trimmed files.

The soft spot VR-014 found is tier labelling, not accuracy: the score drops with
sub-frame misalignment, so 27 of 40 correct alignments were demoted to `loose`.
One frame of slack in the score restores all forty to `audio` with false matches
unmoved.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-002, VR-005, VR-013, VR-014 | SR-002, SR-003
 2026-07-31 16:53:58 +02:00
dtourolleandClaude Opus 5 2a8ee3660b feat(audio): bind the v1 signature and validate offset recovery on real content 
sae_audio exposes the shipped signature to Python. It compiles
audio_signature.cpp directly against FFmpeg rather than linking
sae_gallery: the signature needs no model, no OpenCV and no HDF5, so a
module that dragged those in would make `import sae_audio` depend on a
GPU-capable build of a path that is pure CPU DSP.

The point of binding rather than porting is that a fingerprint is only
useful if every implementation agrees byte for byte. A numpy port would
be a third implementation, and the one nobody checks against the golden
vector.

VR-014 then recovers a known trim from real film audio rather than from
the synthetic tone: 40 random in-cap offsets, every one recovered to the
nearest frame, worst error 46 ms against a 500 ms budget — and 46 ms is
the quantisation floor, not a result, since offsets land on whole
92.88 ms frames.

The soft spot is tier labelling rather than accuracy. Sub-frame
misalignment drags the score down (0.94-0.99 near a frame boundary,
0.69-0.73 at half a frame), demoting 27 of 40 correct alignments to
`loose`. Allowing +/-1 frame of slack in the score fixes it: all 40 back
to `audio` at min 0.906, false matches unmoved at 0.12-0.16, for 81 ms
of the budget.

The module stops at the producer's edge. Sliding one signature against
another is the consumer's algorithm (server SPEC §3, and the jRay
plugin implements it), so a caller writing that slide in numpy is not
duplicating anything this repo owns.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: IR-004, IR-005 | VR-014 | UT-105, UT-106, UT-107, UT-108 | SR-003
 2026-07-31 16:52:11 +02:00
dtourolleandClaude Opus 5 b4318f8d9e fix: belief accumulates across frames (lazy-OR), not once 
A track recognised on 318 of 385 frames was owned on none, so the truth file
named nobody while the matcher was accepting almost continuously.

The correlation discount was an annihilator rather than an attenuator. Weight
was 1 - P(same view), so once a track had one stored view every later frame of
that same face scored ~0.01 and the belief stopped moving. One observation just
over the accept threshold is logit(0.78) ~ 1.27, under the ownership bar — hence
recognised always, owned never.

Two changes, in the order they were found.

Correlated evidence is now attenuated by effective sample size,
n_eff = n / (1 + (n-1)·rho), each frame contributing the marginal gain. That has
the right shape at both ends: uncorrelated evidence accumulates linearly, and a
held pose converges on 1/rho rather than growing without bound. A constant floor
was tried first and rejected — it grows linearly forever, so a long shot could
out-argue genuinely varied evidence purely by lasting longer.

Combination is now weighted lazy-OR: P = 1 - (1-P_old)·(1-p)^w, stored as
log(1-P) so the update is additive and precision stays where it matters as P
approaches 1. Each frame is new evidence that this track is that actor, and the
belief is the probability that at least one sighting was right. It converges
faster than summing log-odds at the same effective count — 2.98 vs 2.53 after
two observations at p=0.78 — which is what a real clip needs.

Note that summing log-odds was already a correct sequential Bayesian update:
the matcher fits with prior 0.5, so logit(p) IS the per-frame log-likelihood
ratio and the running sum carries the prior forward. It was not wrong, it was
slow. What blocked ownership was the discount, not the combination rule.

Also fixes a real correctness bug: the observation count lived on the
discounter, which is shared by every track, so tracks pooled into one effective
sample and each was discounted by how many others happened to be on screen. It
is now a per-track parameter.

The registry's frame scope holds its lock for its lifetime and the mutex is not
recursive, so calling observe() inside a scope self-deadlocks. The pipeline
never does — separate nodes — but the test did, and hung rather than failing.
Documented at the call site.

Verified end to end: the same clip that produced zero actors now identifies
Bing Crosby and Dorothy Lamour with belief 0.97.

Suite: 96 cases, 6142 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-025 | SR-002
 2026-07-31 16:51:08 +02:00
dtourolle af5208035e docs: the RANSAC aligner was a defect, measured 
AR-005 replaced cv::estimateAffinePartial2D(..., RANSAC, 3.0) with Umeyama
least squares over all five points — the estimator InsightFace aligns with, and
so the one the ArcFace/LVFace training crops were produced by.

The first note here assumed the two agree wherever RANSAC keeps all five points,
leaving a small divergence on non-frontal faces. Measured on 400 gallery
headshots with the model held fixed, that was wrong: the crops disagree by a
median 17 source px and 83.5% embed below cos 0.99 of their Umeyama counterpart.
A 4-DoF similarity is exactly determined by two points, so every minimal sample
fits its own pair perfectly and is scored on the other three; real landmarks sit
a median 2.74 canonical px from any similarity fit, so a landmark outside the
3 px band is the common case and RANSAC returns an under-determined transform.

How much that cost in accuracy is a separate question, and the honest answer is
less than those numbers suggest. Rebuilding the full gallery moved the
intra/inter separation the AR-023 calibration is fitted from by 0.583 to 0.590:
the old warp was wrong but self-consistent, gallery and probe both went through
it, and the embedder tolerates framing variation. The sharper evidence is
duplicate detection — the rebuild dropped 1614 near-duplicates against the
original build's ~100, because unstable two-point fits gave near-identical
images visibly different vectors. That instability, not a headline accuracy
delta, is what a tracker accumulating evidence across frames was paying for.

Also records the AR-030 residual's real-data floor: on the most cooperative
images the pipeline sees, it runs a median 2.74 px, so landmark noise occupies
the first few pixels and the synthetic foreshortening ladder is optimistic about
the low end. Any discount curve has to treat that range as uninformative rather
than as mild pose, and VR-012 must set thresholds against the measured
distribution.

Tests carry the tag they verify: the residual's roll/scale invariance and
monotonicity under foreshortening are what make it a pose measure rather than a
pose-and-everything-else measure.

TRACES: AR-005, AR-030 | SR-002
 2026-07-31 16:39:12 +02:00
dtourolleandClaude Opus 5 d3ab598434 feat(artifacts): push and pull the VR-013 corpus 
The cross-source study needs two 4K recordings and a hand-sorted set of
face crops, neither of which belongs in git. Adds an xsource target to
both artifact scripts.

Push uploads the clips as-is (already compressed) and zips labelling/.
Pull fetches both and regenerates frames with ffmpeg rather than
downloading them: ~320 MB of PNG that is deterministic from the clips.
The extraction settings are pinned in the script, not left to the
caller, because the manifests key on frame filenames and on detection
order within each frame — verify_labels.py runs afterwards and fails
loudly if they drift.

Pull refuses to overwrite an existing labelling/. It is human ground
truth: somebody looked at 167 crops and placed each one, and silently
replacing that with a remote copy would destroy the expensive half of
the study.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-013
 2026-07-31 15:58:24 +02:00
dtourolleandClaude Opus 5 66c9ca0a0c refactor(VR-005): drive the study off the sae_embed bindings 
Deletes the Python ports of SCRFDDecoder, ArcFaceEmbedder, align_face,
enhance_for_retry and calibrate_gallery, and calls the shipped C++
instead. 297 lines removed, 108 added.

The ports existed because sae_embed only exposed embed(path), so a
caller could not embed a crop it had degraded. That gap is closed:
detect(), align_face(), enhance_for_retry(), embed_crop()/embed_crops()
and GalleryCalibration are bound now, so there is no longer a reason to
keep a second implementation of any of them.

The calibration is the one that mattered. A parallel copy of the sigmoid
is precisely where "always the calibrated probability, never a raw
cosine" (AR-024) breaks without anyone noticing — the copy goes on
returning plausible numbers after the original has moved. Scoring
through the binding makes the rule structural rather than remembered.

Verified against the committed run: same shape, FPI 0.0% at every size,
same operating point of 32 px. Absolute rates differ by 1-2 points
because this check sampled 100 actors / 574 crops against the original's
258 / 999, not because anything regressed.

Also: --providers and --batch are gone, since provider selection and
batching belong to the backend; embeds are chunked at its max_batch,
because the engine does not split an oversized request and a whole
gallery in one call asks CUDA for a multi-gigabyte buffer. DEDUP_SIM and
MIN_EMB_FOR_POSITIVE stay as mirrored constants — used only to report
the population the C++ fitted on, not to refit it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-005 | AR-024
 2026-07-31 15:51:50 +02:00
dtourolle 81ec77625c Merge branch 'feature/gallery-report' into feature/opencv5 2026-07-31 15:31:07 +02:00
dtourolleandClaude Opus 5 1dfd6fea11 feat: gallery build report 
GR-003 — the calibration fit already computed per-actor dedup counts, how many
actors are eligible for positive pairs, and a 200-bin histogram of the intra and
inter distributions, then discarded all of it to stderr. Nothing persisted, so
nobody could audit whether a gallery was any good.

The report is written alongside the gallery at build time. That is the right
moment: the matcher fits the same sigmoid at analysis time, but by then the
answer is per-run and nobody is looking, whereas build time is when a gallery's
quality is actually decided.

What it surfaces, in order of usefulness:
- actors with no usable image — a silent recall ceiling, since the pipeline can
  never name them and nothing else says why
- actors below the positive-pair threshold — not broken, so nothing complains;
  they just quietly weaken every threshold downstream
- near-duplicate references removed, per actor and total
- the fitted calibration AND the two distributions behind it

That last one is the point. Every threshold in the pipeline is expressed in the
probability space this sigmoid defines, so if the distributions overlap heavily
the calibration is weak and every downstream decision inherits it — while the
gallery still looks fine from the outside.

The gallery-derived prior, intra/(intra+inter), is computed and reported but the
shipped default of 0.5 is deliberately left alone. The spec records these as
disagreeing; now the real value is visible, so the decision can be made on
evidence rather than argument.

Three tests: a zero-image actor is visible in the report, an under-referenced
actor is counted, and the report round-trips through JSON.

Suite: 95 cases, 6142 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: GR-003 | SR-001
 2026-07-31 15:30:46 +02:00
dtourolleandClaude Opus 5 6aabeb9897 feat: provenance attributes on the embedding dump 
VR-010 — a dump made with one detector/embedder pair was byte-indistinguishable
from one made with another, except for the two attributes GR-004 added. Replayed
against a gallery from a different model, cosine similarities are meaningless
but look entirely plausible. The register states the principle directly: a
fixture whose provenance is unknown is worse than no fixture, because it will be
trusted.

Sixteen attributes now record everything that determines the dump's content:
detector model and thresholds, min_face_px, max_faces, cut_threshold,
dense_scale, bbox_upscale, start/end, track_assoc_min_prob, and scene_detect.

scene_detect is the one that matters most. is_scene_boundary is all-zero both
when the detector found nothing and when it never ran, and those mean completely
different things to a consumer — without the flag they are indistinguishable.

No schema_version bump: new root attributes are additive and replay.py already
reads attributes with a default, so older dumps stay readable and the committed
fixtures — which predate this — still load.

Also corrects SCHEMA.md, which claimed bbox was already mapped to original
resolution at dump time. It is not; the upscale is applied downstream in the
matcher, after the dump tap. Harmless while dense_scale is 1 and silently wrong
otherwise, so bbox_upscale is now recorded and the doc says what the code does.

Verified end to end: all sixteen attributes present and correct on a freshly
generated dump.

Suite: 92 cases, 6136 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-010, VR-001 | PR-002
 2026-07-31 15:24:06 +02:00
dtourolleandClaude Opus 5 01d7ead1e7 study(VR-013): cross-source identification probe over input resolution 
Gallery from one recording, probes from another, sweeping the probe's
input resolution end to end. VR-005 asked the same question over gallery
mugshots but degraded an already-aligned 112x112 crop with alignment held
perfect, so it isolates the embedder. Here the whole frame is downscaled
before the detector, so detection and landmark regression degrade with
it — which is most of the difference.

Corpus is two 4096x2160 clips of one shoot, four people, hand-sorted.
Ground truth is sorted by hand and gated by verify_labels.py; labels
carried down the scales geometrically by box position, never by
embedding similarity, which would keep only the faces the embedder
already gets right and drop the ones the sweep exists to find.

Findings, all scored through the production gallery sigmoid at
prob_threshold 0.754 — never a raw cosine:

- Holding 90% of the plateau needs ~50 px end to end, against VR-005's
  ~22 px. min_face_px at 40 looks right; 32 would admit faces in the
  falling region.
- FPI is 0.0% at every scale. Resolution loss goes entirely to TBI.
- The ceiling is cross-view, not resolution: everyone matches themselves
  within a recording (0.55-0.85) and collapses across two (0.14-0.45,
  threshold 0.335). Only the subject with frontal *gallery* references
  identified reliably, whatever their probe pose — so the lever is
  gallery pose coverage, not a better landmark source.
- Averaging SCRFD's overlapping detections instead of discarding them at
  NMS lifts cross-recording TPI 41% -> 49%, for one forward pass and no
  extra model.

Four identities and one shoot, so the shape is the result and the
absolute rates are not. Both clips contain all four people, so there is
no out-of-gallery class and the 10x-weighted out-of-cast misID is
untested here.

Clips, frames, hand-sorted crops and results are gitignored and belong
in the artifact registry — the sorting is human ground truth and
expensive to redo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-013 | AR-002, AR-005, AR-024
 2026-07-31 15:20:18 +02:00
dtourolleandClaude Opus 5 042e424961 study(VR-005): minimum face size from downscaled gallery mugshots 
Holds out one mugshot per actor, degrades that probe to each candidate
face size and matches it against a gallery held at native resolution,
reporting TPI/FPI per size. Replaces AR-002's 66x66 px working estimate
with a measurement. Needs no video and no ground truth beyond the
mugshot cache already on disk.

LVFace-B over 258 actors, 999 gallery embeddings, threshold 0.754:

    px    12    16    20    24    32    40   48+
   TPI   6.6% 46.5% 81.8% 93.4% 98.1% 99.2% 99.2%

FPI is 0.000 at every size — a face too small to identify degrades to
unidentified, never to a wrong name. rank-1 holds at >=99.6% from 24 px
up, so what fails first is the calibrated probability crossing
threshold, not the ranking.

Two limits on reading this. FPI grows with the number of actors
competing, so 258 understates it against a production library. And
detection and alignment run on the native image with only the resulting
112x112 crop degraded, so landmark error at small face sizes is excluded
by construction and the curve is an upper bound — VR-010 measures the
same question end to end, and lands well above these numbers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-005 | AR-002
 2026-07-31 15:17:39 +02:00
dtourolleandClaude Opus 5 9fc2763096 feat: expansion promotion gated on all three discontinuity signals 
AR-019 — promotion may only borrow same-identity evidence from a span where
identity is certain, so every discontinuity signal now clears the buffers rather
than just the histogram cut.

is_scene_boundary was already named in the gate but never set by anything, so
that half of it was dead until AR-010 gave it a producer. It now does what the
spec always said. The third signal, an identity contradiction, needs no code
here: AR-015 closes a track whose belief swapped, so it can no longer promote.

Ownership now comes from the registry rather than a second tally. TrackGallery
was computing its own plurality vote over accepted frames, which meant two
different answers to "who is this track" could coexist in one run — and the
expansion one ignored the Bayesian accumulation entirely, weighting thirty
near-identical looks the same as thirty distinct ones. The local tally survives
only as a fallback for callers with no registry attached, which is the unit
tests and the replay harness.

Suite: 92 cases, 6133 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-019, AR-010, AR-015 | SR-005
 2026-07-31 15:11:56 +02:00
dtourolleandClaude Opus 5 c843c4abe3 feat: banded admission for the per-subject embedding store 
AR-018 — an embedding joins a track's store only if its similarity to something
already there falls inside a band, rather than merely being far from the gallery.

Above the upper bound it is redundant: another look at a pose the store already
covers, teaching the annex nothing while costing a slot a novel view could have
used. Below the lower bound it is suspect: within one track every face is the
same person by construction, so an embedding unlike everything else on the track
is evidence that construction failed — a track-ID collision or a bad detection.
Admitting it is exactly how an actor's annex gets poisoned with someone else's
face.

The old gate had only the upper half of that idea, expressed as a raw cosine
against the gallery. Both bounds are now calibrated probabilities (AR-024), so
the same number means the same thing here as in association and evidence
weighting rather than three different things.

This catches track-ID collisions EARLIER than the spread gate did — at the door
rather than at promotion — so the buffer never becomes two-person in the first
place. The spread gate stays as a second line for a track that drifts gradually
instead of jumping. The existing test was asserting the mechanism rather than
the outcome, so it was rewritten to assert what actually matters: whichever gate
fires, the outsider must not reach the annex.

Rejections are counted. A store that admits nothing is as broken as one that
admits everything, and neither is visible otherwise.

Band defaults 0.90-0.95 are working values pending VR-007; the two bounds fail in
opposite directions and must be swept separately.

Suite: 92 cases, 6133 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-018, AR-024 | SR-005
 2026-07-31 15:10:31 +02:00
dtourolleandClaude Opus 5 cc1bed92d8 perf: back the CPU similarity GEMM with OpenBLAS 
The CPU path was a scalar triple loop. It is the correctness oracle for the GPU
backends, but it is also what CI runs — there is no GPU on the N100 host — and
since AR-003 removed the per-frame face cap, a crowded frame now scores many
faces against a library-scale gallery. Scoring one face against 5000 embeddings
is 2.6 MFLOP; in scalar that does not hold up (AR-027).

S(g,f) viewed as row-major [n_faces x n_gallery] is exactly query * gallery^T,
so the loop nest collapses into a single cblas_sgemm.

OpenBLAS is optional in the build: found via pkg-config, and the scalar path
remains when it is absent so no hard dependency is added and the two can be
diffed when a similarity looks wrong. The configure step warns rather than
failing, since a developer without it should still get a working tree.

The test target links it too. Without that the suite compiles the scalar
fallback while the builder image ships CBLAS, so CI would be verifying a kernel
that is not the one running in production — the same class of mistake as testing
a path the gate never executes.

Recorded as required (not optional) in the DP-007 image, for the same reason.

Suite: 92 cases, 6136 assertions, with CBLAS compiled in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-026, AR-027, DP-007 | SR-001
 2026-07-31 15:04:29 +02:00
dtourolleandClaude Opus 5 13bdc27566 feat: join the decode butterfly so scene boundaries reach the face branch 
AR-010 — is_scene_boundary had no producer: SceneDetectorFunc was a terminal
sink writing scenes.json and never annotating the frames flowing to face
detection. The flag was permanently false, so the boundary half of AR-007's
frame-dependent association was dead code that a test could still exercise
synthetically and appear to verify.

The topology already forks after decode — dense frames to TransNetV2, sampled
frames to face detection — so this is a fork-join. SceneBoundaries is the join:
the detector publishes each window's verdict with a watermark, and an annotator
on the sampled branch stamps the flag.

The watermark is the part that matters. TransNetV2 buffers 100 frames before it
can score any of them, so at any instant it has an opinion up to some time T and
none after. Without recording T a consumer cannot tell "no boundary" from "not
scored yet", and those demand opposite behaviour — treating unscored frames as
boundary-free is exactly what makes a downstream check pass while verifying
nothing.

Buffering alone does not work, which was my first attempt. Channel depth creates
lag only when the consumer is slower, and the face branch runs four orders of
magnitude faster per frame than TransNetV2 (0.01ms vs 400ms), so its channels
drain instantly and no lag accumulates. Measured: 106 of 364 frames outran the
detector. The annotator therefore waits on the watermark explicitly. The
detector signals completion so the tail cannot deadlock, and publishes from
flush_remaining too — without that the final frames arrive with no verdict.

Boundaries are deduped on publish, matching what scenes.json does at write time.
A run of adjacent high-scoring frames is one boundary, not several; leaving them
raw made this view report 357 where the file said 13. Now the two agree exactly.

Frames past the detector's last scored window remain unverified and are counted
as such rather than silently marked boundary-free.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-010 | SR-002
 2026-07-31 14:56:38 +02:00
dtourolleandClaude Opus 5 fa1c494825 docs: AR-010 is blocked on a design decision, not an implementation gap 
Making SceneDetectorFunc a pass-through does not work. TransNetV2 buffers 100
dense frames before it can score any of them and trusts only each window's
centre, so a boundary at time T is not known until roughly 3.3s after T at
30 fps. The face pipeline runs on a parallel branch and has long since passed T.
An association hint that arrives after the association is worthless.

Three options recorded with their costs: two-pass (correct, doubles the decode
that already dominates runtime), delaying the face branch (couples the two
branches' timing, which invites heisenbugs under backpressure), or leaving it
unwired.

Leaving it unwired costs less than it looks, which is what makes this a decision
rather than a defect. The redesign made cuts and boundaries do the same thing —
both say "spatial continuity is broken, associate on embedding" — so TransNetV2
adds nothing over the histogram except on transitions the histogram cannot see:
slow dissolves and fades. That gap is real but narrow.

Where TransNetV2 still earns its cost is AR-019, whose promotion gate wants a
span free of cuts and boundaries. A late answer is fine there, because promotion
happens on track confirmation rather than per frame — so it can be wired
offline against the collected boundary list, off the hot path entirely.

Recommendation: leave the association path on is_cut alone, wire boundaries into
AR-019, and revisit if dissolve-heavy material shows association failures.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-010, AR-019 | SR-002
 2026-07-31 14:27:05 +02:00
dtourolleandClaude Opus 5 f99f1c5ccc feat: no fixed cap on faces per frame 
AR-003 — max_faces defaults to 0, meaning no cap. A fixed cap discards the
SMALLEST faces first, which are exactly the background cast X-Ray still credits
with scene membership, so the pipeline was systematically losing the people it
is supposed to find in crowded scenes.

This is only safe now that AR-004 landed. Previously an uncapped frame would
have pushed more work into channels that dropped on overflow, trading a visible
cap for silent loss. With backpressure the producer slows instead, so per-frame
cost is contained rather than discarded.

The matcher's kMaxFaces used to throw above 32, which made it an accidental
second cap. It sizes the similarity engine's preallocated buffer, so it bounds
memory rather than face count — the frame is now scored in batches of that size.
Memory stays bounded; faces do not.

Largest-first ordering is kept even without the cap, and the comment now says
why: the Hungarian solver tie-breaks on index order, so that ordering is
load-bearing for the replay determinism test rather than a leftover of the cap.

Verified end to end on a real clip: identical output to the capped run (385
frames, 693 faces), which is expected since that footage peaks at 4 faces per
frame — the point is the absence of a regression. The committed fixtures remain
byte-identical and valid.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-003 | SR-002
 2026-07-31 14:22:20 +02:00
dtourolleandClaude Opus 5 3605b8da78 fix: point the KPN submodule at the merged commit 
The recorded pointer was be6e922 — the backpressure fix as originally committed,
before it was rebased onto KPN master. That commit exists on no pushed branch,
so a fresh clone of this branch could not fetch the submodule at all.

Now 6595e6e, the same change on KPN master.

Worth noting for next time: rebasing a submodule commit after the superproject
has already recorded it silently invalidates the pointer. Nothing in the
superproject's status shows it, because the submodule working tree is clean and
at a valid commit — just not the one recorded.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004 | SR-002
 2026-07-31 11:53:27 +02:00
dtourolleandClaude Opus 5 61e487fbee test: replay the real tracker and registry from committed fixtures 
Tier T2 — composition rather than units. The registry tests construct awkward
states directly; these feed the pieces real 480x360 footage with the cuts, gaps
and crowded frames that synthetic input does not produce.

Six cases:
- fixture integrity: exact frame and face counts, contiguous face_offset, and
  the embedder identity each dump carries (GR-004). The counts are asserted
  exactly rather than approximately, which was impossible before AR-004 — what
  a lossy run dropped depended on timing.
- determinism: replaying a fixture twice gives identical track ids and windows.
  This is the property the whole fixture strategy rests on; without it every
  golden output derived from a fixture is unreliable and the CI replay tier is
  worthless.
- every face is assigned a track, and flush leaves nothing open — a track still
  live at EOF is a window that never reaches the output.
- windows are well-formed and inside the clip. A window ends at the last
  sighting, so it can never extend past the footage that produced it.
- a longer extinction window yields fewer, longer tracks. On the sparse fixture
  (140 faces over 385 frames) that is the difference the constant actually
  makes: absorbing a gap versus splitting a window.
- the cut-heavy fixture still contains cuts. This guards the corpus, not the
  code: a regeneration that produced cut-free fixtures would leave the
  association tests passing while silently testing nothing.

Driving the functors directly rather than through a KPN network is deliberate —
no threads, no channels, no scheduling, so the same input gives the same output.

Suite: 86 cases, 6106 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004, AR-012, AR-013, VR-001, VR-002 | SR-002
 2026-07-31 11:23:17 +02:00
dtourolleandClaude Opus 5 c12838b9fd fix: dropped frames fail the run instead of printing a footer 
A drop was reported to stderr and the process exited 0, so a run that discarded
320 frames "succeeded" and produced a truth file that looked complete. The
output in that case is a claim about footage that was never analysed, and
nothing in the file says so.

Now exits 2 and says why. Distinct from 1 (node crash) because the failures are
different: a crash produced no output, a drop produced output that cannot be
trusted.

This is also the regression test for AR-004 that otherwise did not exist. The
backpressure fix is one line in the KPN submodule — easy to lose in an update —
and with data pushes blocking, a drop can no longer occur on the data path. So
any drop now means either that fix regressed or a channel was disabled mid-run,
and both are worth stopping for.

Verified: a clean run still exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004 | SR-002
 2026-07-31 10:45:39 +02:00
dtourolleandClaude Opus 5 d31526cfaf test: committed replay fixtures from the public-domain corpus 
Five HDF5 embedding dumps from bali/ — Road to Bali (1952) — 3.6 MB total,
generated at 5 fps with a 32 px minimum face. CI never calls a model, so
inference happens on a GPU host and CI replays these as data; everything
downstream of embedding is cheap CPU maths.

Public domain is the reason this corpus rather than a convenient one: derived
fixtures can be committed, where anything cut from a copyrighted title could not
live in the repository at all.

The set covers distinct behaviours rather than being five of the same thing:
bali_28 has 9 cuts, so it exercises shot/reverse-shot association (AR-007);
bali_46 is sparse at 140 faces over 385 frames, so it exercises gaps and
extinction (AR-013); bali_13 is the busiest at 4 faces per frame; bali_31 is
short at 29s. All five recorded zero drops.

Both pinned parameters are consequences of measurements, not defaults: 5 fps
because 1 fps over a 77s clip is 77 frames, too thin for an extinction window
measured in tens of seconds; 32 px because that is the VR-005 floor, and the
corpus is 480x360 so a stricter value would reject most of what is there.

make_fixtures.sh regenerates them. Reproducibility is the requirement — a
fixture whose provenance is unknown is worse than none, because it will be
trusted. These are byte-reproducible only because of AR-004: before node
outputs blocked rather than dropped, the same command produced different dumps
run to run.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: VR-001 | PR-002
 2026-07-31 10:41:51 +02:00
dtourolleandClaude Opus 5 09a4650fd9 feat: backpressure — the pipeline slows instead of losing frames 
Picks up the KPN fix: node data outputs block on a full channel rather than
dropping. Sentinels stay out-of-band, so EOF can always overtake a stalled data
path and the hold-and-wait deadlock that comment warns about is not reachable.

Verified on a 77s clip at 5 fps, which should yield 385 sampled frames:

  before  65 written, 320 dropped, 29s, two runs differ
  after   385 written, 0 dropped, 17s, two runs byte-identical

The determinism is the part that matters. Golden fixtures were impossible while
what got dropped depended on timing; VR-001 fixture generation is unblocked by
this, and so is the CI replay strategy that depends on it.

Faster rather than slower, which is worth recording because the intuition runs
the other way: a dropped frame has already cost its decode, and the overflow
exception cost more still.

AR-004 is not fully closed. Channel capacity remains a count of items, while a
face carries a 112x112 crop and a 512-float embedding — so a crowded frame
occupies far more memory per slot than a sparse one. Bounding by bytes in
flight is the remaining half, and it matters once max_faces is removed (AR-003).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004, VR-001 | SR-002
 2026-07-31 10:35:29 +02:00
dtourolleandClaude Opus 5 b98372bad8 docs: AR-004 is a KPN change, with the measurement behind it 
Backpressure cannot be implemented in this repository. Every node output in KPN
uses the dropping push() (pool_node.hpp:404 and :710, plus branch, fanout and
interrupt_node). A lossless push_blocking() already exists on both Channel and
OutputPort — "wait for the consumer to drain instead of dropping; the producer
just runs slower" — and nothing calls it. The fix is a per-channel policy or a
network default in KPN, and this pipeline should select lossless: a dropped
frame here does not degrade a result, it silently changes one.

Measured rather than inferred. One 77s clip at 5 fps should yield ~385 sampled
frames. On CPU it produced 49, ending at 51s, with 285 dropped at camera_pos
and 51 at face_aligner. Rebuilt with CUDA the same clip ran in 29s and reached
EOF correctly, and still dropped 320 at camera_pos, yielding 65. Faster
hardware moves where the queue backs up; it does not change what happens when
it does — which is why this is a correctness requirement rather than a
throughput one.

Raising channel capacity is therefore a stopgap: it lowers the probability of
overflow without changing the behaviour on overflow, and the failure it hides
is silent corruption of the output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-003, AR-004, VR-001 | SR-002
 2026-07-31 10:29:53 +02:00
dtourolleandClaude Opus 5 e0f9c95689 docs: minimum face size is 32px, measured (VR-005) 
Replaces the 66px working estimate with the sweep result. 258 probes degraded
to each size and matched against a native-resolution gallery: 16px 46.5% TPI,
24px 93.4%, 32px 98.1%, 40px 99.2%, flat to 112px. The knee is 24-32 and 32
sits within about a point of the ceiling.

The estimate was roughly twice too strict. At 66px a large share of usable
faces would have been discarded, and on 480x360 sources most of them — which
is exactly the resolution of the fixture corpus.

The more useful finding: false identification was 0.0 at every size, including
12px. Small faces fail by becoming unidentified, never by being attributed to
the wrong actor. That asymmetry is what makes a low threshold safe — the cost
of admitting a marginal face is a miss, not a false claim.

Caveat recorded rather than assumed: FPI grows with gallery size, so 258 actors
understates it against a full library. Treat 0.0 as an observation at this
scale, not a property.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-002, VR-005 | SR-002, PR-002
 2026-07-31 10:22:38 +02:00
dtourolleandClaude Opus 5 9e4cdc4efc docs: bali fixture corpus, and AR-004 blocks reproducible fixtures 
Records `bali/` — five ~77s clips of Road to Bali (1952) — as the fixture
source. Public domain, which is the point rather than a convenience: derived
fixtures can be committed, where anything cut from a copyrighted title could
not live in the repository at all.

Makes explicit what the tier table only implied: CI never calls a model. Not a
preference — the embedder measures ~930 ms/frame on the CPU provider, so a 77s
clip at 5 fps is six minutes of inference. Every model invocation happens
locally and CI consumes the result as data, which is what makes the T1/T2 split
load-bearing rather than stylistic.

Two properties of the corpus to design around: 480x360 puts many faces below
the AR-002 66px minimum, so generation must set and record --min-face-px; and
77s at 1 fps is too thin to exercise an extinction window measured in tens of
seconds, so fixtures want 5 fps.

The finding that matters: a trial dump produced 49 frames of an expected ~385,
stopping at 51s of 77s, with 285 frames dropped at camera_pos and 51 at
face_aligner on channel overflow. Channels drop rather than block, and what
drops depends on timing, so the same command twice can yield different dumps.
Golden fixtures cannot be built on that — AR-004 is a prerequisite for VR-001
fixtures, not just a throughput concern for crowd scenes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-004, VR-001 | PR-002
 2026-07-31 10:19:10 +02:00
dtourolleandClaude Opus 5 08941540cb feat: presence windows come from registry claims (schema_version 2) 
The sink no longer reconstructs presence from per-frame detections. A reaped
track already IS a window — [first_seen, last_seen] of a track an actor owned —
so it is pushed straight to the aggregator when it dies and written out as-is.

AR-012 completed end to end. The annealing pass is deleted, not disabled:
anneal_sec existed only to bridge gaps between isolated accepted frames, and a
track that survives its own gaps leaves it nothing to do. The field is REMOVED
from the output rather than zeroed — a field naming a mechanism the pipeline no
longer has is actively misleading to anyone reading a manifest, and would
outlive everyone who remembers why it reads 0.

IR-002 — schema_version 2, matching jRay/SPEC.md JR-002. Windows become objects
carrying `belief` and `route` rather than bare float pairs, so a consumer can
caveat or filter instead of treating every window as equally certain. The new
`extraction` block carries `extinction_sec` (the successor to anneal_sec, and
what a consumer actually needs to interpret a window) and `gallery_scope` —
global vs limited being the strongest single quality signal when two manifests
compete for one cut, since identical gallery_size can mean very different
recall.

AR-016 wired: a pre-write hook flushes the registry with the last timestamp
seen, so tracks still live at EOF are emitted. A film ends with faces on screen
and those tracks have not timed out; without this the closing scene's cast is
silently dropped, which reads as a recognition miss rather than a bookkeeping
bug.

IR-003 stays In Progress deliberately: the sink now writes after the flush, but
the deferred re-identification pass (AR-020) does not exist yet, so output is
still final at EOF rather than after it.

This is a BREAKING format change and part of the coordinated SR-003 bump — it
must ship together with the jRay reader and the server's acceptance of the new
shape, not ahead of them.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002, SR-003
 2026-07-31 10:10:51 +02:00
dtourolleandClaude Opus 5 fe29d014da feat: identity evidence reaches the registry 
Closes the link that made AR-012 inert: the tracker was maintaining registry
state, but nothing called observe(), so no belief accumulated, no track was ever
owned, and no presence claim could be emitted. Tracking worked and presence did
not.

The matcher now feeds every scored face to the registry as a calibrated
posterior plus its embedding. Deliberately every scored face, not only the ones
clearing prob_threshold: a run of near-misses for one actor is evidence, and
discarding it would leave ownership depending on the per-frame threshold this
redesign exists to stop relying on. The registry discounts for correlation and
decides ownership from the accumulated posterior (AR-025).

The registry is an optional dependency of the matcher. Without one it behaves
exactly as before, which keeps the replay harness and the unit tests working
unchanged rather than forcing every caller to construct a registry it does not
need.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-012, AR-025 | SR-002
 2026-07-31 10:05:59 +02:00
dtourolleandClaude Opus 5 be5f67fa96 feat: tracker on the registry — one pool, calibrated, frame-dependent 
Merges feature/tracker-registry. See e9aea3f for the detail; in summary the
tracker no longer owns track state, association weights embedding over position
whenever position is uninformative, and every similarity comparison is a
calibrated probability.

Four constants retired: track_max_embed_dist, cut_revive_sim,
cut_inactive_max_frames, track_max_frames_missing.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-008, AR-024 | SR-002
 2026-07-31 10:01:31 +02:00
dtourolleandClaude Opus 5 e9aea3fc41 feat: tracker owns no state; association is frame-dependent and calibrated 
Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.

AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.

AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.

AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.

Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.

Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-008, AR-024 | SR-002
 2026-07-31 09:58:27 +02:00
dtourolleandClaude Opus 5 b7c96641a9 docs: no exact tier — the file-hash tier was withdrawn 
A stale reference to 'the runtime/exact tiers' as the fallback for media too
short to carry an audio signature. The exact tier keyed on a file hash and was
withdrawn on legal grounds: it fingerprinted the individual copy a user holds
rather than the cut the timings describe.

The pipeline never emitted a video_hash, so nothing in the code changes — but a
spec that still names a withdrawn tier is what makes the withdrawal look like an
oversight to the next reader, which is exactly how it nearly got re-added.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-31 09:55:50 +02:00
dtourolleandClaude Opus 5 843852e19c feat: registry owns correlation discounting (AR-024, AR-025) 
Moves two responsibilities inside the registry that callers should never have
been trusted with.

AR-025 — per-frame evidence is discounted for correlation by the registry
itself, via EvidenceDiscounter. Log-odds accumulation is only valid for
independent observations, and consecutive frames of one track are anything but:
near-identical pose, lighting and expression. Accumulated naively, thirty frames
of the same face at the same angle drive the posterior to certainty on what is
effectively one measurement.

Each observation is weighted by how much it adds — a view already contributed
counts for ~nothing, a genuinely new pose counts in full. This reuses the
novelty judgement gallery expansion already makes rather than inventing a second
one. The discounter is a separate class the registry holds, so it stays testable
and swappable, but it is a constructor argument rather than an option: there is
no correct way to accumulate without it.

AR-024 — observe() takes a calibrated probability and converts to log-odds
internally. A caller can no longer hand it a raw cosine, which would have been
silently wrong rather than obviously so. Retiring the remaining raw-cosine
constants in the tracker is still open.

DeadTrack now reports effective_obs alongside observations: the raw count and
the evidence that actually counted. A large gap between them is a track the
camera stared at, and worth seeing.

Three tests, one of which is the point: two tracks given the same number of
observations at the same posterior, one repeating a single view and one seeing
eight distinct ones, must not end up equally confident. Without discounting they
would be identical.

Fixed a test that asserted a belief swap on tied evidence. A tie leaves
ownership where it is — a challenger must out-accumulate the incumbent, since
one contrary observation is noise. The original test passed only because it fed
raw log-odds directly.

Suite: 78 cases, 3245 assertions. Coverage 20/63 to 22/63.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-024, AR-025 | SR-002
 2026-07-31 09:15:44 +02:00
dtourolleandClaude Opus 5 f0c7126f80 feat: TrackRegistry — presence follows track extent 
The spine of the redesign. Presence is now the extent of a track an actor owns,
[first_seen, last_seen], rather than the subset of frames in which recognition
happened to succeed. An actor recognised only at the end of a long track is
present for all of it, which is what the scene-scoped ground truth actually
records.

AR-013 — `last_seen` as an optional carries the entire liveness state: unset
means on screen, set means went off at that timestamp and still revivable,
reaped means emitted and erased. No missing-frame counter, no expired flag. It
subsumes the tracker's existing two-pool split, so there is no separate revival
path — matching a dormant track is ordinary inter-frame association.

The asymmetry is the point: interior gaps are claimed, the trailing cool-down is
not. A face lost and re-associated within the timeout never closed its track, so
the gap is presence — someone briefly occluded has not left the scene. But a
track that dies ends at its last sighting, never at the death time. That is
precisely the over-claim the retired extinction_sec keep-alive produced, where
presence ran on into the closing credits.

AR-014 — a belief swap A→B closes the track and opens a successor at the swap
frame. Not a correction: two non-twins both clearing the threshold on one face
is not realistic, whereas a track_id carried across a viewpoint change onto a
different person is. Treating it as a swap-and-continue would emit one window
blending two people; treating it as a boundary yields two that are each right.

AR-015 — two live tracks owned by one actor means at least one is wrong, since a
person cannot be in two places at once. A reverse index catches it on the update
that causes it rather than by scanning. This makes identity a third cut
detector, independent of the histogram and TransNetV2 and firing where those
failed.

AR-016 — flush() closes tracks still live at EOF. Without it a film ending
mid-shot silently drops its closing cast, which presents as a recognition miss
rather than a bookkeeping bug.

Reaping hands the dead track to the aggregator and erases it, so the registry
holds only live tracks and its size is bounded by concurrent on-screen faces
rather than growing with the film.

Locking: a frame's association pass is atomic as a unit via FrameScope, since
per-call locking would let another thread observe a half-updated frame.
owner() reads tally and verdict under one lock — separately, a track could be
both unowned and owned within a single promotion decision. A vote for an
already-reaped track is dropped and counted, because a nonzero count means the
timeout is shorter than the matcher's lag.

11 unit tests, driven directly against the registry with no network and no
fixture — the awkward cases are constructed rather than hunted for. Suite: 75
cases, 3236 assertions. Coverage 14/63 to 20/63.

Not yet wired into FaceTrackerFunc; that is AR-007/AR-008.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-012, AR-013, AR-014, AR-015, AR-016, AR-017 | SR-002
 2026-07-31 09:08:39 +02:00
dtourolleandClaude Opus 5 b35d49c772 docs: tag the implemented core with its requirement IDs 
Adds TRACES tags to code that already satisfies a Done requirement, so coverage
reflects what exists rather than starting from zero:

AR-001 face detection, AR-005 ArcFace alignment, AR-023 calibration fit,
DP-001/DP-002 the single analysis core behind the CLI, IR-001 truth-file
emission, IR-006 the Jellyfin round trip, GR-001/GR-002 gallery build and
incremental merge, VR-001 the embedding dump, VR-002 replay through the real
nodes, VR-003 per-second scoring.

Only Done requirements are tagged. A tag on Planned work would inflate coverage
with fiction that looks plausible — the same failure family as a gate that
cannot fail, and harder to spot.

GR-005 (gallery never leaves the instance) stays untagged deliberately: it is a
prohibition satisfied by the absence of an egress path, so there is no unit that
decides it. Same shape as PR-005 in the system spec, which has no software row
for the same reason. A goal held only by prohibitions cannot be verified by
pointing at code.

Coverage 5/63 to 14/63. The three VR tags are reported as tagged-but-unexecuted
and excluded from the numerator, since their tier cannot run on the CI host —
tagging deliberately cannot raise the number on its own.

Suite still 64 cases, 3199 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-001, AR-005, AR-023, DP-001, DP-002, IR-001, IR-006, GR-001, GR-002, VR-001, VR-002, VR-003
 2026-07-30 21:20:27 +02:00
dtourolleandClaude Opus 5 62396fce75 docs: record why exclude_dirs stays unset, refresh matrix 
The tool's defaults already exclude `vendor`, which covers the submodule at
scripts/vendor/jray-project. Setting the key explicitly is a trap worth
documenting: it REPLACES the defaults rather than extending them, and matching
is on path components rather than prefixes — so ["scripts/vendor"] matches
nothing while silently dropping __pycache__, node_modules, build and the rest.
Verified: the submodule's source is not scanned, and the only vendored path in
the report is the system spec it reads for PR/SR orphan checking.

Coverage after the merges: 5/63, 0 orphans. Every tag names a real requirement,
and nothing claims a requirement that is still Planned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 21:16:15 +02:00
dtourolleandClaude Opus 5 908d166173 feat: bind galleries to the embedder that built them 
GR-004 — a gallery built with one embedding model is meaningless with another.
Cosine similarities across models are garbage but look entirely plausible, so
this fails silently and expensively; every measurement taken against a
mismatched pair would have been quietly wrong.

The stamp is the model basename plus a SHA-256 of its bytes, with embed_dim as
a cheap extra guard. The hash decides and the name explains, because neither
works alone: a name is a promise rather than a fact — models get re-exported in
place under an unchanged filename, which is exactly the case where the weights
differ and nothing else does — while a bare hash mismatch tells an operator
nothing actionable.

Mismatch is fatal in every mode with no bypass. Unstamped only warns, because
unstamped is unknown rather than known-bad, and an error firing on every legacy
gallery trains people to reach for the bypass reflexively. scripts/stamp_gallery.py
binds an existing gallery in place with no re-embedding, so the warning is a
migration step rather than a permanent state; --require-gallery-stamp promotes
it to an error once a site has migrated.

Two gaps found that would have defeated the requirement outright:

- Embedding dumps carried no stamp, so a replay — which has no live embedder —
  had nothing to check the gallery against. Dumps now carry embedder_model and
  embedder_sha256 as root attributes. Additive; schema_version stays 1. This is
  the same gap the dump audit identified independently.
- --merge produced one file holding two embedding spaces, which no later check
  can untangle. Merge paths now verify before writing.

The stamp also survives identity_matcher's calibration write-back, which would
otherwise have stripped it on the first analysis run — the check would have
worked exactly once.

Conflicts resolved additively: both branches appended a source to sae_gallery
and to the test target, and both edited the GR-004 register row.

Merged suite: 64 cases, 3199 assertions, passing on CPU with no GPU.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: GR-004, VR-001 | SR-001
 2026-07-30 19:04:07 +02:00
dtourolleandClaude Opus 5 662a469870 feat: v1 content-derived audio signature 
Implements the server spec §3 construction: a 120 s window centred on the media
midpoint, downmixed and resampled to mono 11025 Hz, a 4096/1024 Hann STFT, 32
log-spaced bands over 300-3000 Hz, one byte per frame carrying a 5-bit peak band
and a 2-bit energy class, base64 with a v1: prefix.

IR-004 — the signature itself, in src/audio_signature.{hpp,cpp}. Decode reuses
the already-linked FFmpeg libraries; libswresample was missing from ffmpeg_libs
and is added. The FFT is written out rather than taken from a library: the
output must be bit-identical against a separate C# implementation, so a
dependency whose version can change the numerics is a liability.

IR-005 — a golden fixture at tests/fixtures/audio/, verified against an
independent Python implementation producing identical bytes. FLAC rather than
WAV because 120 s of 11025 Hz PCM is 2.6 MB and does not compress in git; both
decode to identical samples. The PCM checksum is asserted separately from the
signature so a codec-level divergence is distinguishable from a DSP one.

IR-007 — media under 120 s emits no signature at all, since the centred window
underflows. The rule must be identical in both producers or signatures never
match on exactly the short items most likely to be misidentified.

IR-008 — the v1: prefix is emitted and honoured, so a future change to the DSP
chain is detectable rather than silently non-matching.

Six parameters the spec left undefined had to be pinned to reproduce a byte
stream at all: periodic Hann, band value as the mean of linear magnitudes, ties
to the lowest band, the energy-class definition and its thresholds, byte layout,
and the base64 alphabet. These are now normative in the server spec — left only
in a C++ header, the C# side would have guessed and diverged.

Not yet emitted into the truth file; that is the coordinated schema_version bump
under IR-002/IR-003.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003
 2026-07-30 19:00:08 +02:00
dtourolle 28e3bd9496 docs: generated traceability matrix 
Committed rather than ignored, matching house precedent: coverage becomes
visible to anyone browsing the repo, and its movement over time is real history
worth having in the log.
 2026-07-30 18:59:06 +02:00
dtourolleandClaude Opus 5 d9aaf8fa4e build: consume the shared traceability tooling via submodule 
jray-project is added at scripts/vendor/jray-project and the extractor is used
from there. Only two files are repo-local: traceability.toml, which carries
everything repo-specific, and the CI workflow that invokes the vendored gate.

The extractor is deliberately NOT copied in. One implementation, parameterised
by config — a second copy would drift from the first, and the tool already
proves it works unchanged against all three registers.

Enables system_spec so PR/SR orphan checking runs: previously uncheckable,
because the system spec lived outside every component's checkout.

Gate is green at 0.0% of 63 requirements, which is correct — nothing is tagged
yet. Eleven are flagged unverifiable on this CI host and excluded from the
numerator rather than counted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 18:58:40 +02:00
dtourolleandClaude Opus 5 a2ebdc4cdd docs: GR-004 done — gallery/embedder binding 
Stamp is model basename + SHA-256 + embed_dim: the hash decides, the name
explains. A name alone is a promise rather than a fact — models get re-exported
in place under an unchanged filename, which is exactly the case where weights
differ and nothing else does. A hash alone is unactionable in an error message.

Mismatch is fatal in every mode with no bypass. Unstamped only warns, because
unstamped is unknown rather than known-bad, and an error that fires on every
legacy gallery trains people to reach for the bypass. A migration script binds
existing galleries in place with no re-embedding, so warn is not permanent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: GR-004 | SR-001
 2026-07-30 18:37:07 +02:00
Claude 7db40f430d GR-004: bind galleries to the embedder that built them 
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.

The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.

Mismatch is a hard error in every mode, with no bypass, naming both sides.

Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.

Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.

Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.

Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.

Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 18:35:46 +02:00
dtourolleandClaude Opus 5 020306c94f docs: builder images and per-backend release binaries (DP-008) 
Adds the build/deploy story that was missing: containerised builder images for
cpu / cuda / rocm, and release jobs producing prebuilt binaries so a first
install need not compile.

States explicitly that this does not reverse DP-005. That requirement rejects
Docker as a *runtime* — GPU passthrough is fragile and exists only because of
the container. Using it as a *build* environment is the opposite case, and lets
one machine produce binaries for backends it cannot itself run. Build in a
container, run natively.

Two things deliberately cannot ship, and the installer must not imply otherwise:
TensorRT engines are GPU-architecture and TRT-version specific, so
build_trt_engines.sh still runs on the target; and models are ~725 MB in LFS,
orthogonal to the binary.

The base image is chosen by the OLDEST glibc to be supported, not by
convenience — a binary built in a container runs against the host's glibc, and
getting this wrong fails at load with GLIBC_2.xx not found. Accelerator runtimes
have the same shape of problem, so each image documents its compatible
CUDA/ROCm range and the installer checks it rather than discovering a mismatch
at first inference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: DP-005, DP-007, DP-008 | PR-004
 2026-07-30 18:35:10 +02:00
dtourolleandClaude Opus 5 2919ed68d1 docs: audit findings, verification tiers, CI image, artifact storage 
Corrections from the dump audit and the completed agent work:

- AR-010 is not started, not in progress: is_scene_boundary has no producer
  anywhere. SceneDetectorFunc is a terminal sink writing scenes.json and never
  annotates the frame, so the field is permanently false and the dump column a
  constant 0. A replay test of the frame-dependent track_alpha would pass
  vacuously — the worst failure mode for a verification gate.
- T1 (functor-level) becomes the primary verification tier, not T2. KPN node
  functors are plain callables constructed outside the network, so a node is
  tested by calling operator() with hand-built inputs. That removes four
  hazards at once: fixture provenance, replay-from-frame-0, cross-test state
  leakage, and replay-harness nondeterminism. It also means a dead upstream
  producer no longer blocks testing its consumer.
- VR-010 (dump provenance) and VR-011 (replay harness rewrite) added. A dump
  made with LVFace is currently byte-indistinguishable from one made with
  w600k-R50 — the GR-004 problem again, in the dump.
- Four requirements had no verification tier at all; the traceability gate
  found them.
- DP-007: CI builder image, CPU-only, pinned by tag in the Gitea container
  registry. Corpus fixtures go to the package registry rather than LFS: LFS is
  pulled on clone and would tax every developer for data only CI reads.
- IR-004/005/007/008 marked done; the v1 DSP parameters they had to pin are
  now normative in the server spec.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-010, DP-007, IR-004, IR-005, IR-007, IR-008, VR-001, VR-010, VR-011 | SR-002, SR-003
 2026-07-30 18:33:32 +02:00
dtourolleandClaude Opus 5 45ef7c1916 Add the v1 audio signature to the pipeline (IR-004, IR-005, IR-007, IR-008) 
Implements the content-derived spectral-peak signature from
JRay-public-server/SPEC.md §3 so a truth file is self-identifying: 120 s
window centred on the media midpoint, mono at 11025 Hz, 4096/1024 Hann
STFT, 32 log-spaced bins over 300-3000 Hz, one byte per frame (5-bit peak
band + 2-bit energy class), base64, `v1:` prefix.

Audio decode is a second stream from the FFmpeg libraries the pipeline
already links for video; libswresample is added to the existing
ffmpeg_libs interface target. The FFT is written out rather than pulled
from a library for the same reason the plugin vendors one: the output has
to be bit-identical across two languages, so a dependency whose version
could change the numerics is a liability.

The server spec fixes the geometry but not enough to reproduce a byte
stream — Hann periodicity, band aggregation, the energy-class definition,
tie-breaking and the base64 alphabet are all unconstrained by it. Those
are pinned in audio_signature.hpp and mirrored in the golden fixture, so
the plugin can be implemented from the fixture alone.

IR-005: tests/fixtures/audio/ carries a deterministic 120 s tone (FLAC —
lossless, so identical PCM to the WAV make_fixture.py emits, and 3.5x
smaller in git) plus the signature it must produce, the decoded-PCM
checksum and the full parameter contract. That directory is the artefact
shared with the plugin repo; the PCM checksum is separate from the
signature so a codec-level difference is distinguishable from a DSP one.

IR-007: media under 120 s emits no signature. Same for a file with no
audio stream or one that will not open — UR-9 is an enhancement and must
never be able to break a fetch.

Verified against an independent Python reference implementation: same
bytes. All 32 bands and all 4 energy classes appear in the golden vector,
and the window-centring test wraps the fixture in 90 s of silence either
side and requires the golden value back.

Not wired into the truth-file output yet — that is the schema_version
bump under IR-002/IR-003 and is deliberately out of scope here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 18:25:37 +02:00
dtourolleandClaude Opus 5 43d2c976c3 docs: replace phased plan with a per-requirement one 
The phase structure encoded ordering assumptions that stopped being true as the
design changed, and its Phase 2 still described retuning constants that are now
withdrawn. Ordering is now derived from per-requirement dependencies instead:
anything with no unmet dependency is startable.

Carries over the TrackRegistry design (now keyed to AR-012/AR-013) and records
what was withdrawn from the old plan, including the --presence-mode flag —
comparison against old behaviour uses recorded reference output rather than a
second live code path.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 17:36:09 +02:00
dtourolleandClaude Opus 5 a5299daf6e docs: software spec, requirements register, and implementation plan 
Adds the requirements baseline for the pipeline redesign:

- SPEC.md — software requirements with Current/Gap deltas per item, so the
  document doubles as a work list.
- requirements.md — stable flat IDs (AR/DP/IR/GR/VR) with parent traces,
  priorities, statuses, and a per-requirement verification plan. Replaces the
  thematic A1..E8 scheme, which had already produced an A1a and an out-of-order
  E6; IDs are now permanent and never reused.
- IMPLEMENTATION-PLAN.md — phased work.

The central change is AR-012: presence follows track extent rather than
per-frame recognition, so a window starts when an actor appears rather than
when the recogniser first succeeded. anneal_sec and extinction_sec are
withdrawn rather than retuned — a track that survives its own gaps leaves them
nothing to do.

Verification is shaped by CI running on an N100 with no dGPU: the existing
HDF5 dump makes everything downstream of embedding replayable on CPU, which
covers the bulk of the redesign.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 17:32:44 +02:00
dtourolleandClaude Opus 5 458116f118 fix(trt): drop explicit shapes for static TransNetV2; gallery over-fetch + dedup 
trtexec rejects --minShapes/--optShapes/--maxShapes for a fully static model
("Static model does not take explicit shapes"). TransNetV2's input is fixed at
1x100x27x48x3, so the shape comes from the model itself.

Gallery build now over-fetches TMDB/Wikidata candidates by a configurable
factor: near-duplicate stills (the same photo at different crops or
resolutions) are discarded after embedding, so downloading exactly
images_per_actor left actors short of that many *distinct* embeddings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-07-30 17:32:08 +02:00
dtourolle 2ea5737bbd fix(trt): read ONNX input tensor names instead of hardcoding input.1 
build_trt_engines.sh hardcoded 'input.1' for the ArcFace and SCRFD shape
profiles, which only matches arcface_w600k_{r50,mbf}. Building engines for
any other embedder failed with:

    Cannot find input tensor with name "input.1" in the network inputs!

Input names differ per model: LVFace-B_Glint360K uses 'data', arcface_r18
uses 'input', arcface_w600k_{r50,mbf} use 'input.1'. This matters now that
LVFace-B is the default embedder (src/config.hpp), so ARCFACE_MODEL=<LVFace>
is the expected path.

Read the name from each model via onnxruntime at build time.
 2026-07-30 13:32:56 +02:00
dtourolle 5d2f673a81 build: support OpenCV 5 and TensorRT 10 
OpenCV: distros (Arch/CachyOS) now ship OpenCV 5 as default. The config
package rejects a 5.x install when find_package requests 4, so probe for 5
first and fall back to 4. All components used here (core, imgproc, imgcodecs,
videoio, dnn, objdetect, highgui) exist in both.

TensorRT: nvinfer1::Dims5 was removed in TRT 10 (Dims2..Dims4 remain in
NvInferLegacyDims.h). Build the TransNetV2 rank-5 input shape via the generic
nvinfer1::Dims, which is valid on both 8.x and 10.x.
 2026-07-30 13:07:42 +02:00
dtourolle e5885977df docs: fix Lovelace frame description, Amanda Seyfried's box is a ghost 
Her bbox is frozen at identical coordinates for t=2450 and t=2451; the
dump's own per-frame detections show only one real face at t=2451, and
it matches the Chloë Sevigny box (IoU 1.0), not hers. The frame is one
ghost overlapping one fresh misidentification, not two competing fresh
identities as previously written.
 2026-07-21 09:10:55 +02:00
dtourolle 0bd2747069 docs: full data-grounded rewrite of the performance report 
Replaces narrative claims with verified numbers across all report pages:

- Cross-model held-out validation (LVFace/mbf/r18, all 5 held-out
  films): LVFace wins every film outright, not just "consistent with"
  the training-set pick. r50 dropped from the detailed comparison
  (gallery has ~30% fewer reference images per actor than the other
  three models on identical source photos).
- Per-film training breakdown: LVFace does not win every training
  film (mbf beats it on Lord of War); the 75.3% macro figure hides a
  10.7pp spread.
- Gallery coverage computed per film (20.3%-78.6%) instead of one
  flat 67%-missing average.
- Found and fixed a real scoring bug in optimize.py: a candidate
  whose hardest film's replay timed out was averaged over survivors
  instead of penalized, silently rewarding partial coverage. Affected
  3 of 16 training combos; corrected throughout, and optimize.py now
  scores an incomplete evaluation f1=0.0 instead of averaging over
  whichever films happened to finish.
- Every FPI frame in the deep dive now comes from the proper montage
  renderer (Onscreen/Offscreen panel, ghosts never drawn as boxes),
  never the bare-box debug overlay used earlier.
- Every distinct out-of-cast name across all 9 films gets its own
  frame at its first appearance (9 names, 4 films), not a
  single-example spot check: 2 ground-truth gaps, 1 photograph
  misread as a person, 6 genuine lookalike confusions.
- New methodology.md: the scene-level-vs-per-second scoring mismatch
  that the rest of the report assumes, written out once.
- Cut the deadlock/gdb debugging narrative from the experiment log;
  kept the one fact that matters (KPN's node/network split lets the
  expensive GPU stage run once and the cheap stage replay against
  cached embeddings).
- Plain declarative style throughout, no em dashes, no blog voice.
 2026-07-21 08:55:57 +02:00
dtourolle 4b5557974b docs: montage-renderer imagery, visual polish, README screenshots 
- switch report frames to the scene best/worst montage renderer
  (Onscreen/Offscreen panels + TPI/FPI/FN legend): perfect-second hero,
  wedding couple, funeral 19-of-20, polygraph bridging, crew-scene FN
  ceiling, Robert Patrick ground-truth gap, rapid-cut double label,
  Herbie Hancock on an in-fiction screen
- deep dive restructured: extinction bridging framed as designed
  behavior with a measurable cost (debug overlay draws the boxes; the
  shipped output is presence windows), plus the face-vs-presence
  ceiling and two X-Ray-is-wrong exhibits
- Material polish: light/dark palette toggle, landing-page grid cards,
  figure/caption CSS, how-to-read admonition; site_url set so 404 links
  resolve under the Pages subpath
- README: perfect-second and screen-call frames committed (gitignore
  exceptions), readme_example.jpg retired
- build_site.sh: stage_frame helper downscales montage frames to 1920px
  and pulls any missing montage-frames packages
 2026-07-19 22:27:57 +02:00
dtourolle 93b6827b14 docs: richer report — data figures, success/failure frames, commit-pinned repo links 
- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
 2026-07-19 22:16:38 +02:00
dtourolle b1efefac6f docs: richer report — data figures, success/failure frames, commit-pinned repo links 
- experiment_charts.py generates 4 figures from experiments/ artifacts:
  held-out per-film F1, 16-combo ranking, DE search landscape, and the
  Downton detector-vs-tracker ghost timeline (replaces the blank
  title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
  ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
  as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
  placeholders; build_site.sh pins them to the HEAD commit's raw URLs
  and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
  and to session-memory names; mark artifact-registry paths with their
  pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
  on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
 2026-07-19 22:06:56 +02:00
dtourolle 4925443e56 docs: four focused findings pages (best model, gallery scope, expansion, deep dive) 
Splits the rep4 write-up's key findings into their own linkable pages:
- best-model.md: calibration curves first (discriminative power, independent
  of any threshold), then F1 on the benchmark — LVFace-B Glint360K wins both.
- gallery-scope.md: whole vs. cast-restricted gallery, isolated from model and
  expansion choice — restriction wins on every axis, but isn't a shipped
  runtime feature yet.
- pose-expansion.md: the training-set expand_gallery effect, and the held-out
  replication attempt that found it doesn't reproduce (5 films, 2 models,
  after catching and fixing a replay-timeout truncation bug and a bbox
  first-match-instead-of-best-match bug in the comparison harness itself). An
  honest null result, with the methodology errors documented since they're
  exactly the kind that manufacture a false "it works!" finding.
- lvface-deep-dive.md: the winning model's held-out generalization gap, its
  two failure modes (frozen-bbox ghost tracks), and a verified case (cross-
  checked against Jellyfin's independent cast metadata) where LVFace
  correctly identified an actor that X-Ray's ground truth failed to credit.

Adds a "report-highlights" artifact-registry package (scripts/artifacts/
push_artifacts.sh, pull_artifacts.sh) for hand-picked illustrative frames that
aren't reproducible via the automated best/worst montage selection, and wires
pulling it into scripts/docs/build_site.sh.
 2026-07-19 19:40:19 +02:00
dtourolle d340da755a docs: rep4 bake-off write-up, MkDocs site, artifact-registry-backed experiments 
docs/rep4-optimizer-results.md is the main deliverable: the model bake-off +
threshold re-tune experiment log, including the ROCm teardown deadlock root
cause and fix, DE concurrency tuning, the 16-combo results table, held-out
validation against 5 films never seen by the optimizer (macro F1 67.4% vs.
75.3% training — a real generalization gap), the frozen-bbox "ghost track"
failure mode found via annotated frame evidence, calibration curves per model,
and an isolated-effects breakdown of gallery scope vs. pose expansion.

MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/
pulls referenced images from the artifact registry and generates the
calibration chart at build time (see the tooling commit) rather than
committing images to the repo.

experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every
data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories,
manifests, results) moved to the Gitea package registry. film-lut.template.json
is the committed placeholder for the gitignored file-lut.json (real local
movie paths, never shared — some source filenames carry scene-release tags).

Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for
the new scene-detection path.
 2026-07-19 19:12:22 +02:00
dtourolle 76df2f66aa test: add Catch2 unit test suite (gallery, calibration, tracking, similarity) 
GPU-free, model-free tests for the pure logic: gallery HDF5 save/load
round-trips (actors, embeddings, embedded calibration) and legacy JSON
read back-compat; the calibration sigmoid fit, boundary inversion, and the
in-memory hash-keyed cache reuse/staleness; TrackGallery's diversity-buffer
eviction, novelty/spread safety gates, and promotion; FaceTracker's IoU/
embedding association and cross-cut track revival; and the GEMM similarity
backend (forced to CPU so the suite runs without a GPU).

Verified: all 39 test cases / 1640 assertions pass (cmake -DSAE_BUILD_TESTS=ON).
 2026-07-19 19:10:57 +02:00
dtourolle 6f0ad83a55 feat(tooling): X-Ray threshold optimizer, gallery utilities, artifact registry, docs build 
Optimizer (scripts/optimizer/): replay.py runs the real C++ tracker/matcher/
scene_tracker chain over a dumped-embeddings HDF5 via sae_kpn, so a threshold
sweep never re-decodes video or re-embeds faces. optimize.py drives scipy's
differential_evolution over the knob space, with DE-level parallelism
(multiple population candidates evaluated concurrently via a ThreadPoolExecutor)
on top of per-film replay parallelism. second_score.py is the per-second X-Ray
scoring metric (TPI/FPI/FN, out-of-cast misID weighted 10x, fair recall masked
to gallery-known cast) that superseded an earlier scene-union metric.
dump_error_frames.py / dump_scene_montage.py extract annotated video frames
(bounding boxes, TPI/FPI/FN captions, onscreen-vs-offscreen split) for visual
review of a replay against ground truth. Gallery utilities: cast_restrict.py,
gallery_membership.py, fetch_missing_actors.py, reembed_gallery.py.

scripts/validation/: X-Ray ground-truth loading and provider-agnostic identity
matching (identity.py's keys_for — an actor is the union of every id we can
derive, since pipeline output and ground truth don't share one id space).

scripts/artifacts/: push/pull scripts for the Gitea generic package registry —
galleries, montage frames, and experiment data (manifests/trajectories/results)
are pushed there instead of committed, since none are needed to run the app,
only benchmarks. Versioned by git short-SHA.

scripts/docs/: MkDocs site build (build_site.sh) and the calibration-curve
comparison chart (calibration_chart.py, matplotlib, reads each gallery's
embedded calibration).

Gallery-building scripts (make_jellyfin_gallery.py, make_gallery.py,
filter_gallery.py, run_from_jellyfin.py, movienet_eval.py, movienet_prep.py,
sae_gallery.py) updated to read/write HDF5 galleries exclusively, matching the
engine-side format switch. run_from_jellyfin.py and the optimizer no longer
carry movie source paths in shared manifests (some source filenames include
scene-release tags) — resolved locally via a gitignored file-lut.json instead.
 2026-07-19 19:06:48 +02:00
dtourolle 26139ffe8a feat(engine): add Python replay bindings, gallery pose-expansion, scene detection, embedding dumps 
New C++ sources:
- kpn_bindings.cpp (sae_kpn): assembles the real face_tracker/identity_matcher/
  scene_tracker nodes inside a Python-driven KPN network via nanobind, for
  offline threshold-sweep replay against dumped embeddings (scripts/optimizer/).
- track_gallery.hpp: per-film gallery expansion — promotes a confidently-
  identified track's novel-pose reference views into an in-memory annex so
  later frames/tracks of that actor at similar poses are recognised, without
  touching the baked gallery.
- dump_embeddings.cpp: standalone exe that runs detect→embed only (no gallery,
  no matching) and dumps per-frame face embeddings + metadata to HDF5, so a
  parameter sweep can replay the expensive half once and vary tracking/matching
  config freely downstream.
- scene_detector.hpp / scene_detector_node.hpp: TransNetV2-based shot-boundary
  detection, opt-in alongside the always-on histogram cut detector.
- camera_position_change_detector_node.hpp, embedding_dump_node.hpp: supporting
  nodes for the above.
 2026-07-19 19:05:05 +02:00
dtourolle 41a277bc19 feat(engine): HDF5-native galleries with embedded calibration; TensorRT backends; scene detection 
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.

Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.

Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
 2026-07-19 19:04:03 +02:00
dtourolle aca6147d69 feat(scripts): add scene-gap histogram tool 
scene_gap_hist.py scans scene_analyze output JSONs and, for every actor,
computes the gap (next_scene_start - prev_scene_end) between consecutive
scenes, emitting a text histogram of the distribution. Used to inform the
anneal_sec default.
 2026-07-04 20:42:11 +02:00
dtourolle 65fee74585 perf(movienet): vectorise eval matching; count frames missing from Image.zip 
movienet_eval: replace the per-element dot() with numpy — actor references are
loaded once as an ndarray and scored with a single matmul, keeping a
whole-library gallery fast.

movienet_prep: count and report frames referenced by annotations but absent
from Image.zip instead of skipping them silently.
 2026-07-04 20:41:54 +02:00
dtourolle 1f5acc25df docs: document LVFace embedder support 
LVFace-B_Glint360K.onnx shares ArcFace's I/O contract (112x112 aligned crop ->
L2-normalised 512-d) and input scaling, so it drops in via --arcface-model.
Note the caveat that galleries and calib caches must be rebuilt with the same
embedder used for analysis.
 2026-07-04 20:39:43 +02:00
dtourolle 96b1c22194 feat(cameo): detect recognised actors not credited in a title 
Add two cameo hunters that flag actors recognised in a title but absent from
its cast:
  - cameo_jellyfin.py — pure-Jellyfin cast-membership check (no id cross-walk)
  - cameo_hunt.py     — TMDB filmography check (actor's combined_credits)

run_from_jellyfin.py now stamps the analysed title's Jellyfin item GUID into
the output JSON as top-level 'jellyfin_item_id' (scene_analyze can't know it),
which cameo_jellyfin.py uses to look up the cast in Jellyfin's own id space.
Document that field in the result-sink output schema header.
 2026-07-04 20:39:35 +02:00
dtourolle 3700c763dd tune: raise default anneal_sec from 2s to 10s 
Merge actor windows separated by up to 10s into one epoch, reducing scene
fragmentation from brief detection dropouts.
 2026-07-04 18:55:53 +02:00
dtourolle 66298026e2 feat(pipeline): detect node crashes and tally dropped frames 
Register a KPN event handler in both scene_analyze and scene_preview:
  - Overflow events accumulate per-node dropped-frame counts, printed on exit.
  - A Closed event from any node other than result_sink at EOF means a stage
    died; trip an atomic so the main loop bails out instead of hanging on
    'done' forever, and exit non-zero.

Bumps external/KPN to the commit that exposes set_event_handler / NodeEvent.
 2026-07-04 18:55:47 +02:00
dtourolle 152c34b1f4 refactor(scripts): extract shared sae_* helpers and dedupe gallery builders 
Consolidate copy-pasted logic across the gallery/run scripts into shared
modules:
  - sae_env.py     — zero-dependency .env loader (populates os.environ)
  - sae_tmdb.py    — TMDB API helpers (tmdb_get, person images, id lookups)
  - sae_jellyfin.py— Jellyfin API helpers (jf_get, id/URL normalisation)
  - sae_gallery.py — image download + gallery.json writing

make_gallery, make_jellyfin_gallery and filter_gallery now import these
instead of carrying their own near-identical copies.
 2026-07-04 18:55:37 +02:00
dtourolle aacaefb3dc chore(gitignore): ignore generated calib caches, cameo reports, and plots 
These are regenerable per-run outputs that were polluting the worktree:
calibration caches (*.calib_cache.csv/png), cameo detection run outputs
(cameo_progress.txt, cameo_report.txt), and scene-gap analysis plots.
 2026-07-04 18:54:37 +02:00
dtourolle ea92dd8150 add readme, liscence and use public KPN 2026-06-28 12:09:54 +02:00
dtourolle 0ee131a692 Add AMD support via ort alternative to trt 2026-06-28 11:50:05 +02:00
dtourolle a3ba53ddf7 improved performance 2026-06-13 22:44:44 +02:00
dtourolle fc16d4a0e1 improved jellyfin support 2026-06-12 20:57:33 +02:00
dtourolle a1d6759abc faster calibration curve generation 
jellyfin intergration
 2026-06-12 17:54:23 +02:00
dtourolle d753062c6c Initial commit: scene-actor-extraction pipeline 
Source (KPN++ pipeline nodes, ArcFace embedders, SCRFD/YuNet detectors,
gallery builder), build scripts, and eval artifacts.

- external/KPN as a git submodule (gitea.tourolle.paris/dtourolle/KPN)
- ONNX models tracked via Git LFS (models/*.onnx)
- generated outputs, TensorRT engines, reference repos, and media ignored
 2026-06-12 15:29:01 +02:00
268 changed files with 41544 additions and 17798 deletions
Expand all files Collapse all files
.gitattributes
+1
View File
@@ -0,0 +1 @@
models/*.onnx filter=lfs diff=lfs merge=lfs -text
.gitea/workflows/traceability-check.yml
+154
View File
@@ -0,0 +1,154 @@
name: Traceability Validation
# Mirrors JellyTau's .gitea/workflows/traceability-check.yml. The extractor is
# stdlib Python, so there is no toolchain install step and no jq.
#
# This workflow is component-agnostic: every repo-specific setting - which ID
# prefixes count, which file suffixes are source, which directories to scan,
# the threshold - lives in traceability.toml at the repo root, and the same
# extractor is shared by all three JRay components. Copying this file into
# another component needs no edits.
#
# NOTE: the runner here is an Intel N100 with no discrete GPU. This job is only
# ever static analysis of source comments plus markdown parsing, so it is cheap;
# the requirements it reports as "tagged but unexecuted" are the ones that need
# a GPU host, and they are deliberately never counted as covered.
on:
push:
branches:
- main
- master
- develop
pull_request:
branches:
- main
- master
- develop
jobs:
validate-traces:
runs-on: linux/amd64
name: Check requirement traces
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
submodules: recursive
- name: Check Python is available
run: |
set -e
command -v python3 >/dev/null 2>&1 || {
echo "python3 is missing from the runner image."
echo "The traceability tooling is stdlib-only Python;"
echo "3.9+ with CLI flags, 3.11+ to read traceability.toml."
exit 1
}
python3 --version
# The gate's own arithmetic is the thing being trusted, so its tests run
# before it does. JellyTau's gate was believed for months while it was
# dividing by frozen literals; untested gate logic is how that happens.
- name: Test the extractor
run: python3 scripts/vendor/jray-project/scripts/traceability/test_extract_traces.py
# Threshold policy and every other repo-specific setting live in
# traceability.toml, not here, so local runs and CI runs cannot disagree
# about what "passing" means. Denominators come from docs/requirements.md
# at run time and are never hardcoded -- in this file or anywhere else.
#
# A misconfigured run (zero requirements parsed, zero files scanned) is a
# hard failure rather than a plausible-looking 0%.
- name: Traceability gate
run: sh scripts/vendor/jray-project/scripts/traceability/traceability-gate.sh
# AR-024's register row names its verification tier as "Static check --
# no bare cosine outside a tagged EXCEPTION". This is that check, and it
# belongs here rather than in unit-tests.yml because it is static
# analysis of source text, like everything else in this job, and needs
# no toolchain. It blocks: an untagged bare cosine is a defect by the
# invariant's own wording, not a warning.
- name: AR-024 — no bare cosine outside a recorded exception
run: python3 scripts/ci/check_raw_cosine.py
- name: Check modified files for traces
if: github.event_name == 'pull_request'
run: |
set -e
echo "Checking modified sources for TRACES tags..."
# The extensions come from the report the gate just wrote, which got
# them from traceability.toml. Restating them here would be a second
# place for the source-file definition to live, and the two would
# drift the first time a language is added.
PATTERN=$(python3 -c "
import json, re, sys
suffixes = json.load(open('traces-report.json'))['config']['sourceSuffixes']
print('(' + '|'.join(re.escape(s) + '\$' for s in suffixes) + ')')
")
echo "Source suffixes from traceability.toml: $PATTERN"
CHANGED=$(git diff --name-only "origin/${{ github.base_ref }}...HEAD" \
| grep -E "$PATTERN" || true)
if [ -z "$CHANGED" ]; then
echo "No source files changed."
exit 0
fi
echo "Changed files:"
echo "$CHANGED" | sed 's/^/ /'
echo ""
# Advisory by design: not every file implements a requirement, and a
# tag on every function is noise that rots faster than it helps
# (CLAUDE.md: tag the unit that decides). This step exists to prompt,
# not to block. The blocking checks are in the gate step above.
#
# Piped into the loop rather than a here-string, and `case` rather
# than `[[ == ]]`, so this works under dash as well as bash. The loop
# body runs in a subshell, so misses are recorded in a file.
MISSING=$(mktemp)
echo "$CHANGED" | while IFS= read -r file; do
case "$file" in
*/test_*.py|*_test.py|*Tests.cs|tests/*|*/tests/*) continue ;;
esac
[ -f "$file" ] || continue
if ! grep -q 'TRACES:' "$file"; then
echo " no TRACES tag: $file"
echo "$file" >> "$MISSING"
fi
done
COUNT=$(wc -l < "$MISSING" | tr -d ' ')
rm -f "$MISSING"
if [ "$COUNT" -gt 0 ]; then
echo ""
echo "$COUNT changed file(s) carry no requirement tag."
echo "Format: // TRACES: AR-012, AR-013 | SR-002"
echo " (pipe separates requirement types, comma separates IDs)"
echo "A deliberate invariant exception is tagged separately:"
echo " // EXCEPTION: AR-024 <reason>"
echo "See CLAUDE.md and SPEC.md section 6."
fi
- name: Report summary
if: always()
run: |
echo "Traceability matrix: docs/traceability.md"
echo ""
head -40 docs/traceability.md || true
- name: Save reports
if: always()
uses: actions/upload-artifact@v3
with:
name: traceability-reports
path: |
traces-report.json
docs/traceability.md
retention-days: 30
.gitea/workflows/unit-tests.yml
+140
View File
@@ -0,0 +1,140 @@
name: Unit tests
# TRACES: DP-007 | PR-004
#
# The tier the verification strategy is built on, finally executing.
#
# docs/requirements.md describes a four-tier plan in which T1 (functor unit)
# and T2 (replay) are "the only tiers that can exist in CI at all", and the
# traceability gate reports a CI-scope coverage fraction over exactly those
# tiers. Until this workflow existed, nothing ran them: "covered" meant a
# TRACES tag was present in a file, not that any test had been executed. That
# is the same failure mode as counting a test that cannot run, one level up,
# and the gate cannot detect it because a tag is all it can see.
#
# The runner is an Intel N100 with no discrete GPU. Nothing here calls a model:
# T1 constructs node functors directly, and T2 replays a precomputed HDF5 dump.
# T3 (ORT CPU smoke) and T4 (GPU) are deliberately absent -- the embedder is
# ~930 ms/frame on this hardware, so a 77 s clip at 5 fps would be six minutes
# of inference alone.
on:
push:
branches:
- main
- master
- develop
pull_request:
branches:
- main
- master
- develop
jobs:
unit-tests:
runs-on: linux/amd64
name: Build and run the GPU-free suite
# Pinned by tag, never `latest`, so rebuilding the image cannot silently
# change what a previous green build meant. Bumping the dependency set means
# bumping the tag in scripts/ci/build_builder_image.sh AND here, in one
# commit -- see that script's header.
container:
image: gitea.tourolle.paris/dtourolle/sae-builder-cpu:v1
steps:
- name: Checkout repository
uses: actions/checkout@v4
with:
fetch-depth: 0
# KPN is a submodule and the pipeline does not build without it.
#
# NOTE: this checks out the commit this repo PINS, which is the whole
# point and is also the first thing this job will disagree with a
# developer about. A local KPN working copy that is ahead of
# origin/master builds and passes here while CI builds something else
# entirely; the AR-004 evidence in docs/requirements.md was gathered
# that way. If this job fails on tests that pass locally, check
# `git -C external/KPN log origin/master..HEAD` before suspecting the
# tests.
# LFS is deliberately NOT fetched: SAE_MODELS_DIR is baked into the
# binary as a path string and nothing in T1/T2 opens a model file, so
# pulling ~hundreds of MB of ONNX would cost the job everything and
# buy it nothing.
submodules: recursive
lfs: false
- name: Assert the builder image is the pinned one
run: |
set -e
echo "builder=$SAE_BUILDER version=$SAE_BUILDER_VERSION"
echo "ort=$SAE_ORT_VERSION opencv=$SAE_OPENCV_VERSION"
# The image reports its own tag. A mismatch means the `container:`
# line above and the image that actually landed disagree, which is
# exactly the drift the pinning exists to prevent -- so it fails the
# job rather than building against an unknown toolchain.
[ "$SAE_BUILDER_VERSION" = "v1" ] || {
echo "image reports version '$SAE_BUILDER_VERSION', workflow pins v1" >&2
exit 1
}
- name: Fetch replay fixtures
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
# bash, not sh: the script declares #!/bin/bash and uses `set -o
# pipefail` and arrays, which dash does not have.
run: bash scripts/artifacts/pull_artifacts.sh replay-fixtures latest
# pull_artifacts.sh warns and continues when a package version is missing,
# which is right for a developer pulling one artifact of several and wrong
# here. A T2 test whose fixture never arrived must not look like a pass:
# the dumps are the entire input to the replay tier, and VR-002's claim is
# that replay drives the real nodes over real data.
- name: Verify the fixtures actually arrived
run: |
set -e
missing=0
for f in tests/fixtures/dumps/superhero.h5; do
if [ -s "$f" ]; then
echo " ok: $f ($(wc -c < "$f") bytes)"
else
echo " MISSING: $f" >&2
missing=1
fi
done
if [ "$missing" -ne 0 ]; then
echo "" >&2
echo "Replay fixtures are absent, so the T2 tier cannot run." >&2
echo "They are not in git (tests/fixtures/dumps/.gitignore) -- they" >&2
echo "live in the Gitea generic package registry and are pulled by" >&2
echo "the step above, which needs GITEA_TOKEN to resolve 'latest'." >&2
exit 1
fi
- name: Configure
run: |
set -e
# SAE_GEMM_BACKEND defaults to ROCM and the auto-detect prefers a GPU
# backend where it finds one; CPU is stated explicitly so this job
# cannot start depending on what happens to be installed on the runner.
# The CPU kernel is OpenBLAS in this image (tests/CMakeLists.txt fails
# the configure if it is not), so the suite exercises the kernel the
# CPU release actually ships.
cmake -S . -B build \
-DCMAKE_BUILD_TYPE=Release \
-DSAE_BUILD_TESTS=ON \
-DSAE_GEMM_BACKEND=CPU
- name: Build the test suite
run: cmake --build build --target sae_tests --parallel
- name: Run the tests
run: ctest --test-dir build --output-on-failure
- name: Save test output
if: always()
uses: actions/upload-artifact@v3
with:
name: unit-test-results
path: build/Testing/
retention-days: 30
.gitignore
+120
View File
@@ -0,0 +1,120 @@
# Build
build/
build-*/
cmake-build-*/
CMakeCache.txt
CMakeFiles/
*.cmake
Makefile
install_manifest.txt
compile_commands.json
# Compiled objects
*.o
*.a
*.so
*.dylib
*.json
# Exception: small, curated result summaries backing specific numbers quoted
# in docs/ (cross-model held-out scores, per-film training breakdown, gallery
# coverage). Regenerate with scripts/docs/run_holdout_all_models.py and
# scripts/docs/gallery_coverage_per_film.py.
!docs_data/*.json
# Exception: test fixtures are inputs, not build output. The audio golden
# vector (IR-005) is shared verbatim with the jRay plugin repo, so it has to be
# tracked. Regenerate the media with tests/fixtures/audio/make_fixture.py.
!tests/fixtures/**
# Video files
*.mp4
*.mkv
*.avi
*.mov
# ONNX models in models/ are tracked via Git LFS (see .gitattributes).
# Any stray ONNX elsewhere is generated/downloaded and not tracked.
external/*.onnx
# Generated TensorRT engines (rebuilt by ORT / scripts/build_trt_engines.sh)
trt_cache/
# ORT pre-optimized model cache (generated on first run, provider-specific)
ort_cache/
# Gallery files (generated — HDF5 only, see src/gallery/gallery_store.cpp).
# Legacy JSON galleries from before that switch are also excluded.
gallery.json
gallery_*.json
gallery.h5
gallery_*.h5
# Calibration cache (generated alongside a gallery, per-embedder)
*.calib_cache.csv
*.calib_cache.png
# Cameo detection run outputs (generated by scripts/cameo_*.py)
cameo_progress.txt
cameo_report.txt
# Analysis plot outputs (scene-gap histograms / KDEs, etc.)
scene_gap_*.png
# Per-frame debug images
images/
# Annotations output
annotations.json
*_annotations.json
# MovieNet evaluation data
movienet-ps/
# Eval probe images (data, regenerable)
eval/probe/
# Local reference repos kept for inspiration (each has its own .git)
inspiration/
# experiments/ has its own nested .gitignore for HDF5 galleries/dumps/X-Ray
# corpus (all pushed/pulled via scripts/artifacts/{push,pull}_artifacts.sh to
# the Gitea generic package registry instead of committed).
# Exception to the blanket *.json rule above: the committed placeholder for
# experiments/file-lut.json (see experiments/.gitignore).
!experiments/file-lut.template.json
# Site build output (mkdocs build). Rendered site is deployed to a
# gitea-pages branch, never committed to a working branch.
site/
docs_site/
# Images staged into docs/ from the artifact registry at build time
# (scripts/docs/build_site.sh) — not committed, pulled fresh on each build.
# Exception: pipeline_topology.svg is small and hand-authored (not pulled from
# anywhere) and the README references it directly, so it needs to render on a
# plain Gitea repo view too, not just the built Pages site. (A directory-level
# ignore can't be un-ignored file-by-file below it, so this must NOT blanket-
# ignore docs/assets/ itself — only its contents, minus the one exception.)
docs/assets/images/*
!docs/assets/images/pipeline_topology.svg
# These frames are referenced directly by README.md, which renders on the
# plain Gitea repo view — committed for the same reason as the SVG above.
!docs/assets/images/lovelace_perfect_second.jpg
!docs/assets/images/valerian_screen_call.jpg
# Python
__pycache__/
*.pyc
*.pyo
.venv/
venv/
# Local secrets (API keys — never commit)
.env
# Editor / OS
.vscode/
.idea/
.claude/settings.local.json
*.swp
*.swo
.DS_Store
Thumbs.db
.gitmodules
+7
View File
@@ -0,0 +1,7 @@
[submodule "external/KPN"]
path = external/KPN
url = https://gitea.tourolle.paris/dtourolle/KPN.git
branch = master
[submodule "jray-project"]
path = scripts/vendor/jray-project
url = git@gitea.tourolle.paris:dtourolle/jray-project.git
.nojekyll
View File
404.html
-713
View File
@@ -1,713 +0,0 @@
<!doctype html>
<html lang="en" class="no-js">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="description" content="Face-recognition pipeline for finding on-screen actor presence in film/TV, built on KPN++">
<link rel="icon" href="/dtourolle/scene-actor-extraction/assets/images/favicon.png">
<meta name="generator" content="mkdocs-1.6.1, mkdocs-material-9.7.7">
<title>scene-actor-extraction</title>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/main.ec1eaa64.min.css">
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/assets/stylesheets/palette.ab4e12ef.min.css">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Roboto:300,300i,400,400i,700,700i%7CRoboto+Mono:400,400i,700,700i&display=fallback">
<style>:root{--md-text-font:"Roboto";--md-code-font:"Roboto Mono"}</style>
<link rel="stylesheet" href="/dtourolle/scene-actor-extraction/stylesheets/extra.css">
<script>__md_scope=new URL("/dtourolle/scene-actor-extraction/",location),__md_hash=e=>[...e].reduce(((e,_)=>(e<<5)-e+_.charCodeAt(0)),0),__md_get=(e,_=localStorage,t=__md_scope)=>JSON.parse(_.getItem(t.pathname+"."+e)),__md_set=(e,_,t=localStorage,a=__md_scope)=>{try{t.setItem(a.pathname+"."+e,JSON.stringify(_))}catch(e){}}</script>
</head>
<body dir="ltr" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber">
<input class="md-toggle" data-md-toggle="drawer" type="checkbox" id="__drawer" autocomplete="off">
<input class="md-toggle" data-md-toggle="search" type="checkbox" id="__search" autocomplete="off">
<label class="md-overlay" for="__drawer"></label>
<div data-md-component="skip">
</div>
<div data-md-component="announce">
</div>
<header class="md-header" data-md-component="header">
<nav class="md-header__inner md-grid" aria-label="Header">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-header__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
<label class="md-header__button md-icon" for="__drawer">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M3 6h18v2H3zm0 5h18v2H3zm0 5h18v2H3z"/></svg>
</label>
<div class="md-header__title" data-md-component="header-title">
<div class="md-header__ellipsis">
<div class="md-header__topic">
<span class="md-ellipsis">
scene-actor-extraction
</span>
</div>
<div class="md-header__topic" data-md-component="header-topic">
<span class="md-ellipsis">
</span>
</div>
</div>
</div>
<form class="md-header__option" data-md-component="palette">
<input class="md-option" data-md-color-media="(prefers-color-scheme: dark)" data-md-color-scheme="slate" data-md-color-primary="black" data-md-color-accent="amber" aria-label="Switch to light mode" type="radio" name="__palette" id="__palette_0">
<label class="md-header__button md-icon" title="Switch to light mode" for="__palette_1" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 7a5 5 0 0 1 5 5 5 5 0 0 1-5 5 5 5 0 0 1-5-5 5 5 0 0 1 5-5m0 2a3 3 0 0 0-3 3 3 3 0 0 0 3 3 3 3 0 0 0 3-3 3 3 0 0 0-3-3m0-7 2.39 3.42C13.65 5.15 12.84 5 12 5s-1.65.15-2.39.42zM3.34 7l4.16-.35A7.2 7.2 0 0 0 5.94 8.5c-.44.74-.69 1.5-.83 2.29zm.02 10 1.76-3.77a7.131 7.131 0 0 0 2.38 4.14zM20.65 7l-1.77 3.79a7.02 7.02 0 0 0-2.38-4.15zm-.01 10-4.14.36c.59-.51 1.12-1.14 1.54-1.86.42-.73.69-1.5.83-2.29zM12 22l-2.41-3.44c.74.27 1.55.44 2.41.44.82 0 1.63-.17 2.37-.44z"/></svg>
</label>
<input class="md-option" data-md-color-media="(prefers-color-scheme: light)" data-md-color-scheme="default" data-md-color-primary="black" data-md-color-accent="indigo" aria-label="Switch to dark mode" type="radio" name="__palette" id="__palette_1">
<label class="md-header__button md-icon" title="Switch to dark mode" for="__palette_0" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="m17.75 4.09-2.53 1.94.91 3.06-2.63-1.81-2.63 1.81.91-3.06-2.53-1.94L12.44 4l1.06-3 1.06 3zm3.5 6.91-1.64 1.25.59 1.98-1.7-1.17-1.7 1.17.59-1.98L15.75 11l2.06-.05L18.5 9l.69 1.95zm-2.28 4.95c.83-.08 1.72 1.1 1.19 1.85-.32.45-.66.87-1.08 1.27C15.17 23 8.84 23 4.94 19.07c-3.91-3.9-3.91-10.24 0-14.14.4-.4.82-.76 1.27-1.08.75-.53 1.93.36 1.85 1.19-.27 2.86.69 5.83 2.89 8.02a9.96 9.96 0 0 0 8.02 2.89m-1.64 2.02a12.08 12.08 0 0 1-7.8-3.47c-2.17-2.19-3.33-5-3.49-7.82-2.81 3.14-2.7 7.96.31 10.98 3.02 3.01 7.84 3.12 10.98.31"/></svg>
</label>
</form>
<script>var palette=__md_get("__palette");if(palette&&palette.color){if("(prefers-color-scheme)"===palette.color.media){var media=matchMedia("(prefers-color-scheme: light)"),input=document.querySelector(media.matches?"[data-md-color-media='(prefers-color-scheme: light)']":"[data-md-color-media='(prefers-color-scheme: dark)']");palette.color.media=input.getAttribute("data-md-color-media"),palette.color.scheme=input.getAttribute("data-md-color-scheme"),palette.color.primary=input.getAttribute("data-md-color-primary"),palette.color.accent=input.getAttribute("data-md-color-accent")}for(var[key,value]of Object.entries(palette.color))document.body.setAttribute("data-md-color-"+key,value)}</script>
<label class="md-header__button md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
</label>
<div class="md-search" data-md-component="search" role="dialog">
<label class="md-search__overlay" for="__search"></label>
<div class="md-search__inner" role="search">
<form class="md-search__form" name="search">
<input type="text" class="md-search__input" name="query" aria-label="Search" placeholder="Search" autocapitalize="off" autocorrect="off" autocomplete="off" spellcheck="false" data-md-component="search-query" required>
<label class="md-search__icon md-icon" for="__search">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M9.5 3A6.5 6.5 0 0 1 16 9.5c0 1.61-.59 3.09-1.56 4.23l.27.27h.79l5 5-1.5 1.5-5-5v-.79l-.27-.27A6.52 6.52 0 0 1 9.5 16 6.5 6.5 0 0 1 3 9.5 6.5 6.5 0 0 1 9.5 3m0 2C7 5 5 7 5 9.5S7 14 9.5 14 14 12 14 9.5 12 5 9.5 5"/></svg>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M20 11v2H8l5.5 5.5-1.42 1.42L4.16 12l7.92-7.92L13.5 5.5 8 11z"/></svg>
</label>
<nav class="md-search__options" aria-label="Search">
<button type="reset" class="md-search__icon md-icon" title="Clear" aria-label="Clear" tabindex="-1">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M19 6.41 17.59 5 12 10.59 6.41 5 5 6.41 10.59 12 5 17.59 6.41 19 12 13.41 17.59 19 19 17.59 13.41 12z"/></svg>
</button>
</nav>
</form>
<div class="md-search__output">
<div class="md-search__scrollwrap" tabindex="0" data-md-scrollfix>
<div class="md-search-result" data-md-component="search-result">
<div class="md-search-result__meta">
Initializing search
</div>
<ol class="md-search-result__list" role="presentation"></ol>
</div>
</div>
</div>
</div>
</div>
<div class="md-header__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
</nav>
</header>
<div class="md-container" data-md-component="container">
<nav class="md-tabs" aria-label="Tabs" data-md-component="tabs">
<div class="md-grid">
<ul class="md-tabs__list">
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-tabs__link">
Home
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-tabs__link">
How We Score Against X-Ray
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-tabs__link">
Findings
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-tabs__link">
Full Experiment Log
</a>
</li>
<li class="md-tabs__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-tabs__link">
Service Conversion (proposal)
</a>
</li>
</ul>
</div>
</nav>
<main class="md-main" data-md-component="main">
<div class="md-main__inner md-grid">
<div class="md-sidebar md-sidebar--primary" data-md-component="sidebar" data-md-type="navigation" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--primary md-nav--lifted" aria-label="Navigation" data-md-level="0">
<label class="md-nav__title" for="__drawer">
<a href="/dtourolle/scene-actor-extraction/." title="scene-actor-extraction" class="md-nav__button md-logo" aria-label="scene-actor-extraction" data-md-component="logo">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M12 8a3 3 0 0 0 3-3 3 3 0 0 0-3-3 3 3 0 0 0-3 3 3 3 0 0 0 3 3m0 3.54C9.64 9.35 6.5 8 3 8v11c3.5 0 6.64 1.35 9 3.54 2.36-2.19 5.5-3.54 9-3.54V8c-3.5 0-6.64 1.35-9 3.54"/></svg>
</a>
scene-actor-extraction
</label>
<div class="md-nav__source">
<a href="https://gitea.tourolle.paris/dtourolle/scene-actor-extraction" title="Go to repository" class="md-source" data-md-component="source">
<div class="md-source__icon md-icon">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 448 512"><!--! Font Awesome Free 7.1.0 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license/free (Icons: CC BY 4.0, Fonts: SIL OFL 1.1, Code: MIT License) Copyright 2025 Fonticons, Inc.--><path d="M439.6 236.1 244 40.5c-5.4-5.5-12.8-8.5-20.4-8.5s-15 3-20.4 8.4L162.5 81l51.5 51.5c27.1-9.1 52.7 16.8 43.4 43.7l49.7 49.7c34.2-11.8 61.2 31 35.5 56.7-26.5 26.5-70.2-2.9-56-37.3L240.3 199v121.9c25.3 12.5 22.3 41.8 9.1 55-6.4 6.4-15.2 10.1-24.3 10.1s-17.8-3.6-24.3-10.1c-17.6-17.6-11.1-46.9 11.2-56v-123c-20.8-8.5-24.6-30.7-18.6-45L142.6 101 8.5 235.1C3 240.6 0 247.9 0 255.5s3 15 8.5 20.4l195.6 195.7c5.4 5.4 12.7 8.4 20.4 8.4s15-3 20.4-8.4l194.7-194.7c5.4-5.4 8.4-12.8 8.4-20.4s-3-15-8.4-20.4"/></svg>
</div>
<div class="md-source__repository">
dtourolle/scene-actor-extraction
</div>
</a>
</div>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/." class="md-nav__link">
<span class="md-ellipsis">
Home
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/methodology/" class="md-nav__link">
<span class="md-ellipsis">
How We Score Against X-Ray
</span>
</a>
</li>
<li class="md-nav__item md-nav__item--nested">
<input class="md-nav__toggle md-toggle " type="checkbox" id="__nav_3" >
<label class="md-nav__link" for="__nav_3" id="__nav_3_label" tabindex="0">
<span class="md-ellipsis">
Findings
</span>
<span class="md-nav__icon md-icon"></span>
</label>
<nav class="md-nav" data-md-level="1" aria-labelledby="__nav_3_label" aria-expanded="false">
<label class="md-nav__title" for="__nav_3">
<span class="md-nav__icon md-icon"></span>
Findings
</label>
<ul class="md-nav__list" data-md-scrollfix>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/best-model/" class="md-nav__link">
<span class="md-ellipsis">
Best Model
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/gallery-scope/" class="md-nav__link">
<span class="md-ellipsis">
Gallery Scope (Full vs. Limited)
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/pose-expansion/" class="md-nav__link">
<span class="md-ellipsis">
Pose Expansion
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/lvface-deep-dive/" class="md-nav__link">
<span class="md-ellipsis">
LVFace Deep Dive
</span>
</a>
</li>
</ul>
</nav>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/model-bakeoff/" class="md-nav__link">
<span class="md-ellipsis">
Full Experiment Log
</span>
</a>
</li>
<li class="md-nav__item">
<a href="/dtourolle/scene-actor-extraction/service-conversion/" class="md-nav__link">
<span class="md-ellipsis">
Service Conversion (proposal)
</span>
</a>
</li>
</ul>
</nav>
</div>
</div>
</div>
<div class="md-sidebar md-sidebar--secondary" data-md-component="sidebar" data-md-type="toc" >
<div class="md-sidebar__scrollwrap">
<div class="md-sidebar__inner">
<nav class="md-nav md-nav--secondary" aria-label="Table of contents">
</nav>
</div>
</div>
</div>
<div class="md-content" data-md-component="content">
<article class="md-content__inner md-typeset">
<h1>404 - Not found</h1>
</article>
</div>
<script>var target=document.getElementById(location.hash.slice(1));target&&target.name&&(target.checked=target.name.startsWith("__tabbed_"))</script>
</div>
<button type="button" class="md-top md-icon" data-md-component="top" hidden>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24"><path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8z"/></svg>
Back to top
</button>
</main>
<footer class="md-footer">
<div class="md-footer-meta md-typeset">
<div class="md-footer-meta__inner md-grid">
<div class="md-copyright">
Made with
<a href="https://squidfunk.github.io/mkdocs-material/" target="_blank" rel="noopener">
Material for MkDocs
</a>
</div>
</div>
</div>
</footer>
</div>
<div class="md-dialog" data-md-component="dialog">
<div class="md-dialog__inner md-typeset"></div>
</div>
<script id="__config" type="application/json">{"annotate": null, "base": "/dtourolle/scene-actor-extraction/", "features": ["navigation.tabs", "navigation.sections", "navigation.top", "navigation.footer", "content.code.copy", "content.code.annotate"], "search": "/dtourolle/scene-actor-extraction/assets/javascripts/workers/search.2c215733.min.js", "tags": null, "translations": {"clipboard.copied": "Copied to clipboard", "clipboard.copy": "Copy to clipboard", "search.result.more.one": "1 more on this page", "search.result.more.other": "# more on this page", "search.result.none": "No matching documents", "search.result.one": "1 matching document", "search.result.other": "# matching documents", "search.result.placeholder": "Type to start searching", "search.result.term.missing": "Missing", "select.version": "Select version"}, "version": null}</script>
<script src="/dtourolle/scene-actor-extraction/assets/javascripts/bundle.d7400e89.min.js"></script>
</body>
</html>
CMakeLists.txt
+396
View File
@@ -0,0 +1,396 @@
cmake_minimum_required(VERSION 3.21)
project(scene_actor_extraction VERSION 0.1.0 LANGUAGES CXX)
set(CMAKE_CXX_STANDARD 20)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
# ── Dependencies ──────────────────────────────────────────────────────────────
# KPN++ (pipeline backbone)
set(KPN_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_PYTHON OFF CACHE BOOL "" FORCE)
set(KPN_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
option(SAE_WEB_DEBUG "Enable KPN web debug UI (localhost:9090)" OFF)
if(SAE_WEB_DEBUG)
set(KPN_WEB_DEBUG ON CACHE BOOL "" FORCE)
endif()
add_subdirectory(external/KPN)
# OpenCV (video decode, image ops, DNN inference, face detection)
# Accept 4 or 5: the APIs used here are stable across both, and distros have
# begun shipping 5.x as the default (Arch/CachyOS). find_package's version
# argument is a minimum, but OpenCV's config rejects a 5.x install when 4 is
# requested, so probe for 5 first and fall back to 4.
find_package(OpenCV 5 QUIET COMPONENTS
core imgproc imgcodecs videoio dnn objdetect highgui)
if(NOT OpenCV_FOUND)
find_package(OpenCV 4 REQUIRED COMPONENTS
core imgproc imgcodecs videoio dnn objdetect highgui)
endif()
message(STATUS "OpenCV: ${OpenCV_VERSION}")
# ── Model paths ───────────────────────────────────────────────────────────────
# Defined early so the backend object libraries below can embed it.
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Backend selection ─────────────────────────────────────────────────────────
# Two independent compile-time axes. The core application is agnostic to both:
# only the matching backend .cpp (in src/backends/) is compiled, and the backend
# headers (onnxruntime / NvInfer.h / cublas / rocblas) never reach core TUs.
#
# SAE_INFERENCE_BACKEND ORT → SCRFD + ArcFace via ONNX Runtime (.onnx models)
# TRT → SCRFD + ArcFace via raw TensorRT (.engine files)
# SAE_GEMM_BACKEND ROCM → gallery similarity GEMM via rocBLAS / HIP
# CUDA → gallery similarity GEMM via cuBLAS / CUDA
# CPU → portable reference GEMM (no GPU; CI / testing)
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "Inference backend: ORT | TRT")
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "Gallery GEMM backend: ROCM | CUDA | CPU")
set_property(CACHE SAE_INFERENCE_BACKEND PROPERTY STRINGS ORT TRT)
set_property(CACHE SAE_GEMM_BACKEND PROPERTY STRINGS ROCM CUDA CPU)
# Enable the ORT TensorRT/CUDA execution providers inside the ORT inference
# backend (only meaningful when ORT was built with the TensorRT EP). Off by
# default so ROCm/CPU builds don't reference unavailable EPs.
option(SAE_ORT_TRT_EP "ORT backend: enable TensorRT/CUDA execution providers" OFF)
# AR-026/AR-027: the CPU GEMM path is backed by OpenBLAS, and its absence is a
# configure error rather than a silent downgrade to the scalar loop. Declared at
# top level because the unit-test target compiles the CPU kernel regardless of
# which backend the main build selected, and both must make the same choice.
option(SAE_ALLOW_SCALAR_GEMM
"Permit the scalar-loop GEMM fallback when OpenBLAS is absent" OFF)
# Back-compat: a legacy -DSAE_WITH_TRT=ON/OFF seeds the new vars (ON⇒TRT+CUDA,
# OFF⇒ORT+ROCM) unless the user set them explicitly.
if(DEFINED SAE_WITH_TRT)
if(SAE_WITH_TRT)
set(SAE_INFERENCE_BACKEND "TRT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "CUDA" CACHE STRING "" FORCE)
else()
set(SAE_INFERENCE_BACKEND "ORT" CACHE STRING "" FORCE)
set(SAE_GEMM_BACKEND "ROCM" CACHE STRING "" FORCE)
endif()
message(STATUS "SAE_WITH_TRT=${SAE_WITH_TRT} (legacy) → "
"SAE_INFERENCE_BACKEND=${SAE_INFERENCE_BACKEND} "
"SAE_GEMM_BACKEND=${SAE_GEMM_BACKEND}")
endif()
if(NOT SAE_INFERENCE_BACKEND MATCHES "^(ORT|TRT)$")
message(FATAL_ERROR "SAE_INFERENCE_BACKEND must be ORT or TRT (got '${SAE_INFERENCE_BACKEND}')")
endif()
if(NOT SAE_GEMM_BACKEND MATCHES "^(ROCM|CUDA|CPU)$")
message(FATAL_ERROR "SAE_GEMM_BACKEND must be ROCM, CUDA or CPU (got '${SAE_GEMM_BACKEND}')")
endif()
# CUDA runtime is needed by both TRT inference and CUDA GEMM — find it once.
function(sae_find_cudart)
if(TARGET cudart_dep)
return()
endif()
find_library(CUDART_LIB cudart
HINTS /opt/cuda/lib64 /usr/local/cuda/lib64 /usr/lib)
find_path(CUDART_INCLUDE cuda_runtime_api.h
HINTS /opt/cuda/targets/x86_64-linux/include /opt/cuda/include
/usr/local/cuda/include /usr/include)
if(NOT (CUDART_LIB AND CUDART_INCLUDE))
message(FATAL_ERROR "CUDA runtime not found (cudart=${CUDART_LIB} headers=${CUDART_INCLUDE}).")
endif()
add_library(cudart_dep INTERFACE)
target_include_directories(cudart_dep INTERFACE "${CUDART_INCLUDE}")
target_link_libraries(cudart_dep INTERFACE "${CUDART_LIB}")
set_property(GLOBAL PROPERTY sae_cudart_found TRUE)
endfunction()
# ── Inference backend dependency: builds the `inference_backend` object lib ────
if(SAE_INFERENCE_BACKEND STREQUAL "ORT")
find_library(ORT_LIB onnxruntime REQUIRED
HINTS /usr/lib64/rocm/lib /usr/lib /usr/local/lib)
find_path(ORT_INCLUDE onnxruntime_cxx_api.h
PATH_SUFFIXES onnxruntime
HINTS /usr/lib64/rocm/include/onnxruntime /usr/include/onnxruntime /usr/local/include/onnxruntime
/usr/lib64/rocm/include /usr/include /usr/local/include
REQUIRED)
# The include directive is <onnxruntime/onnxruntime_cxx_api.h>, so we need the
# parent of the onnxruntime/ subdirectory on the include path.
get_filename_component(ORT_INCLUDE_PARENT "${ORT_INCLUDE}" DIRECTORY)
if(NOT EXISTS "${ORT_INCLUDE_PARENT}/onnxruntime")
set(ORT_INCLUDE_PARENT "${ORT_INCLUDE}")
endif()
message(STATUS "Inference backend: ORT (${ORT_LIB} headers: ${ORT_INCLUDE_PARENT})")
add_library(inference_backend OBJECT src/backends/ort_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${ORT_INCLUDE_PARENT}")
target_link_libraries(inference_backend PRIVATE ${OpenCV_LIBS} "${ORT_LIB}")
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
$<$<BOOL:${SAE_ORT_TRT_EP}>:SAE_ORT_WITH_TRT_EP>)
else() # TRT
find_library(NVINFER_LIB nvinfer
HINTS /usr/lib /usr/local/lib /opt/tensorrt/lib)
find_path(NVINFER_INCLUDE NvInfer.h
HINTS /usr/include /usr/local/include /opt/tensorrt/include)
if(NOT (NVINFER_LIB AND NVINFER_INCLUDE))
message(FATAL_ERROR
"TensorRT not found (nvinfer=${NVINFER_LIB} headers=${NVINFER_INCLUDE}). "
"Pass -DSAE_INFERENCE_BACKEND=ORT to load .onnx models without TensorRT.")
endif()
sae_find_cudart()
message(STATUS "Inference backend: TRT (${NVINFER_LIB})")
add_library(inference_backend OBJECT src/backends/trt_backend.cpp)
set_target_properties(inference_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(inference_backend PRIVATE src "${NVINFER_INCLUDE}")
target_link_libraries(inference_backend PRIVATE
${OpenCV_LIBS} "${NVINFER_LIB}" cudart_dep)
target_compile_definitions(inference_backend PRIVATE
SAE_MODELS_DIR="${SAE_MODELS_DIR}")
endif()
# ── GEMM backend dependency: builds the `gemm_backend` object lib ──────────────
if(SAE_GEMM_BACKEND STREQUAL "CPU")
# Portable reference GEMM: no GPU libraries, no headers. Used for CI and as
# the correctness oracle for the CUDA/ROCm backends.
message(STATUS "GEMM backend: CPU (portable reference, no GPU)")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CPU)
# AR-026/AR-027: the CPU path is backed by OpenBLAS, and that is REQUIRED
# rather than opportunistic. The CPU backend is what CI (no GPU) and the cpu
# builder image actually run, so a silent fall back to the scalar loop means
# AR-027 is measured — or worse, believed — on a path no release uses. A
# missing dependency should stop the build and name itself, not degrade into
# a slower answer nobody notices.
#
# The scalar loop survives as the correctness oracle the two backends are
# diffed against; -DSAE_ALLOW_SCALAR_GEMM=ON is how you ask for it, which
# keeps that an explicit, visible choice.
find_package(PkgConfig QUIET)
if(PkgConfig_FOUND)
pkg_check_modules(OPENBLAS QUIET openblas)
endif()
if(OPENBLAS_FOUND)
message(STATUS "GEMM backend: CPU + OpenBLAS ${OPENBLAS_VERSION}")
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CBLAS)
target_include_directories(gemm_backend PRIVATE ${OPENBLAS_INCLUDE_DIRS})
target_link_libraries(gemm_backend PRIVATE ${OPENBLAS_LINK_LIBRARIES})
elseif(SAE_ALLOW_SCALAR_GEMM)
message(WARNING "GEMM backend: CPU scalar fallback (SAE_ALLOW_SCALAR_GEMM=ON) — "
"correct, but slow on a large gallery. Do not measure AR-027 here.")
else()
message(FATAL_ERROR
"OpenBLAS not found, and the CPU GEMM backend requires it (AR-026/AR-027).\n"
" Install it: Fedora dnf install openblas-devel\n"
" Arch pacman -S openblas\n"
" Debian apt install libopenblas-dev\n"
" Or build the scalar fallback deliberately: -DSAE_ALLOW_SCALAR_GEMM=ON")
endif()
elseif(SAE_GEMM_BACKEND STREQUAL "CUDA")
find_library(CUBLAS_LIB cublas
HINTS /opt/cuda/targets/x86_64-linux/lib /opt/cuda/lib64
/usr/local/cuda/lib64 /usr/lib)
if(NOT CUBLAS_LIB)
message(FATAL_ERROR "cuBLAS not found (cublas=${CUBLAS_LIB}).")
endif()
sae_find_cudart()
message(STATUS "GEMM backend: CUDA (${CUBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src)
target_link_libraries(gemm_backend PRIVATE "${CUBLAS_LIB}" cudart_dep)
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_CUDA)
else() # ROCM
find_library(ROCBLAS_LIB rocblas
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(ROCBLAS_INCLUDE rocblas/rocblas.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
find_library(HIP_LIB amdhip64
HINTS /usr/lib64/rocm/lib /usr/lib64 /usr/local/lib)
find_path(HIP_INCLUDE hip/hip_runtime_api.h
HINTS /usr/lib64/rocm/include /usr/include /usr/local/include)
if(NOT (ROCBLAS_LIB AND ROCBLAS_INCLUDE AND HIP_LIB AND HIP_INCLUDE))
message(FATAL_ERROR
"rocBLAS or HIP runtime not found "
"(rocblas=${ROCBLAS_LIB} headers=${ROCBLAS_INCLUDE} "
"hip=${HIP_LIB} headers=${HIP_INCLUDE}). "
"Install rocblas-devel and hip-devel (or pass -DSAE_GEMM_BACKEND=CUDA).")
endif()
message(STATUS "GEMM backend: ROCM (${ROCBLAS_LIB})")
add_library(gemm_backend OBJECT src/backends/gemm_backend.cpp)
set_target_properties(gemm_backend PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(gemm_backend PRIVATE src "${ROCBLAS_INCLUDE}" "${HIP_INCLUDE}")
target_link_libraries(gemm_backend PRIVATE "${ROCBLAS_LIB}" "${HIP_LIB}")
# HIP headers require the platform to be declared explicitly when compiled with g++.
target_compile_definitions(gemm_backend PRIVATE SAE_GEMM_ROCM __HIP_PLATFORM_AMD__)
endif()
# FFmpeg (hwaccel video decode: CUDA/VAAPI, runtime-detected + swscale colour
# conversion). Hwaccel support is built into libavcodec/libavutil; no extra
# libraries are needed here.
# libswresample is the audio side of the same dependency — downmix + resample
# for the audio signature (IR-004, src/audio_signature.cpp). Not a new project
# dependency: it ships with the libav* set already required above.
find_package(PkgConfig REQUIRED)
pkg_check_modules(AVFORMAT REQUIRED libavformat)
pkg_check_modules(AVCODEC REQUIRED libavcodec)
pkg_check_modules(AVUTIL REQUIRED libavutil)
pkg_check_modules(SWSCALE REQUIRED libswscale)
pkg_check_modules(SWRESAMPLE REQUIRED libswresample)
add_library(ffmpeg_libs INTERFACE)
target_compile_options(ffmpeg_libs INTERFACE
${AVFORMAT_CFLAGS_OTHER} ${AVCODEC_CFLAGS_OTHER}
${AVUTIL_CFLAGS_OTHER} ${SWSCALE_CFLAGS_OTHER}
${SWRESAMPLE_CFLAGS_OTHER})
target_include_directories(ffmpeg_libs INTERFACE
${AVFORMAT_INCLUDE_DIRS} ${AVCODEC_INCLUDE_DIRS}
${AVUTIL_INCLUDE_DIRS} ${SWSCALE_INCLUDE_DIRS}
${SWRESAMPLE_INCLUDE_DIRS})
target_link_libraries(ffmpeg_libs INTERFACE
${AVFORMAT_LIBRARIES} ${AVCODEC_LIBRARIES}
${AVUTIL_LIBRARIES} ${SWSCALE_LIBRARIES}
${SWRESAMPLE_LIBRARIES})
message(STATUS "FFmpeg: avformat=${AVFORMAT_VERSION} avcodec=${AVCODEC_VERSION} "
"swresample=${SWRESAMPLE_VERSION}")
# nlohmann/json (gallery + output serialisation)
include(FetchContent)
FetchContent_Declare(
nlohmann_json
GIT_REPOSITORY https://github.com/nlohmann/json.git
GIT_TAG v3.11.3
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nlohmann_json)
# nanobind (Python bindings for the sae_embed module)
find_package(Python 3.8 COMPONENTS Interpreter Development.Module REQUIRED)
FetchContent_Declare(
nanobind
GIT_REPOSITORY https://github.com/wjakob/nanobind.git
GIT_TAG v2.4.0
GIT_SHALLOW TRUE
)
FetchContent_MakeAvailable(nanobind)
# ── Model paths ───────────────────────────────────────────────────────────────
set(SAE_MODELS_DIR "${CMAKE_SOURCE_DIR}/models"
CACHE PATH "Directory containing ONNX model files")
# ── Shared library: gallery store + compiled-in backends ──────────────────────
# The backend object libraries carry their own ORT/TRT/CUDA/ROCm linkage and
# headers; sae_gallery re-exports those object files so every binary that links
# sae_gallery gets the chosen backend without ever seeing its headers.
# HDF5 (C++) — gallery fast-load path + embedding dump. Found here so sae_gallery
# (gallery_store.cpp) can link it; scene_analyze/dump_embeddings reuse the same vars.
find_package(HDF5 REQUIRED COMPONENTS CXX)
add_library(sae_gallery STATIC
src/gallery/gallery_store.cpp
src/gallery/gallery_builder.cpp
src/audio_signature.cpp # IR-004 — content-derived audio signature
src/gallery/embedder_stamp.cpp # GR-004 — gallery/embedder binding
)
set_target_properties(sae_gallery PROPERTIES POSITION_INDEPENDENT_CODE ON)
target_include_directories(sae_gallery PUBLIC src ${HDF5_INCLUDE_DIRS})
target_link_libraries(sae_gallery PUBLIC
kpn
${OpenCV_LIBS}
nlohmann_json::nlohmann_json
inference_backend
gemm_backend
ffmpeg_libs
${HDF5_CXX_LIBRARIES}
)
target_compile_definitions(sae_gallery PUBLIC
SAE_MODELS_DIR="${SAE_MODELS_DIR}"
)
# ── embed_faces — image → embedding JSON (used by gallery builder scripts) ────
add_executable(embed_faces src/embed_faces.cpp)
target_link_libraries(embed_faces PRIVATE sae_gallery)
# ── sae_embed — Python module: load SCRFD+ArcFace once, embed many images ───
nanobind_add_module(sae_embed src/python_bindings.cpp)
target_link_libraries(sae_embed PRIVATE sae_gallery)
# ── sae_kpn — Python module: run the real downstream nodes over dumped embeddings ─
# Assembles face_tracker/identity_matcher/frame_annotation in a Python-driven KPN
# network (KPN_BUILD_PYTHON is enabled per-TU inside the .cpp). Powers the
# threshold-sweep optimizer in scripts/optimizer/.
#
# TRACES: VR-011 | PR-002
# ON again. It was OFF for one commit because it had not compiled since the
# AR-007/AR-008 tracker redesign -- the binding built FaceTrackerFunc from a
# Config alone, and the tracker had required a registry and a calibration since.
# VR-011 replaced the three per-node factories with one `add_pipeline` that
# builds the chain in main.cpp's order, which is the only order that satisfies
# those dependencies, so the failure mode cannot recur from Python.
option(SAE_BUILD_KPN_BINDINGS "Build the sae_kpn Python module" ON)
if(SAE_BUILD_KPN_BINDINGS)
nanobind_add_module(sae_kpn src/kpn_bindings.cpp)
target_link_libraries(sae_kpn PRIVATE sae_gallery)
endif()
# ── sae_audio — Python module: the v1 audio signature (IR-004) ────────────────
# Compiles audio_signature.cpp directly and links only FFmpeg, rather than
# linking sae_gallery: the signature needs no model, no OpenCV and no HDF5, and
# a module that dragged all three in would make `import sae_audio` depend on a
# GPU-capable build of a repo whose audio path is pure CPU DSP. tests/ compiles
# the same source the same way, for the same reason.
nanobind_add_module(sae_audio src/audio_bindings.cpp src/audio_signature.cpp)
target_include_directories(sae_audio PRIVATE src)
target_link_libraries(sae_audio PRIVATE ffmpeg_libs)
# HDF5 already found above (before sae_gallery); vars HDF5_CXX_LIBRARIES / _INCLUDE_DIRS
# are reused by scene_analyze / dump_embeddings below.
# ── analyze — main analysis binary ───────────────────────────────────────────
add_executable(scene_analyze src/main.cpp)
target_link_libraries(scene_analyze PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze PRIVATE ${HDF5_INCLUDE_DIRS})
# ── analyze_debug — same binary with debug frame/crop output ─────────────────
add_executable(scene_analyze_debug src/main.cpp)
target_link_libraries(scene_analyze_debug PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(scene_analyze_debug PRIVATE ${HDF5_INCLUDE_DIRS})
target_compile_definitions(scene_analyze_debug PRIVATE SAE_DEBUG=1)
# ── dump_embeddings — standalone embedding dumper, NO gallery/matcher ─────────
# Front-half only (decode→detect→align→embed→HDF5) for the optimizer replay corpus
# and model bake-off. Skips gallery load + calibration (~24s/run faster).
add_executable(dump_embeddings src/dump_embeddings.cpp)
target_link_libraries(dump_embeddings PRIVATE sae_gallery ${HDF5_CXX_LIBRARIES})
target_include_directories(dump_embeddings PRIVATE ${HDF5_INCLUDE_DIRS})
# ── scene_preview — live annotated display while analysing ───────────────────
add_executable(scene_preview src/scene_preview.cpp)
target_link_libraries(scene_preview PRIVATE sae_gallery)
# ── build_gallery — offline gallery construction tool ────────────────────────
add_executable(build_gallery src/build_gallery.cpp)
target_link_libraries(build_gallery PRIVATE sae_gallery)
# ── Optional: web debug UI for pipeline introspection ────────────────────────
if(SAE_WEB_DEBUG)
kpn_target_enable_web_debug(scene_analyze)
kpn_target_enable_web_debug(scene_analyze_debug)
kpn_target_enable_web_debug(scene_preview)
endif()
# ── Tests ─────────────────────────────────────────────────────────────────────
option(SAE_BUILD_TESTS "Build unit tests (GPU-free)" OFF)
if(SAE_BUILD_TESTS)
enable_testing()
add_subdirectory(tests)
endif()
message(STATUS "OpenCV ${OpenCV_VERSION} found")
message(STATUS "Models dir: ${SAE_MODELS_DIR}")
Dockerfile.builder-cpu
+342
View File
@@ -0,0 +1,342 @@
# sae-builder-cpu — the CI build image
#
# TRACES: DP-007 | PR-004
#
# Build/push: scripts/ci/build_builder_image.sh --push
# Consumed by: .gitea/workflows/unit-tests.yml (pinned by tag, never :latest)
# Docs: docs/ci-image.md
#
# This is the CPU corner of the DP-008 builder matrix and the DP-007 CI image at
# the same time — one artifact, two uses. The CUDA and ROCm siblings differ only
# in the accelerator stack layered on top of this dependency set.
#
# CI runs on an Intel N100 with no discrete GPU. Everything here is chosen so
# that `-DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=CPU -DSAE_BUILD_TESTS=ON`
# configures, builds and runs without a GPU, without a model, and without
# reaching GitHub.
# ─── Base image ──────────────────────────────────────────────────────────────
#
# Chosen for the OLDEST glibc to be supported, not for recency. A binary built
# in a container runs against the *host's* glibc; glibc is backward compatible
# but not forward, so the build base sets the floor for every machine DP-008's
# binaries can ever run on. Building on a newer base than the oldest supported
# host produces the classic `GLIBC_2.xx not found` failure at load time.
#
# Debian 12 "bookworm" = glibc 2.36 (Aug 2022). What that floor covers:
#
# Distro glibc Covered?
# Arch / CachyOS (rolling) 2.41+ yes
# Fedora 37 and later 2.36+ yes ← DP-005's targets are Fedora+Arch
# Debian 12 / 13 2.36+ yes
# Ubuntu 24.04 LTS 2.39 yes
# Ubuntu 22.04 LTS 2.35 NO
# RHEL / Rocky / Alma 9 2.34 NO
# Debian 11 2.31 NO
#
# The three misses are accepted deliberately: DP-005 puts Debian/Ubuntu out of
# installer scope and names Fedora + Arch as the supported distros, and every
# supported Fedora is 2.36 or newer. Going lower costs the toolchain rather than
# buying reach — Debian 11 ships GCC 10 (incomplete C++20) and Python 3.9, which
# has no `tomllib` and therefore cannot read the traceability gate's
# traceability.toml.
#
# Escape hatch, recorded now so it is not rediscovered under pressure: if the
# floor must drop to glibc 2.28 (RHEL 8 / manylinux_2_28 — the same baseline the
# ONNX Runtime and PyTorch wheels target), the move is a Rocky 8 base plus
# gcc-toolset-13, and OpenCV/FFmpeg/HDF5 all leave apt for source or
# EPEL/RPM Fusion. That is a different image, not a flag on this one.
#
# Not a glibc problem but worth stating: the binaries this image produces also
# link OpenCV, FFmpeg and HDF5 shared objects by soname. Making a *portable*
# release binary (DP-008) is a separate question from the glibc floor, and is
# answered by static linking or bundling, not by the base image.
FROM debian:12-slim
# Pins. Every version this image installs from source is an ARG so a rebuild is
# a one-line diff and `docker history` records what a given tag actually holds.
#
# ORT 1.28.0 and OpenCV 5.0.0 match the developer machine, so CI and local
# builds exercise the same libraries rather than merely similar ones.
# Catch2 / nlohmann_json / nanobind match the FetchContent pins in
# CMakeLists.txt:248 and tests/CMakeLists.txt:12 exactly — a vendored copy at a
# different version would be a silent divergence, not a convenience.
ARG ORT_VERSION=1.28.0
ARG OPENCV_VERSION=5.0.0
ARG CATCH2_VERSION=v3.5.3
ARG NLOHMANN_JSON_VERSION=v3.11.3
ARG NANOBIND_VERSION=v2.4.0
# Stamped so a build can prove which image it ran in, and so a green tick can be
# traced back to a specific dependency set. See the "Confirm the builder image"
# step in .gitea/workflows/unit-tests.yml.
ARG IMAGE_TAG=dev
ENV SAE_BUILDER=cpu \
SAE_BUILDER_VERSION=${IMAGE_TAG} \
SAE_ORT_VERSION=${ORT_VERSION} \
SAE_OPENCV_VERSION=${OPENCV_VERSION} \
DEBIAN_FRONTEND=noninteractive
# ─── System dependencies ─────────────────────────────────────────────────────
#
# One layer, ordered by why it is here rather than alphabetically.
RUN apt-get update && apt-get install -y --no-install-recommends \
# Toolchain. bookworm's default gcc is 12.2 — enough for the C++20 the
# project sets unconditionally (CMakeLists.txt:4). cmake is 3.25, above the
# 3.21 minimum. Ninja because the N100 has four cores and every second of
# build scheduling shows.
build-essential \
cmake \
ninja-build \
pkg-config \
git \
ca-certificates \
curl \
# Gitea's act_runner executes JS actions (actions/checkout, upload-artifact)
# with the `node` found *inside* the container. Without this the job cannot
# even check the repository out. Same reason as the kpnpp-builder image.
nodejs \
# HDF5 with the C++ API: galleries are HDF5-native and it is also the VR-001
# dump format. find_package(HDF5 COMPONENTS CXX) at CMakeLists.txt:273.
libhdf5-dev \
# FFmpeg decode. swresample is on this list deliberately: the audio
# signature (IR-004) downmixes to mono and resamples to 11025 Hz, and
# tests/test_audio_signature.cpp decodes the golden FLAC fixture, so the
# test build needs it as much as the main build does.
libavformat-dev \
libavcodec-dev \
libavutil-dev \
libswscale-dev \
libswresample-dev \
# OpenBLAS — required here, not optional. CI has no GPU, so SAE_GEMM_BACKEND
# =CPU is the only path it ever exercises, and without OpenBLAS the CPU GEMM
# falls back to a scalar loop that does not scale against a library-sized
# gallery (AR-027). The build only *warns* when it is missing so a developer
# without it still gets a working tree; the image must never be that case.
# Both the main build (CMakeLists.txt:162) and the test target
# (tests/CMakeLists.txt:42) discover it through pkg-config `openblas`.
libopenblas-dev \
# Python: the build itself needs the interpreter and headers
# (find_package(Python COMPONENTS Interpreter Development.Module) at
# CMakeLists.txt:254, for the nanobind modules). numpy/h5py/scipy are for
# the Python-side tooling — fixture generation, replay, validation scripts.
# From apt rather than pip: bookworm marks the environment externally
# managed (PEP 668), and apt's h5py is already linked against the same
# libhdf5 installed above. bookworm's python3 is 3.11, which has tomllib —
# the traceability gate needs it to read traceability.toml.
python3 \
python3-dev \
python3-numpy \
python3-h5py \
python3-scipy \
# Image codecs for the OpenCV build below. Without these OpenCV silently
# builds an imgcodecs that cannot read a JPEG, which fails at run time in a
# gallery build rather than at compile time here.
libjpeg62-turbo-dev \
libpng-dev \
libtiff-dev \
libwebp-dev \
libopenjp2-7-dev \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*
# Fail the image build, not the CI run, if OpenBLAS or swresample are not
# discoverable the way CMakeLists.txt discovers them. An image that ships
# libopenblas but no openblas.pc would compile the scalar fallback in silence.
RUN set -eux; \
pkg-config --exists openblas; \
echo "openblas $(pkg-config --modversion openblas)"; \
pkg-config --exists libswresample; \
echo "swresample $(pkg-config --modversion libswresample)"
# ─── ONNX Runtime, CPU provider only ─────────────────────────────────────────
#
# The official prebuilt linux-x64 tarball is the CPU build: no CUDA, no
# TensorRT, no ROCm execution providers. That is the whole requirement here —
# excluding the GPU providers is not a size optimisation, it is the point.
#
# Verified against the 1.28.0 tarball: the shared object's highest versioned
# symbol requirement is GLIBC_2.27 / GLIBCXX_3.4.21, well under this base's
# 2.36, so ORT does not raise the floor set above.
#
# Installed to /usr/local/{lib,include/onnxruntime} because CMakeLists.txt
# includes <onnxruntime/onnxruntime_cxx_api.h> and needs the *parent* of that
# directory on the include path (CMakeLists.txt:108-113).
#
# CI never calls a model — the embedder measures ~930 ms/frame on this CPU
# provider — so ORT is present to satisfy the link, not to run inference.
RUN set -eux; \
curl -fsSL -o /tmp/ort.tgz \
"https://github.com/microsoft/onnxruntime/releases/download/v${ORT_VERSION}/onnxruntime-linux-x64-${ORT_VERSION}.tgz"; \
mkdir -p /tmp/ort; \
tar -xzf /tmp/ort.tgz -C /tmp/ort --strip-components=1; \
cp -a /tmp/ort/lib/libonnxruntime.so* /usr/local/lib/; \
mkdir -p /usr/local/include/onnxruntime; \
cp -a /tmp/ort/include/. /usr/local/include/onnxruntime/; \
ldconfig; \
rm -rf /tmp/ort /tmp/ort.tgz; \
test -f /usr/local/include/onnxruntime/onnxruntime_cxx_api.h
# ─── OpenCV 5, from source ───────────────────────────────────────────────────
#
# This is the reason the image is prebuilt at all. CMakeLists.txt:25 probes for
# OpenCV 5 first and falls back to 4; the branch targets 5, which no Debian
# release ships (bookworm has 4.6), and building it inside every CI run would
# dominate the run on an N100.
#
# BUILD_LIST is exactly the seven components find_package asks for
# (CMakeLists.txt:25-29) — OpenCV resolves their internal dependencies itself.
# Everything else is off: tests, samples, Java/Python bindings, and the apps.
#
# No GUI backend. highgui still builds (find_package REQUIREs the component) but
# with a stub — CI never calls imshow, and pulling GTK/Qt into a headless build
# image buys nothing. scene_preview is a developer tool, not a CI target.
#
# CUDA/cuDNN explicitly off: DP-007 excludes the GPU stack outright.
#
# The source tree and build tree are removed in the same layer, so the ~3 GB of
# intermediates cost nothing in the published image.
RUN set -eux; \
curl -fsSL -o /tmp/opencv.tar.gz \
"https://github.com/opencv/opencv/archive/refs/tags/${OPENCV_VERSION}.tar.gz"; \
mkdir -p /tmp/opencv-src; \
tar -xzf /tmp/opencv.tar.gz -C /tmp/opencv-src --strip-components=1; \
cmake -S /tmp/opencv-src -B /tmp/opencv-build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DBUILD_LIST=core,imgproc,imgcodecs,videoio,dnn,objdetect,highgui \
-DBUILD_SHARED_LIBS=ON \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_DOCS=OFF \
-DBUILD_opencv_apps=OFF \
-DBUILD_JAVA=OFF \
-DBUILD_opencv_python3=OFF \
-DWITH_FFMPEG=ON \
-DWITH_GTK=OFF \
-DWITH_QT=OFF \
-DWITH_OPENGL=OFF \
-DWITH_CUDA=OFF \
-DWITH_CUDNN=OFF \
-DOPENCV_GENERATE_PKGCONFIG=ON \
-DCMAKE_INSTALL_RPATH=/usr/local/lib; \
cmake --build /tmp/opencv-build --parallel; \
cmake --install /tmp/opencv-build; \
ldconfig; \
rm -rf /tmp/opencv-src /tmp/opencv-build /tmp/opencv.tar.gz
# ─── Vendored dependencies: Catch2, nlohmann/json, nanobind ──────────────────
#
# All three are FetchContent'ed by the build today, which makes every CI run
# depend on GitHub being reachable — a network outage would present as a code
# failure. Baking them in removes that dependency entirely.
#
# Catch2 is *installed*, so tests/CMakeLists.txt:6 `find_package(Catch2 3 QUIET)`
# succeeds and the FetchContent fallback is never reached. Its source is kept as
# well so the override below can cover the case where find_package somehow does
# not fire.
#
# nanobind must be cloned with submodules: its `ext/robin_map` is a git
# submodule, and a GitHub source tarball does not contain it. This is the one
# dependency where "download the tarball" produces a tree that configures and
# then fails to compile.
RUN set -eux; \
mkdir -p /opt/vendor; \
git clone --depth 1 --branch "${NLOHMANN_JSON_VERSION}" \
https://github.com/nlohmann/json.git /opt/vendor/nlohmann_json; \
git clone --depth 1 --branch "${NANOBIND_VERSION}" --recurse-submodules \
https://github.com/wjakob/nanobind.git /opt/vendor/nanobind; \
git clone --depth 1 --branch "${CATCH2_VERSION}" \
https://github.com/catchorg/Catch2.git /opt/vendor/Catch2; \
cmake -S /opt/vendor/Catch2 -B /tmp/catch2-build -G Ninja \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_INSTALL_PREFIX=/usr/local \
-DBUILD_TESTING=OFF; \
cmake --build /tmp/catch2-build --parallel; \
cmake --install /tmp/catch2-build; \
rm -rf /tmp/catch2-build; \
find /opt/vendor -maxdepth 2 -name .git -exec rm -rf {} +; \
ldconfig
# The initial-cache script the build is configured with. It lives in the image,
# not in the workflow, so the vendor paths have exactly one owner: move a
# directory here and no consumer needs editing.
#
# FETCHCONTENT_FULLY_DISCONNECTED=ON is the load-bearing line. With it, any
# FetchContent dependency that is *not* covered by an override above is a hard
# configure error instead of a silent download — so "this build does not touch
# GitHub" is enforced by the build system rather than asserted in a comment.
RUN set -eux; \
printf '%s\n' \
'# Baked into sae-builder-cpu. Use with: cmake -C /opt/vendor/vendored-deps.cmake ...' \
'# TRACES: DP-007' \
'set(FETCHCONTENT_SOURCE_DIR_NLOHMANN_JSON "/opt/vendor/nlohmann_json" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_SOURCE_DIR_NANOBIND "/opt/vendor/nanobind" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_SOURCE_DIR_CATCH2 "/opt/vendor/Catch2" CACHE PATH "vendored in the CI image")' \
'set(FETCHCONTENT_FULLY_DISCONNECTED ON CACHE BOOL "no CI build may fetch from the network")' \
> /opt/vendor/vendored-deps.cmake; \
cat /opt/vendor/vendored-deps.cmake
# ─── Self-check ──────────────────────────────────────────────────────────────
#
# Run the project's own dependency discovery — the same find_package and
# pkg_check_modules calls CMakeLists.txt makes — against this image, at image
# build time. An image that cannot satisfy them should fail here, loudly, once,
# rather than in every CI run that pulls it.
#
# Deliberately not a build of the project: the image must be buildable without
# the repository, and the repository's own configure step is what CI is for.
RUN set -eux; \
mkdir -p /tmp/selfcheck; \
printf '%s\n' \
'cmake_minimum_required(VERSION 3.21)' \
'project(sae_image_selfcheck LANGUAGES CXX)' \
'set(CMAKE_CXX_STANDARD 20)' \
'set(CMAKE_CXX_STANDARD_REQUIRED ON)' \
'find_package(OpenCV 5 REQUIRED COMPONENTS core imgproc imgcodecs videoio dnn objdetect highgui)' \
'message(STATUS "OpenCV ${OpenCV_VERSION}")' \
'find_package(HDF5 REQUIRED COMPONENTS CXX)' \
'message(STATUS "HDF5 ${HDF5_VERSION}")' \
'find_package(Catch2 3 REQUIRED)' \
'message(STATUS "Catch2 ${Catch2_VERSION}")' \
'find_package(Python 3.8 REQUIRED COMPONENTS Interpreter Development.Module)' \
'find_package(PkgConfig REQUIRED)' \
'pkg_check_modules(AVFORMAT REQUIRED libavformat)' \
'pkg_check_modules(AVCODEC REQUIRED libavcodec)' \
'pkg_check_modules(AVUTIL REQUIRED libavutil)' \
'pkg_check_modules(SWSCALE REQUIRED libswscale)' \
'pkg_check_modules(SWRESAMPLE REQUIRED libswresample)' \
'pkg_check_modules(OPENBLAS REQUIRED openblas)' \
'find_library(ORT_LIB onnxruntime REQUIRED HINTS /usr/lib /usr/local/lib)' \
'find_path(ORT_INCLUDE onnxruntime_cxx_api.h PATH_SUFFIXES onnxruntime' \
' HINTS /usr/include/onnxruntime /usr/local/include/onnxruntime /usr/local/include REQUIRED)' \
'message(STATUS "ORT ${ORT_LIB} / ${ORT_INCLUDE}")' \
> /tmp/selfcheck/CMakeLists.txt; \
cmake -S /tmp/selfcheck -B /tmp/selfcheck/build -G Ninja; \
rm -rf /tmp/selfcheck
# Python-side tooling the fixture and validation scripts import. Checked here so
# a missing wheel is an image failure rather than a mid-run traceback.
RUN python3 -c "import numpy, h5py, scipy; print('numpy', numpy.__version__, 'h5py', h5py.__version__, 'scipy', scipy.__version__)"
# ─── What is deliberately NOT here ───────────────────────────────────────────
#
# CUDA, TensorRT, ROCm, and the ORT GPU execution providers
# No GPU to use them. They belong to the sae-builder-cuda and
# sae-builder-rocm siblings (DP-008).
#
# The ONNX models
# Seven files, ~725 MB, in Git LFS. T1/T2 tests are model-free by design
# (tests/CMakeLists.txt:1-4), so the CI image needs none of them, and
# baking them in would inflate the image roughly tenfold to serve the T3
# smoke tests alone. Those pull the model they need via LFS in a separate
# job. The CI workflow checks out with LFS off for the same reason.
#
# The repository
# Nothing from the source tree is COPYed in. The image is a toolchain, and
# a toolchain that embeds the code it builds has to be rebuilt whenever the
# code changes — which is exactly the per-run cost this image exists to
# avoid.
WORKDIR /src
LICENSE
+41
View File
@@ -0,0 +1,41 @@
MIT License
Copyright (c) 2026 Duncan Tourolle
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
---
NOTE ON MODELS AND THIRD-PARTY COMPONENTS
The MIT license above applies only to the source code in this repository. It
does NOT cover:
* Machine-learning model weights (the ONNX files in models/). These weights
are the property of their respective authors and are governed by their own
licenses, not by the MIT license above. The models — including the
InsightFace "buffalo" packs (ArcFace / SCRFD), YuNet, and LVFace — are
redistributed here for convenience under those upstream licenses. Several,
notably the InsightFace models, are licensed for NON-COMMERCIAL RESEARCH
USE ONLY. You are responsible for reviewing and complying with each model's
license before use.
* Third-party libraries this software links against (OpenCV, ONNX Runtime,
FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind, and others), each of which
carries its own license.
README.md
+259
View File
@@ -0,0 +1,259 @@
# Scene Actor Extraction
Identifies actors in movie files and produces X-ray-style scene annotations compatible with [Jellyfin](https://jellyfin.org/). Built on a KPN++ pipeline with ArcFace/LVFace embeddings and a tracked-identity matcher.
**67.4% macro-F1 against Amazon X-Ray ground truth**, on 5 films never seen by
the optimizer (89.7% P / 65.4% R training-set; see the generalization-gap
discussion in the [deep dive](https://pages.tourolle.paris/dtourolle/scene-actor-extraction/lvface-deep-dive/)).
Full benchmark write-up, model comparison, and failure-mode analysis:
**https://pages.tourolle.paris/dtourolle/scene-actor-extraction/**
![A perfect X-Ray second on a held-out film](docs/assets/images/lovelace_perfect_second.jpg)
*A perfect X-Ray second on a held-out film (never used for threshold tuning):
every visible face named at 100%, the background extra honestly left unnamed,
and the two credited cast without a visible face correctly carried as present
off-screen. Bottom panels show the per-second verdict against Amazon X-Ray
(green = correct, orange = wrong, blue = missed).*
It also doesn't care whether the face is in the room:
![Herbie Hancock identified on an in-fiction video-call screen](docs/assets/images/valerian_screen_call.jpg)
*Herbie Hancock at 98% — as a face on a screen inside the movie, under a
sci-fi HUD overlay.*
## How it works
1. **Build a gallery** — download actor headshots from TMDB/IMDB, embed them with ArcFace or LVFace (`build_gallery` / `scripts/make_gallery.py`).
2. **Analyze a movie** — `scene_analyze` decodes frames at configurable FPS, detects faces (SCRFD), tracks them across cuts, matches identities against the gallery using calibrated similarity, and writes time-window JSON.
3. **Output** — minimal mode produces Jellyfin-ready actor name + time-window JSON; standard mode adds per-frame bbox, similarity, and track data.
![Pipeline topology](docs/assets/images/pipeline_topology.svg)
## Dependencies
| Dependency | Role |
|---|---|
| KPN++ | Pipeline backbone (nodes, networks) |
| OpenCV 4 | Video decode, image ops, DNN inference |
| ONNX Runtime | SCRFD face detector (dynamic shape nodes unsupported by cv::dnn) |
| TensorRT + CUDA runtime + cuBLAS | Optional TRT engines for SCRFD/ArcFace (`--detector-engine`/`--arcface-engine`); identity_matcher's GPU gallery scan |
| FFmpeg (libav*) | NVDEC hardware video decode + colour conversion |
| nlohmann/json | JSON I/O |
| nanobind | Python bindings for `sae_embed` |
## Build
```bash
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
```
This also builds `sae_embed`, a Python module (via nanobind) that loads the
SCRFD detector and ArcFace embedder once and exposes a reusable `embed()`
method. The gallery-builder scripts (`make_gallery.py`,
`make_jellyfin_gallery.py`, `movienet_eval.py`) import it directly — there is
no subprocess fallback, so if it's missing they exit with a build instruction:
```bash
cmake --build build --target sae_embed
```
Optional flags:
| Flag | Default | Effect |
|---|---|---|
| `-DSAE_WEB_DEBUG=ON` | OFF | Enables KPN web debug UI at `localhost:9090` |
## Models
The ONNX model weights live in `models/` (tracked via Git LFS):
- `LVFace-B_Glint360K.onnx` — LVFace embedder (ViT backbone, ICCV 2025), the default
(best F1 in the rep4 model bake-off, see `docs/rep4-optimizer-results.md`)
- `arcface_w600k_r50.onnx` — ArcFace embedder, previous default
- `arcface_w600k_mbf.onnx`, `arcface_r18.onnx` — lighter ArcFace alternatives
- `face_detection_yunet_2023mar.onnx` — YuNet face detector
- `scrfd_500m_bnkps.onnx` — SCRFD face detector
### LVFace
[LVFace](https://github.com/bytedance/LVFace) is a Vision-Transformer face
recognition model. The `LVFace-B_Glint360K.onnx` export shares ArcFace's I/O
contract (112×112 aligned BGR crop → L2-normalised 512-d embedding) and its
`(x − 127.5)/128` input scaling, so it slots straight into the existing embedder
— just point `--arcface-model` at it:
```bash
./build/scene_analyze --arcface-model models/LVFace-B_Glint360K.onnx \
--gallery gallery.h5 --movie movie.mp4
```
> **Important:** embeddings from different recognition models are not
> interchangeable. A gallery (and its calibration cache) must be built with the
> **same** embedder used for analysis — rebuild the gallery with
> `--arcface models/LVFace-B_Glint360K.onnx` before analysing with LVFace.
If they are missing (e.g. LFS not fetched), re-download them with:
```bash
bash scripts/download_models.sh
```
> **Model licensing:** the model weights carry their own licenses, separate
> from this project's MIT license, and are redistributed here under those
> upstream terms. Several — notably the InsightFace "buffalo" models (ArcFace /
> SCRFD) — are licensed for **non-commercial research use only**. Review and
> comply with each model's license before use.
## Binaries
| Binary | Description |
|---|---|
| `scene_analyze` | Main analysis pipeline, writes JSON output |
| `scene_analyze_debug` | Same as above + per-frame annotated JPEGs (`SAE_DEBUG=1`) |
| `scene_preview` | Live OpenCV display window while analysing |
| `build_gallery` | Offline gallery builder from a directory of images |
| `embed_faces` | CLI: image(s) → embedding JSON, used by gallery-builder scripts |
| `sae_embed` | Python module (nanobind) used by gallery-builder scripts — loads SCRFD+ArcFace once |
### `scene_analyze`
```bash
./build/scene_analyze --gallery gallery.h5 --movie movie.mp4 [options]
```
Key options:
| Flag | Default | Description |
|---|---|---|
| `--fps` | 1 | Frames per second to sample (5–10 recommended for tracking) |
| `--prob-threshold` | 0.5 | Minimum calibrated match probability |
| `--match-threshold` | — | Raw cosine similarity threshold (fallback) |
| `--extinction` | 5s | How long a track persists after last detection |
| `--track-alpha` | — | IoU vs. embedding weight in Hungarian assignment |
| `--track-min-iou` | — | Minimum IoU gate for spatial assignment |
| `--track-max-embed` | — | Maximum embedding distance gate |
| `--track-max-missing` | — | Frames a track survives without a detection |
### Gallery builders
**Per-movie (TMDB):**
```bash
python3 scripts/make_gallery.py --tmdb-key <TMDB_KEY> --movie-id <TMDB_ID> --output gallery.h5
```
Fetches cast images from TMDB and embeds them via `sae_embed`.
**Whole-library (Jellyfin):**
```bash
python3 scripts/make_jellyfin_gallery.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--output gallery.h5
```
Scans every Movie/Series in Jellyfin, collects the unique cast across the
whole library, downloads each actor's headshot directly from Jellyfin (no
TMDB key needed), and embeds them via `sae_embed` into one global
gallery.h5. Since `identity_matcher` scores faces against the entire
gallery, `scene_analyze` can then recognise any actor from your library in
any film — not just the cast listed for that one title. Pass `--merge` on
later runs to only embed actors newly added to the library. Pass
`--tmdb-key` to fall back to TMDB profile images for actors with no usable
image cached in Jellyfin.
Jellyfin/TMDB lookups and image downloads for different actors run
concurrently (`--workers`, default 8). Embedding is GPU-bound, so it's
gated separately via `--embed-concurrency` (default 1) — only that many
embed calls run at once while other actors' downloads continue in the
background.
To restrict a single-title run to that title's credited cast (faster, fewer
look-alike mismatches), filter the global gallery first:
```bash
python3 scripts/filter_gallery.py \
--gallery gallery.h5 \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--output gallery_matrix.h5
```
## Running directly from Jellyfin
`scripts/run_from_jellyfin.py` resolves a title to its media file via the
Jellyfin API, filters the gallery to that title's cast, and runs
`scene_analyze` in one step. Requires this tool to run on a host that shares
Jellyfin's media mount (it uses the item's on-disk `Path`, not a stream URL):
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <API_KEY> \
--title "The Matrix" \
--gallery gallery.h5 \
-- --fps 5 --verbosity 2
```
Anything after `--` is passed through to `scene_analyze` unchanged. Pass
`--no-filter` to use the gallery as-is (skip per-title cast filtering), or
`--item-id` instead of `--title` to skip the search.
After a successful run, the output JSON is pushed to the [JRay Jellyfin
plugin](https://gitea.tourolle.paris/dtourolle/jRay)'s Truth endpoint
(`PUT /Plugins/JRay/Items/{itemId}/Truth`) so
Jellyfin picks it up immediately, using `--api-key` (must be an
**Administrator** key for the push to succeed). Pass `--no-push` to skip
this and only write `--output` locally (e.g. for local debugging).
### Worker mode
Pass `--worker` instead of `--item-id`/`--title` to run this as an extraction
worker: it polls the JRay plugin's `GET /Plugins/JRay/Tasks/Pending` endpoint
for a random batch of items with no truth data yet, processes each one, and
pushes the result back. The endpoint's sampling spreads work across the
backlog without any server-side task tracking, so any number of workers can
poll the same library concurrently.
```bash
python3 scripts/run_from_jellyfin.py \
--jellyfin-url http://jellyfin.local:8096 \
--api-key <ADMIN_API_KEY> \
--gallery whole_gallery.h5 \
--worker \
-- --fps 5
```
- `--poll-limit` — batch size requested from `Tasks/Pending` (default 10, max 100)
- `--poll-interval` — seconds to sleep between polls when the backlog is empty (default 60)
- `--once` — process a single batch and exit instead of looping forever
A failure on one item (bad path, push rejected, etc.) is logged and the
worker moves on to the next item rather than exiting.
## Output format
**Minimal** (default) — Jellyfin-ready:
```json
[
{ "actor": "Name", "start": 12.0, "end": 45.5 }
]
```
**Standard** — per-frame detail with bounding boxes, similarity scores, and track IDs.
## Evaluation
Scripts in `eval/` and `scripts/movienet_*.py` support benchmarking against the MovieNet dataset.
## License
The source code in this repository is licensed under the [MIT License](LICENSE).
The MIT license covers **only the code**. The model weights in `models/` (see
[Models](#models)) are redistributed under their own licenses — several for
non-commercial research use only. Third-party libraries this software links
against (OpenCV, ONNX Runtime, FFmpeg, TensorRT/CUDA, nlohmann/json, nanobind,
and others) likewise carry their own licenses.
assets/images/cafe_society_rapid_cut.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 200 KiB

assets/images/calibration_curves.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 94 KiB

assets/images/de_search_landscape.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 137 KiB

assets/images/downton_crew_fn.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 219 KiB

assets/images/downton_funeral_19of20.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 164 KiB

assets/images/downton_ghost_timeline.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 69 KiB

assets/images/downton_wedding_couple.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 201 KiB

assets/images/favicon.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 1.8 KiB

assets/images/germar_beats_xray.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 311 KiB

assets/images/holdout_f1_by_film.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 83 KiB

assets/images/lord_of_war_fpi_reagan_photo.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 155 KiB

assets/images/lord_of_war_fpi_reddick.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 176 KiB

assets/images/lord_of_war_fpi_shumbris.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 161 KiB

assets/images/lovelace_fpi_sevigny.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 142 KiB

assets/images/lovelace_polygraph_bridged.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 234 KiB

assets/images/lovelace_robert_patrick_fpi.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 163 KiB

assets/images/many_saints_fpi_balsam.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 174 KiB

assets/images/many_saints_fpi_deschanel.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 124 KiB

assets/images/many_saints_fpi_gardner.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 111 KiB

assets/images/many_saints_fpi_yates.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 134 KiB

assets/images/many_saints_outofcast_fpi.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 141 KiB

assets/images/rep4_matrix_f1.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 103 KiB

assets/images/scarface_fpi_alley.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 40 KiB

assets/javascripts/bundle.d7400e89.min.js
Vendored
-16
View File
File diff suppressed because one or more lines are too long
assets/javascripts/bundle.d7400e89.min.js.map
-7
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.ar.min.js
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.da.min.js
Vendored
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Danish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.da=function(){this.pipeline.reset(),this.pipeline.add(e.da.trimmer,e.da.stopWordFilter,e.da.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.da.stemmer))},e.da.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.da.trimmer=e.trimmerSupport.generateTrimmer(e.da.wordCharacters),e.Pipeline.registerFunction(e.da.trimmer,"trimmer-da"),e.da.stemmer=function(){var r=e.stemmerSupport.Among,i=e.stemmerSupport.SnowballProgram,n=new function(){function e(){var e,r=f.cursor+3;if(d=f.limit,0<=r&&r<=f.limit){for(a=r;;){if(e=f.cursor,f.in_grouping(w,97,248)){f.cursor=e;break}if(f.cursor=e,e>=f.limit)return;f.cursor++}for(;!f.out_grouping(w,97,248);){if(f.cursor>=f.limit)return;f.cursor++}d=f.cursor,d<a&&(d=a)}}function n(){var e,r;if(f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(c,32),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del();break;case 2:f.in_grouping_b(p,97,229)&&f.slice_del()}}function t(){var e,r=f.limit-f.cursor;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.find_among_b(l,4)?(f.bra=f.cursor,f.limit_backward=e,f.cursor=f.limit-r,f.cursor>f.limit_backward&&(f.cursor--,f.bra=f.cursor,f.slice_del())):f.limit_backward=e)}function s(){var e,r,i,n=f.limit-f.cursor;if(f.ket=f.cursor,f.eq_s_b(2,"st")&&(f.bra=f.cursor,f.eq_s_b(2,"ig")&&f.slice_del()),f.cursor=f.limit-n,f.cursor>=d&&(r=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,e=f.find_among_b(m,5),f.limit_backward=r,e))switch(f.bra=f.cursor,e){case 1:f.slice_del(),i=f.limit-f.cursor,t(),f.cursor=f.limit-i;break;case 2:f.slice_from("løs")}}function o(){var e;f.cursor>=d&&(e=f.limit_backward,f.limit_backward=d,f.ket=f.cursor,f.out_grouping_b(w,97,248)?(f.bra=f.cursor,u=f.slice_to(u),f.limit_backward=e,f.eq_v_b(u)&&f.slice_del()):f.limit_backward=e)}var a,d,u,c=[new r("hed",-1,1),new r("ethed",0,1),new r("ered",-1,1),new r("e",-1,1),new r("erede",3,1),new r("ende",3,1),new r("erende",5,1),new r("ene",3,1),new r("erne",3,1),new r("ere",3,1),new r("en",-1,1),new r("heden",10,1),new r("eren",10,1),new r("er",-1,1),new r("heder",13,1),new r("erer",13,1),new r("s",-1,2),new r("heds",16,1),new r("es",16,1),new r("endes",18,1),new r("erendes",19,1),new r("enes",18,1),new r("ernes",18,1),new r("eres",18,1),new r("ens",16,1),new r("hedens",24,1),new r("erens",24,1),new r("ers",16,1),new r("ets",16,1),new r("erets",28,1),new r("et",-1,1),new r("eret",30,1)],l=[new r("gd",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("elig",1,1),new r("els",-1,1),new r("løst",-1,2)],w=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],p=[239,254,42,3,0,0,0,0,0,0,0,0,0,0,0,0,16],f=new i;this.setCurrent=function(e){f.setCurrent(e)},this.getCurrent=function(){return f.getCurrent()},this.stem=function(){var r=f.cursor;return e(),f.limit_backward=r,f.cursor=f.limit,n(),f.cursor=f.limit,t(),f.cursor=f.limit,s(),f.cursor=f.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return n.setCurrent(e),n.stem(),n.getCurrent()}):(n.setCurrent(e),n.stem(),n.getCurrent())}}(),e.Pipeline.registerFunction(e.da.stemmer,"stemmer-da"),e.da.stopWordFilter=e.generateStopWordFilter("ad af alle alt anden at blev blive bliver da de dem den denne der deres det dette dig din disse dog du efter eller en end er et for fra ham han hans har havde have hende hendes her hos hun hvad hvis hvor i ikke ind jeg jer jo kunne man mange med meget men mig min mine mit mod ned noget nogle nu når og også om op os over på selv sig sin sine sit skal skulle som sådan thi til ud under var vi vil ville vor være været".split(" ")),e.Pipeline.registerFunction(e.da.stopWordFilter,"stopWordFilter-da")}});
assets/javascripts/lunr/min/lunr.de.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.du.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.el.min.js
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.es.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.fi.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.fr.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.he.min.js
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.hi.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hi=function(){this.pipeline.reset(),this.pipeline.add(e.hi.trimmer,e.hi.stopWordFilter,e.hi.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.hi.stemmer))},e.hi.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿa-zA-Zａ-ｚＡ-Ｚ0-9０-９",e.hi.trimmer=e.trimmerSupport.generateTrimmer(e.hi.wordCharacters),e.Pipeline.registerFunction(e.hi.trimmer,"trimmer-hi"),e.hi.stopWordFilter=e.generateStopWordFilter("अत अपना अपनी अपने अभी अंदर आदि आप इत्यादि इन इनका इन्हीं इन्हें इन्हों इस इसका इसकी इसके इसमें इसी इसे उन उनका उनकी उनके उनको उन्हीं उन्हें उन्हों उस उसके उसी उसे एक एवं एस ऐसे और कई कर करता करते करना करने करें कहते कहा का काफ़ी कि कितना किन्हें किन्हों किया किर किस किसी किसे की कुछ कुल के को कोई कौन कौनसा गया घर जब जहाँ जा जितना जिन जिन्हें जिन्हों जिस जिसे जीधर जैसा जैसे जो तक तब तरह तिन तिन्हें तिन्हों तिस तिसे तो था थी थे दबारा दिया दुसरा दूसरे दो द्वारा न नके नहीं ना निहायत नीचे ने पर पहले पूरा पे फिर बनी बही बहुत बाद बाला बिलकुल भी भीतर मगर मानो मे में यदि यह यहाँ यही या यिह ये रखें रहा रहे ऱ्वासा लिए लिये लेकिन व वग़ैरह वर्ग वह वहाँ वहीं वाले वुह वे वो सकता सकते सबसे सभी साथ साबुत साभ सारा से सो संग ही हुआ हुई हुए है हैं हो होता होती होते होना होने".split(" ")),e.hi.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.hi.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var t=i.toString().toLowerCase().replace(/^\s+/,"");return r.cut(t).split("|")},e.Pipeline.registerFunction(e.hi.stemmer,"stemmer-hi"),e.Pipeline.registerFunction(e.hi.stopWordFilter,"stopWordFilter-hi")}});
assets/javascripts/lunr/min/lunr.hu.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.hy.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.hy=function(){this.pipeline.reset(),this.pipeline.add(e.hy.trimmer,e.hy.stopWordFilter)},e.hy.wordCharacters="[A-Za-z԰-֏ff-ﭏ]",e.hy.trimmer=e.trimmerSupport.generateTrimmer(e.hy.wordCharacters),e.Pipeline.registerFunction(e.hy.trimmer,"trimmer-hy"),e.hy.stopWordFilter=e.generateStopWordFilter("դու և եք էիր էիք հետո նաև նրանք որը վրա է որ պիտի են այս մեջ ն իր ու ի այդ որոնք այն կամ էր մի ես համար այլ իսկ էին ենք հետ ին թ էինք մենք նրա նա դուք եմ էի ըստ որպես ում".split(" ")),e.Pipeline.registerFunction(e.hy.stopWordFilter,"stopWordFilter-hy"),e.hy.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}(),e.Pipeline.registerFunction(e.hy.stemmer,"stemmer-hy")}});
assets/javascripts/lunr/min/lunr.it.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.ja.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.ja=function(){this.pipeline.reset(),this.pipeline.add(e.ja.trimmer,e.ja.stopWordFilter,e.ja.stemmer),r?this.tokenizer=e.ja.tokenizer:(e.tokenizer&&(e.tokenizer=e.ja.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.ja.tokenizer))};var t=new e.TinySegmenter;e.ja.tokenizer=function(i){var n,o,s,p,a,u,m,l,c,f;if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t.toLowerCase()):t.toLowerCase()});for(o=i.toString().toLowerCase().replace(/^\s+/,""),n=o.length-1;n>=0;n--)if(/\S/.test(o.charAt(n))){o=o.substring(0,n+1);break}for(a=[],s=o.length,c=0,l=0;c<=s;c++)if(u=o.charAt(c),m=c-l,u.match(/\s/)||c==s){if(m>0)for(p=t.segment(o.slice(l,c)).filter(function(e){return!!e}),f=l,n=0;n<p.length;n++)r?a.push(new e.Token(p[n],{position:[f,p[n].length],index:a.length})):a.push(p[n]),f+=p[n].length;l=c+1}return a},e.ja.stemmer=function(){return function(e){return e}}(),e.Pipeline.registerFunction(e.ja.stemmer,"stemmer-ja"),e.ja.wordCharacters="一二三四五六七八九十百千万億兆一-龠々〆ヵヶぁ-んァ-ヴーｱ-ﾝﾞa-zA-Zａ-ｚＡ-Ｚ0-9０-９",e.ja.trimmer=e.trimmerSupport.generateTrimmer(e.ja.wordCharacters),e.Pipeline.registerFunction(e.ja.trimmer,"trimmer-ja"),e.ja.stopWordFilter=e.generateStopWordFilter("これ それ あれ この その あの ここ そこ あそこ こちら どこ だれ なに なん 何 私 貴方 貴方方 我々 私達 あの人 あのかた 彼女 彼 です あります おります います は が の に を で え から まで より も どの と し それで しかし".split(" ")),e.Pipeline.registerFunction(e.ja.stopWordFilter,"stopWordFilter-ja"),e.jp=e.ja,e.Pipeline.registerFunction(e.jp.stemmer,"stemmer-jp"),e.Pipeline.registerFunction(e.jp.trimmer,"trimmer-jp"),e.Pipeline.registerFunction(e.jp.stopWordFilter,"stopWordFilter-jp")}});
assets/javascripts/lunr/min/lunr.jp.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
module.exports=require("./lunr.ja");
assets/javascripts/lunr/min/lunr.kn.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.kn=function(){this.pipeline.reset(),this.pipeline.add(e.kn.trimmer,e.kn.stopWordFilter,e.kn.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.kn.stemmer))},e.kn.wordCharacters="ಀ-಄ಅ-ಔಕ-ಹಾ-ೌ಼-ಽೕ-ೖೝ-ೞೠ-ೡೢ-ೣ೤೥೦-೯ೱ-ೳ",e.kn.trimmer=e.trimmerSupport.generateTrimmer(e.kn.wordCharacters),e.Pipeline.registerFunction(e.kn.trimmer,"trimmer-kn"),e.kn.stopWordFilter=e.generateStopWordFilter("ಮತ್ತು ಈ ಒಂದು ರಲ್ಲಿ ಹಾಗೂ ಎಂದು ಅಥವಾ ಇದು ರ ಅವರು ಎಂಬ ಮೇಲೆ ಅವರ ತನ್ನ ಆದರೆ ತಮ್ಮ ನಂತರ ಮೂಲಕ ಹೆಚ್ಚು ನ ಆ ಕೆಲವು ಅನೇಕ ಎರಡು ಹಾಗು ಪ್ರಮುಖ ಇದನ್ನು ಇದರ ಸುಮಾರು ಅದರ ಅದು ಮೊದಲ ಬಗ್ಗೆ ನಲ್ಲಿ ರಂದು ಇತರ ಅತ್ಯಂತ ಹೆಚ್ಚಿನ ಸಹ ಸಾಮಾನ್ಯವಾಗಿ ನೇ ಹಲವಾರು ಹೊಸ ದಿ ಕಡಿಮೆ ಯಾವುದೇ ಹೊಂದಿದೆ ದೊಡ್ಡ ಅನ್ನು ಇವರು ಪ್ರಕಾರ ಇದೆ ಮಾತ್ರ ಕೂಡ ಇಲ್ಲಿ ಎಲ್ಲಾ ವಿವಿಧ ಅದನ್ನು ಹಲವು ರಿಂದ ಕೇವಲ ದ ದಕ್ಷಿಣ ಗೆ ಅವನ ಅತಿ ನೆಯ ಬಹಳ ಕೆಲಸ ಎಲ್ಲ ಪ್ರತಿ ಇತ್ಯಾದಿ ಇವು ಬೇರೆ ಹೀಗೆ ನಡುವೆ ಇದಕ್ಕೆ ಎಸ್ ಇವರ ಮೊದಲು ಶ್ರೀ ಮಾಡುವ ಇದರಲ್ಲಿ ರೀತಿಯ ಮಾಡಿದ ಕಾಲ ಅಲ್ಲಿ ಮಾಡಲು ಅದೇ ಈಗ ಅವು ಗಳು ಎ ಎಂಬುದು ಅವನು ಅಂದರೆ ಅವರಿಗೆ ಇರುವ ವಿಶೇಷ ಮುಂದೆ ಅವುಗಳ ಮುಂತಾದ ಮೂಲ ಬಿ ಮೀ ಒಂದೇ ಇನ್ನೂ ಹೆಚ್ಚಾಗಿ ಮಾಡಿ ಅವರನ್ನು ಇದೇ ಯ ರೀತಿಯಲ್ಲಿ ಜೊತೆ ಅದರಲ್ಲಿ ಮಾಡಿದರು ನಡೆದ ಆಗ ಮತ್ತೆ ಪೂರ್ವ ಆತ ಬಂದ ಯಾವ ಒಟ್ಟು ಇತರೆ ಹಿಂದೆ ಪ್ರಮಾಣದ ಗಳನ್ನು ಕುರಿತು ಯು ಆದ್ದರಿಂದ ಅಲ್ಲದೆ ನಗರದ ಮೇಲಿನ ಏಕೆಂದರೆ ರಷ್ಟು ಎಂಬುದನ್ನು ಬಾರಿ ಎಂದರೆ ಹಿಂದಿನ ಆದರೂ ಆದ ಸಂಬಂಧಿಸಿದ ಮತ್ತೊಂದು ಸಿ ಆತನ ".split(" ")),e.kn.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.kn.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var n=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(n).split("|")},e.Pipeline.registerFunction(e.kn.stemmer,"stemmer-kn"),e.Pipeline.registerFunction(e.kn.stopWordFilter,"stopWordFilter-kn")}});
assets/javascripts/lunr/min/lunr.ko.min.js
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.multi.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){e.multiLanguage=function(){for(var t=Array.prototype.slice.call(arguments),i=t.join("-"),r="",n=[],s=[],p=0;p<t.length;++p)"en"==t[p]?(r+="\\w",n.unshift(e.stopWordFilter),n.push(e.stemmer),s.push(e.stemmer)):(r+=e[t[p]].wordCharacters,e[t[p]].stopWordFilter&&n.unshift(e[t[p]].stopWordFilter),e[t[p]].stemmer&&(n.push(e[t[p]].stemmer),s.push(e[t[p]].stemmer)));var o=e.trimmerSupport.generateTrimmer(r);return e.Pipeline.registerFunction(o,"lunr-multi-trimmer-"+i),n.unshift(o),function(){this.pipeline.reset(),this.pipeline.add.apply(this.pipeline,n),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add.apply(this.searchPipeline,s))}}}});
assets/javascripts/lunr/min/lunr.nl.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.no.min.js
Vendored
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Norwegian` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.no=function(){this.pipeline.reset(),this.pipeline.add(e.no.trimmer,e.no.stopWordFilter,e.no.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.no.stemmer))},e.no.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.no.trimmer=e.trimmerSupport.generateTrimmer(e.no.wordCharacters),e.Pipeline.registerFunction(e.no.trimmer,"trimmer-no"),e.no.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,i=new function(){function e(){var e,r=w.cursor+3;if(a=w.limit,0<=r||r<=w.limit){for(s=r;;){if(e=w.cursor,w.in_grouping(d,97,248)){w.cursor=e;break}if(e>=w.limit)return;w.cursor=e+1}for(;!w.out_grouping(d,97,248);){if(w.cursor>=w.limit)return;w.cursor++}a=w.cursor,a<s&&(a=s)}}function i(){var e,r,n;if(w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(m,29),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:n=w.limit-w.cursor,w.in_grouping_b(c,98,122)?w.slice_del():(w.cursor=w.limit-n,w.eq_s_b(1,"k")&&w.out_grouping_b(d,97,248)&&w.slice_del());break;case 3:w.slice_from("er")}}function t(){var e,r=w.limit-w.cursor;w.cursor>=a&&(e=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,w.find_among_b(u,2)?(w.bra=w.cursor,w.limit_backward=e,w.cursor=w.limit-r,w.cursor>w.limit_backward&&(w.cursor--,w.bra=w.cursor,w.slice_del())):w.limit_backward=e)}function o(){var e,r;w.cursor>=a&&(r=w.limit_backward,w.limit_backward=a,w.ket=w.cursor,e=w.find_among_b(l,11),e?(w.bra=w.cursor,w.limit_backward=r,1==e&&w.slice_del()):w.limit_backward=r)}var s,a,m=[new r("a",-1,1),new r("e",-1,1),new r("ede",1,1),new r("ande",1,1),new r("ende",1,1),new r("ane",1,1),new r("ene",1,1),new r("hetene",6,1),new r("erte",1,3),new r("en",-1,1),new r("heten",9,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",12,1),new r("s",-1,2),new r("as",14,1),new r("es",14,1),new r("edes",16,1),new r("endes",16,1),new r("enes",16,1),new r("hetenes",19,1),new r("ens",14,1),new r("hetens",21,1),new r("ers",14,1),new r("ets",14,1),new r("et",-1,1),new r("het",25,1),new r("ert",-1,3),new r("ast",-1,1)],u=[new r("dt",-1,-1),new r("vt",-1,-1)],l=[new r("leg",-1,1),new r("eleg",0,1),new r("ig",-1,1),new r("eig",2,1),new r("lig",2,1),new r("elig",4,1),new r("els",-1,1),new r("lov",-1,1),new r("elov",7,1),new r("slov",7,1),new r("hetslov",9,1)],d=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,48,0,128],c=[119,125,149,1],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,i(),w.cursor=w.limit,t(),w.cursor=w.limit,o(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return i.setCurrent(e),i.stem(),i.getCurrent()}):(i.setCurrent(e),i.stem(),i.getCurrent())}}(),e.Pipeline.registerFunction(e.no.stemmer,"stemmer-no"),e.no.stopWordFilter=e.generateStopWordFilter("alle at av bare begge ble blei bli blir blitt både båe da de deg dei deim deira deires dem den denne der dere deres det dette di din disse ditt du dykk dykkar då eg ein eit eitt eller elles en enn er et ett etter for fordi fra før ha hadde han hans har hennar henne hennes her hjå ho hoe honom hoss hossen hun hva hvem hver hvilke hvilken hvis hvor hvordan hvorfor i ikke ikkje ikkje ingen ingi inkje inn inni ja jeg kan kom korleis korso kun kunne kva kvar kvarhelst kven kvi kvifor man mange me med medan meg meget mellom men mi min mine mitt mot mykje ned no noe noen noka noko nokon nokor nokre nå når og også om opp oss over på samme seg selv si si sia sidan siden sin sine sitt sjøl skal skulle slik so som som somme somt så sånn til um upp ut uten var vart varte ved vere verte vi vil ville vore vors vort vår være være vært å".split(" ")),e.Pipeline.registerFunction(e.no.stopWordFilter,"stopWordFilter-no")}});
assets/javascripts/lunr/min/lunr.pt.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.ro.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.ru.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.sa.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sa=function(){this.pipeline.reset(),this.pipeline.add(e.sa.trimmer,e.sa.stopWordFilter,e.sa.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sa.stemmer))},e.sa.wordCharacters="ऀ-ःऄ-एऐ-टठ-यर-िी-ॏॐ-य़ॠ-९॰-ॿ꣠-꣱ꣲ-ꣷ꣸-ꣻ꣼-ꣽꣾ-ꣿᆰ0-ᆰ9",e.sa.trimmer=e.trimmerSupport.generateTrimmer(e.sa.wordCharacters),e.Pipeline.registerFunction(e.sa.trimmer,"trimmer-sa"),e.sa.stopWordFilter=e.generateStopWordFilter('तथा अयम्‌ एकम्‌ इत्यस्मिन्‌ तथा तत्‌ वा अयम्‌ इत्यस्य ते आहूत उपरि तेषाम्‌ किन्तु तेषाम्‌ तदा इत्यनेन अधिकः इत्यस्य तत्‌ केचन बहवः द्वि तथा महत्वपूर्णः अयम्‌ अस्य विषये अयं अस्ति तत्‌ प्रथमः विषये इत्युपरि इत्युपरि इतर अधिकतमः अधिकः अपि सामान्यतया ठ इतरेतर नूतनम्‌ द न्यूनम्‌ कश्चित्‌ वा विशालः द सः अस्ति तदनुसारम् तत्र अस्ति केवलम्‌ अपि अत्र सर्वे विविधाः तत्‌ बहवः यतः इदानीम्‌ द दक्षिण इत्यस्मै तस्य उपरि नथ अतीव कार्यम्‌ सर्वे एकैकम्‌ इत्यादि। एते सन्ति उत इत्थम्‌ मध्ये एतदर्थं . स कस्य प्रथमः श्री. करोति अस्मिन् प्रकारः निर्मिता कालः तत्र कर्तुं समान अधुना ते सन्ति स एकः अस्ति सः अर्थात् तेषां कृते . स्थितम् विशेषः अग्रिम तेषाम्‌ समान स्रोतः ख म समान इदानीमपि अधिकतया करोतु ते समान इत्यस्य वीथी सह यस्मिन् कृतवान्‌ धृतः तदा पुनः पूर्वं सः आगतः किम्‌ कुल इतर पुरा मात्रा स विषये उ अतएव अपि नगरस्य उपरि यतः प्रतिशतं कतरः कालः साधनानि भूत तथापि जात सम्बन्धि अन्यत्‌ ग अतः अस्माकं स्वकीयाः अस्माकं इदानीं अन्तः इत्यादयः भवन्तः इत्यादयः एते एताः तस्य अस्य इदम् एते तेषां तेषां तेषां तान् तेषां तेषां तेषां समानः सः एकः च तादृशाः बहवः अन्ये च वदन्ति यत् कियत् कस्मै कस्मै यस्मै यस्मै यस्मै यस्मै न अतिनीचः किन्तु प्रथमं सम्पूर्णतया ततः चिरकालानन्तरं पुस्तकं सम्पूर्णतया अन्तः किन्तु अत्र वा इह इव श्रद्धाय अवशिष्यते परन्तु अन्ये वर्गाः सन्ति ते सन्ति शक्नुवन्ति सर्वे मिलित्वा सर्वे एकत्र"'.split(" ")),e.sa.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var r=e.wordcut;r.init(),e.sa.tokenizer=function(t){if(!arguments.length||null==t||void 0==t)return[];if(Array.isArray(t))return t.map(function(r){return isLunr2?new e.Token(r.toLowerCase()):r.toLowerCase()});var i=t.toString().toLowerCase().replace(/^\s+/,"");return r.cut(i).split("|")},e.Pipeline.registerFunction(e.sa.stemmer,"stemmer-sa"),e.Pipeline.registerFunction(e.sa.stopWordFilter,"stopWordFilter-sa")}});
assets/javascripts/lunr/min/lunr.stemmer.support.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(r,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(r.lunr)}(this,function(){return function(r){r.stemmerSupport={Among:function(r,t,i,s){if(this.toCharArray=function(r){for(var t=r.length,i=new Array(t),s=0;s<t;s++)i[s]=r.charCodeAt(s);return i},!r&&""!=r||!t&&0!=t||!i)throw"Bad Among initialisation: s:"+r+", substring_i: "+t+", result: "+i;this.s_size=r.length,this.s=this.toCharArray(r),this.substring_i=t,this.result=i,this.method=s},SnowballProgram:function(){var r;return{bra:0,ket:0,limit:0,cursor:0,limit_backward:0,setCurrent:function(t){r=t,this.cursor=0,this.limit=t.length,this.limit_backward=0,this.bra=this.cursor,this.ket=this.limit},getCurrent:function(){var t=r;return r=null,t},in_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},in_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e<=s&&e>=i&&(e-=i,t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},out_grouping:function(t,i,s){if(this.cursor<this.limit){var e=r.charCodeAt(this.cursor);if(e>s||e<i)return this.cursor++,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor++,!0}return!1},out_grouping_b:function(t,i,s){if(this.cursor>this.limit_backward){var e=r.charCodeAt(this.cursor-1);if(e>s||e<i)return this.cursor--,!0;if(e-=i,!(t[e>>3]&1<<(7&e)))return this.cursor--,!0}return!1},eq_s:function(t,i){if(this.limit-this.cursor<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor+s)!=i.charCodeAt(s))return!1;return this.cursor+=t,!0},eq_s_b:function(t,i){if(this.cursor-this.limit_backward<t)return!1;for(var s=0;s<t;s++)if(r.charCodeAt(this.cursor-t+s)!=i.charCodeAt(s))return!1;return this.cursor-=t,!0},find_among:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=l;m<_.s_size;m++){if(n+l==u){f=-1;break}if(f=r.charCodeAt(n+l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n+_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n+_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},find_among_b:function(t,i){for(var s=0,e=i,n=this.cursor,u=this.limit_backward,o=0,h=0,c=!1;;){for(var a=s+(e-s>>1),f=0,l=o<h?o:h,_=t[a],m=_.s_size-1-l;m>=0;m--){if(n-l==u){f=-1;break}if(f=r.charCodeAt(n-1-l)-_.s[m])break;l++}if(f<0?(e=a,h=l):(s=a,o=l),e-s<=1){if(s>0||e==s||c)break;c=!0}}for(;;){var _=t[s];if(o>=_.s_size){if(this.cursor=n-_.s_size,!_.method)return _.result;var b=_.method();if(this.cursor=n-_.s_size,b)return _.result}if((s=_.substring_i)<0)return 0}},replace_s:function(t,i,s){var e=s.length-(i-t),n=r.substring(0,t),u=r.substring(i);return r=n+s+u,this.limit+=e,this.cursor>=i?this.cursor+=e:this.cursor>t&&(this.cursor=t),e},slice_check:function(){if(this.bra<0||this.bra>this.ket||this.ket>this.limit||this.limit>r.length)throw"faulty slice operation"},slice_from:function(r){this.slice_check(),this.replace_s(this.bra,this.ket,r)},slice_del:function(){this.slice_from("")},insert:function(r,t,i){var s=this.replace_s(r,t,i);r<=this.bra&&(this.bra+=s),r<=this.ket&&(this.ket+=s)},slice_to:function(){return this.slice_check(),r.substring(this.bra,this.ket)},eq_v_b:function(r){return this.eq_s_b(r.length,r)}}}},r.trimmerSupport={generateTrimmer:function(r){var t=new RegExp("^[^"+r+"]+"),i=new RegExp("[^"+r+"]+$");return function(r){return"function"==typeof r.update?r.update(function(r){return r.replace(t,"").replace(i,"")}):r.replace(t,"").replace(i,"")}}}}});
assets/javascripts/lunr/min/lunr.sv.min.js
Vendored
-18
View File
@@ -1,18 +0,0 @@
/*!
* Lunr languages, `Swedish` language
* https://github.com/MihaiValentin/lunr-languages
*
* Copyright 2014, Mihai Valentin
* http://www.mozilla.org/MPL/
*/
/*!
* based on
* Snowball JavaScript Library v0.3
* http://code.google.com/p/urim/
* http://snowball.tartarus.org/
*
* Copyright 2010, Oleg Mazko
* http://www.mozilla.org/MPL/
*/
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.sv=function(){this.pipeline.reset(),this.pipeline.add(e.sv.trimmer,e.sv.stopWordFilter,e.sv.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.sv.stemmer))},e.sv.wordCharacters="A-Za-zªºÀ-ÖØ-öø-ʸˠ-ˤᴀ-ᴥᴬ-ᵜᵢ-ᵥᵫ-ᵷᵹ-ᶾḀ-ỿⁱⁿₐ-ₜKÅℲⅎⅠ-ↈⱠ-ⱿꜢ-ꞇꞋ-ꞭꞰ-ꞷꟷ-ꟿꬰ-ꭚꭜ-ꭤff-stＡ-Ｚａ-ｚ",e.sv.trimmer=e.trimmerSupport.generateTrimmer(e.sv.wordCharacters),e.Pipeline.registerFunction(e.sv.trimmer,"trimmer-sv"),e.sv.stemmer=function(){var r=e.stemmerSupport.Among,n=e.stemmerSupport.SnowballProgram,t=new function(){function e(){var e,r=w.cursor+3;if(o=w.limit,0<=r||r<=w.limit){for(a=r;;){if(e=w.cursor,w.in_grouping(l,97,246)){w.cursor=e;break}if(w.cursor=e,w.cursor>=w.limit)return;w.cursor++}for(;!w.out_grouping(l,97,246);){if(w.cursor>=w.limit)return;w.cursor++}o=w.cursor,o<a&&(o=a)}}function t(){var e,r=w.limit_backward;if(w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(u,37),w.limit_backward=r,e))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.in_grouping_b(d,98,121)&&w.slice_del()}}function i(){var e=w.limit_backward;w.cursor>=o&&(w.limit_backward=o,w.cursor=w.limit,w.find_among_b(c,7)&&(w.cursor=w.limit,w.ket=w.cursor,w.cursor>w.limit_backward&&(w.bra=--w.cursor,w.slice_del())),w.limit_backward=e)}function s(){var e,r;if(w.cursor>=o){if(r=w.limit_backward,w.limit_backward=o,w.cursor=w.limit,w.ket=w.cursor,e=w.find_among_b(m,5))switch(w.bra=w.cursor,e){case 1:w.slice_del();break;case 2:w.slice_from("lös");break;case 3:w.slice_from("full")}w.limit_backward=r}}var a,o,u=[new r("a",-1,1),new r("arna",0,1),new r("erna",0,1),new r("heterna",2,1),new r("orna",0,1),new r("ad",-1,1),new r("e",-1,1),new r("ade",6,1),new r("ande",6,1),new r("arne",6,1),new r("are",6,1),new r("aste",6,1),new r("en",-1,1),new r("anden",12,1),new r("aren",12,1),new r("heten",12,1),new r("ern",-1,1),new r("ar",-1,1),new r("er",-1,1),new r("heter",18,1),new r("or",-1,1),new r("s",-1,2),new r("as",21,1),new r("arnas",22,1),new r("ernas",22,1),new r("ornas",22,1),new r("es",21,1),new r("ades",26,1),new r("andes",26,1),new r("ens",21,1),new r("arens",29,1),new r("hetens",29,1),new r("erns",21,1),new r("at",-1,1),new r("andet",-1,1),new r("het",-1,1),new r("ast",-1,1)],c=[new r("dd",-1,-1),new r("gd",-1,-1),new r("nn",-1,-1),new r("dt",-1,-1),new r("gt",-1,-1),new r("kt",-1,-1),new r("tt",-1,-1)],m=[new r("ig",-1,1),new r("lig",0,1),new r("els",-1,1),new r("fullt",-1,3),new r("löst",-1,2)],l=[17,65,16,1,0,0,0,0,0,0,0,0,0,0,0,0,24,0,32],d=[119,127,149],w=new n;this.setCurrent=function(e){w.setCurrent(e)},this.getCurrent=function(){return w.getCurrent()},this.stem=function(){var r=w.cursor;return e(),w.limit_backward=r,w.cursor=w.limit,t(),w.cursor=w.limit,i(),w.cursor=w.limit,s(),!0}};return function(e){return"function"==typeof e.update?e.update(function(e){return t.setCurrent(e),t.stem(),t.getCurrent()}):(t.setCurrent(e),t.stem(),t.getCurrent())}}(),e.Pipeline.registerFunction(e.sv.stemmer,"stemmer-sv"),e.sv.stopWordFilter=e.generateStopWordFilter("alla allt att av blev bli blir blivit de dem den denna deras dess dessa det detta dig din dina ditt du där då efter ej eller en er era ert ett från för ha hade han hans har henne hennes hon honom hur här i icke ingen inom inte jag ju kan kunde man med mellan men mig min mina mitt mot mycket ni nu när någon något några och om oss på samma sedan sig sin sina sitta själv skulle som så sådan sådana sådant till under upp ut utan vad var vara varför varit varje vars vart vem vi vid vilka vilkas vilken vilket vår våra vårt än är åt över".split(" ")),e.Pipeline.registerFunction(e.sv.stopWordFilter,"stopWordFilter-sv")}});
assets/javascripts/lunr/min/lunr.ta.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.ta=function(){this.pipeline.reset(),this.pipeline.add(e.ta.trimmer,e.ta.stopWordFilter,e.ta.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.ta.stemmer))},e.ta.wordCharacters="஀-உஊ-ஏஐ-ஙச-ட஠-னப-யர-ஹ஺-ிீ-௉ொ-௏ௐ-௙௚-௟௠-௩௪-௯௰-௹௺-௿a-zA-Zａ-ｚＡ-Ｚ0-9０-９",e.ta.trimmer=e.trimmerSupport.generateTrimmer(e.ta.wordCharacters),e.Pipeline.registerFunction(e.ta.trimmer,"trimmer-ta"),e.ta.stopWordFilter=e.generateStopWordFilter("அங்கு அங்கே அது அதை அந்த அவர் அவர்கள் அவள் அவன் அவை ஆக ஆகவே ஆகையால் ஆதலால் ஆதலினால் ஆனாலும் ஆனால் இங்கு இங்கே இது இதை இந்த இப்படி இவர் இவர்கள் இவள் இவன் இவை இவ்வளவு உனக்கு உனது உன் உன்னால் எங்கு எங்கே எது எதை எந்த எப்படி எவர் எவர்கள் எவள் எவன் எவை எவ்வளவு எனக்கு எனது எனவே என் என்ன என்னால் ஏது ஏன் தனது தன்னால் தானே தான் நாங்கள் நாம் நான் நீ நீங்கள்".split(" ")),e.ta.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.ta.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.ta.stemmer,"stemmer-ta"),e.Pipeline.registerFunction(e.ta.stopWordFilter,"stopWordFilter-ta")}});
assets/javascripts/lunr/min/lunr.te.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,t){"function"==typeof define&&define.amd?define(t):"object"==typeof exports?module.exports=t():t()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.te=function(){this.pipeline.reset(),this.pipeline.add(e.te.trimmer,e.te.stopWordFilter,e.te.stemmer),this.searchPipeline&&(this.searchPipeline.reset(),this.searchPipeline.add(e.te.stemmer))},e.te.wordCharacters="ఀ-ఄఅ-ఔక-హా-ౌౕ-ౖౘ-ౚౠ-ౡౢ-ౣ౦-౯౸-౿఼ఽ్ౝ౷౤౥",e.te.trimmer=e.trimmerSupport.generateTrimmer(e.te.wordCharacters),e.Pipeline.registerFunction(e.te.trimmer,"trimmer-te"),e.te.stopWordFilter=e.generateStopWordFilter("అందరూ అందుబాటులో అడగండి అడగడం అడ్డంగా అనుగుణంగా అనుమతించు అనుమతిస్తుంది అయితే ఇప్పటికే ఉన్నారు ఎక్కడైనా ఎప్పుడు ఎవరైనా ఎవరో ఏ ఏదైనా ఏమైనప్పటికి ఒక ఒకరు కనిపిస్తాయి కాదు కూడా గా గురించి చుట్టూ చేయగలిగింది తగిన తర్వాత దాదాపు దూరంగా నిజంగా పై ప్రకారం ప్రక్కన మధ్య మరియు మరొక మళ్ళీ మాత్రమే మెచ్చుకో వద్ద వెంట వేరుగా వ్యతిరేకంగా సంబంధం".split(" ")),e.te.stemmer=function(){return function(e){return"function"==typeof e.update?e.update(function(e){return e}):e}}();var t=e.wordcut;t.init(),e.te.tokenizer=function(r){if(!arguments.length||null==r||void 0==r)return[];if(Array.isArray(r))return r.map(function(t){return isLunr2?new e.Token(t.toLowerCase()):t.toLowerCase()});var i=r.toString().toLowerCase().replace(/^\s+/,"");return t.cut(i).split("|")},e.Pipeline.registerFunction(e.te.stemmer,"stemmer-te"),e.Pipeline.registerFunction(e.te.stopWordFilter,"stopWordFilter-te")}});
assets/javascripts/lunr/min/lunr.th.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var r="2"==e.version[0];e.th=function(){this.pipeline.reset(),this.pipeline.add(e.th.trimmer),r?this.tokenizer=e.th.tokenizer:(e.tokenizer&&(e.tokenizer=e.th.tokenizer),this.tokenizerFn&&(this.tokenizerFn=e.th.tokenizer))},e.th.wordCharacters="[฀-๿]",e.th.trimmer=e.trimmerSupport.generateTrimmer(e.th.wordCharacters),e.Pipeline.registerFunction(e.th.trimmer,"trimmer-th");var t=e.wordcut;t.init(),e.th.tokenizer=function(i){if(!arguments.length||null==i||void 0==i)return[];if(Array.isArray(i))return i.map(function(t){return r?new e.Token(t):t});var n=i.toString().replace(/^\s+/,"");return t.cut(n).split("|")}}});
assets/javascripts/lunr/min/lunr.tr.min.js
Vendored
-18
View File
File diff suppressed because one or more lines are too long
assets/javascripts/lunr/min/lunr.vi.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r():r()(e.lunr)}(this,function(){return function(e){if(void 0===e)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===e.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");e.vi=function(){this.pipeline.reset(),this.pipeline.add(e.vi.stopWordFilter,e.vi.trimmer)},e.vi.wordCharacters="[A-Za-ẓ̀͐́͑̉̃̓ÂâÊêÔôĂ-ăĐ-đƠ-ơƯ-ư]",e.vi.trimmer=e.trimmerSupport.generateTrimmer(e.vi.wordCharacters),e.Pipeline.registerFunction(e.vi.trimmer,"trimmer-vi"),e.vi.stopWordFilter=e.generateStopWordFilter("là cái nhưng mà".split(" "))}});
assets/javascripts/lunr/min/lunr.zh.min.js
Vendored
-1
View File
@@ -1 +0,0 @@
!function(e,r){"function"==typeof define&&define.amd?define(r):"object"==typeof exports?module.exports=r(require("@node-rs/jieba")):r()(e.lunr)}(this,function(e){return function(r,t){if(void 0===r)throw new Error("Lunr is not present. Please include / require Lunr before this script.");if(void 0===r.stemmerSupport)throw new Error("Lunr stemmer support is not present. Please include / require Lunr stemmer support before this script.");var i="2"==r.version[0];r.zh=function(){this.pipeline.reset(),this.pipeline.add(r.zh.trimmer,r.zh.stopWordFilter,r.zh.stemmer),i?this.tokenizer=r.zh.tokenizer:(r.tokenizer&&(r.tokenizer=r.zh.tokenizer),this.tokenizerFn&&(this.tokenizerFn=r.zh.tokenizer))},r.zh.tokenizer=function(n){if(!arguments.length||null==n||void 0==n)return[];if(Array.isArray(n))return n.map(function(e){return i?new r.Token(e.toLowerCase()):e.toLowerCase()});t&&e.load(t);var o=n.toString().trim().toLowerCase(),s=[];e.cut(o,!0).forEach(function(e){s=s.concat(e.split(" "))}),s=s.filter(function(e){return!!e});var u=0;return s.map(function(e,t){if(i){var n=o.indexOf(e,u),s={};return s.position=[n,e.length],s.index=t,u=n,new r.Token(e,s)}return e})},r.zh.wordCharacters="\\w一-龥",r.zh.trimmer=r.trimmerSupport.generateTrimmer(r.zh.wordCharacters),r.Pipeline.registerFunction(r.zh.trimmer,"trimmer-zh"),r.zh.stemmer=function(){return function(e){return e}}(),r.Pipeline.registerFunction(r.zh.stemmer,"stemmer-zh"),r.zh.stopWordFilter=r.generateStopWordFilter("的 一 不 在 人 有 是 为 為 以 于 於 上 他 而 后 後 之 来 來 及 了 因 下 可 到 由 这 這 与 與 也 此 但 并 並 个 個 其 已 无 無 小 我 们 們 起 最 再 今 去 好 只 又 或 很 亦 某 把 那 你 乃 它 吧 被 比 别 趁 当 當 从 從 得 打 凡 儿 兒 尔 爾 该 該 各 给 給 跟 和 何 还 還 即 几 幾 既 看 据 據 距 靠 啦 另 么 麽 每 嘛 拿 哪 您 凭 憑 且 却 卻 让 讓 仍 啥 如 若 使 谁 誰 虽 雖 随 隨 同 所 她 哇 嗡 往 些 向 沿 哟 喲 用 咱 则 則 怎 曾 至 致 着 著 诸 諸 自".split(" ")),r.Pipeline.registerFunction(r.zh.stopWordFilter,"stopWordFilter-zh")}});
assets/javascripts/lunr/tinyseg.js
-206
View File
@@ -1,206 +0,0 @@
/**
* export the module via AMD, CommonJS or as a browser global
* Export code from https://github.com/umdjs/umd/blob/master/returnExports.js
*/
;(function (root, factory) {
if (typeof define === 'function' && define.amd) {
// AMD. Register as an anonymous module.
define(factory)
} else if (typeof exports === 'object') {
/**
* Node. Does not work with strict CommonJS, but
* only CommonJS-like environments that support module.exports,
* like Node.
*/
module.exports = factory()
} else {
// Browser globals (root is window)
factory()(root.lunr);
}
}(this, function () {
/**
* Just return a value to define the module export.
* This example returns an object, but the module
* can return a function as the exported value.
*/
return function(lunr) {
// TinySegmenter 0.1 -- Super compact Japanese tokenizer in Javascript
// (c) 2008 Taku Kudo <taku@chasen.org>
// TinySegmenter is freely distributable under the terms of a new BSD licence.
// For details, see http://chasen.org/~taku/software/TinySegmenter/LICENCE.txt
function TinySegmenter() {
var patterns = {
"[一二三四五六七八九十百千万億兆]":"M",
"[一-龠々〆ヵヶ]":"H",
"[ぁ-ん]":"I",
"[ァ-ヴーｱ-ﾝﾞｰ]":"K",
"[a-zA-Zａ-ｚＡ-Ｚ]":"A",
"[0-9０-９]":"N"
}
this.chartype_ = [];
for (var i in patterns) {
var regexp = new RegExp(i);
this.chartype_.push([regexp, patterns[i]]);
}
this.BIAS__ = -332
this.BC1__ = {"HH":6,"II":2461,"KH":406,"OH":-1378};
this.BC2__ = {"AA":-3267,"AI":2744,"AN":-878,"HH":-4070,"HM":-1711,"HN":4012,"HO":3761,"IA":1327,"IH":-1184,"II":-1332,"IK":1721,"IO":5492,"KI":3831,"KK":-8741,"MH":-3132,"MK":3334,"OO":-2920};
this.BC3__ = {"HH":996,"HI":626,"HK":-721,"HN":-1307,"HO":-836,"IH":-301,"KK":2762,"MK":1079,"MM":4034,"OA":-1652,"OH":266};
this.BP1__ = {"BB":295,"OB":304,"OO":-125,"UB":352};
this.BP2__ = {"BO":60,"OO":-1762};
this.BQ1__ = {"BHH":1150,"BHM":1521,"BII":-1158,"BIM":886,"BMH":1208,"BNH":449,"BOH":-91,"BOO":-2597,"OHI":451,"OIH":-296,"OKA":1851,"OKH":-1020,"OKK":904,"OOO":2965};
this.BQ2__ = {"BHH":118,"BHI":-1159,"BHM":466,"BIH":-919,"BKK":-1720,"BKO":864,"OHH":-1139,"OHM":-181,"OIH":153,"UHI":-1146};
this.BQ3__ = {"BHH":-792,"BHI":2664,"BII":-299,"BKI":419,"BMH":937,"BMM":8335,"BNN":998,"BOH":775,"OHH":2174,"OHM":439,"OII":280,"OKH":1798,"OKI":-793,"OKO":-2242,"OMH":-2402,"OOO":11699};
this.BQ4__ = {"BHH":-3895,"BIH":3761,"BII":-4654,"BIK":1348,"BKK":-1806,"BMI":-3385,"BOO":-12396,"OAH":926,"OHH":266,"OHK":-2036,"ONN":-973};
this.BW1__ = {",と":660,",同":727,"B1あ":1404,"B1同":542,"、と":660,"、同":727,"」と":1682,"あっ":1505,"いう":1743,"いっ":-2055,"いる":672,"うし":-4817,"うん":665,"から":3472,"がら":600,"こう":-790,"こと":2083,"こん":-1262,"さら":-4143,"さん":4573,"した":2641,"して":1104,"すで":-3399,"そこ":1977,"それ":-871,"たち":1122,"ため":601,"った":3463,"つい":-802,"てい":805,"てき":1249,"でき":1127,"です":3445,"では":844,"とい":-4915,"とみ":1922,"どこ":3887,"ない":5713,"なっ":3015,"など":7379,"なん":-1113,"にし":2468,"には":1498,"にも":1671,"に対":-912,"の一":-501,"の中":741,"ませ":2448,"まで":1711,"まま":2600,"まる":-2155,"やむ":-1947,"よっ":-2565,"れた":2369,"れで":-913,"をし":1860,"を見":731,"亡く":-1886,"京都":2558,"取り":-2784,"大き":-2604,"大阪":1497,"平方":-2314,"引き":-1336,"日本":-195,"本当":-2423,"毎日":-2113,"目指":-724,"Ｂ１あ":1404,"Ｂ１同":542,"｣と":1682};
this.BW2__ = {"..":-11822,"11":-669,"――":-5730,"−−":-13175,"いう":-1609,"うか":2490,"かし":-1350,"かも":-602,"から":-7194,"かれ":4612,"がい":853,"がら":-3198,"きた":1941,"くな":-1597,"こと":-8392,"この":-4193,"させ":4533,"され":13168,"さん":-3977,"しい":-1819,"しか":-545,"した":5078,"して":972,"しな":939,"その":-3744,"たい":-1253,"たた":-662,"ただ":-3857,"たち":-786,"たと":1224,"たは":-939,"った":4589,"って":1647,"っと":-2094,"てい":6144,"てき":3640,"てく":2551,"ては":-3110,"ても":-3065,"でい":2666,"でき":-1528,"でし":-3828,"です":-4761,"でも":-4203,"とい":1890,"とこ":-1746,"とと":-2279,"との":720,"とみ":5168,"とも":-3941,"ない":-2488,"なが":-1313,"など":-6509,"なの":2614,"なん":3099,"にお":-1615,"にし":2748,"にな":2454,"によ":-7236,"に対":-14943,"に従":-4688,"に関":-11388,"のか":2093,"ので":-7059,"のに":-6041,"のの":-6125,"はい":1073,"はが":-1033,"はず":-2532,"ばれ":1813,"まし":-1316,"まで":-6621,"まれ":5409,"めて":-3153,"もい":2230,"もの":-10713,"らか":-944,"らし":-1611,"らに":-1897,"りし":651,"りま":1620,"れた":4270,"れて":849,"れば":4114,"ろう":6067,"われ":7901,"を通":-11877,"んだ":728,"んな":-4115,"一人":602,"一方":-1375,"一日":970,"一部":-1051,"上が":-4479,"会社":-1116,"出て":2163,"分の":-7758,"同党":970,"同日":-913,"大阪":-2471,"委員":-1250,"少な":-1050,"年度":-8669,"年間":-1626,"府県":-2363,"手権":-1982,"新聞":-4066,"日新":-722,"日本":-7068,"日米":3372,"曜日":-601,"朝鮮":-2355,"本人":-2697,"東京":-1543,"然と":-1384,"社会":-1276,"立て":-990,"第に":-1612,"米国":-4268,"１１":-669};
this.BW3__ = {"あた":-2194,"あり":719,"ある":3846,"い.":-1185,"い。":-1185,"いい":5308,"いえ":2079,"いく":3029,"いた":2056,"いっ":1883,"いる":5600,"いわ":1527,"うち":1117,"うと":4798,"えと":1454,"か.":2857,"か。":2857,"かけ":-743,"かっ":-4098,"かに":-669,"から":6520,"かり":-2670,"が,":1816,"が、":1816,"がき":-4855,"がけ":-1127,"がっ":-913,"がら":-4977,"がり":-2064,"きた":1645,"けど":1374,"こと":7397,"この":1542,"ころ":-2757,"さい":-714,"さを":976,"し,":1557,"し、":1557,"しい":-3714,"した":3562,"して":1449,"しな":2608,"しま":1200,"す.":-1310,"す。":-1310,"する":6521,"ず,":3426,"ず、":3426,"ずに":841,"そう":428,"た.":8875,"た。":8875,"たい":-594,"たの":812,"たり":-1183,"たる":-853,"だ.":4098,"だ。":4098,"だっ":1004,"った":-4748,"って":300,"てい":6240,"てお":855,"ても":302,"です":1437,"でに":-1482,"では":2295,"とう":-1387,"とし":2266,"との":541,"とも":-3543,"どう":4664,"ない":1796,"なく":-903,"など":2135,"に,":-1021,"に、":-1021,"にし":1771,"にな":1906,"には":2644,"の,":-724,"の、":-724,"の子":-1000,"は,":1337,"は、":1337,"べき":2181,"まし":1113,"ます":6943,"まっ":-1549,"まで":6154,"まれ":-793,"らし":1479,"られ":6820,"るる":3818,"れ,":854,"れ、":854,"れた":1850,"れて":1375,"れば":-3246,"れる":1091,"われ":-605,"んだ":606,"んで":798,"カ月":990,"会議":860,"入り":1232,"大会":2217,"始め":1681,"市":965,"新聞":-5055,"日,":974,"日、":974,"社会":2024,"ｶ月":990};
this.TC1__ = {"AAA":1093,"HHH":1029,"HHM":580,"HII":998,"HOH":-390,"HOM":-331,"IHI":1169,"IOH":-142,"IOI":-1015,"IOM":467,"MMH":187,"OOI":-1832};
this.TC2__ = {"HHO":2088,"HII":-1023,"HMM":-1154,"IHI":-1965,"KKH":703,"OII":-2649};
this.TC3__ = {"AAA":-294,"HHH":346,"HHI":-341,"HII":-1088,"HIK":731,"HOH":-1486,"IHH":128,"IHI":-3041,"IHO":-1935,"IIH":-825,"IIM":-1035,"IOI":-542,"KHH":-1216,"KKA":491,"KKH":-1217,"KOK":-1009,"MHH":-2694,"MHM":-457,"MHO":123,"MMH":-471,"NNH":-1689,"NNO":662,"OHO":-3393};
this.TC4__ = {"HHH":-203,"HHI":1344,"HHK":365,"HHM":-122,"HHN":182,"HHO":669,"HIH":804,"HII":679,"HOH":446,"IHH":695,"IHO":-2324,"IIH":321,"III":1497,"IIO":656,"IOO":54,"KAK":4845,"KKA":3386,"KKK":3065,"MHH":-405,"MHI":201,"MMH":-241,"MMM":661,"MOM":841};
this.TQ1__ = {"BHHH":-227,"BHHI":316,"BHIH":-132,"BIHH":60,"BIII":1595,"BNHH":-744,"BOHH":225,"BOOO":-908,"OAKK":482,"OHHH":281,"OHIH":249,"OIHI":200,"OIIH":-68};
this.TQ2__ = {"BIHH":-1401,"BIII":-1033,"BKAK":-543,"BOOO":-5591};
this.TQ3__ = {"BHHH":478,"BHHM":-1073,"BHIH":222,"BHII":-504,"BIIH":-116,"BIII":-105,"BMHI":-863,"BMHM":-464,"BOMH":620,"OHHH":346,"OHHI":1729,"OHII":997,"OHMH":481,"OIHH":623,"OIIH":1344,"OKAK":2792,"OKHH":587,"OKKA":679,"OOHH":110,"OOII":-685};
this.TQ4__ = {"BHHH":-721,"BHHM":-3604,"BHII":-966,"BIIH":-607,"BIII":-2181,"OAAA":-2763,"OAKK":180,"OHHH":-294,"OHHI":2446,"OHHO":480,"OHIH":-1573,"OIHH":1935,"OIHI":-493,"OIIH":626,"OIII":-4007,"OKAK":-8156};
this.TW1__ = {"につい":-4681,"東京都":2026};
this.TW2__ = {"ある程":-2049,"いった":-1256,"ころが":-2434,"しょう":3873,"その後":-4430,"だって":-1049,"ていた":1833,"として":-4657,"ともに":-4517,"もので":1882,"一気に":-792,"初めて":-1512,"同時に":-8097,"大きな":-1255,"対して":-2721,"社会党":-3216};
this.TW3__ = {"いただ":-1734,"してい":1314,"として":-4314,"につい":-5483,"にとっ":-5989,"に当た":-6247,"ので,":-727,"ので、":-727,"のもの":-600,"れから":-3752,"十二月":-2287};
this.TW4__ = {"いう.":8576,"いう。":8576,"からな":-2348,"してい":2958,"たが,":1516,"たが、":1516,"ている":1538,"という":1349,"ました":5543,"ません":1097,"ようと":-4258,"よると":5865};
this.UC1__ = {"A":484,"K":93,"M":645,"O":-505};
this.UC2__ = {"A":819,"H":1059,"I":409,"M":3987,"N":5775,"O":646};
this.UC3__ = {"A":-1370,"I":2311};
this.UC4__ = {"A":-2643,"H":1809,"I":-1032,"K":-3450,"M":3565,"N":3876,"O":6646};
this.UC5__ = {"H":313,"I":-1238,"K":-799,"M":539,"O":-831};
this.UC6__ = {"H":-506,"I":-253,"K":87,"M":247,"O":-387};
this.UP1__ = {"O":-214};
this.UP2__ = {"B":69,"O":935};
this.UP3__ = {"B":189};
this.UQ1__ = {"BH":21,"BI":-12,"BK":-99,"BN":142,"BO":-56,"OH":-95,"OI":477,"OK":410,"OO":-2422};
this.UQ2__ = {"BH":216,"BI":113,"OK":1759};
this.UQ3__ = {"BA":-479,"BH":42,"BI":1913,"BK":-7198,"BM":3160,"BN":6427,"BO":14761,"OI":-827,"ON":-3212};
this.UW1__ = {",":156,"、":156,"「":-463,"あ":-941,"う":-127,"が":-553,"き":121,"こ":505,"で":-201,"と":-547,"ど":-123,"に":-789,"の":-185,"は":-847,"も":-466,"や":-470,"よ":182,"ら":-292,"り":208,"れ":169,"を":-446,"ん":-137,"・":-135,"主":-402,"京":-268,"区":-912,"午":871,"国":-460,"大":561,"委":729,"市":-411,"日":-141,"理":361,"生":-408,"県":-386,"都":-718,"｢":-463,"･":-135};
this.UW2__ = {",":-829,"、":-829,"〇":892,"「":-645,"」":3145,"あ":-538,"い":505,"う":134,"お":-502,"か":1454,"が":-856,"く":-412,"こ":1141,"さ":878,"ざ":540,"し":1529,"す":-675,"せ":300,"そ":-1011,"た":188,"だ":1837,"つ":-949,"て":-291,"で":-268,"と":-981,"ど":1273,"な":1063,"に":-1764,"の":130,"は":-409,"ひ":-1273,"べ":1261,"ま":600,"も":-1263,"や":-402,"よ":1639,"り":-579,"る":-694,"れ":571,"を":-2516,"ん":2095,"ア":-587,"カ":306,"キ":568,"ッ":831,"三":-758,"不":-2150,"世":-302,"中":-968,"主":-861,"事":492,"人":-123,"会":978,"保":362,"入":548,"初":-3025,"副":-1566,"北":-3414,"区":-422,"大":-1769,"天":-865,"太":-483,"子":-1519,"学":760,"実":1023,"小":-2009,"市":-813,"年":-1060,"強":1067,"手":-1519,"揺":-1033,"政":1522,"文":-1355,"新":-1682,"日":-1815,"明":-1462,"最":-630,"朝":-1843,"本":-1650,"東":-931,"果":-665,"次":-2378,"民":-180,"気":-1740,"理":752,"発":529,"目":-1584,"相":-242,"県":-1165,"立":-763,"第":810,"米":509,"自":-1353,"行":838,"西":-744,"見":-3874,"調":1010,"議":1198,"込":3041,"開":1758,"間":-1257,"｢":-645,"｣":3145,"ｯ":831,"ｱ":-587,"ｶ":306,"ｷ":568};
this.UW3__ = {",":4889,"1":-800,"−":-1723,"、":4889,"々":-2311,"〇":5827,"」":2670,"〓":-3573,"あ":-2696,"い":1006,"う":2342,"え":1983,"お":-4864,"か":-1163,"が":3271,"く":1004,"け":388,"げ":401,"こ":-3552,"ご":-3116,"さ":-1058,"し":-395,"す":584,"せ":3685,"そ":-5228,"た":842,"ち":-521,"っ":-1444,"つ":-1081,"て":6167,"で":2318,"と":1691,"ど":-899,"な":-2788,"に":2745,"の":4056,"は":4555,"ひ":-2171,"ふ":-1798,"へ":1199,"ほ":-5516,"ま":-4384,"み":-120,"め":1205,"も":2323,"や":-788,"よ":-202,"ら":727,"り":649,"る":5905,"れ":2773,"わ":-1207,"を":6620,"ん":-518,"ア":551,"グ":1319,"ス":874,"ッ":-1350,"ト":521,"ム":1109,"ル":1591,"ロ":2201,"ン":278,"・":-3794,"一":-1619,"下":-1759,"世":-2087,"両":3815,"中":653,"主":-758,"予":-1193,"二":974,"人":2742,"今":792,"他":1889,"以":-1368,"低":811,"何":4265,"作":-361,"保":-2439,"元":4858,"党":3593,"全":1574,"公":-3030,"六":755,"共":-1880,"円":5807,"再":3095,"分":457,"初":2475,"別":1129,"前":2286,"副":4437,"力":365,"動":-949,"務":-1872,"化":1327,"北":-1038,"区":4646,"千":-2309,"午":-783,"協":-1006,"口":483,"右":1233,"各":3588,"合":-241,"同":3906,"和":-837,"員":4513,"国":642,"型":1389,"場":1219,"外":-241,"妻":2016,"学":-1356,"安":-423,"実":-1008,"家":1078,"小":-513,"少":-3102,"州":1155,"市":3197,"平":-1804,"年":2416,"広":-1030,"府":1605,"度":1452,"建":-2352,"当":-3885,"得":1905,"思":-1291,"性":1822,"戸":-488,"指":-3973,"政":-2013,"教":-1479,"数":3222,"文":-1489,"新":1764,"日":2099,"旧":5792,"昨":-661,"時":-1248,"曜":-951,"最":-937,"月":4125,"期":360,"李":3094,"村":364,"東":-805,"核":5156,"森":2438,"業":484,"氏":2613,"民":-1694,"決":-1073,"法":1868,"海":-495,"無":979,"物":461,"特":-3850,"生":-273,"用":914,"町":1215,"的":7313,"直":-1835,"省":792,"県":6293,"知":-1528,"私":4231,"税":401,"立":-960,"第":1201,"米":7767,"系":3066,"約":3663,"級":1384,"統":-4229,"総":1163,"線":1255,"者":6457,"能":725,"自":-2869,"英":785,"見":1044,"調":-562,"財":-733,"費":1777,"車":1835,"軍":1375,"込":-1504,"通":-1136,"選":-681,"郎":1026,"郡":4404,"部":1200,"金":2163,"長":421,"開":-1432,"間":1302,"関":-1282,"雨":2009,"電":-1045,"非":2066,"駅":1620,"１":-800,"｣":2670,"･":-3794,"ｯ":-1350,"ｱ":551,"ｸﾞ":1319,"ｽ":874,"ﾄ":521,"ﾑ":1109,"ﾙ":1591,"ﾛ":2201,"ﾝ":278};
this.UW4__ = {",":3930,".":3508,"―":-4841,"、":3930,"。":3508,"〇":4999,"「":1895,"」":3798,"〓":-5156,"あ":4752,"い":-3435,"う":-640,"え":-2514,"お":2405,"か":530,"が":6006,"き":-4482,"ぎ":-3821,"く":-3788,"け":-4376,"げ":-4734,"こ":2255,"ご":1979,"さ":2864,"し":-843,"じ":-2506,"す":-731,"ず":1251,"せ":181,"そ":4091,"た":5034,"だ":5408,"ち":-3654,"っ":-5882,"つ":-1659,"て":3994,"で":7410,"と":4547,"な":5433,"に":6499,"ぬ":1853,"ね":1413,"の":7396,"は":8578,"ば":1940,"ひ":4249,"び":-4134,"ふ":1345,"へ":6665,"べ":-744,"ほ":1464,"ま":1051,"み":-2082,"む":-882,"め":-5046,"も":4169,"ゃ":-2666,"や":2795,"ょ":-1544,"よ":3351,"ら":-2922,"り":-9726,"る":-14896,"れ":-2613,"ろ":-4570,"わ":-1783,"を":13150,"ん":-2352,"カ":2145,"コ":1789,"セ":1287,"ッ":-724,"ト":-403,"メ":-1635,"ラ":-881,"リ":-541,"ル":-856,"ン":-3637,"・":-4371,"ー":-11870,"一":-2069,"中":2210,"予":782,"事":-190,"井":-1768,"人":1036,"以":544,"会":950,"体":-1286,"作":530,"側":4292,"先":601,"党":-2006,"共":-1212,"内":584,"円":788,"初":1347,"前":1623,"副":3879,"力":-302,"動":-740,"務":-2715,"化":776,"区":4517,"協":1013,"参":1555,"合":-1834,"和":-681,"員":-910,"器":-851,"回":1500,"国":-619,"園":-1200,"地":866,"場":-1410,"塁":-2094,"士":-1413,"多":1067,"大":571,"子":-4802,"学":-1397,"定":-1057,"寺":-809,"小":1910,"屋":-1328,"山":-1500,"島":-2056,"川":-2667,"市":2771,"年":374,"庁":-4556,"後":456,"性":553,"感":916,"所":-1566,"支":856,"改":787,"政":2182,"教":704,"文":522,"方":-856,"日":1798,"時":1829,"最":845,"月":-9066,"木":-485,"来":-442,"校":-360,"業":-1043,"氏":5388,"民":-2716,"気":-910,"沢":-939,"済":-543,"物":-735,"率":672,"球":-1267,"生":-1286,"産":-1101,"田":-2900,"町":1826,"的":2586,"目":922,"省":-3485,"県":2997,"空":-867,"立":-2112,"第":788,"米":2937,"系":786,"約":2171,"経":1146,"統":-1169,"総":940,"線":-994,"署":749,"者":2145,"能":-730,"般":-852,"行":-792,"規":792,"警":-1184,"議":-244,"谷":-1000,"賞":730,"車":-1481,"軍":1158,"輪":-1433,"込":-3370,"近":929,"道":-1291,"選":2596,"郎":-4866,"都":1192,"野":-1100,"銀":-2213,"長":357,"間":-2344,"院":-2297,"際":-2604,"電":-878,"領":-1659,"題":-792,"館":-1984,"首":1749,"高":2120,"｢":1895,"｣":3798,"･":-4371,"ｯ":-724,"ｰ":-11870,"ｶ":2145,"ｺ":1789,"ｾ":1287,"ﾄ":-403,"ﾒ":-1635,"ﾗ":-881,"ﾘ":-541,"ﾙ":-856,"ﾝ":-3637};
this.UW5__ = {",":465,".":-299,"1":-514,"E2":-32768,"]":-2762,"、":465,"。":-299,"「":363,"あ":1655,"い":331,"う":-503,"え":1199,"お":527,"か":647,"が":-421,"き":1624,"ぎ":1971,"く":312,"げ":-983,"さ":-1537,"し":-1371,"す":-852,"だ":-1186,"ち":1093,"っ":52,"つ":921,"て":-18,"で":-850,"と":-127,"ど":1682,"な":-787,"に":-1224,"の":-635,"は":-578,"べ":1001,"み":502,"め":865,"ゃ":3350,"ょ":854,"り":-208,"る":429,"れ":504,"わ":419,"を":-1264,"ん":327,"イ":241,"ル":451,"ン":-343,"中":-871,"京":722,"会":-1153,"党":-654,"務":3519,"区":-901,"告":848,"員":2104,"大":-1296,"学":-548,"定":1785,"嵐":-1304,"市":-2991,"席":921,"年":1763,"思":872,"所":-814,"挙":1618,"新":-1682,"日":218,"月":-4353,"査":932,"格":1356,"機":-1508,"氏":-1347,"田":240,"町":-3912,"的":-3149,"相":1319,"省":-1052,"県":-4003,"研":-997,"社":-278,"空":-813,"統":1955,"者":-2233,"表":663,"語":-1073,"議":1219,"選":-1018,"郎":-368,"長":786,"間":1191,"題":2368,"館":-689,"１":-514,"Ｅ２":-32768,"｢":363,"ｲ":241,"ﾙ":451,"ﾝ":-343};
this.UW6__ = {",":227,".":808,"1":-270,"E1":306,"、":227,"。":808,"あ":-307,"う":189,"か":241,"が":-73,"く":-121,"こ":-200,"じ":1782,"す":383,"た":-428,"っ":573,"て":-1014,"で":101,"と":-105,"な":-253,"に":-149,"の":-417,"は":-236,"も":-206,"り":187,"る":-135,"を":195,"ル":-673,"ン":-496,"一":-277,"中":201,"件":-800,"会":624,"前":302,"区":1792,"員":-1212,"委":798,"学":-960,"市":887,"広":-695,"後":535,"業":-697,"相":753,"社":-507,"福":974,"空":-822,"者":1811,"連":463,"郎":1082,"１":-270,"Ｅ１":306,"ﾙ":-673,"ﾝ":-496};
return this;
}
TinySegmenter.prototype.ctype_ = function(str) {
for (var i in this.chartype_) {
if (str.match(this.chartype_[i][0])) {
return this.chartype_[i][1];
}
}
return "O";
}
TinySegmenter.prototype.ts_ = function(v) {
if (v) { return v; }
return 0;
}
TinySegmenter.prototype.segment = function(input) {
if (input == null || input == undefined || input == "") {
return [];
}
var result = [];
var seg = ["B3","B2","B1"];
var ctype = ["O","O","O"];
var o = input.split("");
for (i = 0; i < o.length; ++i) {
seg.push(o[i]);
ctype.push(this.ctype_(o[i]))
}
seg.push("E1");
seg.push("E2");
seg.push("E3");
ctype.push("O");
ctype.push("O");
ctype.push("O");
var word = seg[3];
var p1 = "U";
var p2 = "U";
var p3 = "U";
for (var i = 4; i < seg.length - 3; ++i) {
var score = this.BIAS__;
var w1 = seg[i-3];
var w2 = seg[i-2];
var w3 = seg[i-1];
var w4 = seg[i];
var w5 = seg[i+1];
var w6 = seg[i+2];
var c1 = ctype[i-3];
var c2 = ctype[i-2];
var c3 = ctype[i-1];
var c4 = ctype[i];
var c5 = ctype[i+1];
var c6 = ctype[i+2];
score += this.ts_(this.UP1__[p1]);
score += this.ts_(this.UP2__[p2]);
score += this.ts_(this.UP3__[p3]);
score += this.ts_(this.BP1__[p1 + p2]);
score += this.ts_(this.BP2__[p2 + p3]);
score += this.ts_(this.UW1__[w1]);
score += this.ts_(this.UW2__[w2]);
score += this.ts_(this.UW3__[w3]);
score += this.ts_(this.UW4__[w4]);
score += this.ts_(this.UW5__[w5]);
score += this.ts_(this.UW6__[w6]);
score += this.ts_(this.BW1__[w2 + w3]);
score += this.ts_(this.BW2__[w3 + w4]);
score += this.ts_(this.BW3__[w4 + w5]);
score += this.ts_(this.TW1__[w1 + w2 + w3]);
score += this.ts_(this.TW2__[w2 + w3 + w4]);
score += this.ts_(this.TW3__[w3 + w4 + w5]);
score += this.ts_(this.TW4__[w4 + w5 + w6]);
score += this.ts_(this.UC1__[c1]);
score += this.ts_(this.UC2__[c2]);
score += this.ts_(this.UC3__[c3]);
score += this.ts_(this.UC4__[c4]);
score += this.ts_(this.UC5__[c5]);
score += this.ts_(this.UC6__[c6]);
score += this.ts_(this.BC1__[c2 + c3]);
score += this.ts_(this.BC2__[c3 + c4]);
score += this.ts_(this.BC3__[c4 + c5]);
score += this.ts_(this.TC1__[c1 + c2 + c3]);
score += this.ts_(this.TC2__[c2 + c3 + c4]);
score += this.ts_(this.TC3__[c3 + c4 + c5]);
score += this.ts_(this.TC4__[c4 + c5 + c6]);
// score += this.ts_(this.TC5__[c4 + c5 + c6]);
score += this.ts_(this.UQ1__[p1 + c1]);
score += this.ts_(this.UQ2__[p2 + c2]);
score += this.ts_(this.UQ3__[p3 + c3]);
score += this.ts_(this.BQ1__[p2 + c2 + c3]);
score += this.ts_(this.BQ2__[p2 + c3 + c4]);
score += this.ts_(this.BQ3__[p3 + c2 + c3]);
score += this.ts_(this.BQ4__[p3 + c3 + c4]);
score += this.ts_(this.TQ1__[p2 + c1 + c2 + c3]);
score += this.ts_(this.TQ2__[p2 + c2 + c3 + c4]);
score += this.ts_(this.TQ3__[p3 + c1 + c2 + c3]);
score += this.ts_(this.TQ4__[p3 + c2 + c3 + c4]);
var p = "O";
if (score > 0) {
result.push(word);
word = "";
p = "B";
}
p1 = p2;
p2 = p3;
p3 = p;
word += seg[i];
}
result.push(word);
return result;
}
lunr.TinySegmenter = TinySegmenter;
};
}));
assets/javascripts/lunr/wordcut.js
-6708
View File
File diff suppressed because one or more lines are too long
assets/javascripts/workers/search.2c215733.min.js
Vendored
-42
View File
File diff suppressed because one or more lines are too long
assets/javascripts/workers/search.2c215733.min.js.map
-7
View File
File diff suppressed because one or more lines are too long
assets/stylesheets/main.ec1eaa64.min.css
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/stylesheets/main.ec1eaa64.min.css.map
-1
View File
File diff suppressed because one or more lines are too long
assets/stylesheets/palette.ab4e12ef.min.css
Vendored
-1
View File
File diff suppressed because one or more lines are too long
assets/stylesheets/palette.ab4e12ef.min.css.map
-1
View File
@@ -1 +0,0 @@
{"version":3,"sources":["src/templates/assets/stylesheets/palette/_scheme.scss","../../../../src/templates/assets/stylesheets/palette.scss","src/templates/assets/stylesheets/palette/_accent.scss","src/templates/assets/stylesheets/palette/_primary.scss","src/templates/assets/stylesheets/utilities/_break.scss"],"names":[],"mappings":"AA2BA,cAGE,6BAME,sDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CACA,mDAAA,CACA,6DAAA,CACA,+DAAA,CACA,gEAAA,CAGA,mDAAA,CACA,gDAAA,CACA,yDAAA,CACA,4DAAA,CAGA,0BAAA,CACA,mCAAA,CAGA,iCAAA,CACA,kCAAA,CACA,mCAAA,CACA,mCAAA,CACA,kCAAA,CACA,iCAAA,CACA,+CAAA,CACA,6DAAA,CACA,gEAAA,CACA,4DAAA,CACA,4DAAA,CACA,6DAAA,CAGA,6CAAA,CAGA,+CAAA,CAGA,uDAAA,CACA,6DAAA,CACA,2DAAA,CAGA,iCAAA,CAGA,yDAAA,CACA,iEAAA,CAGA,mDAAA,CACA,mDAAA,CAGA,qDAAA,CACA,uDAAA,CAGA,8DAAA,CAKA,8DAAA,CAKA,0DAAA,CAzEA,iBCiBF,CD6DE,kHAEE,YC3DJ,CDkFE,yDACE,4BChFJ,CD+EE,2DACE,4BC7EJ,CD4EE,gEACE,4BC1EJ,CDyEE,2DACE,4BCvEJ,CDsEE,yDACE,4BCpEJ,CDmEE,0DACE,4BCjEJ,CDgEE,gEACE,4BC9DJ,CD6DE,0DACE,4BC3DJ,CD0DE,2OACE,4BC/CJ,CDsDA,+FAGE,iCCpDF,CACF,CCjDE,2BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD6CN,CCvDE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDoDN,CC9DE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD2DN,CCrEE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDkEN,CC5EE,8BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDyEN,CCnFE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDgFN,CC1FE,kCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDuFN,CCjGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD8FN,CCxGE,4BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDqGN,CC/GE,6BACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCD4GN,CCtHE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDmHN,CC7HE,4BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD6HN,CCpIE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDoIN,CC3IE,6BACE,yBAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCD2IN,CClJE,8BACE,4BAAA,CACA,2CAAA,CAIE,8BAAA,CACA,qCDkJN,CCzJE,mCACE,4BAAA,CACA,2CAAA,CAOE,yBAAA,CACA,qCDsJN,CE3JE,4BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwJN,CEnKE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgKN,CE3KE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwKN,CEnLE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgLN,CE3LE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwLN,CEnME,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgMN,CE3ME,mCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwMN,CEnNE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgNN,CE3NE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwNN,CEnOE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgON,CE3OE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwON,CEnPE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmPN,CE3PE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2PN,CEnQE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCFmQN,CE3QE,+BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAIE,+BAAA,CACA,sCF2QN,CEnRE,oCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFgRN,CE3RE,8BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCFwRN,CEnSE,6BACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BF4RN,CE5SE,kCACE,6BAAA,CACA,oCAAA,CACA,mCAAA,CAOE,0BAAA,CACA,sCAAA,CAKA,4BFqSN,CEtRE,sEACE,4BFyRJ,CE1RE,+DACE,4BF6RJ,CE9RE,iEACE,4BFiSJ,CElSE,gEACE,4BFqSJ,CEtSE,iEACE,4BFySJ,CEhSA,8BACE,mDAAA,CACA,4DAAA,CACA,0DAAA,CACA,oDAAA,CACA,2DAAA,CAGA,4BFiSF,CE9RE,yCACE,+BFgSJ,CE7RI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCFiSN,CG7MI,mCD1EA,+CACE,8CF0RJ,CEvRI,qDACE,8CFyRN,CEpRE,iEACE,mCFsRJ,CACF,CGxNI,sCDvDA,uCACE,oCFkRJ,CACF,CEzQA,8BACE,kDAAA,CACA,4DAAA,CACA,wDAAA,CACA,oDAAA,CACA,6DAAA,CAGA,4BF0QF,CEvQE,yCACE,+BFyQJ,CEtQI,kDAEE,0CAAA,CACA,sCAAA,CAFA,mCF0QN,CEnQE,yCACE,6CFqQJ,CG9NI,0CDhCA,8CACE,gDFiQJ,CACF,CGnOI,0CDvBA,iFACE,6CF6PJ,CACF,CG3PI,sCDKA,uCACE,6CFyPJ,CACF","file":"palette.css"}
best-model/index.html
-1140
View File
File diff suppressed because it is too large Load Diff
docs/SPEC.md
+1912
View File
File diff suppressed because it is too large Load Diff
assets/images/lovelace_perfect_second.jpg → docs/assets/images/lovelace_perfect_second.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 239 KiB

After

Width:  |  Height:  |  Size: 239 KiB

assets/images/pipeline_topology.svg → docs/assets/images/pipeline_topology.svg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 2.8 KiB

After

Width:  |  Height:  |  Size: 2.8 KiB

assets/images/valerian_screen_call.jpg → docs/assets/images/valerian_screen_call.jpg
View File
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 165 KiB

After

Width:  |  Height:  |  Size: 165 KiB

docs/benchmark.md
+344
View File
@@ -0,0 +1,344 @@
# Benchmark — SuperHero
The reference film for end-to-end accuracy. Replaces Road to Bali, which was
withdrawn for the reason in [Why not Road to Bali](#why-not-road-to-bali).
TRACES: AR-011, AR-012, AR-013 | VR-001, VR-005 | SR-002
---
## The film
SuperHero, from the [NIST TRECVID Deep Video Understanding development
set](https://www-nlpir.nist.gov/projects/trecvid/dvu/dvu.development.dataset/).
14 films are asserted Creative Commons and need no data agreement; only the 5
KinoLorber test films are gated.
| | |
|---|---|
| Runtime | 1025.5 s (17.1 min), 10 scenes |
| Resolution | 640×360 |
| Ground truth | Per-scene presence, from the scene knowledge graphs |
| Gallery | 5 characters, 14 references |
The DVU set is what makes this workable: it ships **character** face crops cut
from the film itself, so ground truth and gallery are both in character space
and scoring needs no actor→character mapping.
**Licence caveat.** NIST links licence evidence for only 4 of the 14 films, and
SuperHero is not one of them — its end credits carry no copyright or CC notice,
list a "Temporary Musical Score" and a SAG cast, and it has no traceable online
release. Fine for internal benchmarking; do not redistribute frames from it.
Valkaama is the one film with an independently documented licence (CC BY-SA 3.0)
if provenance ever has to be defended.
---
## Reproducing it
```sh
# 1. Annotations, character mugshots, scene segmentation.
# NIST names the same film three different ways, hence the overrides.
KG_DIR=superHero KG_FILE=superhero scripts/fetch_dvu.sh SuperHero ../dvu-hero
# 2. Scene clips (movie.shots), then fuse them into one stream.
# Fusing matters — see "Run it as one film" below.
# SuperHero-1.webm … SuperHero-10.webm from
# <dataset>/movie.shots/, then:
ffmpeg -f concat -safe 0 -i concat.txt -c copy SuperHero_full.webm
# 3. Gallery, with the face-size floor that keeps references in distribution.
./build/build_gallery --root ../dvu-hero/root \
--output ../dvu-hero/hero66.h5 --min-face-px 66
# 4. Run, on the GPU path (see "Check you are on the GPU").
./build/scene_analyze --movie hero/SuperHero_full.webm \
--gallery ../dvu-hero/hero66.h5 \
--detector-engine trt_cache/scrfd.scrfd_500m_bnkps.640.fp16.engine \
--arcface-engine trt_cache/arcface.LVFace-B_Glint360K.b4.fp16.engine \
--fps 5 --min-face-px 32 --expand-gallery \
--output pred.json
```
Nothing here is in git: the clips are ~130 MB and the annotations are
regenerable. Replay fixtures derived from the run ship through the artifact
registry instead:
```sh
scripts/artifacts/push_artifacts.sh replay-fixtures
scripts/artifacts/pull_artifacts.sh replay-fixtures [version]
```
The gallery travels in the same archive as the dumps deliberately — a dump only
replays meaningfully against the gallery it was produced with, and pairing one
with a different gallery silently changes every identity decision in it.
---
## Results
Measured on the fused film, gallery expansion on.
| Metric | Value |
|---|---|
| Precision | **1.00** |
| Recall | 0.65 |
| F1 | 0.79 |
| True positives | 13 |
| False positives | **0** |
| False negatives | 7 |
Six of ten scenes scored exactly right, including the three-character scenes 4
and 5.
**Zero false positives is the result worth keeping.** Every out-of-gallery
character — Beast, Mighty Celestial, Ms. Johnson, Doctor, two Masked Persons —
was declined rather than forced onto a nearest match. That is the calibrated
probability (AR-024) doing its job, and it is the right failure direction for an
X-Ray overlay: a miss is a gap, an invention is a lie.
**The misses have a shape.** Scenes 1, 2, 3 and 8 were missed, and 1–3 are the
three shortest scenes in the film (14 s, 38 s, 27 s). That is consistent with
per-track Bayesian accumulation (AR-025) needing enough sightings before belief
crosses threshold. Scene 8 is 65 s and does not fit that story — it is the one
to look at first when improving recall.
Running the same scenes as isolated clips did *not* do better, so cross-scene
gallery expansion is not currently compensating for short scenes.
### Run it as one film, not as clips
Per-scene clips defeat per-film gallery expansion (AR-019), which grows a
temporary gallery from track continuity across the whole film and re-assesses
unknown tracks at the end. Ten isolated clips give it nothing to work with, and
pay model and gallery load ten times over.
Fusing also makes presence windows cross real scene boundaries, which is how
SR-002's scene-scoped question is asked in production. Note the joins are
artificial cuts — consecutive scenes were never contiguous footage — so presence
bleeding across a boundary may be the join rather than a tracking fault.
---
## Throughput
| Path | Realtime factor | Sampled fps | 17-min film |
|---|---|---|---|
| `build/` (TensorRT) | **8.25×** | 41.3 | **2.1 min** |
| `build-ort/` (ORT) | 0.54× | 2.7 | ~32 min |
TensorRT figure re-measured 2026-08-04 over the whole film at `--fps 5
--min-face-px 32 --expand-gallery`: 5129 frames, 1025.4 s of film in 124.2 s
wall. Two runs agreed to 0.4% (124.2 s clean, 124.7 s under gdb). It supersedes
an earlier 2.0×; that figure predates the current tree and was not re-derived
here, so treat the gain as measured rather than explained.
Throughput varies strongly with face density, and **a short window is not a
sample of the film**. The opening 60 s benchmarks at 23.9× — decode there costs
4-6 ms/frame against a 12.35 ms whole-film mean (n=510), because seeking forward
in VP8/WebM gets dearer the deeper you go, and there are few faces. Always quote
the whole-film average.
### Where the time goes (VR-015)
Measured over the whole film, 2026-08-04:
| node | cpu_s | % of pipeline CPU | cpu/f | exec/f | stall/f | in% | out% |
|---|---|---|---|---|---|---|---|
| **embedder** | **91.0** | **60%** | 17.74 | 21.61 | 3.87 | 12 | 0 |
| **face_detector** ▶ | 41.4 | 27% | 8.07 | 24.20 | **16.14** | **99** | **0** |
| frame_source | 12.1 | 8% | 2.36 | 11.26 | 8.90 | — | 97 |
| camera_pos | 3.2 | 2% | 0.63 | 0.64 | 0.01 | 97 | 99 |
| face_aligner | 1.7 | 1% | 0.33 | 0.34 | 0.01 | 0 | 12 |
| identity_matcher | 1.3 | 1% | 0.26 | 0.34 | 0.09 | 0 | 0 |
| tracker / sink | 0.5 | <1% | — | — | — | 0 | 0 |
**`face_detector` paces the run**: its input channel is 97.8% full while its
output is 99.4% empty — everything upstream jammed, everything downstream
starved. It occupies 5129 × 24.20 ms ≈ 124.1 s of a 124.2 s run, essentially
100% wall occupancy, yet only 33% of that is CPU. The other 16.14 ms/frame is
device wait.
**The embedder is the larger cost but not the constraint**: 60% of all pipeline
CPU, 73% of wall as thread-busy. Whether that is real work or a spinning
`cudaStreamSynchronize` is unresolved — see the sync caveat below, which is a
one-line experiment.
**`frame_source` is the trap this table exists to defuse.** It reports
`exec/f = 11.26 ms` against `cpu/f = 2.36 ms`, and its output channel is 97%
full: it is backpressured, not expensive. The old KPN `ema` reading made it look
like the most costly node in the pipeline at 141.899 ms/frame.
`--benchmark <path>` writes a per-node timing report and prints a table at
shutdown. `hero/run_bench.sh` is `run_trt.sh` with it switched on:
```bash
./build/scene_analyze … --benchmark $H/bench_trt.json --output $H/pred_bench.json
```
**Do not read the `ema` column of the old KPN diagnostics block as a cost.** KPN
times a node across `fire_once`, which wraps the functor *and* the push to the
next channel, and a push parks when that channel is full (AR-004). A
backpressured node therefore bills its waiting to itself. On this film that
produced a genuinely inverted answer:
```
│ frame_source frames=5132 ema=141.899ms ← reported cost
[frame_source] decode avg=16.6127ms fps=60.19 ← actual decode
```
The source is not expensive; it is idle, holding a frame nobody has taken yet.
Optimising against that number means optimising the fastest node in the graph.
The benchmark report separates the two:
| Column | Meaning | Blind spot |
|---|---|---|
| `cpu_s`, `cpu%tot` | thread CPU time, and this node's share of all of it | a GPU wait looks like idleness |
| `cpu/f` | CPU ms per frame — backpressure cannot inflate it | as above |
| `exec/f` | wall ms per frame in the node, **including parked pushes** | overstates a blocked node |
| `stall/f` | `exec/f − cpu/f`: parked, or waiting on a device | does not say which |
| `in%`, `out%` | mean fill of the node's input and output channels | — |
| `press` | `in% − out%`; **the node marked ▶ is pacing the run** | not a cost, an ordering |
Read `press` first: work queues up in front of the bottleneck and starves
everything after it, so the pacing node is the one with a full input and an empty
output. Then read `cpu%run` to decide the repair — a saturated thread means the
work itself must get cheaper, while an idle thread under pressure means the node
is waiting on the GPU or the disk, where batch size and engine precision are the
knobs and the C++ is not.
Channel fills are sampled every 100 ms (`--benchmark-interval-ms`) because
`current_fill` is instantaneous: by shutdown every channel has drained, so a
single read at the end reports an idle pipeline no matter how congested it was.
#### Check the GPU is not throttled before comparing anything
**On this hardware, thermal state moves the result more than any code change
we are likely to make.** The same binary measured **8.25× cool and 3.12× once
heat-soaked** — a 2.6× swing — because the laptop RTX 3050 hits `SW Thermal
Slowdown` and pins the SM clock to **210 MHz out of 2100**:
```
$ nvidia-smi -q -d PERFORMANCE | grep -E "SW Power Cap|SW Thermal"
SW Power Cap : Active
SW Thermal Slowdown : Active
```
A number recorded without its clock state is not comparable to any other
number, and back-to-back full-film runs guarantee the later ones are throttled.
`run_bench.sh` now records `nvidia-smi` either side of the run into
`bench_gpu.txt`; check it before believing a regression. Let the GPU idle back
to full clock between measurements, and never A/B two runs across a heat-soak.
This one cost real time here: a 2.7× "regression" was attributed to a code
change and reverted on that basis, when the change was innocent and the GPU had
simply warmed up between the two measurements.
#### `cpu_s` on a GPU node is mostly spin — measured
CUDA's default sync policy (`cudaDeviceScheduleAuto`) spin-waits before it
yields, so `cudaStreamSynchronize` charges the *calling thread's* CPU while the
GPU works. A GPU-bound node therefore reports a large `cpu_s` and reads as
CPU-bound.
`SAE_CUDA_BLOCKING_SYNC=1` switches to a blocking wait. Measured over 300 s of
film, four cases, identical otherwise:
| case | realtime | total CPU | embedder CPU |
|---|---|---|---|
| baseline | 3.29× | 103 s | 66 s |
| **`SAE_CUDA_BLOCKING_SYNC=1`** | 3.29× | **23 s** | **4 s** |
| `SAE_CV_THREADS=1` | 3.30× | 101 s | 66 s |
| both | 3.29× | 25 s | 5 s |
**94% of the embedder's CPU was spin, not work**, and 78% of the pipeline's.
Throughput is unchanged, so this is free CPU — which matters for a service
sharing a box (DP-003) and makes `cpu_s` mean what it says. Prefer it for any
run where the CPU numbers are being read.
`SAE_CV_THREADS=1` does nothing measurable: the only OpenCV-heavy node is
`face_aligner` at 1-2% of the pipeline, so the TBB arena is not worth removing
and `warpAffine` is not worth replacing.
**Caveat: measured with the GPU clamped at 210 MHz** (see below). A device at
full clock spends less time in the sync, so the absolute spin figure will fall;
the ranking should not.
#### `cpu_s` counts one thread — mind the TBB arena
OpenCV 5 here is built against TBB, and every OpenCV module links it, so
`cv::parallel_for_` dispatches onto a TBB arena of `nproc − 1` workers (19 on the
20-core dev box; visible as `libtbb.so.12` frames in a thread dump). Since
`CLOCK_THREAD_CPUTIME_ID` is per-thread, work a node fans out that way is billed
to the TBB workers, **not** to the node.
So a node using `warpAffine`, a histogram compare or a colour conversion reads
cheaper in `cpu_s` than it really is, and the missing time appears in `stall/f`,
where it looks identical to a GPU wait. `exec/f` does capture it — the functor
does not return until the parallel region joins — so the tell is a node whose
`exec/f` far exceeds its `cpu/f` **while its output channel is empty**: that is
fan-out, not blocking.
Worth knowing for its own sake, too: 9 KPN node threads plus 19 TBB workers plus
the CUDA and NVDEC threads is heavy oversubscription on 20 cores.
The JSON carries the same data plus the run's configuration, so two runs can be
diffed directly — which is the point, when sweeping `--embed-batch`, `--fps` or
an engine precision.
### Check you are on the GPU
ORT's CUDA execution provider fails to load on this machine and **silently falls
back to CPU**:
```
Failed to load library libonnxruntime_providers_cuda.so:
undefined symbol: cudnnGetConvolutionBackwardDataAlgorithm_v7
```
That symbol was removed in cuDNN 9; the packaged ORT is built against cuDNN 8.
ORT logs this once at startup and then runs happily on CPU, so a `build-ort`
timing is a CPU number wearing a GPU label — a 15× error with no symptom other
than a figure you have no baseline for. Grep the log for `Failed to load
library` before trusting any throughput measurement.
The TensorRT path (`build/`) needs prebuilt engines from
`scripts/build_trt_engines.sh` and reports what it loaded:
```
[TrtScrfd] loaded: … [TrtArcFace] loaded: … max_batch=4
[similarity] cuBLAS/CUDA engine: gallery resident on GPU
```
---
## Why not Road to Bali
Bali was chosen because DVU ships character mugshots for it. It was withdrawn on
**face scale**, measured on its own reference crops:
| | Bali | SuperHero |
|---|---|---|
| Median detected face | 27 px | **69 px** |
| Maximum detected face | 69 px | **241 px** |
| References ≥66 px | 2 of 69 | 14 of 27 |
The DVU images are scene crops, not mugshots, so the crop dimensions say nothing
about face scale — the face has to be detected and measured. Bali's median
reference was being upscaled roughly 4× to reach ArcFace's 112×112, and the
worst 7×, which violates AR-011: every model gets the input it was trained for.
A model run off-distribution returns confident, plausible, wrong output.
In a gallery that error is permanent. A bad frame costs one frame; a poisoned
reference corrupts every future match against that identity.
No threshold rescued it. At 66 px only 2 of 69 references survived — the largest
face in the entire set is 69 px — so there was no cut that both kept references
in distribution and left enough of them to calibrate. SuperHero's gallery builds
at a 66 px floor and calibrates on its own (`a=15.2867 b=-4.98633`, 100 % train
accuracy) rather than borrowing constants.
Any accuracy figure recorded against Bali predates this and should be treated as
measuring upscaling artifacts as much as the pipeline.
docs/best-model.md
+103
View File
@@ -0,0 +1,103 @@
# Which embedding model is best?
Three ArcFace variants (w600k-R50, R18, w600k-MBF) and LVFace-B (Glint360K,
455MB) were compared. r50 is excluded from the training/held-out comparison
below; its gallery has roughly 30% fewer reference images per actor than the
other three on the identical source photos, which confounds a direct score
comparison (see [the full experiment log](model-bakeoff.md) for detail). It
remains in the calibration comparison, which does not depend on the gallery
image count.
## First signal: calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | cosine similarity) =
σ(a·sim + b)`, stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This is a property of the embedding space alone, computed from intra- and
inter-actor reference-image pairs with no tracking or scene logic involved,
so it is a clean first read on discriminative power before running a
benchmark.
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
| model | a (steepness) | boundary at P=0.5 |
|---|---|---|
| LVFace-B Glint360K | 17.7 | sim 0.228 |
| ArcFace w600k-MBF | 16.2 | sim 0.267 |
| ArcFace w600k-R50 | 15.4 | sim 0.301 |
| ArcFace R18 | 15.3 | sim 0.309 |
LVFace has both the steepest transition and the lowest decision boundary,
separating same-actor from different-actor reference pairs more confidently
at a lower similarity than any ArcFace variant.
## Second signal: held-out F1
Each model's own tuned `full_exp` config, replayed against the 5 films the
optimizer never saw and scored the same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on all 5 held-out films; the ranking never flips
between models. Total misID count across the 5 films: LVFace 1032, mbf
2197, r18 1224. LVFace has less than half mbf's misID total and still
scores higher on every film.
Held-out results are stronger evidence than training results, because
training numbers can reflect what the optimizer was tuned to fit rather
than general performance. On training data, the ordering is not as clean:
| film | LVFace F1 | mbf F1 | r18 F1 | best |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
mbf beats LVFace on Lord of War (77.2% vs 75.6%), the only film in either
table where LVFace does not score highest. LVFace's training-set macro
average (75.3%, see [the full experiment log](model-bakeoff.md)) is not a
uniform win across every film it contributes to; the held-out result, where
LVFace wins all 5 films outright, is the stronger claim.
This reverses an earlier, superseded benchmarking pass that used a
scene-union metric and found the three models statistically
indistinguishable (around 85% each), concluding LVFace was not worth its
size. That metric masked out-of-cast false positives behind a
gallery-intersect-cast recall filter; the per-second metric used here does
not.
## Full training-matrix picture
![All 12 combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
Best full-gallery combo per model (all three are `full_exp`), from the
training matrix in [the full experiment log](model-bakeoff.md):
| model | F1 | P | R | misID |
|---|---|---|---|---|
| LVFace-B Glint360K | 75.3% | 89.7% | 65.4% | 232 |
| ArcFace w600k-MBF | 72.0% | 87.7% | 61.4% | 240 |
| ArcFace R18 | 69.1% | 87.6% | 57.7% | 242 |
LVFace leads within both the restricted and full gallery modes, visible
directly in the chart above without reading the table. The three models'
misID counts on the full gallery are nearly identical (232/240/242); LVFace's
lead here is a precision-and-recall lead, not a misID one.
## Operational note
Switching the default embedder is not a config change alone; the gallery
is model-specific, since embeddings from different models are not
comparable. Any existing gallery built against a different model must be
rebuilt from source images before the new default takes effect.
[`scripts/optimizer/reembed_gallery.py`](https://REPOLINK/scripts/optimizer/reembed_gallery.py)
does this from a reference gallery's cached source images without
re-downloading anything.
docs/gallery-scope.md
+69
View File
@@ -0,0 +1,69 @@
# Whole gallery vs. cast-restricted gallery
Two ways to run the matcher. Full mode scores every detected face against
the entire 2418-actor gallery. Restricted mode pre-filters each film's
gallery down to just its Jellyfin-credited cast (typically around 15
top-billed actors) before the matcher runs.
## Result
Averaged across the 3 compared models (r50 excluded, see
[the full experiment log](model-bakeoff.md)) and both expansion settings, on
the 4 training films:
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once, not a precision/recall trade:
+4.8pp F1, +6.0pp recall, roughly a quarter the total misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections.
Every model's best-scoring combo in the training matrix uses the
restricted gallery:
![All combos ranked by training-set F1, filled dots are restricted](assets/images/rep4_matrix_f1.png)
See [the full experiment log](model-bakeoff.md) for the complete table. One
combo reaches zero true out-of-cast misidentifications,
`arcface_w600k_mbf_restricted_exp` (F1 76.2%), and it is a restricted one,
consistent with restriction, not expansion, being what suppresses cross-film
confusions.
The restriction effect (+4.8pp averaged across models) is larger than the
model-choice effect: LVFace beats r18 by 6.2pp in full mode but beats mbf by
3.3pp. Restriction is the single strongest lever in the matrix.
## Why this is not the shipped default
Cast restriction is implemented today only as an offline optimizer
technique
([`scripts/optimizer/cast_restrict.py`](https://REPOLINK/scripts/optimizer/cast_restrict.py)):
it pre-builds a filtered gallery file per film using Jellyfin's cast list
before the benchmark calls the matcher. There is no runtime "restrict to
this title's credited cast" switch in the shipped application;
`scene_analyze` always matches against whatever single gallery file it is
given.
Building this as a real feature requires:
- A live Jellyfin cast lookup at analysis time. The title is already known,
and [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)
already performs this lookup for its own `filter_gallery`-based
restriction path; it is not wired into `scene_analyze` as a first-class
option.
- A decision on the fallback case: what happens to a real, uncredited
cameo (see the Germar Terrell Gardner and Talia Balsam cases in the
[LVFace deep dive](lvface-deep-dive.md#where-lvface-beat-x-ray)) if the
restricted gallery never includes them at all.
- Regenerating the restricted-gallery cache whenever a title's Jellyfin
cast list changes.
The shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp) defaults use
the full-mode winner (`LVFace-B_Glint360K_full_exp`, F1 75.3% training,
67.4% held-out macro) rather than the higher-scoring `restricted_exp`
(78.3%), because 78.3% describes a capability the application does not
have yet.
docs/index.md
+91
View File
@@ -0,0 +1,91 @@
# scene-actor-extraction
A face-recognition pipeline that finds when each actor appears on screen in
a film or TV episode, built on [KPN++](https://gitea.tourolle.paris/dtourolle/KPN)
(a C++20 Kahn Process Network library) for the detect, track, match, and
scene pipeline, with a Jellyfin-integrated gallery and an X-Ray-validated
optimizer.
This is a correctly scored second from a held-out film, one the optimizer
never saw during tuning:
![A perfect X-Ray second: three faces named at 100%, two more correctly carried off-screen](assets/images/lovelace_perfect_second.jpg)
Every visible face is named at 100% confidence (Chris Noth, Hank Azaria,
Bobby Cannavale), the background extra is correctly left unnamed, and the
two credited cast members without a visible face are correctly reported
present but not visible. This matches Amazon X-Ray's own record for this
second exactly.
Results are not uniform across films. The hardest held-out film scores 46%
F1. This report documents why: one tunable trade (extinction bridging at
hard cuts), one structural limit (X-Ray credits people whose faces never
appear on screen), and a small number of cases where the pipeline is
correct and X-Ray's ground truth is not. Read
[how we score against X-Ray](methodology.md) first. X-Ray's ground truth is
scene-level; the pipeline's output is per-second. That difference shapes
every finding below.
## Findings
<div class="grid cards" markdown>
- :material-trophy:{ .lg .middle } **[Which model is best?](best-model.md)**
---
Calibration curves first, independent of any threshold, then held-out
F1 across three models. LVFace-B Glint360K wins both, and wins on every
held-out film.
- :material-filter:{ .lg .middle } **[Whole vs. cast-restricted gallery](gallery-scope.md)**
---
Restricting the matcher to a film's credited cast improves F1,
recall, and misID rate at once, but is not a shipped runtime feature
yet.
- :material-account-convert:{ .lg .middle } **[Does pose expansion help?](pose-expansion.md)**
---
A training-set effect that did not reproduce on 5 held-out films once
two methodology bugs in the comparison harness were found and fixed.
- :material-magnify-expand:{ .lg .middle } **[Deep dive: LVFace-B Glint360K](lvface-deep-dive.md)**
---
The held-out generalization gap, the two mechanisms behind its errors,
and every distinct case where it names someone outside the film's
credited cast.
</div>
## Full experiment log
- **[Full experiment log](model-bakeoff.md)**: the complete log behind the
four pages above, including how replaying against cached embeddings
inside the same KPN network makes a full model and configuration
comparison practical, the full results table, and every caveat. This is
where the shipped [`src/config.hpp`](https://REPOLINK/src/config.hpp)
defaults come from.
- **[Service conversion (proposal)](service-conversion.md)**: design
sketch for a native idle-GPU worker gated on screen lock, not yet built.
## Reproducing the benchmarks
Gallery `.h5` files, embedding dumps, the X-Ray corpus, montage frame
images, and DE trajectories are not committed to this repository. They are
pushed to the Gitea package registry and pulled on demand:
```bash
scripts/artifacts/pull_artifacts.sh galleries
scripts/artifacts/pull_artifacts.sh experiment-data
scripts/artifacts/pull_artifacts.sh montage-frames <film-slug>
```
See [`scripts/artifacts/push_artifacts.sh`](https://REPOLINK/scripts/artifacts/push_artifacts.sh)
for the upload side, which requires a `GITEA_TOKEN` with package write
scope.
docs/lvface-deep-dive.md
+303
View File
@@ -0,0 +1,303 @@
# Deep dive: LVFace-B Glint360K
LVFace won the model comparison (see [Which model is best?](best-model.md))
and is the shipped default embedder. This page reports how it performs in
detail: a baseline of correct output, the two mechanisms behind its errors,
and every distinct case where it names someone who is not in the film's
credited cast.
Read [How we score against X-Ray](methodology.md) first. X-Ray's ground truth
is scene-level, not per-frame. A name marked correct in the Offscreen column
below is the pipeline correctly reporting scene membership, not a workaround.
!!! note "How to read the frames on this page"
The top of each image is the film frame, with a box and name on every
face the pipeline matched to a real detection. The panels below are the
per-second result against X-Ray. **Onscreen** lists names attached to a
visible face this second. **Offscreen** lists names the pipeline reports
present without a currently visible face. Colors mark the verdict:
<span style="color:#0ca30c">**green**</span> correct (TPI),
<span style="color:#eb6834">**orange**</span> wrong (FPI),
<span style="color:#3987e5">**blue**</span> missed (FN).
## Baseline: correctly scored seconds
![Wedding couple correctly identified, Downton Abbey: A New Era](assets/images/downton_wedding_couple.jpg)
Six faces on screen, all six named correctly, including Penelope Wilton at
the edge of the pews and a partly occluded Michelle Dockery. Thirteen more
cast members X-Ray lists as present in the scene are correctly reported
Offscreen. One miss: Maggie Smith (blue). Score for this second: 0.86.
![19 of 20 correct in the funeral crowd](assets/images/downton_funeral_19of20.jpg)
The same film's funeral scene: dark clothing, hats, half the faces turned
away. Nineteen of the twenty cast members X-Ray lists for this scene score
correct: seven named on screen at up to 100% confidence, twelve more reported
correctly as present but not visible.
![Herbie Hancock identified on an in-fiction video call](assets/images/valerian_screen_call.jpg)
The pipeline does not require a live face. This is Herbie Hancock at 98%
confidence, identified from a face displayed on a screen inside the film, on
a video call under a science-fiction HUD overlay.
## Training vs. held-out: the generalization gap
The shipped config (`prob_threshold=0.754`, `anneal_sec=35.54`,
`extinction_sec=57.43`, `expand_gallery=true`) was tuned on 4 films. Scored
on the 5 films the optimizer never saw:
![Held-out per-film F1 vs. the training-set fit](assets/images/holdout_f1_by_film.png)
| film | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| Benny & Joon | 83.0% | 89.1% | 77.7% | 15125 | 1846 | 0 | 4337 |
| Lovelace | 77.5% | 90.3% | 67.9% | 14990 | 1085 | 58 | 7085 |
| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 18663 | 548 | 0 | 12467 |
| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 52027 | 1173 | 0 | 80084 |
| The Many Saints of Newark | 46.3% | 54.7% | 40.1% | 15922 | 4394 | 974 | 23791 |
| macro average | 67.4% | 85.8% | 57.0% | | | | |
The `P` column is misID-weighted (each out-of-film name counts 10x in the
denominator; see [methodology](methodology.md#precision-recall-and-the-misid-weighting)).
That weighting is why Many Saints reads 54.7% here despite naming mostly real,
present faces: its raw (unweighted) precision is **78.4%**, and the gap is
entirely its 974 misIDs paying the 10x penalty. The three zero-misID films
(Benny & Joon, Downton, Valerian) have identical weighted and raw precision;
Lovelace, with 58 misIDs, sits 3pp below its raw 93.3%.
Held-out F1 is 67.4%, against 75.3% on training, an 8pp drop. The spread
between the best and worst held-out film is 37pp. This is not unique to
LVFace: [the full experiment log](model-bakeoff.md#held-out-validation-all-3-models)
shows mbf and r18 with the same shape of spread on the same films, at a
uniformly lower level. Two mechanisms explain the spread. Both are shown
below with frame-level evidence.
## Mechanism 1: extinction bridging
The extinction window keeps a name reported as present for up to
`extinction_sec` after its last real detection. This is deliberate: most
gaps in face visibility are short (a turned head, an occlusion, a cut to a
reaction shot), and the window bridges them.
![Two faces on screen, six more correctly bridged](assets/images/lovelace_polygraph_bridged.jpg)
Lovelace's polygraph scene: only Eric Roberts and Amanda Seyfried have
visible faces. X-Ray lists eight cast members present. All eight score
correct; the other six are reported Offscreen through a stretch where the
camera never shows them. The extinction window is why.
The same mechanism fails at a hard cut into a long stretch with no faces at
all. Downton Abbey's recall (39.4%, the worst of the five held-out films) is
dominated by this failure. It is verified directly against the raw
per-frame stream and the dump's own detection counts, not inferred from the
score. Plotting the dump's per-second `face_count` (detector output,
independent of the tracker) against what the tracker reports, through
Downton Abbey's hard cut into its closing credits:
![Detector vs. tracker through Downton Abbey's cut to credits](assets/images/downton_ghost_timeline.png)
From the cut onward the detector reports zero faces for close to a minute.
The tracker continues reporting the previous shot's 15 identities for the
same span (verified for Hugh Bonneville: bbox `(1743.2, 0.0, 171.3, 317.8)`,
unchanged to the pixel, at every sampled second for 57 seconds). The
staircase at the right edge is the extinction window expiring, actor by
actor. This is `SceneTrackerFunc::active_[actor_idx].last_bbox`
([`src/nodes/scene_tracker_node.hpp`](https://REPOLINK/src/nodes/scene_tracker_node.hpp))
re-emitted as designed. `extinction_sec=57.4` was tuned long because
bridging is correct on most footage, as in the polygraph scene above. The
training films did not contain a faceless stretch long enough to expose the
cost side; the held-out set did.
The extinction window is a scoring concept, not something drawn on screen.
The shipped output is presence windows with no bounding boxes. Even the
debug overlay used for this report never draws a box for a bridged name: a
name inside its extinction window with no current detection appears only as
a name in the Offscreen column, the same as every correctly bridged name
above.
A related, smaller effect shows up at rapid cuts:
![Two labels on one face after a shot/reverse-shot cut](assets/images/cafe_society_rapid_cut.jpg)
Café Society (a training film), a shot/reverse-shot dialog. The box on Steve
Carell's face carries two labels: his own, and Jesse Eisenberg's, left over
from the counter-shot a moment earlier. Both names score correct, because
both actors are present in this scene per X-Ray. The box position is
briefly wrong; the presence claim, which is what the pipeline ships, is
right.
## Mechanism 2: the face-vs-presence ceiling
Downton Abbey's recall did not collapse because faces were misread. It
collapsed because for most of its 80084 false-negative seconds there was no
face to read.
![22 cast credited, nobody facing the camera](assets/images/downton_crew_fn.jpg)
A newsreel crew moves equipment through the hall. X-Ray credits 22 cast
members as present in this scene. None face the camera. Eight still score
correct, carried by presence windows from adjacent shots. The other fourteen
are missed, and no face-recognition system can recover them, because there
is no face in the frame. X-Ray records scene membership; the pipeline
measures visible faces. In ensemble scenes these two quantities diverge, and
that gap accounts for most of the false-negative count.
## Every distinct out-of-cast name
Many Saints of Newark has the largest misID count of any held-out film: 974
seconds, weighted. Rather than characterize this from a single frame, the
raw replay stream was searched directly for every name the pipeline reports
that is not in the film's credited cast. The same search was run on all 9
films in the benchmark, one rule applied uniformly: **find the first second
each distinct out-of-cast name appears, and render that exact second.**
Five films produce no such name anywhere in their runtime: Benny & Joon,
Café Society, Downton Abbey, Sound of Metal, Valerian. Zero out-of-cast
names across their entire length. Four films produce nine distinct names
between them, shown below in full, not a sample.
### The Many Saints of Newark: 4 names
![Germar Terrell Gardner, first out-of-cast name in Many Saints](assets/images/many_saints_fpi_gardner.jpg)
Germar Terrell Gardner, t=848s, 78% confidence. A real, clearly visible
background actor. He is not in X-Ray's cast list for this film, but he is
credited in Jellyfin's independent cast metadata (see
[Where LVFace beat X-Ray](#where-lvface-beat-x-ray) below). This is a
ground-truth gap, not a model error.
![Archie Yates, second out-of-cast name in Many Saints](assets/images/many_saints_fpi_yates.jpg)
Archie Yates, t=2521s, 78% confidence. A real detected face, a genuine
lookalike confusion.
![Zooey Deschanel, third out-of-cast name in Many Saints](assets/images/many_saints_fpi_deschanel.jpg)
Zooey Deschanel, t=2819s, 99% confidence. A real detected face at a dinner
table, high-confidence lookalike confusion.
![Talia Balsam, fourth out-of-cast name in Many Saints](assets/images/many_saints_fpi_balsam.jpg)
Talia Balsam, t=4551s, 93% confidence. A real detected face. Talia Balsam
plays Mrs. Jarecki, a guidance counselor, in this film; she is confirmed
on screen by direct inspection of the frame. She does not appear in X-Ray's
`people.csv` for this title. This is a second ground-truth gap in the same
film, not a model error.
Two of these four names are ground-truth gaps (Gardner, Balsam), not
misidentifications. The other two (Yates, Deschanel) are genuine embedding
errors on real faces.
### Lord of War: 3 names
![David Shumbris, first out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_shumbris.jpg)
David Shumbris, t=418s, 81% confidence. A real face in a dim, low-detail
shot under a train track. A genuine lookalike confusion in poor lighting.
![Ronald Reagan, second out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reagan_photo.jpg)
Ronald Reagan, t=1003s, 100% confidence. This is not a lookalike confusion.
The detected face is a photograph of Reagan appearing within the shot, not a
living actor. The detector and matcher both did their job correctly on the
image content in front of them; the error is that a photograph inside the
scene is not the same thing as an actor present in the scene, and the
pipeline has no way to draw that distinction from a face crop alone.
![Lance Reddick, third out-of-cast name in Lord of War](assets/images/lord_of_war_fpi_reddick.jpg)
Lance Reddick, t=6424s, 78% confidence. A small, distant, low-detail face at
the edge of frame. A marginal, low-confidence lookalike confusion.
### Lovelace: 1 name
![Chloë Sevigny, out-of-cast name in Lovelace](assets/images/lovelace_fpi_sevigny.jpg)
Chloë Sevigny, t=2451s, 100% confidence. Amanda Seyfried's track is real and
well-tracked through most of this shot, but her bbox is frozen at the exact
same coordinates for t=2450 and t=2451, one second where her box stopped
updating from a fresh detection. Only one real face is detected at t=2451
(confirmed against the dump's own per-frame detections), and it is a tight
IoU-1.0 fit under the Chloë Sevigny box, not the Seyfried one. So the green
Seyfried box in this frame is a ghost, re-emitting her last known position
for that one second, and the fresh, wrong detection is Sevigny, landing on
top of it. Not two competing fresh identities on one crop: one ghost and
one fresh misidentification happening to overlap.
### Scarface: 1 name
![Kirstie Alley, out-of-cast name in Scarface](assets/images/scarface_fpi_alley.jpg)
Kirstie Alley, t=2451s, 89% confidence. Al Pacino is correctly identified in
the foreground at 100%; a background face in the same shot is wrongly
labeled Kirstie Alley. (The t=2451s here and the Lovelace Chloë Sevigny case
above landing on the identical second is a genuine coincidence, verified from
each film's raw stream by [`first_fpi_frames.py`](https://REPOLINK/scripts/docs/first_fpi_frames.py),
not a transcription slip, two unrelated films whose *first* out-of-cast name
happens to fall at the same timestamp.)
### Summary of the nine
| film | name | t (s) | confidence | classification |
|---|---|---|---|---|
| Many Saints of Newark | Germar Terrell Gardner | 848 | 78% | ground-truth gap |
| Many Saints of Newark | Archie Yates | 2521 | 78% | lookalike confusion |
| Many Saints of Newark | Zooey Deschanel | 2819 | 99% | lookalike confusion |
| Many Saints of Newark | Talia Balsam | 4551 | 93% | ground-truth gap |
| Lord of War | David Shumbris | 418 | 81% | lookalike confusion |
| Lord of War | Ronald Reagan | 1003 | 100% | photo-in-frame |
| Lord of War | Lance Reddick | 6424 | 78% | lookalike confusion, marginal |
| Lovelace | Chloë Sevigny | 2451 | 100% | lookalike confusion |
| Scarface | Kirstie Alley | 2451 | 89% | lookalike confusion |
Of nine distinct out-of-cast names across four films, two are ground-truth
gaps, one is a photograph misread as a person, and six are genuine
embedding-space confusions on real detected faces. None trace to extinction
bridging: every one of these nine is a fresh detection on a real face crop
at the second it first appears.
## Where LVFace beat X-Ray
Not every name marked wrong is actually wrong.
[`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)
scores strictly against X-Ray, and X-Ray has gaps of its own.
![LVFace correctly identifies Germar Terrell Gardner, uncredited by X-Ray](assets/images/germar_beats_xray.jpg)
Germar Terrell Gardner, the same name from the table above, does not appear
in X-Ray's `people.csv` for The Many Saints of Newark. Jellyfin's
independent cast metadata does credit him for this film (cross-checked
against `experiments/manifests/jellyfin_casts.json` from the
`experiment-data` artifact package, a data source entirely separate from
X-Ray). Talia Balsam is the same case: confirmed on screen, absent from
X-Ray's cast list for this title.
![Robert Patrick, clearly on screen, scored wrong by a ground-truth gap](assets/images/lovelace_robert_patrick_fpi.jpg)
This extends past uncredited background actors. This is Robert Patrick,
top-billed in Lovelace, clearly on screen reading a newspaper, identified at
100%. The frame is scored wrong because X-Ray's people-in-scene list for
this specific scene omits him, despite crediting him elsewhere in the film.
The identification is correct; the ground truth is missing an entry.
X-Ray is a large, convenient ground truth. It is not a complete one. The
misID and FPI counts reported throughout this document include some fixed
amount of noise from gaps in X-Ray itself, in both directions.
## Summary
LVFace wins the model comparison on every held-out film. It correctly names
19 of 20 people in a crowded funeral scene and correctly identifies a face
displayed on a screen inside the film. Its errors resolve into two
mechanisms: extinction bridging, which is correct on most footage and fails
specifically at hard cuts into long faceless stretches, and the
face-versus-presence ceiling, where X-Ray credits scene membership for
people whose faces never appear on screen. Of the nine distinct
out-of-cast identifications found across the benchmark, two trace to gaps in
X-Ray's own cast data, one is a photograph misread as a person, and six are
genuine lookalike confusions on real faces. The held-out generalization gap,
75.3% training to 67.4% held-out, is real and should be treated as the
expected operating point, not the training-set figure.
docs/methodology.md
+134
View File
@@ -0,0 +1,134 @@
# How we score against X-Ray
Every number in this report, every F1 and misID count, comes from one
comparison. The comparison has a mismatch at its core that shapes nearly
every finding in this report: the ground truth is scene-level, the
pipeline's output is per-second, and the two do not mean the same thing.
This page documents that comparison once, so the findings pages can rely on
it without re-explaining it.
## What Amazon X-Ray records
X-Ray ships three tables per film: `scenes.csv` (a list of `[start, end]`
timespans), `people_in_scenes.csv` (which actors are credited in each
scene), and `people.csv` (actor identities). There is no per-frame or
per-second annotation anywhere in X-Ray. A scene might run 45 seconds, and
X-Ray records one cast list for the entire span, not "on screen from
second 12 to second 30."
To compare this against per-second predictions, `second_score.py` expands
every scene into per-second ground truth by copying the whole scene's cast
list onto every second inside it:
```python
for sn, (t0, t1) in spans.items():
cast = scene_cast.get(sn, [])
for t in range(int(t0), int(t1)):
timeline[t] = cast
```
That is the entire mechanism. If X-Ray credits five actors to a 30-second
scene, all five count as ground truth present for all 30 seconds, including
seconds where only one of them is on screen. This is not a simplification
introduced by the pipeline; it is the only reading of X-Ray's data that is
possible, because X-Ray itself does not record anything finer-grained.
## Why an offscreen name can be scored correct
A name listed under Offscreen with a correct (green) label is not the
pipeline guessing or padding its score. It is the pipeline correctly
answering the question X-Ray actually asks: is this actor part of this
scene. It answers that question using a presence window (`[start, end]`,
held open across cuts by `anneal_sec` and `extinction_sec`), which matches
X-Ray's scene-level semantics more closely than a raw per-frame detection
would.
A system that only reported "this actor is visible in this exact frame"
would score worse against X-Ray's scene-level ground truth, producing a
false negative every time the camera cuts away from a character who is
still present in the scene. Not because it is wrong about the world, but
because it would be answering a stricter, different question than the one
X-Ray's data supports. The presence-window design exists specifically to
answer X-Ray's actual question.
## What this resolves and what it does not
This resolves the semantic mismatch between a scene and an instant. It does
not resolve two other limitations, both discussed in the
[LVFace deep dive](lvface-deep-dive.md).
**The face-vs-presence ceiling.** X-Ray credits scene membership regardless
of whether a face is ever visible: background crew, characters shot from
behind, voice-only presence. No amount of bridging recovers a face that
never appears on screen. This is a hard ceiling on recall, not a defect.
**Extinction bridging can overshoot.** The same presence-window mechanism
that correctly answers "still in this scene" during a normal cut can also
bridge across a scene boundary it has no way to detect. A hard cut into a
different scene with no faces, such as closing credits, carries the
previous scene's identities forward until the window expires. This is the
mechanism behind Downton Abbey's recall collapse, documented in the deep
dive.
## Precision, recall, and the misID weighting
Per sampled second `t`:
**TPI** (true positive instances): actors both X-Ray and the pipeline agree
are present.
**FPI** (false positive instances): actors the pipeline reports that are
not in X-Ray's cast for this second. Split into two categories:
- **FPI_incast**: the actor is in the film's cast, just not credited to
this particular scene. A timing or boundary slip.
- **FPI_misid**: the actor is not in the film's cast at all. A genuine
wrong-identity error, weighted 10x in the precision objective, because
naming someone who is not even in the film is a categorically worse
error than a few seconds of scene-boundary slop.
!!! note "Every headline `P` and `F1` is misID-weighted"
The precision reported throughout this report, and therefore the F1
derived from it, puts each `FPI_misid` into the denominator **10 times**
(`precision = TPI / (TPI + FPI_incast + 10·FPI_misid)`,
[`second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
This is deliberate: the whole point is to punish naming an out-of-film
actor far harder than a scene-boundary slip. But it means the `P` column
is not raw precision, and a misID-heavy film's `P` is depressed
super-linearly. `second_score.py` also emits an unweighted `precision_raw`
(always ≥ the weighted `P`); where the gap matters, The Many Saints of
Newark, weighted `P` 54.7% vs. raw 78.4%, the [LVFace deep dive](lvface-deep-dive.md)
reports both. When comparing `P` across films, remember you are comparing a
quantity that penalizes misIDs, not just a hit rate.
**FN** (false negatives): actors X-Ray lists that the pipeline never
reports, counted only for actors who have a gallery reference embedding.
Across the 9-film benchmark, coverage of X-Ray's credited cast ranges from
20% to 79% by film (see
[the full experiment log](model-bakeoff.md#gallery-coverage-per-film)); an
actor with no reference photo can never be recognized regardless of model
quality, and counting them as a miss would penalize gallery coverage, not
recognition accuracy.
Two further numbers are reported alongside F1:
**agreement_rate**: mean per-second Jaccard overlap
(`|Pred ∩ GT| / |Pred ∪ GT|`), partial credit. Naming 2 of 3 present actors
scores 2/3, not 0.
**exact_match_rate**: the fraction of sampled seconds where the pipeline's
named set exactly equals X-Ray's, no partial credit. Far harsher, and
dominated by recall, since any single missed actor zeroes that second.
## Reproduce
```bash
python3 scripts/optimizer/second_score.py \
--pred pred.json --xray experiments/xray/.../<xray_dir> \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5
```
See also [the full experiment log](model-bakeoff.md) for how `pred.json` is
produced, and the [LVFace deep dive](lvface-deep-dive.md) for what these
mechanisms look like frame by frame.
docs/model-bakeoff.md
+346
View File
@@ -0,0 +1,346 @@
# Full experiment log
This page reports how the pipeline performs across three questions: which
embedding model is best, whether restricting the gallery to a film's
credited cast helps, and whether promoting confidently identified poses into
a per-film gallery annex helps. It also documents the replay architecture
that made testing all three questions in one pass practical, and every
caveat needed to trust the numbers.
Read [How we score against X-Ray](methodology.md) first for what F1,
precision, recall, and misID mean in this report. All numbers below use the
per-second metric
([`scripts/optimizer/second_score.py`](https://REPOLINK/scripts/optimizer/second_score.py)).
r50 (ArcFace w600k-R50) is excluded from the detailed comparison below. Its
gallery was built with roughly 30% fewer reference images per actor than the
other three models on the identical source photos (10808 vs 15055 total
embeddings across the same 2418 actors), which confounds any direct
comparison of its scores against the others. It remains in the
[calibration curve comparison](best-model.md#first-signal-calibration-curves),
which does not depend on the training benchmark.
## Why replay makes this affordable
Decoding video and running face detection, alignment, and embedding is the
expensive part of this pipeline. Everything downstream of that (tracking,
identity matching, scene aggregation) is cheap. KPN++'s node/network
structure means those two stages are separate components connected by
typed channels, so the expensive stage can run once per film, cache its
output, and the cheap stage can be re-run against that cache as many times
as needed with different Config values.
`scene_analyze --dump-embeddings out.h5` runs the expensive half once per
film and writes per-frame face detections and embeddings to HDF5
([`scripts/optimizer/SCHEMA.md`](https://REPOLINK/scripts/optimizer/SCHEMA.md)).
[`scripts/optimizer/replay.py`](https://REPOLINK/scripts/optimizer/replay.py)
then re-assembles the real C++ `face_tracker`, `identity_matcher`, and
`scene_tracker` nodes into a Python-driven KPN network and replays a
film's cached embeddings through them, varying `prob_threshold`,
`anneal_sec`, `extinction_sec`, and `expand_gallery` freely. No GPU
inference and no video decode happen during a replay; each one completes
in seconds. This is what makes a 512-evaluation differential-evolution
search per model, per gallery mode, per expansion setting, tractable, and
what made the full held-out validation across three models in this report
possible in one session rather than requiring three full re-encodes of the
benchmark set.
`optimize.py` runs `differential_evolution` over this replay function as its
objective, with DE-level parallelism (multiple candidate configs evaluated
concurrently, each spawning its own replay subprocesses) on top of it. The
practical ceiling on this machine's GPU was 8 concurrent replay processes;
9 silently degraded every score to 0.0% (well-formed output, wrong numbers,
not a crash), so `optimize.py` was run at `REPLAY_WORKERS=4 DE_WORKERS=2`.
## Search space
`popsize=10, maxiter=15` per combo (3 parameters, up to 512 evaluations,
usually stopping earlier on DE's convergence tolerance).
`anneal_sec`/`extinction_sec` bounds were widened from 1-30/1-15 to 1-60/1-60
partway through the sweep. r50's 4 combos finished before the widening and
used the old, narrower bounds; this is one more reason r50 is excluded from
direct comparison here.
## Training films and held-out films
9 films have dumped embeddings across all 4 models. 4 were used for
optimization:
- Café Society (62-cast)
- Lord of War (64-cast)
- Scarface (67-cast)
- Sound of Metal (14-cast)
5 were held out, never seen by any optimizer run:
- Benny & Joon
- Downton Abbey: A New Era
- Lovelace
- The Many Saints of Newark
- Valerian and the City of a Thousand Planets
## Gallery coverage per film
The gallery has reference embeddings for 2418 actors, but coverage of any
given film's credited cast varies widely. This was previously reported as
one flat number (67% of X-Ray cast lacking a reference embedding, averaged
across the whole benchmark); the per-film breakdown is:
| film | cast credited | in gallery | coverage |
|---|---|---|---|
| Lord of War | 64 | 13 | 20.3% |
| Scarface | 67 | 15 | 22.4% |
| The Many Saints of Newark | 48 | 13 | 27.1% |
| Café Society | 62 | 17 | 27.4% |
| Lovelace | 42 | 15 | 35.7% |
| Valerian and the City of a Thousand Planets | 36 | 13 | 36.1% |
| Benny & Joon | 23 | 12 | 52.2% |
| Downton Abbey: A New Era | 36 | 22 | 61.1% |
| Sound of Metal | 14 | 11 | 78.6% |
Two training films (Lord of War, Scarface) have the worst coverage in the
set, 20-22%. Their training-set F1 numbers below are partly capped by
missing references, not purely by model quality. Downton Abbey has 61%
coverage, the second-best in the benchmark, yet the worst held-out recall
of any film (39.4%, LVFace). Its recall problem is not primarily a coverage
problem; it is the extinction-bridging failure documented in the
[LVFace deep dive](lvface-deep-dive.md#mechanism-1-extinction-bridging).
Reproduce with `scripts/docs/gallery_coverage_per_film.py`.
## Training results, 3 models × 2 gallery modes × 2 expansion settings
Ranked by F1. misid = FPI_misid, the count of true wrong-actor
identifications (naming someone not in the film's cast at all), distinct
from FPI, which also includes in-cast timing slips.
Each combo's row is its best **full-coverage** evaluation: the highest-F1 DE
evaluation in which all 4 training films replayed without a timeout (see
[Dropped-film scoring](#a-scoring-bug-worth-recording-dropped-film-evaluations)
below for why this qualifier is load-bearing and not the same as `argmax F1`
over the raw sweep).
| combo | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| LVFace-B_Glint360K_restricted_exp | 78.3% | 91.0% | 68.9% | 42830 | 3782 | 60 | 19492 |
| LVFace-B_Glint360K_restricted_noexp | 76.7% | 91.5% | 66.2% | 41149 | 3400 | 59 | 21173 |
| arcface_w600k_mbf_restricted_exp | 76.2% | 90.0% | 66.2% | 64328 | 7480 | 0 | 33234 |
| arcface_r18_restricted_exp | 75.5% | 87.6% | 66.5% | 41399 | 5666 | 60 | 20923 |
| LVFace-B_Glint360K_full_exp | 75.3% | 89.7% | 65.4% | 47757 | 3407 | 232 | 26966 |
| arcface_w600k_mbf_restricted_noexp | 75.0% | 91.1% | 63.9% | 39752 | 3465 | 60 | 22570 |
| arcface_r18_restricted_noexp | 73.5% | 91.3% | 61.7% | 38299 | 3220 | 60 | 24023 |
| LVFace-B_Glint360K_full_noexp | 72.3% | 88.3% | 61.8% | 40363 | 3503 | 244 | 25850 |
| arcface_w600k_mbf_full_exp | 72.0% | 87.7% | 61.4% | 39875 | 3729 | 240 | 26338 |
| arcface_w600k_mbf_full_noexp | 71.0% | 93.2% | 57.9% | 41699 | 2472 | 56 | 33024 |
| arcface_r18_full_exp | 69.1% | 87.6% | 57.7% | 37342 | 3119 | 242 | 28871 |
| arcface_r18_full_noexp | 66.6% | 91.3% | 53.1% | 34314 | 2362 | 107 | 31899 |
![All combos ranked by training-set F1](assets/images/rep4_matrix_f1.png)
The two clearest patterns: every model's best-scoring combo uses the
restricted gallery, and LVFace leads within both gallery modes. `full_exp`
(the shipped combination) is the best-scoring option that uses only
features the running application currently supports; restriction is not
wired into the application yet (see
[Whole vs. cast-restricted gallery](gallery-scope.md)).
### A scoring bug worth recording: dropped-film evaluations
The numbers above are corrected ones. The raw `rep4_best_*.json` files, and an
earlier version of this table, reported a different `arcface_w600k_mbf_full_noexp`
row: **74.2% F1 at TPI 12645**, a third the TPI of every sibling combo. That was
not a better config; it was an artifact of how the optimizer aggregates.
`optimize.py` builds each candidate's score from only the films whose replay
subprocess returned (`per_film = [m for m in ex.map(_one, films) if m is not
None]`), then **averages** F1/precision/recall and **sums** TPI/FPI/misID over
just those survivors. When a film's replay times out (the sweep ran near the
8-process concurrency ceiling, so this happened intermittently), that film
silently drops from both. A candidate whose hardest film timed out is therefore
scored on an easier subset, and differential evolution, maximizing that score,
will happily converge onto exactly such a candidate. For `mbf_full_noexp` the
reported winner was one of 7 evaluations (out of 512) whose TPI had collapsed to
a partial-film subset; its median-coverage evaluations sit around 51686 TPI.
The fix here was to re-derive each combo's best row from its DE trajectory
(`experiments/trajectories/rep4_*.jsonl`), keeping only evaluations within 30% of
that combo's median TPI (full 4-film coverage) before taking the best F1. This
needs no re-running, the honest best configuration was already in the sweep,
just not the one `argmax F1` selected. Three combos moved: `mbf_full_noexp`
74.2% → **71.0%**, `LVFace_full_noexp` 72.4% → **72.3%** (and its misID, 0 → 244,
was itself a dropped-film artifact), `mbf_restricted_exp` 76.5% → **76.2%**. The
shipped LVFace `full_exp` winner was unaffected, its reported evaluation already
had full coverage (TPI 47757 ≈ median). `experiment_charts.py` applies the same
`clean_best` filter, so every figure on this page matches the corrected table.
The underlying `optimize.py` aggregation is also being fixed so a dropped-film
evaluation can never be selected as a winner again.
### Per-film training breakdown
The 75.3% LVFace training figure is a macro average across 4 films, not a
uniform result:
| film | LVFace F1 | mbf F1 | r18 F1 | best model |
|---|---|---|---|---|
| Café Society | 68.1% | 62.2% | 60.1% | LVFace |
| Lord of War | 75.6% | 77.2% | 75.6% | mbf |
| Scarface | 71.5% | 68.6% | 64.1% | LVFace |
| Sound of Metal | 78.8% | 76.5% | 71.6% | LVFace |
LVFace does not win every training film. mbf scores higher on Lord of War
(77.2% vs 75.6%). LVFace's own training-film range is 68.1% to 78.8%, a
10.7pp spread, smaller than the 37pp spread seen on held-out films but real.
Reproduce with `scripts/docs/run_holdout_all_models.py --films training`.
## Held-out validation, all 3 models
The training matrix above is training-set fit. Each model's own tuned
`full_exp` config was replayed against the 5 held-out films, scored the
same way:
| film | LVFace F1 | mbf F1 | r18 F1 |
|---|---|---|---|
| Benny & Joon | 83.0% | 78.5% | 77.1% |
| Lovelace | 77.5% | 73.7% | 72.2% |
| Valerian and the City of a Thousand Planets | 74.1% | 70.2% | 71.0% |
| Downton Abbey: A New Era | 56.2% | 55.0% | 53.0% |
| The Many Saints of Newark | 46.3% | 44.5% | 42.1% |
| **macro average** | **67.4%** | **64.4%** | **63.1%** |
LVFace scores highest on every one of the 5 held-out films; the ranking
never flips. Total misIDs across the 5 films: LVFace 1032, mbf 2197, r18
1224. LVFace has less than half mbf's misID count while also scoring
higher on every film. This directly confirms the model choice out of
sample; it is not inferred from the training numbers alone. See the
[LVFace deep dive](lvface-deep-dive.md) for frame-level detail on where and
why LVFace still fails on the two worst films. Reproduce with
`scripts/docs/run_holdout_all_models.py`.
## Two effects in isolation: gallery scope and pose expansion
Averaging across the 3 compared models (r50 excluded) isolates each variable
from model choice.
**Gallery scope**, averaged over both expansion settings and all 3 models
(6 evaluations per row):
| scope | F1 | P | R | total misID |
|---|---|---|---|---|
| full | 71.1% | 89.6% | 59.6% | 1121 |
| restricted | 75.9% | 90.4% | 65.6% | 299 |
Restriction improves every metric at once. This is not a precision/recall
trade: +4.8pp F1, +6.0pp recall, and roughly a quarter the misIDs. Fewer
candidates in the matcher's search space means fewer opportunities for a
lookalike false match, and the recall gain shows this does not cost real
detections. Restriction is currently an offline optimizer technique, not a
runtime feature of the application; see
[Whole vs. cast-restricted gallery](gallery-scope.md) for what building it
into the application would require.
**Pose expansion** (promoting a confidently identified track's novel-pose
views into a per-film gallery annex,
[`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)):
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion is a clean win: +1.6pp F1, +3.3pp recall,
misID drops. The annex only competes against the film's own roughly 15-actor
cast, so a new pose of a known actor is unlikely to be confused with someone
else. In full mode, expansion buys +2.1pp F1 and +3.9pp recall but at a real
cost: misID rises from 407 to 714 as the same new-pose view now competes
against the full 2418-actor gallery, where a confidently learned pose is more
likely to match the wrong person. On the full gallery it is a recall-vs-misID
trade, not a free gain. This training-set effect
did not reproduce on held-out data; see
[Does pose expansion help?](pose-expansion.md) for the full held-out test
and the two methodology bugs caught while checking it.
## Calibration curves
Each gallery carries a fitted Platt sigmoid `P(match | sim) = σ(a·sim + b)`,
stored directly in the gallery HDF5
([`src/gallery/gallery_calibration.hpp`](https://REPOLINK/src/gallery/gallery_calibration.hpp)).
This measures discriminative power independent of whatever
`prob_threshold` a given run used:
![Calibrated P(match|similarity) for all four models](assets/images/calibration_curves.png)
LVFace has the steepest curve (`a=17.7` vs 15.3-16.2 for the ArcFace
variants) and the lowest P=0.5 decision boundary (similarity 0.23 vs
0.27-0.31), separating same-actor from different-actor pairs more
confidently at a lower similarity than any ArcFace variant tested,
including r50. Generated by
[`scripts/docs/calibration_chart.py`](https://REPOLINK/scripts/docs/calibration_chart.py).
## Extinction and anneal window search
Every one of the 512 DE evaluations for the winning LVFace `full_exp`
combo, plotted over the `prob_threshold` × `extinction_sec` plane:
![DE search landscape: 512 evaluations over prob_threshold × extinction_sec](assets/images/de_search_landscape.png)
Nearly everything scoring well sits at `extinction_sec` above 50, across a
wide range of thresholds. Short extinction windows are uniformly weaker:
under a strict threshold, there is no good configuration in that region of
the search space. The optimizer converged with `anneal_sec=59.2,
extinction_sec=59.2`, about 99% of the widened 60s bound, which raises an
open question not resolved in this round: does performance keep improving
past 60s, or does it plateau there. Not chased further this pass.
## Caveats
- r50's 4 combos used the older, narrower search bounds (1-30/1-15 instead
of 1-60/1-60) and are further confounded by its thinner gallery. Excluded
from all comparisons above except calibration.
- The shipped defaults use `full_exp` (75.3% training F1), not the
higher-scoring `restricted_exp` (78.3%), because cast restriction is not
a runtime feature of the application yet.
- `expand_gallery` is mode-dependent, not a free win. Averaged across models
on the full gallery it trades misIDs for recall (see the pose-expansion
table). For LVFace specifically, though, `full_exp` beats `full_noexp` on
every axis at once (F1 75.3 vs 72.3, precision 89.7 vs 88.3, recall 65.4 vs
61.8, misID 232 vs 244), so the shipped `full_exp` is a clean choice for
this model, not an F1-vs-safety trade. (An earlier version of this page
reported `full_noexp` at 72.4% with zero misIDs and higher precision, which
made it look like the safer option; that was the dropped-film artifact
described above, not a real property of the config.)
- Switching the default model is an operational change: any gallery built
from a different model's embeddings must be rebuilt before the new
default takes effect.
## Reproduce
```bash
# 4-film training matrix, all 4 models × 2 gallery modes × 2 expansion settings
bash experiments/run_rep4_subprocess.sh
# single combo
SAE_EXPAND=1 REPLAY_WORKERS=4 DE_WORKERS=2 python3 scripts/optimizer/optimize.py \
--manifest experiments/manifests/rep4_LVFace-B_Glint360K_full.json \
--gallery experiments/galleries/gallery_LVFace-B_Glint360K.h5 \
--params prob_threshold:0.5:0.999 anneal_sec:1:60 extinction_sec:1:60 \
--popsize 10 --maxiter 15 --trajectory traj.jsonl --out best.json
# held-out validation, all 3 models, 5 films
python3 scripts/docs/run_holdout_all_models.py --out docs_data/holdout_all_models.json
# per-film training breakdown, all 3 models, 4 films
python3 scripts/docs/run_holdout_all_models.py --films training --out docs_data/training_per_film.json
# gallery coverage per film
python3 scripts/docs/gallery_coverage_per_film.py --out docs_data/gallery_coverage_per_film.json
# regenerate this page's charts from experiments/ artifacts
python3 scripts/docs/experiment_charts.py --out-dir docs/assets/images
# one frame per distinct out-of-cast name across all 9 films (used in the deep dive)
python3 scripts/docs/first_fpi_frames.py
```
See also the session log
[`experiments/SESSION_STATE.md`](https://REPOLINK/experiments/SESSION_STATE.md).
docs/plan.md
+413
View File
@@ -0,0 +1,413 @@
# Implementation plan — per requirement
One entry per requirement that needs work. Requirements marked `Done` in
[`requirements.md`](requirements.md) are omitted.
**Ordering is derived from dependencies, not assigned to phases.** Each entry
lists what it depends on; anything with no unmet dependency is startable. This
replaces the earlier phase-based plan, which encoded ordering assumptions that
stopped being true as the design changed.
Verification for each requirement is specified in
[`requirements.md`](requirements.md) — this document covers *how to build it*,
not how to prove it.
---
## Startable now (no unmet dependencies)
`GR-004` · `IR-004` · `IR-005` · `IR-007` · `IR-008` · `VR-005` · `AR-011` ·
`AR-023` extension · tooling port
These touch disjoint files and can proceed concurrently.
## Blocked on the registry
Everything in `AR-007` … `AR-022` depends on `AR-012`/`AR-013` landing first,
because they all read or write track state. **This group is one coherent
refactor, not parallel work** — splitting it across concurrent efforts produces
incompatible designs in the same files.
---
# Algorithm
## AR-012, AR-013 — TrackRegistry (the spine)
**Depends on:** nothing. **Blocks:** AR-007, AR-008, AR-014 … AR-022.
Everything else in Part A waits on this, so it goes first.
### Ownership: a shared resource, not a node
The registry is **external to the dataflow network**, created in `main` and
handed to each node that needs it as `std::shared_ptr<TrackRegistry>`. Lifetime
is guaranteed by refcount rather than by the "object must outlive the node"
convention, so no ordering assumption exists between network teardown and
registry destruction.
This is idiomatic here: node functors are already constructed outside the network
and passed by reference (`main.cpp:186-207`), and KPN provides `SharedResource<T>`
for state shared across nodes (KPN SPEC §163, §445).
Not a node, because ownership is not a stage in the stream — it is state several
stages read and write, whose final answer is only known when a track dies.
Not inside `TrackGallery`, because that would couple presence to `expand_gallery`,
a switchable feature.
**The registry *is* the tracker's state.** `FaceTrackerFunc` does not keep its own
`tracks_`/`inactive_` maps and mirror them in — it operates on the registry
directly. Two parallel copies could disagree, and every divergence would surface
as wrong presence windows, silently.
### Per-track state
```
Track
first_seen : double set once, at creation
last_seen : optional<double> UNSET while on screen; set to the last
on-screen timestamp when the face is lost
actor : optional<int> set when a posterior crosses the threshold
belief : {actor_idx -> accumulated_logodds} Bayesian, not a tally
embedding : Embedding running directional mean, for association
```
`last_seen` carries the entire liveness state. Unset = on screen; set = went off
at T. No separate missing-frames counter, no expired flag — the optional *is* the
state machine, and it subsumes the current two-pool split (`tracks_` = unset,
`inactive_` = set).
### Lifecycle
```
face detected, no match → new track, first_seen = t, last_seen = unset
actor identified → update belief; set actor when threshold crossed
face lost → last_seen = t_last_on_screen (stays revivable)
face seen again, embedding match → last_seen = unset (same track continues)
tick(t), t - last_seen > timeout → emit to aggregator, DELETE the entry
```
A presence window is `[first_seen, last_seen]`. Nothing else.
**Interior gaps are claimed; the trailing cool-down is not.** A face lost at t₁
and re-acquired at t₂ within the timeout never closed its track, so the actor is
present across `[t₁, t₂]` — correct, since someone briefly occluded or off-camera
has not left the scene. But a track that dies ends at `last_seen`, not at the
moment of death. That asymmetry is what removes the old `extinction_sec`
over-claim.
**Reaping is a handoff, not a deletion into a holding pen.** The dead track goes
to the result aggregator immediately and the registry drops it, so the registry
holds only live tracks and its size is bounded by concurrent on-screen faces.
### Interface
```
TrackRegistry
tick(timestamp) ← FaceTrackerFunc, every frame
candidates() -> span<Track&> → all live tracks
create(timestamp, embedding) -> track_id
mark_seen(track_id, timestamp, embedding) → updates mean, clears last_seen
mark_lost(track_id, last_on_screen_timestamp)
on_vote(track_id, actor_idx, posterior) ← IdentityMatcherFunc
owner(track_id) -> optional<actor_idx> → TrackGallery
on_track_dead : callback(DeadTrack) → ResultSinkFunc
flush() ← at EOF
```
`candidates()` returns **one pool**; `last_seen` tells the caller whether IoU
applies. There is no separate revival path — matching a dormant track is ordinary
inter-frame association.
`tick()` advances the clock so dead tracks are reaped independently of detection
activity; without it a track only dies when some *other* face happens to appear.
### Locking
The tracker mutates registry state across a frame's association pass, so that
pass holds the lock for its duration (a `frame_scope()` handle). Every other
caller's operations must be individually atomic. A single `std::mutex` over the
whole registry is the right start — contention is a few small updates per frame
against per-frame work measured in GPU milliseconds.
Two cases constrain the API:
- `owner()` is a **read-modify-read** in disguise: `TrackGallery` calls it while
`IdentityMatcher` may be voting on the same track. Tally and verdict must be
read under one lock as a snapshot, or a track can be both unowned and owned
within a single promotion decision.
- `on_vote()` arrives downstream of the tracker's `tick()` for the same frame, so
a vote may land after the clock moved on. **Rule: a vote for a known track
always lands on its tally, regardless of clock.** Only reaping is clock-driven.
A vote for an already-reaped track is dropped and **counted** — a nonzero count
means the timeout is shorter than the matcher's lag.
`on_track_dead` fires from inside `tick()` while the frame lock is held, so the
callback must not re-enter the registry. Keep it to a push onto the aggregator's
storage.
## AR-016 — EOF flush
**Depends on:** AR-012.
`flush()` emits every still-live track through the same callback, closing at
`last_seen` if set and the final tick timestamp otherwise. Idempotent, leaving the
registry empty; the sink's `written_.exchange(true)` guard
(`result_sink_node.hpp:66`) shows the shape.
Must run on **every** termination path that produces output. Not SIGTERM during
opportunistic runs (DP-004) — those push no partial result, so there is nothing
to flush.
Without it a film ending mid-shot silently drops its closing cast, which looks
like a recognition miss rather than a bookkeeping bug.
## AR-014, AR-015 — Contradiction rules
**Depends on:** AR-012, AR-025.
| Condition | Meaning | Action |
|---|---|---|
| Belief on one track swaps A → B | `track_id` carried across a viewpoint change onto a different person | Close at `last_seen`, open a new track for B at the swap frame |
| Two **live** tracks owned by one actor | One person split in two, or an identity attached to the wrong track | Treat as a detected cut: reset affected state, re-associate on embedding |
The second makes identity a **third cut detector**, independent of histogram and
TransNetV2, firing where those failed. Detect it via a reverse index
`actor_idx → live track_ids`, so the condition is caught on the update that
causes it rather than by scanning.
Both counted and reported — the rates measure how often tracking is silently
wrong, which nothing currently reveals.
## AR-007, AR-008 — Tracker on one pool
**Depends on:** AR-012, AR-024.
`FaceTrackerFunc` is constructed with the registry and uses it as state; its
`tracks_`/`inactive_` maps and the cross-cut revival branch collapse into one
pool keyed on `last_seen`. Per frame: `tick()`, association over `candidates()`,
then `create`/`mark_seen`/`mark_lost`.
`track_alpha` becomes **frame-dependent** — normal frames use the tuned blend,
frames flagged `is_cut`/`is_scene_boundary` drop toward embedding-only.
## AR-024 — Probability space everywhere
**Depends on:** AR-023. **Blocks:** AR-007, AR-018, AR-021, AR-025.
Cuts across tracker, matcher and expansion, so it lands with the registry work
rather than after it. Retires `track_max_embed_dist`, `cut_revive_sim`,
`expand_novelty_sim`, `expand_track_spread_max`.
Enforcement is a **static grep check** for bare cosine outside a tagged
`EXCEPTION` — a unit test cannot prove absence across a codebase.
## AR-025 — Bayesian accumulation
**Depends on:** AR-023, AR-024.
Log-odds per candidate actor, added per frame. `on_vote()` is an *update*, not an
increment.
**The independence problem must be handled explicitly.** Consecutive frames are
highly correlated; naive accumulation drives the posterior to certainty on what is
effectively one observation. Preferred mitigation: update only on sufficiently
novel observations, reusing the diversity buffer's existing judgement rather than
inventing a second one. The registry should receive already-discounted evidence.
## AR-017 — Claims carry belief and route
**Depends on:** AR-012, AR-025. `DeadTrack` carries posterior plus how it was
identified (live / deferred / pooled).
## AR-018 … AR-021 — Expansion, deferred pass, clustering
**Depends on:** AR-012, AR-024, AR-026.
Ordering within the group: AR-018 (banded store) → AR-019 (annex) → AR-020 (TBI
queue + deferred pass) → AR-021 (clustering).
AR-021 needs the temporal cannot-link constraint from track extents, so it cannot
start before AR-012. The annex must be a **contiguous matrix** with promotions
appended (AR-026), not a list.
**Output timing changes:** the sink can no longer finalise at EOF — the deferred
pass runs after and may add windows (IR-003).
## AR-022 — Unidentified capture
**Depends on:** AR-020. Unidentified = TBI entries surviving the deferred pass.
Context crops opt-in behind `--dump-unidentified-crops`.
## AR-001 … AR-004 — Detection and backpressure
**Depends on:** nothing (AR-002, AR-011); AR-004 blocks AR-003.
- **AR-002** — `min_face_px` stays **40** (VR-013 measured it end to end) but must
be expressed in original resolution rather than decoded-frame space. The value
is already right in `config.hpp`; the change is the coordinate space.
- **AR-011** — feed TransNetV2 at native rate; derive the dedup window from
source fps rather than the hardcoded `0.04 s`.
- **AR-004** — backpressure. `kMaxFaces` (`identity_matcher_node.hpp:133`)
currently **throws**; channel capacities of 16 (`main.cpp:204-207`) were sized
against ≤10 faces/frame. Must block on bytes in flight, not item counts.
- **AR-003** — remove `max_faces`. **Gated on AR-004**, not a follow-up to it.
## AR-026, AR-027 — GEMM and scale
**Depends on:** nothing to start. The annex CPU loop has moved into the GEMM
path: the annex is a contiguous matrix, promotions are appended to the engine's
resident gallery, and the CPU backend now requires OpenBLAS. What is left of
AR-026 is call site 3, the deferred pass — so the rest of AR-026 lands *with*
AR-020 rather than before it.
---
# Gallery
## GR-004 — Model binding — **DONE**
**Depended on:** nothing. Landed before any measurement work, as intended.
Stamp = model basename + SHA-256 of the ONNX, written as the `/embedder` group at
build time (`gallery_builder.cpp`, `sae_gallery.save_gallery_hdf5`) and verified
at load in `scene_analyze`, `scene_preview`, the `sae_kpn` matcher binding,
`replay.py`, `optimize.py` and `movienet_eval.py`. Mismatch is a hard error naming
both sides, with no bypass. Embedding dumps carry the same stamp, since a replay
has no live embedder to check against.
Unstamped legacy galleries **warn loudly and proceed** rather than failing:
unknown is not known-bad, and hard-failing every pre-existing gallery would turn
the check into something people disable. `--require-gallery-stamp` /
`SAE_REQUIRE_GALLERY_STAMP=1` promotes that to a hard error — measurement runs
should set it. `scripts/stamp_gallery.py` re-binds an existing gallery without
re-embedding, so the warning state is cheap to leave.
Cross-model similarities are meaningless but *look* plausible — this fails
silently and expensively, and it would corrupt every measurement taken during the
rest of this work.
## GR-003 — Coverage reporting
**Depends on:** nothing. Surface what calibration already computes and discards
(`kHistBins = 200`): zero-image actors, under-referenced actors, dedup counts,
and the intra/inter PDFs.
## GR-006 … GR-008 — Provenance tiers
**Depends on:** AR-019. Tier per embedding (baked / harvested / confirmed);
harvested persisted but flagged; bell-curve outlier check
(`EXCEPTION: AR-024`).
---
# Integration
## IR-004, IR-005, IR-007, IR-008 — Audio signature
**Depends on:** nothing. **Fully independent — no existing pipeline file is
touched.** Best candidate for concurrent work.
Implement server spec §3 exactly. Audio decode is a second stream from the
already-linked FFmpeg. Media < 120 s: no signature, no offset. Emit and honour
the `v1:` prefix.
The golden-vector fixture is shared with the plugin repo and runs on CPU, so the
one place two implementations must agree bit-for-bit is verifiable in CI.
## IR-001 … IR-003 — Truth file
**Depends on:** AR-017 (belief), AR-020 (output timing).
Windows carry belief and route; `extraction.*` gains `extinction_sec` and
`gallery_scope`; `anneal_sec` removed. All breaking → **one** coordinated
`schema_version` bump with IR-004 (SR-003).
---
# Validation
## VR-005 — Minimum face size study
**Depends on:** nothing. Standalone Python, no C++ contact. **Done** — knee at
24–32 px. It measures the embedder with alignment held perfect, so it bounds the
answer from below rather than setting it; AR-002's floor comes from **VR-013**,
which sweeps input resolution end to end and lands at 40 px.
## VR-013 — Cross-source identification probe
**Depends on:** `sae_embed` exposing `detect()`, `align_face()`, `embed_crop()`
and the gallery calibration — it drives the shipped C++ rather than reimplementing
it, which is what VR-005 could not do.
Gallery from one recording, probes from another, sweeping the probe's **input
resolution before the detector**, so detection and landmark regression degrade
with the frame. `experiments/xsource/`.
**Findings.** Holding 90% of the plateau needs ~50 px end to end against VR-005's
~22 px; `min_face_px` 40 is right and 32 would admit faces in the falling region.
FPI is 0.0% at every scale — resolution loss goes entirely to TBI, never to a
wrong name. The ceiling is **cross-view, not resolution**: everyone matches
themselves within a recording (0.55–0.85) and collapses across two (0.14–0.45),
and only the subject with frontal *gallery* references identified reliably — so
the lever is gallery pose coverage (`docs/pose-expansion.md`), not a better
landmark model. Averaging SCRFD's NMS-discarded landmark estimates lifts
cross-clip TPI 41% → 49% for one forward pass.
**Open.** Four identities and one shoot, so the shape is the result and the
absolute rates are not. Both clips hold all four people, so there is no
out-of-gallery class and the 10×-weighted out-of-cast misID is untested — holding
one identity out of the gallery would fix that.
## VR-014 — Audio-signature offset recovery
**Depends on:** `sae_audio` exposing `compute_signature()` and
`signature_from_mono()` — it drives the shipped C++, as VR-013 does, so the
thing measured is the thing that ships.
`scripts/validation/test_audio_offset.py` over
`tests/fixtures/audio/superhero_offset_200s.flac`: 200 s of public-domain film audio
(the same SuperHero clips the replay fixtures use), long enough for a 120 s
window to slide past the ±600-frame search cap. The slide itself is numpy here
on purpose — matching belongs to the consumer, so writing it out keeps this a
test of the signature rather than of somebody's matcher.
**Findings.** Alignment is a solved problem here: the offset is the nearest frame
in every in-cap trial, worst error **46 ms against a 500 ms budget**, and 46 ms is
the quantisation floor — offsets are whole 92.88 ms frames, so no correct answer
can be worse. The `runtime/2` anchor's factor of two holds through real trimmed
files, and out-of-cap offsets and unrelated content are both declined.
**The score is where the slack is, and it costs a tier rather than accuracy.** It
tracks sub-frame misalignment — 0.94–0.99 near a frame boundary, 0.69–0.73 at
half a frame — so two thirds of correct alignments miss the server's 0.85 `audio`
threshold and land in `loose`. UT-108 measures the fix rather than proposing one:
±1 frame of slack in the score returns all 40 to `audio` (min 0.906) with false
matches unmoved at 0.12–0.16, costing 81 ms of the budget. See
[`SPEC.md`](SPEC.md) IR-004 — the score is normative in the server spec, so the
change is theirs to make.
**Open.** One source, one language, one era of recording. The shape (offset exact,
score set by sub-frame phase) should hold generally, but the absolute scores are
this fixture's.
## VR-001 — Dump audit
**Depends on:** nothing. Read-only investigation: confirm the HDF5 dump preserves
everything needed to reconstruct tracks deterministically, including the
park/revive path. **Prerequisite for the CI strategy**, since T2 replay is how
most of AR-007 … AR-022 is verified.
## VR-006 … VR-009
**Depends on:** their subjects landing. VR-009 (posterior calibration holds)
depends on AR-025 and is what stops the Bayesian accumulation being decoration.
---
# Withdrawn from the old plan
The phase structure, the `--presence-mode {frame,track}` flag, and "Phase 2 —
retune `anneal_sec`/`extinction_sec`". Those constants are withdrawn rather than
retuned; comparison against old behaviour uses recorded reference output instead
of a second live code path.
docs/pose-expansion.md
+110
View File
@@ -0,0 +1,110 @@
# Pose expansion: does promoting new poses mid-film help?
`expand_gallery`
([`src/gallery/track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp))
promotes a confidently identified track's novel-pose reference views into a
per-film, in-memory gallery annex. The idea: once the pipeline is confident
about an identity, a pose it has not seen before (turned head, different
lighting) becomes an extra reference for recognizing that actor again later
in the same film, without touching the baked gallery.
## Training-set signal
Averaged across the 3 compared models (r50 excluded), on the 4 films used
for optimization. These are the corrected, full-coverage figures, see the
[dropped-film note](model-bakeoff.md#a-scoring-bug-worth-recording-dropped-film-evaluations)
in the experiment log for why an earlier version of this table overstated the
full-mode misID jump (209 → 864) that was itself partly a truncation artifact:
| scope | expansion | F1 | R | misID |
|---|---|---|---|---|
| full | off | 70.0% | 57.6% | 407 |
| full | on | 72.1% | 61.5% | 714 |
| restricted | off | 75.1% | 63.9% | 179 |
| restricted | on | 76.7% | 67.2% | 120 |
In restricted mode, expansion looks like a clean win: +1.6pp F1, +3.3pp
recall, lower misID. In full mode it looks like a recall-for-misID trade:
+2.1pp F1, +3.9pp recall, but misID rises from 407 to 714. See
[the full experiment log](model-bakeoff.md) for the per-model breakdown.
This asymmetry motivated the question below: does turning expansion on
change what gets recognized frame by frame, or is the aggregate F1 shift
coming from something else.
## Held-out test
Same model, same tuned config, `expand_gallery` toggled on vs. off, nothing
else changed, full gallery mode, per-second scoring against X-Ray. This
isolates expansion from every other variable that differs between the
training-set rows above.
LVFace-B Glint360K, all 5 held-out films:
| film | F1 (exp) | F1 (noexp) | TPI delta | FN delta |
|---|---|---|---|---|
| Benny & Joon | 83.0% | 83.0% | -2 | +2 |
| Downton Abbey: A New Era | 56.1% | 56.2% | -7 | +7 |
| Lovelace | 77.5% | 77.4% | +33 | -33 |
| The Many Saints of Newark | 46.3% | 46.3% | +2 | -2 |
| Valerian and the City of a Thousand Planets | 74.1% | 74.1% | +2 | -2 |
ArcFace R18, Benny & Joon, r18's own tuned config: F1 77.1% for both, TPI
and FN identical, FPI differs by 2.
Every film, both models tested: F1 differs by 0.1-0.2pp, TPI/FN swings are
in the tens out of tens of thousands. This is noise, not a signal.
Expansion made no measurable difference to per-second on-screen
identification on any held-out film tested.
## Two methodology bugs caught during this check
Getting to the table above required catching two wrong turns, both worth
recording because they are exactly the kind of error that produces a false
positive "expansion helped" finding.
1. **Timeout truncation.** The first Downton Abbey `exp` replay was cut off
by a 60-second subprocess timeout at about 76% through the film (5589 of
7368 expected seconds). This silent data loss produced a large,
convincing-looking TPI gap (47938 vs 52032) purely because one run was
missing a quarter of the film. Caught by comparing `n_seconds` between
runs before trusting any score delta; fixed by re-running with a longer
timeout.
2. **Bbox-matching bug.** An early per-second raw-annotation diff matched
each `exp` detection to the first `noexp` detection with IoU above 0.5,
not the best-overlapping one. With 3 faces close together in frame, this
produced spurious disagreements (for example "exp says Aidan Quinn,
noexp says Johnny Depp" at the same second) that vanished once the match
used the best-IoU candidate instead of the first one. Both configs had
actually output the same three names at the same three boxes.
Both bugs independently pointed toward "expansion is doing something," and
both were artifacts of the comparison harness, not the pipeline. Before
trusting a dramatic before/after diff, check that both runs cover the same
seconds and that entities are matched by best overlap, not first found.
## Conclusion
The training-set aggregate effect, particularly the full-mode misID
increase, does not reproduce on held-out data. At minimum it
is far smaller than the training-set numbers suggested; it may be sampling
variation from only 4 training films rather than a generalizable
mechanism. Note the same *class* of harness bug appears twice in this
investigation, the timeout truncation in bug #1 above, and the dropped-film
aggregation that inflated the raw training-set misID figures. Both make an
inert config look consequential; both are reasons to distrust a dramatic
training-set delta until it survives on held-out films, which this one did
not. This does not mean `expand_gallery` never does anything: the
mechanism is real, and
[`track_gallery.hpp`](https://REPOLINK/src/gallery/track_gallery.hpp)'s
promotion logging confirms tracks get confirmed and views get promoted
into the annex on every film tested. It means whatever effect expansion
has on final per-second identification was too small to detect against 5
held-out films with this scoring method. A cleaner test would need either
more held-out films or a metric that can see the annex's direct
contribution, such as tagging which reference embedding won each match;
neither was in scope for this pass.
Do not treat the training-set exp/noexp numbers in
[the full experiment log](model-bakeoff.md) as proof that expansion changes
real-world behavior in either direction. On the evidence gathered so far,
it does not move the needle enough to see.
docs/requirements.md
+409
View File
@@ -0,0 +1,409 @@
# scene-actor-extraction — requirements register
Stable IDs for every requirement in [`SPEC.md`](SPEC.md), which holds the prose.
This file is the **authoritative list**; the CI gate reads its denominators from
here (see [`../../SPEC.md`](../../SPEC.md) §6).
**IDs are permanent.** A withdrawn requirement is marked `Withdrawn` and its
number is never reused — renumbering is what produces orphan TRACES tags. This
register replaces the earlier thematic `A1…E8` scheme, which had already produced
an `A1a` and an out-of-order `E6`.
Tag code with `// TRACES: AR-012 | SR-002`.
| Type | Scope |
|---|---|
| `AR` | Algorithm — the extraction pipeline itself |
| `DP` | Deployment — how it runs |
| `IR` | Integration — contracts with other components |
| `GR` | Gallery — building and maintaining actor references |
| `VR` | Validation — parameter studies and benchmarks |
| `UT` / `IT` | Unit / integration tests |
Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
---
## Algorithm (AR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| AR-001 | Detect faces in sampled frames; emit bbox, confidence, 5-point landmarks in original pixel space | SR-002 | High | Done |
| AR-002 | Minimum face size **40×40 px** (VR-013 measured end to end; VR-005's 32 px is an embedder-only upper bound), expressed in **original** resolution (decoupled from `dense_scale`) | SR-002 | High | **Done** — `FaceDetectorFunc::drop_undersized()`. The threshold is divided by `bbox_upscale` rather than every box multiplied, which keeps the comparison on the detector's own numbers and means turning `dense_scale` on cannot silently raise the minimum face the pipeline accepts. Verified at the threshold and at `dense_scale` 0.5 (UT-002), and end to end on the fixture (IT-001) — the superhero dump's smallest side is *exactly* its recorded 32 px, so the filter is binding there rather than vacuously satisfied |
| AR-003 | No fixed per-frame face cap — crowd scenes must not lose background cast | SR-002 | Medium | **Done** — `max_faces` defaults to 0 (no cap); the matcher batches through its GEMM buffer instead of throwing |
| AR-004 | Backpressure: unbounded faces/frame absorbed by slowing, never by dropping or throwing | SR-002 | High | **Mostly** — node outputs *park* on a full channel: the value is held, the worker released, and a channel space-callback resumes the node. Replaces `push_blocking`, which parked a scheduler worker inside the push and, with one thread per node, stopped that node draining its own input. Verified: 385/385 frames, 0 drops. Holes closed since, in the order they surfaced: **(a)** `FanoutNode` dropped on overflow rather than waiting, so the AR-010 scene join shed frames exactly when the dense branch fell behind — measured at **9 of 2192 items delivered** to the slower of two branches, now lossless with the fast branch throttled to within its buffering; **(b)** the residual hang, recorded as ~1 run in 20 at a 300 s timeout, was a **startup** lost wake, not a mid-stream one — `start()` enables a node's inputs several statements before it installs the push callback, and a producer firing into that gap is accepted by the ring while waking nobody, since `Channel::push` signals only the empty→non-empty edge. Signature is zero items delivered, never a partial stall. Reproduced 7 times in 24 under CPU contention and 0 in 10 without; `start()` now closes with the level-triggered `on_input_ready()`, giving 0 in 24 on the same harness — though the *cause* was narrower than recorded there and is fixed properly in **(e)**; **(c)** `FilterNode` and `RouterNode` were the last data paths still using the throwing `push()` with the exception swallowed, so a full output discarded the value — including the **EOF sentinel**. The decimator passes EOF by predicate (`if (f.eof) return true;`) but its output is reliably full, the embedder being the slowest node in the chain, so the token was discarded, nothing downstream ever shut down, and the run had to be killed. **This is the wedge.** Both now route sentinels out-of-band and retry data until taken; the regression case delivers 6 of 40 values and never sets `saw_eof` before, 40 and terminating after; **(d)** the sentinel could be delivered *ahead of* a value still queued behind it — `pop()` observed the ring empty and then took the sentinel, and a producer can push a value *and* publish the sentinel inside that window, so any consumer treating EOF as a hard stop loses the tail. `take_sentinel` now re-checks emptiness *after* observing `has_eof_`, which is sound because the sentinel is published with a release store after the ring pushes. ~1 run in 15 before, 0 in 25 after; **(e)** two `fire_once` invocations for one node could overlap, because the submit gate was released before the firing had finished touching node state. That breaks the one-slot park the whole scheme rests on — a parked value can be overwritten by the other firing, with no drop recorded anywhere. ThreadSanitizer caught it as a race on `pending_done_`; the release is now the last act of a firing. The same sweep found the callbacks themselves being written while a running neighbour read them (ten TSan races), which is the *actual* cause of the startup lost wake in **(b)** — callbacks are now installed in a `prepare()` pass before any node starts. **New constraint:** a channel carries at most **one undelivered sentinel**; a second offered before the first is taken is refused and reported, never queued and never overwritten, since two control tokens on one channel means the stream ended twice. Single-shot EOF today, live the moment a pipeline is reused for a second input. **Consequence to hold onto:** a lossless fanout makes join depth a correctness precondition — one branch can now run ahead of another only by the slower branch's buffering, so `kSceneJoinDepth` must exceed the TransNetV2 window. Making the decimator lossless also makes it a backpressure point rather than a relief valve: the source now throttles to the face branch instead of quietly thinning it. Correct under this requirement, but it changes the shape of a loaded run and is **not yet benchmarked**. **Gap:** capacity is still counted in *items*, not bytes, so a crowd frame carrying 60 crops occupies one slot exactly as an empty one does — the memory ceiling the plan asks for is unenforced |
| AR-005 | Align to 112×112 via ArcFace 5-point similarity transform, fitted by **Umeyama least squares over all five points** (as InsightFace does) — never a robust fit, which would discard the landmarks AR-030 reads | SR-002 | High | **Done** — `umeyama_similarity()`. The RANSAC fit it replaces disagreed by a median 17 source px on 400 headshots, 83.5% of crops embedding below cos 0.99, and was unstable and RNG-driven: rebuilding caught 1614 near-duplicates against the original build's ~100. **All galleries rebuilt** (2456 actors, 10254 embeddings); measured separation gain is small (0.583 → 0.590), so recorded accuracy figures should be re-run but are not expected to move far |
| AR-006 | 512-d L2-normalised embeddings, batched | SR-002 | High | Done |
| AR-007 | Associate detections by IoU + embedding, with **frame-dependent** weighting | SR-002 | High | **Done** — `track_alpha` is the base for ordinary frames; drops to embedding-only on cut/boundary and for dormant tracks |
| AR-008 | One track pool keyed on `last_seen`; no separate revival path | SR-002 | High | **Done** — one pool keyed on `last_seen`; park/revive branch deleted |
| AR-009 | Camera-cut detection (histogram) as an association hint | SR-002 | High | Done |
| AR-010 | Scene-boundary detection (TransNetV2) as an association hint | SR-002 | Medium | **Done** — decode butterfly joined via `SceneBoundaries`; the sampled branch waits for the detector's watermark. Frames past its last scored window are counted as unverified, never assumed boundary-free |
| AR-011 | **Every model is fed the input it was trained for** — cost reduced by running less often, never by degrading one inference | SR-002 | High | **Done** — both violations SPEC.md named are closed. (1) `scene_decode_fps` defaults to 0 (native): at 12 fps a 100-frame `kWindow` spanned ~8.3 s instead of the ~4 s TransNetV2 was trained on, half-speed motion over twice its temporal context. (2) The boundary dedup window is derived from the cadence the detector was actually fed (`SceneDetectorFunc::dedup_window_sec()`, median observed interval, halved) rather than the literal 0.04 s — one frame at 25 fps, and at 30 fps wider than a frame, so two cuts on consecutive frames merged into one and the loss was invisible: the file simply had fewer boundaries. Derivation checked at 24/25/30 fps and under a seek (UT-003). **Consequence, not a gap:** `scene_threshold` 0.60 was fitted against the 12 fps input and is now certainly wrong — VR-006 re-fits it, and until then boundary recall at native rate is untuned rather than better. Dense decode is the cost driver, so this is not free; `dense_scale` and `scene_stride` remain the reductions that do not run the model off-distribution. **Half-applied until now:** the derived window reached `scenes.json` and nothing else. `SceneBoundaries` — the path that actually feeds `is_scene_boundary` to the tracker — kept the literal 0.04 s under a comment claiming the two views agreed. They did not. The detector now supplies the window it derived to both |
| AR-012 | Presence follows **track extent**, not per-frame recognition | **SR-002** | High | **Done** — `src/track_registry.hpp`; window is `[first_seen, last_seen]` of an owned track |
| AR-013 | `last_seen` optional state machine; window ends at last sighting, never after | SR-002 | High | **Done** — `last_seen` optional is the whole state machine; interior gaps absorbed, trailing cool-down never claimed |
| AR-014 | Belief swap A→B terminates the track and starts a new one | SR-002 | Medium | **Done** — swap closes at `last_seen` and opens a successor at the swap frame; counted |
| AR-015 | Two live tracks owned by one actor ⇒ treat as a detected cut, re-associate | SR-002 | Medium | **Done** — reverse index detects it on the causing update; counted |
| AR-016 | All tracks closed at EOF — a film ends with faces on screen | SR-002 | High | **Done** — `flush()`, idempotent, closes at last sighting or final tick |
| AR-017 | Every presence claim carries its belief and identification route | SR-002 | High | **Done** — `DeadTrack` carries belief, observation count, and now a `route` enum. The route was previously the literal string `"live"` written at serialisation time, so the published field could not distinguish anything and AR-017's own edge case ("deferred and pooled routes distinguishable") was unmeetable. Only `live` occurs until AR-020 lands; `deferred` exists so that pass has somewhere to write instead of a schema change to make |
| AR-018 | Per-subject embedding store with banded admission (novel enough, safe enough) | SR-005 | Medium | **Done** — banded admission in probability space, bounds from `expand_band_lo/hi`; the lower bound re-asked pairwise at promotion, since `admit` compares only against the nearest member and a drifting track can chain past it. Retires `expand_novelty_sim` and `expand_track_spread_max` — novelty is now the eviction ordering, not a threshold. Rejections counted. Bounds unswept (VR-007) |
| AR-019 | Per-film gallery annex from owned tracks; acquires the non-frontal views TMDB lacks | SR-005 | Medium | **Done** — all three discontinuity signals clear the buffers; ownership comes from the registry, not a second local tally. **Correction:** the local tally was still there and still deciding. Promotion fired on a local accepted-frame count and fell back to a local per-actor plurality whenever the registry had not yet claimed the track — which is the common case, since three accepted frames arrive well before a posterior crosses `ownership_logodds`. So in practice the plurality usually decided, and it could not see the AR-025 discounting it was supposed to defer to. Promotion now requires the registry's verdict; the accepted-frame count is an explicit evidence floor. `forget()`, which had no callers under a comment claiming the matcher called it, is replaced by `prune_dead` against the registry's own liveness |
| AR-020 | Deferred re-identification of unknown tracks against the final expanded gallery | SR-005 | High | Planned |
| AR-021 | Cluster unknown tracks into one entity per person, under temporal cannot-link constraints | SR-005 | Medium | Planned |
| AR-022 | Capture still-unidentified tracks: embeddings, metadata, **context crops** for human review | §4 | Medium | Planned |
| AR-023 | Fit sigmoid calibration from intra/inter similarity distributions | SR-002 | High | **Done** — and the meaning of "the fit failed" is now uniform. `valid=false` used to send the matcher to a raw-cosine accept rule while `same_person_probability` sent every other stage to the untuned default sigmoid: one run, two policies, no announcement. Both now take the default sigmoid and warn loudly that the probabilities are not meaningful |
| AR-024 | **Always the calibrated probability, never a raw cosine** — exceptions recorded | SR-002 | High | **Done** — association, accumulation and expansion all in probability space; `track_max_embed_dist`, `cut_revive_sim`, `expand_novelty_sim`, `expand_track_spread_max` retired. Enforcement now exists rather than being asserted: `scripts/ci/check_raw_cosine.py` blocks in CI. It immediately caught a live violation — the matcher's no-calibration fallback thresholded raw cosine distance **and fed `max(0, cosine)` into `TrackRegistry::observe`**, whose contract says in terms that it cannot be handed an uncalibrated number by a careless caller. `match_threshold`, `match_ratio` and `match_ratio_ceil` are retired with it, and `TrackGallery`'s `max(0, cosine)` default calibration is now a hard error. One exception recorded, in the calibration's own dedup |
| AR-025 | Per-track Bayesian accumulation in log-odds, with correlated-observation discounting | SR-002 | High | **Done** — log-odds accumulation with correlation discounting owned by the registry, `src/evidence_discount.hpp`. The four constants governing this — `ownership_logodds`, `rho_max`, `admit_below`, `max_views` — were unreachable in-class defaults until now; see VR-007 |
| AR-026 | All similarity computed as GEMM, including annex and deferred pass | SR-001 | High | **In Progress** — two of the three call sites done. Baked gallery was already GEMM; the annex now is too — it is a contiguous row-major matrix (`track_gallery.hpp`) whose promoted rows are appended to the engine's resident matrix (`ISimilarityEngine::append_rows`), so one multiply covers baked and promoted references and the host-side cosine loop is gone. CPU path requires OpenBLAS (scalar fallback now opt-in behind `SAE_ALLOW_SCALAR_GEMM`). Remaining: the deferred pass, which does not exist until AR-020 |
| AR-027 | Throughput acceptable for **arbitrary** gallery size | SR-001 | High | Planned |
| AR-028 | **Embedding input quality assessed and carried** — every face scored on size, sharpness and visibility before its embedding is used as identity evidence; the vector travels with the face and reaches the VR-001 dump | SR-002 | High | **Done** — filled in by `FaceAlignerFunc`, where both measured axes come free from the warp; carried on `DetectedFace` and written to the dump as `faces/sharpness` + `faces/alignment_residual`, taking it to `schema_version` 2. Size is `bbox`, not duplicated into a field that would drift. No face is admitted unscored (-1 sentinel), and the degenerate-fit case is now counted and reported rather than silently dropped. **Carried, not consumed** — no discount and no threshold, which is AR-030 and VR-012. Verified UT-137, UT-138 (aligner) and UT-139…UT-141 (dump round-trip, version, sentinel). The committed fixtures are still v1, so they carry no vector until `make_fixtures.sh` is re-run on a GPU host |
| AR-029 | Sharpness measure on the **aligned crop** (scale-normalised, so it cannot re-measure size) | SR-002 | Medium | **Done** — `crop_sharpness()`: variance of the Laplacian over variance of the crop, so contrast cannot leak in the way it does for the raw textbook measure. Both blur ladders monotone, Gaussian and motion. Three properties recorded on the function for VR-012 rather than corrected here: the contrast invariance is exact in the algebra but bends at the 8-bit quantisation floor (a dim *and* soft crop reads sharper than it is — 148% high at σ 2.5), `BORDER_CONSTANT` fill from a frame-edge face adds a step edge, and the measure conflates focus with intrinsic texture. Verified UT-130…UT-136 |
| AR-030 | Visibility measure from the AR-001 5-point landmarks — extreme pose or occlusion **discounts the observation, never deletes the detection** | SR-002 | Medium | **In Progress** — measure is the AR-005 alignment residual (`estimate_alignment()`), carried on `DetectedFace`; roll/scale invariance and monotonicity under foreshortening asserted. Nothing consumes it as a discount yet |
## Deployment (DP)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| DP-001 | One analysis core; modes are front-ends and must not fork pipeline logic | PR-004 | High | **Done, after a repair.** `scene_preview` had forked the construction sequence and then rotted: it built `FaceTrackerFunc{cfg}` against a signature that stopped existing with the AR-007/AR-008 redesign, so **it had not compiled since**, and it never wired registry claims into its sink. It now mirrors `main.cpp` exactly — matcher, then registry, then tracker. The lesson is that "must not fork" needs the build to notice; a front-end nothing compiles is a fork that rots in silence |
| DP-002 | Batch CLI over one title | PR-004 | High | Done |
| DP-003 | On-demand resident service with bounded, observable queue | PR-004 | Medium | Planned |
| DP-004 | Opportunistic/idle mode: external trigger, hard stop, implicit re-queue | PR-004 | Medium | Planned |
| DP-005 | Native installer, no Docker; Fedora + Arch | PR-004 | Medium | Planned |
| DP-006 | Background incremental gallery refresh on a timer | PR-003 | Medium | Planned |
| DP-007 | CI builder image, CPU-only, pinned by tag in the Gitea container registry | PR-004 | High | **Mostly** — image and publish script exist (`Dockerfile.builder-cpu`, `scripts/ci/build_builder_image.sh`) and `.gitea/workflows/unit-tests.yml` now consumes it, pinned to `v1` and asserting at run time that the image reports that tag. **Gap:** the image is built and pushed by hand from an authenticated host; nothing rebuilds it on a change to the Dockerfile |
| DP-008 | Builder images + release jobs per backend (cpu / cuda / rocm); ship binaries, not engines | PR-004 | Medium | Planned |
## Integration (IR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| IR-001 | Emit the JRay truth format as sibling `.jray.json` | SR-003 | High | Done |
| IR-002 | Windows carry belief + route; `extraction.*` carries `extinction_sec`, `gallery_scope` | SR-003 | High | **Done** — `schema_version: 2`; windows are objects with `belief` + `route`; `extraction.*` carries `extinction_sec` and `gallery_scope`; `anneal_sec` removed |
| IR-003 | Output written **after** the deferred pass, not at EOF | SR-003 | High | **In Progress** — sink builds windows from registry claims and flushes at EOF; the deferred pass (AR-020) does not exist yet, so output is still final at EOF |
| IR-004 | Compute the audio signature exactly per server spec §3 | SR-003 | Medium | **Done** — `src/audio_signature.*`; not yet emitted into the truth file (IR-002). One real defect found and fixed since: the resampler's `AVChannelLayout`s were not zero-initialised, and `av_channel_layout_copy` uninitialises its destination first, so `av_freep` was handed stack garbage. It aborted about 1 run in 4 of UT-103 — invisible in the aggregate test binary, where the case usually passes, and absent under a sanitizer build because it is stack-dependent. `ctest`, one process per case, is what turned it into a reproducible failure |
| IR-005 | Golden-vector fixture shared with the plugin repo to prove bit-exactness | SR-003 | High | **Done** — `tests/fixtures/audio/`; v1 parameters now normative in server spec §3 |
| IR-007 | Media < 120 s: emit no signature, apply no sync offset — identical rule in both producers | SR-003 | Low | **Done** |
| IR-008 | Emit and honour the signature's own `v1:` version prefix | SR-003 | Low | **Done** |
| IR-006 | Jellyfin round-trip: pull pending queue, push complete results only | SR-001 | High | Done |
## Gallery (GR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| GR-001 | Build gallery from Jellyfin library cast, TMDB profile fallback | SR-001, SR-005 | High | Done |
| GR-002 | Incremental `--merge` refresh without re-embedding known actors | PR-003 | High | Done |
| GR-003 | Report coverage: zero-image actors, under-referenced actors, dedup, calibration PDFs | SR-001 | Medium | **Done** — `gallery/gallery_report.hpp`, written next to the gallery by `build_gallery`. Zero-usable-image actors come from the build audit, which a stored gallery cannot reconstruct; also `distinct_references`, `duplicates_removed`, and the intra/inter distributions the calibration fits and would otherwise discard |
| GR-004 | Stamp embedder identity into the gallery; **hard startup error** on mismatch | SR-001 | High | **Done** — basename + SHA-256 + `embed_dim`; mismatch fatal with no bypass, unstamped warns unless `--require-gallery-stamp`; `scripts/stamp_gallery.py` migrates in place |
| GR-005 | Gallery data never leaves the instance | **SR-005** | High | Done |
| GR-006 | Provenance tiers: baked / harvested / confirmed, distinguishable per embedding | SR-005 | High | Planned |
| GR-007 | Persist harvested embeddings **flagged and reviewable**, never silently equal to baked | SR-005 | Medium | Planned |
| GR-008 | Flag distributional outliers among an actor's references (poisoning guard) — `EXCEPTION: AR-024` | SR-005 | Medium | Planned |
| GR-009 | Human-confirmed associations persist and improve future extractions | §4 | Medium | TBD |
## Validation (VR)
| ID | Requirement | Traces to | Priority | Status |
|---|---|---|---|---|
| VR-001 | HDF5 post-inference dump at the embedded-frame boundary | PR-002 | High | Done |
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | **Done** — including the sink, as of VR-011. Worth recording what the reimplementation was hiding: `build_minimal` rebuilt windows in Python from per-frame annotations, which never consult the registry, so it kept producing plausible output while registry-based presence in replay was returning **nothing at all**. The first run of the real chain emitted 0 actors on a film where 1647 frames carried an identified face. A reimplementation does not merely risk disagreeing with the pipeline; it can conceal the pipeline being broken |
| VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done |
| VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done |
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | **Done** — knee at 24–32 px; 32 px gives 98.1% TPI, 0.0 FPI at every size. Degrades an already-aligned 112×112 crop, so it isolates the embedder and is an **upper bound**; VR-013 measures the same question end to end and AR-002 takes its number, not this one |
| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | **Medium** | **Planned, now unblocked** — native-rate decode landed with AR-011, so the prerequisite is met and the current 0.60 is a value fitted against input the pipeline no longer produces. Raised from Low for that reason: it is no longer a refinement, it is a stale constant |
| VR-007 | Expansion band, clustering threshold, deferred-pass ablation, **and the AR-025 accumulation knobs** | PR-002 | Medium | **Planned — scope corrected.** `rho_max`'s own comment already deferred to this row, and four constants it names were unreachable: `ownership_logodds` on `TrackRegistry::Config`, and `max_views`/`admit_below`/`rho_max` on `EvidenceDiscounter::Config`, which `main` built through the one-argument constructor. No sweep could vary them. They are in `Config` with CLI flags now, so this row can be run. `ownership_logodds` is the one to start with: below it a track makes **no presence claim at all**, so it decides whether an actor is reported rather than how confidently |
| VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned |
| VR-009 | Verify accumulated posteriors are calibrated against held-out tracks | PR-002 | High | Planned |
| VR-010 | Dump provenance attributes — embedder model, detector settings, `dense_scale`, `scene_detect`, sample rate | PR-002 | **High** | **Done** — `DumpProvenance` in `embedding_dump_node.hpp`, written as root attributes and read back tolerantly. Every field is optional so a pre-VR-010 dump reads as *unknown* rather than as a default; a silently-defaulted `detector_conf` is exactly the fabricated provenance this exists to prevent. This row said `Planned` while five VR-010 tags sat in the code — stale in the opposite direction to the rest of this register |
| VR-011 | Rewrite the replay harness for the post-AR-012 output contract | PR-002 | High | **Done** — `sae_kpn` compiles again and the replay drives the whole chain including `ResultSinkFunc`, so presence comes from `TrackRegistry` claims rather than being rebuilt in Python. The three per-node factories are replaced by one `add_pipeline` that mirrors `main.cpp`'s construction order — the ordering constraint (matcher fits the calibration, registry needs a discounter from it, tracker needs both, sink needs the claims) is what a factory-per-node API could not express, and is why the tracker factory kept building `FaceTrackerFunc{cfg}` against a signature that had stopped existing. `build_minimal` and `anneal_sec` are gone. Verified end to end on the SuperHero fixture: 5 actors, 32 windows, 0 dropped votes |
| VR-012 | Quality-knee study — TPI/FPI vs sharpness and vs pose, as VR-005 did for size; also settles whether the 5-point pose proxy needs a dedicated landmark model | PR-002 | Medium | Planned |
| VR-014 | Audio-signature **offset recovery on real content** — a known trim recovered from film audio, not from the synthetic golden tone | PR-002 | Medium | **Done** — 40 random in-cap offsets, every one recovered to the nearest frame: **worst error 46 ms against a 500 ms budget**, and 46 ms is the floor rather than a result, since the offset is quantised to whole 92.88 ms frames. The `runtime/2` anchor confirmed through real head-trimmed files (a `delta` trim moves the window by `delta/2`). The one soft spot is **tier labelling, not accuracy**: the score falls with sub-frame misalignment (0.94–0.99 near a frame boundary, 0.69–0.73 at half a frame), so 27/40 correct alignments were demoted to `loose`. ±1 frame of slack in the *score* fixes it — measured, all 40 back to `audio` (min 0.906), false matches unmoved at 0.12–0.16, costing 81 ms of the budget |
| VR-015 | Per-node cost and bottleneck attribution for a run — where the time actually goes | PR-004 | High | **Done** — `--benchmark <path>` on `scene_analyze`; `src/benchmark.hpp`. Reports cumulative CPU and wall time per node, and locates the pacing node from sampled channel occupancy rather than from time-in-node, which backpressure inflates. Verified UT-120…UT-124 |
| VR-013 | Cross-source identification probe — gallery from one recording, probes from another, swept over input resolution end to end | PR-002 | Medium | **In Progress** — holding 90% of the plateau needs ~50 px end to end against VR-005's ~22 px, the gap being detection and landmark error; **`min_face_px` 40, since 32 admits faces in the falling region** (AR-002). FPI 0.0% at every scale. Ceiling is cross-view, not resolution |
| VR-016 | **Cut-detection cadence study** — is `cut_threshold` 0.70 meaningful at the rate `camera_pos` is actually fed? | PR-002 | Medium | **Planned.** The histogram cut detector is the one always-on signal with no recorded provenance, and its input rate is not the rate it was fitted at. With `--scene-detect` off, `camera_pos` sits downstream of a source already decimated to `sample_fps`, so at the 1.0 default it compares frames **one second apart** — inside a single shot those differ enormously, and 0.70 correlation is a low bar to clear. With `--scene-detect` on it sees native-rate frames instead, so the same constant means two different things depending on an unrelated flag. This is AR-011's argument ("every model gets the input it was trained for") applied to a non-neural detector, and it matters because `is_cut` drives `track_alpha` to 0 and clears every expansion buffer. Cheap first measurement: run `camera_pos` over a `hero/` clip at 1/2/5 fps and compare cut counts against `tests/fixtures/dumps/scene_bounds.json`. The committed 5 fps dump shows 2.6% of frames flagged; nobody has measured 1 fps |
| VR-017 | **Vote-lag study** — how often does the matcher fall more than `track_extinction_sec` behind the tracker on real content? | PR-002 | **High** | **Planned.** Channel depth is a correctness parameter between `face_tracker` and `identity_matcher`, and the constraint runs opposite to the scene join's: there `kSceneJoinDepth` must EXCEED the TransNetV2 window, here the depth must be UNDER `track_extinction_sec × sample_fps`. Backpressure is what makes it bite — it is working, and a lossless channel converts depth into lag by design. Both nodes are 16 deep in `main.cpp`, which at the default `sample_fps` 1.0 is ~16 s of lag against a 5 s window, so `scene_analyze` can drop identity votes and until now said nothing. It now reports `dropped_votes` at shutdown; this row is the measurement that decides whether that should be fatal, and whether the right fix is bounding the depth or removing the coupling (reap on the matcher's clock rather than the tracker's, so a vote cannot be late by construction) |
---
## Verification strategy
**CI runs on an Intel N100 with no discrete GPU.** That is a hard constraint on
how each requirement can be verified, and it shapes the test design rather than
merely limiting it.
Four tiers, in decreasing order of preference:
| Tier | Runs in CI | What it covers |
|---|---|---|
| **T1 — Functor unit** | Yes | A KPN node's `operator()` driven directly with hand-built inputs |
| **T2 — Replay** | Yes | The composed pipeline driven from an HDF5 fixture — no GPU, no video |
| **T3 — CPU inference** | Yes, slowly | ORT CPU provider over a handful of frames; smoke tests only |
| **T4 — GPU** | **No** | Throughput, TRT engines, large-gallery GEMM |
### T1 is the primary tier, and KPN is why
**Node functors are plain callable structs, constructed independently of the
network that wraps them** (`main.cpp:186-207` builds them as stack objects;
`ObjectNode` merely adapts them). So a node is testable by constructing it and
calling `operator()` — no channels, no threads, no network, no fixture.
This is already the established pattern, not a proposal:
`tests/test_face_tracker.cpp` "drives the node's `operator()` with hand-built
`EmbeddedSceneFrame`s and inspects the emitted `track_ids`", and does so
"pure, GPU-free, model-free".
The consequence is that most of the redesign is verifiable **without any
fixture at all**: construct exactly the awkward state — a belief swap, two live
tracks converging on one actor, a film ending mid-track, a gap one frame under
the timeout — rather than hunting for a clip that happens to exhibit it.
Four hazards this removes outright:
- **No fixture-provenance risk** for these tests — the inputs are synthetic and
explicit.
- **No "fixture must be replayed from frame 0"** concern — state is constructed
directly.
- **No cross-test state leakage** (e.g. a tracker's `next_id_` persisting) — each
test constructs a fresh functor.
- **No replay-harness nondeterminism** — no channels, so no EOF-tail heuristics
or silent drops.
It also means **a dead upstream producer does not block testing a downstream
consumer.** `is_scene_boundary` currently has no producer (see AR-010), which
would make a *replay* test of the frame-dependent `track_alpha` pass vacuously —
but a T1 test simply constructs a frame with `is_scene_boundary = true` and
asserts the weighting changes. The producer gap is a pipeline defect to fix, not
a verification blocker.
### T2 covers what T1 cannot
Replay remains necessary for **composition** — that the nodes wired together
behave as the sum of their parts — and for realistic data at scale, which
synthetic inputs cannot honestly imitate. It is the tier that would catch a
wiring error, a channel-capacity problem, or an ordering assumption that only
appears under concurrency.
The HDF5 dump (VR-001) captures state after decode → detect → align → embed, so
replay needs no GPU and no video. That was built for the optimizer; it doubles as
CI, which is a strong argument for keeping the schema honest and for replay
driving the *real* nodes rather than a reimplementation (VR-002).
**Fixtures and studies are generated locally**, on the development machine where
the models, galleries and media already exist. CI consumes them; it never
produces them.
**Small committed fixtures are required.** A few HDF5 dumps covering the awkward
cases — a cut, a belief swap, two live tracks converging, a film ending
mid-track, an unknown track that only resolves after expansion — are worth more
than a large corpus, and they are small enough to commit.
**T4 requirements cannot pass in CI, and the gate must not pretend otherwise.**
For these, CI verifies that a test *exists and is tagged*, not that it passes;
the run happens on a GPU host, nightly or manually, and reports separately. A
requirement whose only evidence is a test that never executes should be visible
as such rather than counted as covered.
| Requirement | Tier | Note |
|---|---|---|
| AR-001, AR-005, AR-006 | T3 | Smoke only — correctness of detection/embedding is a model property, not ours |
| AR-002 | T2 | Size filtering is arithmetic on dumped bboxes |
| AR-003, AR-004 | T1 + T4 | Backpressure logic is unit-testable; saturation behaviour needs real load |
| AR-007 … AR-017 | **T2** | The core of the redesign — fully replayable |
| AR-018 … AR-022 | **T2** | Expansion, deferred pass, clustering: all post-embedding |
| AR-023 … AR-025 | T1 | Calibration fit and log-odds accumulation are pure maths |
| AR-026, AR-027 | T4 | GEMM throughput and scaling — GPU host only |
| DP-* | T1 + manual | Lifecycle logic unit-tested; install paths are manual |
| IR-001 … IR-003 | T1 | Serialisation against a golden truth file |
| IR-004, IR-005 | **T1** | Audio signature is CPU DSP — the golden-vector fixture runs anywhere, which is precisely why it is the right cross-repo check |
| GR-001 … GR-005 | T1 + T3 | Gallery assembly is I/O and bookkeeping; embedding is T3 smoke |
| GR-006 … GR-008 | T1 | Tiering and outlier detection operate on stored embeddings |
| VR-* | Out of CI | Studies are run deliberately and their results committed as documents |
| VR-014 | **T2** | The exception, and the reason the blanket row above is not the whole story: its fixture is committed and its signature is CPU-only DSP, so the study *is* a test a CI host can run — not a measurement someone has to remember to repeat |
**One consequence worth stating:** AR-027 (arbitrary gallery scale) is
structurally unverifiable on the CI host. It needs a GPU host and a synthetic
large gallery, so it is the requirement most likely to silently regress. Its
benchmark (VR-008) should run on a schedule rather than on demand.
### CI never calls a model
**Not "should not" — cannot.** The N100 has no GPU, and even the ONNX Runtime CPU
provider is impractical: a measured run of the embedder on this hardware sits at
~930 ms per frame, so a 77 s clip at 5 fps would take roughly six minutes of
inference alone. Every model invocation therefore happens **locally, ahead of
time**, and CI consumes the result as data.
This is what makes the T1/T2 split load-bearing rather than a preference: T1 and
T2 are the only tiers that can exist in CI at all.
### Fixture corpus — `hero/`
Five clips of **SuperHero (1952)**, ~77 s each, 480×360, 30 fps, 42 MB total.
Public domain, and that is the reason to use it rather than a convenience:
**derived fixtures — dumps, crops, golden outputs — can be committed without the
rights question that rules out sharing gallery data (SR-005).** A fixture cut
from a copyrighted title could not live in the repository at all.
Two properties to design around rather than discover:
- **480×360 means small faces.** At this resolution a face is often 40–80 px, so
the AR-002 minimum of 40 px (original resolution) sits at the very bottom of
that range: the filter is close to binding, and anything shot wider is lost.
Fixture generation must set `--min-face-px` explicitly and record it, or the
dumps will be sparse for reasons unrelated to what is being tested.
- **77 s is short.** At 1 fps that is 77 frames — too thin to exercise an
extinction window measured in tens of seconds. Generate at 5 fps (≈385 frames,
~1 MB) and record the rate in provenance, since the behaviour under test
changes with it.
> **AR-004 blocks reproducible fixture generation.** A trial run of one clip
> produced 49 frames of an expected ~385, ending at 51 s of 77 s, with the
> diagnostics reporting 285 frames dropped at `camera_pos` and 51 at
> `face_aligner`. Channels overflow and **drop** rather than blocking, and what
> gets dropped depends on timing — so the same command run twice can produce
> different dumps. Golden fixtures cannot be built on that. AR-004 is therefore
> a prerequisite for VR-001 fixtures, not merely a throughput concern for crowd
> scenes.
### Fixtures — precomputed inference, pulled by CI
The N100 cannot run inference at any useful rate, so **inference output is
precomputed on a GPU host and consumed by CI as data.** This converts most of
what looks like GPU work into pure CPU replay.
| Fixture | Contents | Size | Storage |
|---|---|---|---|
| **Edge-case dumps** | ~6 short clips (30–60 s), one per awkward behaviour | ~0.1–1 MB each | **Committed in-repo** |
| **Corpus dumps** | Full-length titles from the validation corpus | ~21–38 MB each | **Gitea package registry**, pinned by version + checksum |
| **Synthetic gallery** | Random unit-norm embeddings, fixed seed | small | Generated at test time |
| **Golden truth files** | Expected output for each edge-case dump | KB | Committed |
| **Audio golden vectors** | FLAC + expected signature + parameter contract | ~600 KB | Committed, **shared with the plugin repo** |
Edge-case dumps are small enough to commit, and being in-repo means they version
with the code that reads them.
**Corpus dumps go to the Gitea package registry, not Git LFS.** Both are
available — the models already use LFS — but their fetch semantics differ in a
way that matters here. LFS objects are pulled on clone unless a developer
explicitly skips them, so ~38 MB per title behind LFS taxes everyone who clones,
forever, for data that only CI and the optimizer ever read. Registry artifacts
are fetched on demand by the job that needs them.
Rule of thumb: **LFS for what the build needs; the package registry for what a
particular job needs.** Models are the former; corpus dumps and the CI image
(DP-007) are the latter.
Pin by version and verify by checksum on fetch. A fixture that changes silently
under CI is worse than a missing one, because the failure presents as a code
regression.
**Generation must be reproducible and versioned.** A script, run on a GPU host,
regenerates every fixture from source clips; it is re-run when the VR-001 schema
version bumps. A fixture whose provenance is unknown is worse than no fixture,
because it will be trusted.
> **The limitation that must stay visible:** replay fixtures freeze upstream
> behaviour. A test driven from a dump verifies AR-007 onward *given those
> embeddings* — it cannot detect a regression in detection, alignment or
> embedding, because those produced the fixture. Nothing in CI can. That gap is
> covered only by the T3 smoke test and the scheduled GPU run, and it should not
> be papered over by a high replay-coverage number.
### Per-requirement verification plan
| ID | Tier | Test asserts | Edge cases to cover |
|---|---|---|---|
| AR-001 | T3 | Detector returns plausible boxes on a known frame | — smoke only |
| AR-002 | T2 | Faces below 40 px (original res) are dropped | Exactly at threshold; with `dense_scale` 0.5 — the interaction that motivated the requirement |
| AR-003 | T2 | No cap applied; a 40-face frame yields 40 | Crowd frame |
| AR-004 | T1 | Saturated input blocks rather than drops or throws | Bounded queue at capacity; **byte-based** limit with large crops; SIGTERM mid-block. Cases the KPN suite now pins, each of which failed before being written: a fanout feeding an unequal pair loses nothing *and* throttles the fast branch (either assertion alone passes on a broken implementation); a filter delivers EOF into a saturated output; a sentinel is never delivered ahead of a queued value; a twice-parked value keeps its payload; and a node started with data already in its input still fires — the startup lost wake, which needs no contention to reproduce once the state is constructed directly |
| AR-005 | T1 | Known landmarks → expected 112×112 warp; the fit never mirrors | Landmarks near frame edge; degenerate/collinear points; a mirrored set — SVD returns a reflection unless the determinant guard rejects it |
| AR-006 | T3 | Embeddings are unit-norm | Batch smaller than, equal to, larger than `embed_batch_size` |
| AR-007 | T2 | Association picks the right track | Two faces crossing paths; one leaving frame as another enters |
| AR-008 | T2 | One pool; dormant tracks match on embedding, not IoU | Dormant track whose old bbox overlaps a *different* new face — must not match on position |
| AR-009/010 | T2 | Cut/boundary shifts weighting toward embedding | Cut with same people; cut with all-new people |
| AR-011 | T1 | TransNetV2 receives native-rate frames | Source at 24/25/30 fps — dedup window derived, not assumed |
| AR-012 | **T2** | Window spans full track extent, not first recognition | Actor recognised only at track end — window must still start at `first_seen` |
| AR-013 | **T2** | `last_seen` set/unset; window ends at last sighting | Gap just under vs just over timeout; reappearance after timeout → two windows |
| AR-014 | T2 | Belief swap closes one window, opens another | No blended window; no overlap at the swap frame |
| AR-015 | T2 | Two live tracks on one actor trigger re-association | Counter increments |
| AR-016 | **T2** | Every track closed at EOF | Film ending mid-shot — window ends at final frame, not dropped |
| AR-017 | T1 | Claim carries posterior and route | Deferred and pooled routes distinguishable — now possible: `route` is an enum on `DeadTrack` rather than the literal `"live"` the sink used to write. Only `live` occurs until AR-020 exists, so the test that matters today is that the field survives serialisation |
| AR-018 | T1 | Band admits only within bounds | At each bound exactly; store never admits below lower bound |
| AR-019 | T2 | Promotion only when all three signals quiet | Cut mid-track blocks promotion |
| AR-020 | **T2** | Unknown resolved after expansion | Track failing at minute 12, resolved at EOF — the ordering-independence claim |
| AR-021 | T2 | Clustering merges same person, respects cannot-link | **Temporally overlapping tracks never merge**; measure how many merges the constraint rejects |
| AR-022 | T1 | Context crops retained, bounded per track | Track running for minutes |
| AR-023 | T1 | Sigmoid fit on synthetic separable data | Too few positive pairs → `valid=false`, and the fallback that engages is the **default sigmoid**, not the retired cosine rule. Assert the warning fires: an unfitted sigmoid returns plausible-looking probabilities, so nothing downstream can tell |
| AR-024 | **Static check** | No bare cosine outside a tagged `EXCEPTION` | `scripts/ci/check_raw_cosine.py`, blocking in the traceability workflow. Honest about its reach: it catches direct `cosine_similarity()` uses not routed through a calibration and **cannot follow a cosine through a variable across statements**, which is a convention backed by review rather than by the tool. Scans `src` only — a test legitimately asserts properties of the metric space, and sweeping those in would produce blanket exceptions that devalue the tag |
| AR-025 | T1 | Log-odds accumulate; correlated frames discounted | 30 identical frames must **not** reach the certainty of 30 diverse ones |
| AR-026 | T1 + T4 | GEMM path produces same result as reference loop | Equivalence on small input in CI; throughput on GPU host |
| AR-027 | **T4** | Throughput at 10²…10⁵ actors | Scheduled, not on-demand |
| AR-028 | **T2** | No embedding reaches the matcher unscored; the vector survives into the dump | Face failing exactly one axis; all three healthy; a face whose landmarks are degenerate — dropped for want of a crop to score, but **counted** rather than silently vanished (UT-138) |
| AR-029 | T1 | Synthetic blur ladder → monotonically falling sharpness | Gaussian vs motion blur; **small sharp face vs large soft one** — size must not leak into this axis. The blur ladder must be measured on a **1/f texture**: on a flat-spectrum one the motion ladder *rises*, since an anisotropic smear takes energy out of numerator and denominator together (UT-131). Contrast must not leak either — exact in the algebra, and the 8-bit floor that bends it is pinned by UT-133 |
| AR-030 | T1 | Alignment residual rises monotonically with foreshortening | **In-plane roll, scale and translation must leave it at zero** — the property that makes it a pose measure rather than a pose-and-everything-else measure; face size must not shift it; degenerate landmarks report not-ok rather than a number |
| VR-012 | **T4** | Knee located per axis on held-out films | Report each candidate threshold's cost in **lost true presence**, not only its gain in precision — a gate that improves misID by discarding half the cast has not helped |
| VR-013 | **T4** | Identification holds across two recordings of the same people, and degrades to TBI rather than to a wrong name as input resolution falls | Gallery and probes must come from *different* recordings — a hold-one-out over one recording measures a much easier problem and will not surface the cross-view failure. Ground truth is hand-sorted; labels propagated by embedding similarity would keep only the faces the embedder already gets right |
| VR-016 | **T2** | Cut rate as a function of the cadence `camera_pos` is fed | Same clip at 1/2/5 fps, `--scene-detect` on and off. The dump already records `cut_threshold` and `sample_fps` (VR-010), so a replay can score this without re-decoding. A finding of "0.70 is fine at every rate" is a real result and should be recorded as one |
| IR-001/002 | T1 | Serialised output matches golden file | Zero-length window; actor with many windows |
| IR-003 | T1 | Output written after deferred pass | Not at EOF |
| IR-004/005 | **T1** | Signature matches golden vector bit-for-bit | Identical result in both producer repos |
| VR-014 | **T2** | A known trim offset is recovered from **real film audio**, to the nearest frame | An offset past the ±600-frame cap and unrelated content must both be *declined*, never given a best-effort alignment. Fixture and signature are both CPU-only, so unlike the other VR rows this one is CI-*executable* — though the repo's only workflow today is the traceability gate, so nothing runs it there yet. The signature comes from the shipped C++ through `sae_audio`; a numpy port would be a third implementation nobody checks against the golden vector |
| IR-006 | T1 + manual | Queue pull and result push against a stubbed Jellyfin API | Partial result never pushed; push only after the deferred pass |
| IR-007 | **T1** | Media < 120 s emits no signature at all | Exactly 120 s; just under; zero-length audio. Must match the plugin's cutoff exactly — a caller-varying window length is what SR-004 forbids |
| IR-008 | T1 | `v1:` prefix emitted and honoured on read | Unknown prefix rejected, not guessed |
| GR-009 | T1 | Human-confirmed associations persist and are tier-tagged | Survives a gallery rebuild; distinguishable from baked and harvested |
| GR-004 | T1 | Mismatched embedder → hard startup error | Error names both sides; **unstamped warns, and errors under `SAE_REQUIRE_GALLERY_STAMP`**; same filename + different SHA-256 must still be a mismatch |
| GR-008 | T1 | Outlier flagged among an actor's references | Injected poisoned embedding detected |
| VR-009 | T1 | Posterior calibration holds | A 0.99 posterior is wrong ~1% of the time on held-out tracks |
Three of these are worth singling out because they verify claims that would
otherwise be assertions: **AR-012** (window starts at `first_seen` even when
recognition comes late) is the entire point of the redesign; **AR-020** (a track
failing mid-film resolves at EOF) is the claim that ordering stops mattering; and
**AR-025** (30 identical frames ≠ 30 diverse ones) is what stops the Bayesian
accumulation from being decoration.
---
## Withdrawn
| ID | Requirement | Reason |
|---|---|---|
| — | `anneal_sec` window merging | Superseded by AR-012/AR-013: a track survives its own gaps, so there is nothing to anneal |
| — | `extinction_sec` actor keep-alive | Superseded by AR-013: windows end at last sighting, which is what this over-claimed |
Both are now deleted rather than retained at zero — a field naming a mechanism
the pipeline no longer has is actively misleading (see `SPEC.md` A6.6).
**This paragraph was false for some time, and the failure is worth keeping.** It
was written in the present perfect as though the removal had happened. It had
not: `Config::extinction_sec` (57.4) and `Config::anneal_sec` (35.5) were still
there, `--extinction` and `--anneal` still parsed, and `SceneTrackerFunc` still
ran its keep-alive in both shipped pipelines, printing its timeout at every
startup. `SPEC.md`'s removal list ends "grep for both names and expect no
survivors"; there were about forty.
Nothing in the tooling could have caught it. The traceability gate reads tags,
not behaviour, and a withdrawn requirement has no tag to be orphaned — the
register simply asserted a state of the code, and no test asked. The general
form is worth stating: **a status column is a claim, and the only claims this
project can check automatically are the ones a test or a static check makes.**
The same pattern produced three other rows corrected in this pass (AR-011,
AR-017, AR-019), each recorded as done and done in one place out of two.
`SceneTrackerFunc` is replaced by the stateless `FrameAnnotationFunc`. One
visible consequence: `--verbosity standard`'s `frames[].identified` used to
include every actor inside the keep-alive window, and now lists what was matched
in that frame. Minimal and xray output never consulted the node.
---
## Notes on coverage
- **VR-*** traces to PR-002 (scene-granularity answers) rather than to a system
requirement: parameter studies are single-repo work serving accuracy, and this
is correct rather than a gap.
- **PR-005** (leak nothing) has no `AR`/`DP` row. It is satisfied *structurally*
by SR-004 and GR-005 — the server holds no binary, the gallery never leaves the
instance — not by any component doing something. It cannot be verified by
pointing at code, and it dies the moment either prohibition is relaxed.
docs/service-conversion.md
+215
View File
@@ -0,0 +1,215 @@
# Conversion to service — a native idle-GPU worker
Status: **design / proposal**. Nothing here is built yet.
## The idea
Turn the CLI tools into a **turnkey batch worker that uses the machine's idle
GPU**: it analyses newly-added Jellyfin media when you're not using the computer
(screen locked), and stops the instant you come back. It's an overnight job on
your own Linux box.
**No Docker.** This runs on your own machine with your own drivers, so a container
buys little and costs a lot: GPU passthrough (nvidia-container-toolkit, or
`/dev/kfd`+`/dev/dri`+`video` group for ROCm) is the single most fragile part of a
containerised setup, and it exists *only* because of the container. Natively, the
GPU just works with the drivers you already have, and the media paths Jellyfin
reports are just real paths — no re-mounting. So we ship a **native installer**
instead of an image builder.
Two deliverables:
1. **An installer** — `scripts/build_install.py`. Detects your distro, ensures the
GPU/build dependencies are present (via `dnf`/`pacman`), compiles `scene_analyze`
for your GPU, and installs the binary + Python glue + two systemd **user**
units under `~/.local`.
2. **A screen-lock gate** — one of those systemd units watches logind lock/unlock
and starts/stops the worker. Lock → analyse. Unlock → stop.
## What already exists (reuse, don't rebuild)
The processing loop is already implemented — this is packaging, building, and
lock-gating, not new pipeline logic.
| Piece | Where | What it does |
|---|---|---|
| Analysis engine | `build/scene_analyze` | Video → face detect/align/embed → gallery match → result JSON |
| Backend selection | [`CMakeLists.txt`](https://REPOLINK/CMakeLists.txt) (`SAE_INFERENCE_BACKEND`, `SAE_GEMM_BACKEND`) | ORT/TRT + ROCm/CUDA, chosen **at build time** |
| New-media queue | JRay plugin → `GET /Plugins/JRay/Tasks/Pending` | Backlog of items with no results yet |
| Worker loop | [`scripts/run_from_jellyfin.py`](https://REPOLINK/scripts/run_from_jellyfin.py)` --worker` | Poll Pending → run `scene_analyze` → push results |
| Result push | `PUT /Plugins/JRay/Items/{id}/Truth` | Stores per-actor scene windows back in Jellyfin |
| Incremental gallery | [`scripts/make_jellyfin_gallery.py`](https://REPOLINK/scripts/make_jellyfin_gallery.py)` --merge` | Embeds only cast not already in the gallery |
| Secrets loader | `.env` via [`scripts/sae_env.py`](https://REPOLINK/scripts/sae_env.py) | `JELLYFIN_URL`, `JELLYFIN_API_KEY`, `TMDB_API_KEY` |
## Installer config
One file. Build-time settings (fixed when we compile) vs. run-time settings (in the
worker's `.env`, editable without recompiling).
```yaml
# install.yaml — consumed by scripts/build_install.py
platform: nvidia # nvidia | amd | cpu → picks the cmake backend
model:
arcface: LVFace-B_Glint360K.onnx # embedder compiled against; gallery MUST match
schedule:
gallery_scan_interval: 24h # incremental --merge cadence; 0 disables the scanner
prefix: ~/.local # install root (bin, share, systemd user units)
# runtime (written to the worker .env, not compiled in):
runtime:
jellyfin_url: http://localhost:8096
# JELLYFIN_API_KEY / TMDB_API_KEY are filled into .env by hand after install
```
**Secrets never go in the repo or a build artifact** — the installer writes a
`.env` under the install prefix with blanks for the keys, and you fill them in
once. `sae_env.py` already loads it.
**Model ⇄ gallery coupling (guard, don't just document):** embeddings from
different recognition models aren't interchangeable. We compile against one
embedder; the gallery must be built with the same one. Stamp the embedder name
into `gallery.json`, and have the worker **refuse to start** if the gallery's
embedder ≠ the configured `model.arcface`, rather than silently mismatching.
## Dependencies via the system package manager
The heavy build/runtime deps (OpenCV, ffmpeg, the GPU stack) are best provided by
the distro, not vendored. The installer ships a per-distro dependency list and
either installs them or prints the exact command. Targets: **Fedora (dnf)** and
**Arch (pacman)** first.
| Dependency | Fedora (dnf) | Arch (pacman) |
|---|---|---|
| OpenCV | `opencv-devel` | `opencv` |
| ffmpeg | `ffmpeg-free`/`ffmpeg` (RPM Fusion) | `ffmpeg` |
| CMake / toolchain | `cmake gcc-c++` | `cmake gcc` |
| CUDA + TensorRT (nvidia) | NVIDIA CUDA repo + `libnvinfer-*` | `cuda`, `tensorrt` |
| ROCm (amd) | `rocm-hip-sdk` / `rocblas-devel` | `rocm-hip-sdk`, `rocblas` |
| ONNX Runtime | **not packaged** — installer fetches a pinned release tarball into the prefix | AUR `onnxruntime` (or same pinned-tarball fallback) |
So the flow is: **detect distro → check each package → install via the native
manager (or print `sudo dnf install …` / `sudo pacman -S …`)**, with ONNX Runtime
as the one known gap the installer fills itself (a pinned upstream release
extracted under the install prefix, so it doesn't depend on a system package that
may not exist). CUDA/ROCm being present is *assumed* — you already run a GPU
desktop; the installer verifies and points you at the vendor repo if not.
## What `build_install.py` does
```
build_install.py install.yaml
│
├─ detect distro (dnf vs pacman) and platform from config
├─ ensure deps: install via manager, or print the exact command; fetch ONNX Runtime if needed
├─ cmake + build scene_analyze with the platform's backend flags:
│ nvidia → -DSAE_INFERENCE_BACKEND=TRT -DSAE_GEMM_BACKEND=CUDA
│ amd → -DSAE_INFERENCE_BACKEND=ORT -DSAE_GEMM_BACKEND=ROCM
│ cpu → -DSAE_INFERENCE_BACKEND=ORT (CPU EP; slow, for smoke tests)
├─ install into <prefix>:
│ bin/sae-scene-analyze the compiled binary
│ share/sae-worker/ Python glue + a venv (requests, etc.), models/
│ share/sae-worker/.env runtime config (keys blank, url from config)
├─ install systemd --user units:
│ sae-worker.service runs the worker + gallery-scan supervisor
│ sae-lock-gate.service watches logind lock/unlock, start/stops the worker
└─ print next steps (edit .env, `systemctl --user enable --now sae-lock-gate`)
```
## The worker service (supervisor)
`sae-worker.service` runs a small Python supervisor as its main process:
- starts the **worker loop** (`run_from_jellyfin.py --worker`) — the hot path,
- starts a **gallery-scan timer** — sleeps `gallery_scan_interval`, runs
`make_jellyfin_gallery.py --merge`, repeats,
- exits cleanly on SIGTERM (see re-queue below).
## The lock gate
`sae-lock-gate.service` runs a tiny watcher that subscribes to logind
lock/unlock signals and drives the worker service:
```
screen locks → systemctl --user start sae-worker.service
screen unlocks → systemctl --user stop sae-worker.service (SIGTERM)
```
**Screen-lock is the only signal — deliberately.** We don't also gate on GPU/CPU
load, because our own worker *is* the load: a load threshold would form a feedback
loop (worker starts → GPU spikes → threshold trips → worker stops → load drops →
restart → …). Lock state is external to what the worker does, so it can't
oscillate.
Signal source is desktop-dependent: logind `Lock`/`Unlock` (GNOME/KDE via
`loginctl`/D-Bus) covers most setups; a `swayidle`/`xss-lock` hook is the fallback
for wlroots/X-only compositors. The installer picks based on what's present.
## On resume: hard stop + re-queue (it's free)
Stopping the worker mid-analysis costs nothing to reschedule, because of how the
JRay queue works: **an item only leaves `/Tasks/Pending` once its results are
pushed** (`push_truth`). A worker stopped mid-`scene_analyze` simply leaves that
item Pending — next lock picks it up again. No re-queue bookkeeping.
Two small correctness requirements (the only worker changes needed):
1. **Never push a partial result.** Already true — `push_truth` runs only after
`scene_analyze` returns; a killed run pushes nothing. ✓ (keep it that way).
2. **Clean up on signal.** `process_item` writes a temp filtered-gallery file and
unlinks it in a `finally`; a SIGKILL skips `finally`. Fix: write temps under a
dir the worker wipes on start, and/or a SIGTERM handler that unlinks before
exit. Minor.
Accepted trade-off: a partially-analysed title restarts from scratch next lock.
Fine for an overnight/idle workload; no mid-video checkpointing.
## The end-to-end UX
```bash
# once: build + install for your GPU + model
./scripts/build_install.py install.yaml
# detects Fedora/Arch, ensures deps, compiles, installs units under ~/.local
# once: set your keys, enable the gate
$EDITOR ~/.local/share/sae-worker/.env # JELLYFIN_API_KEY, TMDB_API_KEY
systemctl --user enable --now sae-lock-gate.service
# from then on: nothing. Lock your screen → it analyses. Unlock → it stops.
```
No Docker, no GPU passthrough config, no media re-mounting — the worker sees the
same filesystem and GPU as everything else on the box.
## Implementation plan (follow-up commits)
Ordered so each step stands alone:
1. **installer skeleton** — `scripts/build_install.py`: parse `install.yaml`,
distro detect, dependency check/print (start with cpu platform so it builds
without a GPU), cmake+build, copy into prefix.
2. **supervisor + cleanup** — `scripts/service.py` (worker loop + gallery-scan
timer + SIGTERM); temp-file cleanup fix in `run_from_jellyfin.py`.
3. **systemd units + lock gate** — generate/install `sae-worker.service`,
`sae-lock-gate.service`, and the logind lock watcher.
4. **gallery/model guard** — stamp embedder into `gallery.json`; startup mismatch
check.
5. **platform + distro matrix** — nvidia/amd backends; dnf/pacman dep lists; ONNX
Runtime fetch fallback.
6. **docs** — README "Run on your idle GPU" section.
## Settled decisions
- **ONNX Runtime build** — the installer fetches the **ROCm ORT** release. It
serves the `amd` platform, and its CPU execution provider covers the `cpu`
smoke-test fallback too, so one download handles both. (nvidia uses raw TRT and
doesn't need ORT.)
- **`dnf`/`pacman` invocation** — **auto-install.** The installer runs `sudo dnf
install …` / `sudo pacman -S …` itself (prompting for sudo), rather than only
printing the command. It still prints what it's about to install first.
- **Distro coverage** — **Fedora + Arch only** for now. Debian/Ubuntu (`apt`) is
out of scope.
## Open questions
*(none blocking — the spec above is buildable as-is.)*
stylesheets/extra.css → docs/stylesheets/extra.css
View File
docs/traceability.md
+1230
View File
File diff suppressed because it is too large Load Diff
docs_data/holdout_all_models.json
+269
View File
@@ -0,0 +1,269 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15119,
"FPI": 1845,
"FPI_misid": 0,
"FPI_incast": 1845,
"FN": 4343,
"precision": 0.8912402735203961,
"precision_raw": 0.8912402735203961,
"recall": 0.7768471893947179,
"f1": 0.8301213418986437,
"agreement_rate": 0.7239983093829193,
"exact_match_rate": 0.41098901098901097,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 52022,
"FPI": 1160,
"FPI_misid": 0,
"FPI_incast": 1160,
"FN": 80089,
"precision": 0.9781881087586025,
"precision_raw": 0.9781881087586025,
"recall": 0.39377493168623356,
"f1": 0.5615106884771687,
"agreement_rate": 0.4057686401759256,
"exact_match_rate": 0.033084311632870865,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14988,
"FPI": 1086,
"FPI_misid": 58,
"FPI_incast": 1028,
"FN": 7087,
"precision": 0.9031091829356471,
"precision_raw": 0.9324374766703994,
"recall": 0.6789580973952435,
"f1": 0.775154508546456,
"agreement_rate": 0.7204967829586512,
"exact_match_rate": 0.3597703211914588,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15928,
"FPI": 4394,
"FPI_misid": 974,
"FPI_incast": 3420,
"FN": 23785,
"precision": 0.5475797579757976,
"precision_raw": 0.7837811239051274,
"recall": 0.40107773273235464,
"f1": 0.46301652592258835,
"agreement_rate": 0.3705156874642392,
"exact_match_rate": 0.04588936642173853,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18658,
"FPI": 548,
"FPI_misid": 0,
"FPI_incast": 548,
"FN": 12472,
"precision": 0.9714672498177653,
"precision_raw": 0.9714672498177653,
"recall": 0.5993575329264376,
"f1": 0.7413382072472983,
"agreement_rate": 0.5877853464704299,
"exact_match_rate": 0.21980294368081743,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 15219,
"FPI": 2463,
"FPI_misid": 180,
"FPI_incast": 2283,
"FN": 4243,
"precision": 0.7884675163195524,
"precision_raw": 0.8607058025110281,
"recall": 0.7819854074606927,
"f1": 0.7852130843050252,
"agreement_rate": 0.7072306082196138,
"exact_match_rate": 0.34911242603550297,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 53043,
"FPI": 2383,
"FPI_misid": 604,
"FPI_incast": 1779,
"FN": 79068,
"precision": 0.8715290328940882,
"precision_raw": 0.9570057373795692,
"recall": 0.4015032813316075,
"f1": 0.5497453011561202,
"agreement_rate": 0.4094114144765671,
"exact_match_rate": 0.032817502668089645,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 14606,
"FPI": 1337,
"FPI_misid": 180,
"FPI_incast": 1157,
"FN": 7469,
"precision": 0.8316346865569664,
"precision_raw": 0.916138744276485,
"recall": 0.6616534541336353,
"f1": 0.7369695746505879,
"agreement_rate": 0.6907677036961077,
"exact_match_rate": 0.31742329086667864,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 15223,
"FPI": 4574,
"FPI_misid": 994,
"FPI_incast": 3580,
"FN": 24490,
"precision": 0.52962460425147,
"precision_raw": 0.768954892155377,
"recall": 0.383325359454083,
"f1": 0.44475283393712756,
"agreement_rate": 0.3554753116932427,
"exact_match_rate": 0.03715513655899071,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 18472,
"FPI": 853,
"FPI_misid": 239,
"FPI_incast": 614,
"FN": 12658,
"precision": 0.860122927919538,
"precision_raw": 0.9558602846054334,
"recall": 0.5933825891423065,
"f1": 0.7022773067710907,
"agreement_rate": 0.5795914643682625,
"exact_match_rate": 0.18817662084904513,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Benny___Joon": {
"name": "Benny & Joon",
"TPI": 13580,
"FPI": 1666,
"FPI_misid": 60,
"FPI_incast": 1606,
"FN": 5882,
"precision": 0.86025592296972,
"precision_raw": 0.8907254361799817,
"recall": 0.697770013359367,
"f1": 0.7705401724920563,
"agreement_rate": 0.6547675401521545,
"exact_match_rate": 0.32578191039729504,
"n_seconds": 5915,
"duration_sec": 5915.0
},
"Downton_Abbey__A_New_Era": {
"name": "Downton Abbey: A New Era",
"TPI": 48545,
"FPI": 1066,
"FPI_misid": 180,
"FPI_incast": 886,
"FN": 83566,
"precision": 0.9475708067381078,
"precision_raw": 0.9785128298159682,
"recall": 0.3674561542944948,
"f1": 0.5295567845883649,
"agreement_rate": 0.3815368792000116,
"exact_match_rate": 0.032950907150480255,
"n_seconds": 7496,
"duration_sec": 7496.0
},
"Lovelace": {
"name": "Lovelace",
"TPI": 13615,
"FPI": 963,
"FPI_misid": 120,
"FPI_incast": 843,
"FN": 8460,
"precision": 0.8695235662281262,
"precision_raw": 0.933941555768967,
"recall": 0.6167610419026047,
"f1": 0.7216494845360825,
"agreement_rate": 0.6506356469257915,
"exact_match_rate": 0.2894311860757222,
"n_seconds": 5573,
"duration_sec": 5573.0
},
"The_Many_Saints_of_Newark": {
"name": "The Many Saints of Newark",
"TPI": 13489,
"FPI": 3757,
"FPI_misid": 796,
"FPI_incast": 2961,
"FN": 26224,
"precision": 0.5526013928717739,
"precision_raw": 0.7821523831613127,
"recall": 0.3396620753909299,
"f1": 0.42072267361165266,
"agreement_rate": 0.3229817885335633,
"exact_match_rate": 0.04422570359073894,
"n_seconds": 7213,
"duration_sec": 7213.0
},
"Valerian_and_the_City_of_a_Thousand_Plan": {
"name": "Valerian and the City of a Thousand Planets",
"TPI": 17692,
"FPI": 397,
"FPI_misid": 68,
"FPI_incast": 329,
"FN": 13438,
"precision": 0.9460456660071654,
"precision_raw": 0.9780529603626513,
"recall": 0.5683263732733698,
"f1": 0.710080070638759,
"agreement_rate": 0.5633540120828806,
"exact_match_rate": 0.13404695292543486,
"n_seconds": 8221,
"duration_sec": 8221.0
}
}
}
}
docs_data/training_per_film.json
+221
View File
@@ -0,0 +1,221 @@
{
"LVFace-B_Glint360K": {
"config": {
"prob_threshold": 0.7540024664611272,
"anneal_sec": 35.53996030397922,
"extinction_sec": 57.43359645269811
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 14499,
"FPI": 1380,
"FPI_misid": 0,
"FPI_incast": 1380,
"FN": 12231,
"precision": 0.9130927640279615,
"precision_raw": 0.9130927640279615,
"recall": 0.5424242424242425,
"f1": 0.6805604449764134,
"agreement_rate": 0.57285804629501,
"exact_match_rate": 0.18947003810183582,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13893,
"FPI": 1654,
"FPI_misid": 174,
"FPI_incast": 1480,
"FN": 5737,
"precision": 0.811838952842868,
"precision_raw": 0.8936129156750499,
"recall": 0.7077432501273561,
"f1": 0.7562256756388972,
"agreement_rate": 0.7005158404089996,
"exact_match_rate": 0.38715420432758146,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 20518,
"FPI": 1078,
"FPI_misid": 58,
"FPI_incast": 1020,
"FN": 14722,
"precision": 0.9276607288181572,
"precision_raw": 0.9500833487682904,
"recall": 0.5822360953461975,
"f1": 0.7154363820216882,
"agreement_rate": 0.6297735703976657,
"exact_match_rate": 0.25910733470065433,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 13349,
"FPI": 677,
"FPI_misid": 0,
"FPI_incast": 677,
"FN": 6504,
"precision": 0.9517324967916726,
"precision_raw": 0.9517324967916726,
"recall": 0.6723920818012391,
"f1": 0.7880397886596416,
"agreement_rate": 0.7112222835587533,
"exact_match_rate": 0.40311896218603366,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_w600k_mbf": {
"config": {
"prob_threshold": 0.8371114538930519,
"anneal_sec": 48.80011450565114,
"extinction_sec": 59.3110040220424
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 13456,
"FPI": 1434,
"FPI_misid": 180,
"FPI_incast": 1254,
"FN": 13274,
"precision": 0.8150211992731677,
"precision_raw": 0.9036937541974479,
"recall": 0.5034044145155256,
"f1": 0.622386679000925,
"agreement_rate": 0.5463565775524695,
"exact_match_rate": 0.19154832005542086,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13778,
"FPI": 1740,
"FPI_misid": 60,
"FPI_incast": 1680,
"FN": 5852,
"precision": 0.8580146967243741,
"precision_raw": 0.8878721484727413,
"recall": 0.7018848700967907,
"f1": 0.7721362923111411,
"agreement_rate": 0.6881858851455998,
"exact_match_rate": 0.36469460421802247,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 18863,
"FPI": 862,
"FPI_misid": 0,
"FPI_incast": 862,
"FN": 16377,
"precision": 0.956299112801014,
"precision_raw": 0.956299112801014,
"recall": 0.535272417707151,
"f1": 0.6863640498499044,
"agreement_rate": 0.594178111391709,
"exact_match_rate": 0.2357652114464303,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12642,
"FPI": 554,
"FPI_misid": 0,
"FPI_incast": 554,
"FN": 7211,
"precision": 0.9580175810851773,
"precision_raw": 0.9580175810851773,
"recall": 0.6367803354656727,
"f1": 0.7650458410239341,
"agreement_rate": 0.687468948385309,
"exact_match_rate": 0.3789677063207287,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
},
"arcface_r18": {
"config": {
"prob_threshold": 0.8955101189489445,
"anneal_sec": 59.08214397442713,
"extinction_sec": 59.29474134414983
},
"films": {
"Caf\u00e9_Society": {
"name": "Caf\u00e9 Society",
"TPI": 12207,
"FPI": 1119,
"FPI_misid": 60,
"FPI_incast": 1059,
"FN": 14523,
"precision": 0.88035482475119,
"precision_raw": 0.9160288158487168,
"recall": 0.45667789001122333,
"f1": 0.6013892994383683,
"agreement_rate": 0.5125626845924863,
"exact_match_rate": 0.1674748874263942,
"n_seconds": 5774,
"duration_sec": 5774.0
},
"Lord_of_War": {
"name": "Lord of War",
"TPI": 13122,
"FPI": 1409,
"FPI_misid": 60,
"FPI_incast": 1349,
"FN": 6508,
"precision": 0.870678787074514,
"precision_raw": 0.9030348909228546,
"recall": 0.6684666327050433,
"f1": 0.7562894441082388,
"agreement_rate": 0.6698963754222389,
"exact_match_rate": 0.3389482333607231,
"n_seconds": 7302,
"duration_sec": 7302.0
},
"Scarface": {
"name": "Scarface",
"TPI": 16961,
"FPI": 685,
"FPI_misid": 0,
"FPI_incast": 685,
"FN": 18279,
"precision": 0.961181004193585,
"precision_raw": 0.961181004193585,
"recall": 0.48129965947786607,
"f1": 0.6414173883447416,
"agreement_rate": 0.5440709115628456,
"exact_match_rate": 0.2017775173356773,
"n_seconds": 10239,
"duration_sec": 10239.0
},
"Sound_of_Metal": {
"name": "Sound of Metal",
"TPI": 12017,
"FPI": 591,
"FPI_misid": 122,
"FPI_incast": 469,
"FN": 7836,
"precision": 0.8767692981176127,
"precision_raw": 0.953125,
"recall": 0.6052989472623784,
"f1": 0.7161715188176049,
"agreement_rate": 0.6594534915815532,
"exact_match_rate": 0.3573005796301408,
"n_seconds": 7246,
"duration_sec": 7246.0
}
}
}
}
eval/gt.json
+362
View File
@@ -0,0 +1,362 @@
[
{
"crop": "eval/probe/nm0001589_0000.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0119_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0001.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0125_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0002.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0128_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0003.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0225_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0004.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0306_img_2.jpg"
},
{
"crop": "eval/probe/nm0001589_0005.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0308_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0006.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0311_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0007.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0315_img_0.jpg"
},
{
"crop": "eval/probe/nm0001589_0008.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0317_img_1.jpg"
},
{
"crop": "eval/probe/nm0001589_0009.jpg",
"imdb_id": "nm0001589",
"actor_name": "Michael Palin",
"source_frame": "tt0079470/shot_0319_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0000.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0001.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0074_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0002.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0078_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0003.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0079_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0004.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0080_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0005.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0638_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0006.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0101410/shot_0641_img_0.jpg"
},
{
"crop": "eval/probe/nm0000114_0007.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0017_img_1.jpg"
},
{
"crop": "eval/probe/nm0000114_0008.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0022_img_2.jpg"
},
{
"crop": "eval/probe/nm0000114_0009.jpg",
"imdb_id": "nm0000114",
"actor_name": "Steve Buscemi",
"source_frame": "tt0105236/shot_0024_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0000.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0097_img_2.jpg"
},
{
"crop": "eval/probe/nm0005042_0001.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0100_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0002.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0108_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0003.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0110_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0004.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0121_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0005.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0123_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0006.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0151_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0007.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0158_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0008.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0178_img_0.jpg"
},
{
"crop": "eval/probe/nm0005042_0009.jpg",
"imdb_id": "nm0005042",
"actor_name": "Jason Isaacs",
"source_frame": "tt0119081/shot_0180_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0000.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0154_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0001.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0157_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0002.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0161_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0003.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0163_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0004.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0164_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0005.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0167_img_0.jpg"
},
{
"crop": "eval/probe/nm0175916_0006.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0199_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0007.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0200_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0008.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0201_img_2.jpg"
},
{
"crop": "eval/probe/nm0175916_0009.jpg",
"imdb_id": "nm0175916",
"actor_name": "Paddy Considine",
"source_frame": "tt0440963/shot_0202_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0000.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0005_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0001.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0009_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0002.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0010_img_2.jpg"
},
{
"crop": "eval/probe/nm1385871_0003.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0013_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0004.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0014_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0005.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0543_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0006.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0585_img_1.jpg"
},
{
"crop": "eval/probe/nm1385871_0007.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0628_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0008.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0631_img_0.jpg"
},
{
"crop": "eval/probe/nm1385871_0009.jpg",
"imdb_id": "nm1385871",
"actor_name": "Olga Kurylenko",
"source_frame": "tt1483013/shot_0635_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0000.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0019_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0001.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0032_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0002.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0055_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0003.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0057_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0004.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0059_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0005.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0063_img_1.jpg"
},
{
"crop": "eval/probe/nm2057859_0006.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0072_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0007.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0074_img_0.jpg"
},
{
"crop": "eval/probe/nm2057859_0008.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0075_img_2.jpg"
},
{
"crop": "eval/probe/nm2057859_0009.jpg",
"imdb_id": "nm2057859",
"actor_name": "Andrea Riseborough",
"source_frame": "tt1483013/shot_0080_img_0.jpg"
}
]
eval/predictions_mbf.json
+1562
View File
File diff suppressed because it is too large Load Diff
eval/predictions_r18.json
+1562
View File
File diff suppressed because it is too large Load Diff
eval/predictions_r50.json
+1562
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More