AR-024's register row gives its verification tier as "Static check -- no
bare cosine outside a tagged EXCEPTION". No such check existed, so the
invariant was enforced by reading, and reading had missed a live
violation.
scripts/ci/check_raw_cosine.py is that check, wired into the
traceability workflow as a blocking step. It is honest about its reach:
it catches direct cosine_similarity() uses not routed through a
calibration, and it cannot follow a cosine through a variable across
statements. That limit is documented in the script rather than left for
someone to discover after trusting a pass.
What it caught, and what this commit removes with it:
The identity matcher's no-calibration fallback thresholded raw cosine
distance (match_threshold) plus a ratio test (match_ratio,
match_ratio_ceil). Worse than the invariant breach: it fed
max(0, cosine) into TrackRegistry::observe, whose contract reads
"posterior is a calibrated probability, never a raw cosine (AR-024) ...
so the accumulation cannot be fed an uncalibrated number by a careless
caller". It could, and did. And it disagreed with the rest of the
pipeline about what "the fit failed" means -- same_person_probability
answers that with the untuned default sigmoid and a loud warning, so
association stayed in probability space while matching alone left it.
One run, two policies, no announcement.
Now one rule: cal_.probability() always, with a warning when the fit is
not real. A worse answer than a fitted calibration, a better one than a
number whose units nothing else shares.
TrackGallery::set_calibration is mandatory for the same reason. Its
default was max(0, cosine), which made expand_band_lo = 0.90 mean
"cosine > 0.9" in a test and "P(same person) > 0.9" in production.
FaceTrackerFunc already threw without one; the expansion store now
matches.
One exception is recorded, in the calibration's own dedup. It is not a
close call: at 1 - 1e-7 it asks whether two vectors are the same vector,
and it runs on the fit's input, so a calibrated comparison there would
have to be calibrated by the fit it is feeding.
Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys
are read with a contains() check, so each one had been silently inert
since the field behind it was deleted -- a sweep varying one of them
measured nothing and reported an ordinary-looking F1.
TRACES: AR-024, AR-023 | SR-002
--benchmark <path> reports cumulative CPU and wall time per node and
names the node pacing the run. The pacing node is located from sampled
channel occupancy, not from time-in-node: backpressure inflates
time-in-node for everything downstream of the real bottleneck, so the
obvious measure names the victim rather than the cause.
Sampling starts with the network and stops before it is destroyed.
Channel fill is instantaneous and everything has drained by shutdown, so
a single read at the end reports an idle pipeline however congested it
was.
kill -USR1 dumps the table from a running or wedged process. Channel
occupancy identifies a stalled node -- full input, empty output --
without a debug build or a debugger, which is the difference between
diagnosing the AR-004 hang in seconds and reproducing it under gdb.
Two knobs this exposes for measurement rather than sets: SAE_CV_THREADS,
because OpenCV's TBB arena and KPN's thread-per-node are two schedulers
unaware of each other on the same cores; and SAE_CUDA_BLOCKING_SYNC,
because the default spin-wait held the embedder thread at 99.7% user
time while nvidia-powerd cut the GPU's clock from 1005 to 210 MHz.
Neither default changes until a measurement says it should.
TRACES: VR-015 | PR-004
Closes both violations SPEC.md named under "Every model gets the input it
was trained for". They are one bug, not two.
The dense stream defaulted to 12 fps, so a 100-frame TransNetV2 window
spanned ~8.3 s against the ~4 s it was trained on: half-speed motion over
twice its temporal context. Boundary timestamps stayed correct throughout,
which is exactly why the degradation was invisible and why the compressed
separation it produced (~0.50 baseline against ~0.7+ peaks) was read as a
property of the ONNX export rather than of the input.
Dedup then merged boundaries closer than a literal 0.04 s — one frame at
25 fps, and wider than a frame at 30, so two cuts on consecutive frames
became one. Nothing in scenes.json showed it; the file simply had fewer
boundaries. Native rate is where that constant did the most damage, which
is why fixing the decode rate without fixing the dedup would have made
things worse.
dedup_window_sec() now takes the median interval the detector was actually
fed and halves it. Half a frame rather than a whole one: the only thing
being merged is one frame scored by two overlapping windows, and two
distinct frames are a full interval apart.
Cost is real — dense decode is the pipeline's cost driver. It is accepted;
dense_scale and scene_stride remain the reductions that do not run the
model off-distribution. scene_threshold 0.60 was fitted against the 12 fps
input and is now stale, so VR-006 goes from Low to Medium: it is no longer
a refinement, it is a constant that no longer describes the input.
AR-002 rides along because it was already implemented, just untagged and
unverified — the register said Planned while the code was correct. The size
filter becomes FaceDetectorFunc::drop_undersized(), tested at the threshold
and at dense_scale 0.5, and checked end to end against the superhero dump,
whose smallest face is exactly its recorded 32 px minimum, so the fixture
check cannot pass vacuously.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-002, AR-011 | SR-002 | UT-002, UT-003, IT-001
Adopts the KPN backpressure fix (28e0667) and registers the application
error listener it exposes.
`push_blocking` parked a scheduler worker inside the push. Each ObjectNode
owns a private single-thread pool, so the parked thread was the only one
that could drain that node's own input: under sustained backpressure
frame_source, camera_pos, face_detector and face_aligner all slept in
nanosleep at once and the pipeline stopped. Nodes now hold the value,
release the worker, and resume on a channel space-callback.
main.cpp registers set_error_handler so a node that throws names itself
and its exception. Previously the exception was discarded at the node
boundary and survived only as "node 'x' stopped unexpectedly", which says
that a node died but not why — the missing detail that made this slow to
diagnose.
AR-004 drops from Done to Mostly. Two gaps are recorded rather than
claimed fixed: a hang surviving at roughly 1 run in 20 against a 300 s
timeout (down from every run failing), and FanoutNode still dropping on
overflow instead of parking, which sheds frames on the AR-010 scene join
precisely when the dense branch falls behind.
TRACES: AR-004 | SR-002
A track recognised on 318 of 385 frames was owned on none, so the truth file
named nobody while the matcher was accepting almost continuously.
The correlation discount was an annihilator rather than an attenuator. Weight
was 1 - P(same view), so once a track had one stored view every later frame of
that same face scored ~0.01 and the belief stopped moving. One observation just
over the accept threshold is logit(0.78) ~ 1.27, under the ownership bar — hence
recognised always, owned never.
Two changes, in the order they were found.
Correlated evidence is now attenuated by effective sample size,
n_eff = n / (1 + (n-1)·rho), each frame contributing the marginal gain. That has
the right shape at both ends: uncorrelated evidence accumulates linearly, and a
held pose converges on 1/rho rather than growing without bound. A constant floor
was tried first and rejected — it grows linearly forever, so a long shot could
out-argue genuinely varied evidence purely by lasting longer.
Combination is now weighted lazy-OR: P = 1 - (1-P_old)·(1-p)^w, stored as
log(1-P) so the update is additive and precision stays where it matters as P
approaches 1. Each frame is new evidence that this track is that actor, and the
belief is the probability that at least one sighting was right. It converges
faster than summing log-odds at the same effective count — 2.98 vs 2.53 after
two observations at p=0.78 — which is what a real clip needs.
Note that summing log-odds was already a correct sequential Bayesian update:
the matcher fits with prior 0.5, so logit(p) IS the per-frame log-likelihood
ratio and the running sum carries the prior forward. It was not wrong, it was
slow. What blocked ownership was the discount, not the combination rule.
Also fixes a real correctness bug: the observation count lived on the
discounter, which is shared by every track, so tracks pooled into one effective
sample and each was discounted by how many others happened to be on screen. It
is now a per-track parameter.
The registry's frame scope holds its lock for its lifetime and the mutex is not
recursive, so calling observe() inside a scope self-deadlocks. The pipeline
never does — separate nodes — but the test did, and hung rather than failing.
Documented at the call site.
Verified end to end: the same clip that produced zero actors now identifies
Bing Crosby and Dorothy Lamour with belief 0.97.
Suite: 96 cases, 6142 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-025 | SR-002
AR-010 — is_scene_boundary had no producer: SceneDetectorFunc was a terminal
sink writing scenes.json and never annotating the frames flowing to face
detection. The flag was permanently false, so the boundary half of AR-007's
frame-dependent association was dead code that a test could still exercise
synthetically and appear to verify.
The topology already forks after decode — dense frames to TransNetV2, sampled
frames to face detection — so this is a fork-join. SceneBoundaries is the join:
the detector publishes each window's verdict with a watermark, and an annotator
on the sampled branch stamps the flag.
The watermark is the part that matters. TransNetV2 buffers 100 frames before it
can score any of them, so at any instant it has an opinion up to some time T and
none after. Without recording T a consumer cannot tell "no boundary" from "not
scored yet", and those demand opposite behaviour — treating unscored frames as
boundary-free is exactly what makes a downstream check pass while verifying
nothing.
Buffering alone does not work, which was my first attempt. Channel depth creates
lag only when the consumer is slower, and the face branch runs four orders of
magnitude faster per frame than TransNetV2 (0.01ms vs 400ms), so its channels
drain instantly and no lag accumulates. Measured: 106 of 364 frames outran the
detector. The annotator therefore waits on the watermark explicitly. The
detector signals completion so the tail cannot deadlock, and publishes from
flush_remaining too — without that the final frames arrive with no verdict.
Boundaries are deduped on publish, matching what scenes.json does at write time.
A run of adjacent high-scoring frames is one boundary, not several; leaving them
raw made this view report 357 where the file said 13. Now the two agree exactly.
Frames past the detector's last scored window remain unverified and are counted
as such rather than silently marked boundary-free.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-010 | SR-002
A drop was reported to stderr and the process exited 0, so a run that discarded
320 frames "succeeded" and produced a truth file that looked complete. The
output in that case is a claim about footage that was never analysed, and
nothing in the file says so.
Now exits 2 and says why. Distinct from 1 (node crash) because the failures are
different: a crash produced no output, a drop produced output that cannot be
trusted.
This is also the regression test for AR-004 that otherwise did not exist. The
backpressure fix is one line in the KPN submodule — easy to lose in an update —
and with data pushes blocking, a drop can no longer occur on the data path. So
any drop now means either that fix regressed or a channel was disabled mid-run,
and both are worth stopping for.
Verified: a clean run still exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004 | SR-002
The sink no longer reconstructs presence from per-frame detections. A reaped
track already IS a window — [first_seen, last_seen] of a track an actor owned —
so it is pushed straight to the aggregator when it dies and written out as-is.
AR-012 completed end to end. The annealing pass is deleted, not disabled:
anneal_sec existed only to bridge gaps between isolated accepted frames, and a
track that survives its own gaps leaves it nothing to do. The field is REMOVED
from the output rather than zeroed — a field naming a mechanism the pipeline no
longer has is actively misleading to anyone reading a manifest, and would
outlive everyone who remembers why it reads 0.
IR-002 — schema_version 2, matching jRay/SPEC.md JR-002. Windows become objects
carrying `belief` and `route` rather than bare float pairs, so a consumer can
caveat or filter instead of treating every window as equally certain. The new
`extraction` block carries `extinction_sec` (the successor to anneal_sec, and
what a consumer actually needs to interpret a window) and `gallery_scope` —
global vs limited being the strongest single quality signal when two manifests
compete for one cut, since identical gallery_size can mean very different
recall.
AR-016 wired: a pre-write hook flushes the registry with the last timestamp
seen, so tracks still live at EOF are emitted. A film ends with faces on screen
and those tracks have not timed out; without this the closing scene's cast is
silently dropped, which reads as a recognition miss rather than a bookkeeping
bug.
IR-003 stays In Progress deliberately: the sink now writes after the flush, but
the deferred re-identification pass (AR-020) does not exist yet, so output is
still final at EOF rather than after it.
This is a BREAKING format change and part of the coordinated SR-003 bump — it
must ship together with the jRay reader and the server's acceptance of the new
shape, not ahead of them.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002, SR-003
Closes the link that made AR-012 inert: the tracker was maintaining registry
state, but nothing called observe(), so no belief accumulated, no track was ever
owned, and no presence claim could be emitted. Tracking worked and presence did
not.
The matcher now feeds every scored face to the registry as a calibrated
posterior plus its embedding. Deliberately every scored face, not only the ones
clearing prob_threshold: a run of near-misses for one actor is evidence, and
discarding it would leave ownership depending on the per-frame threshold this
redesign exists to stop relying on. The registry discounts for correlation and
decides ownership from the accumulated posterior (AR-025).
The registry is an optional dependency of the matcher. Without one it behaves
exactly as before, which keeps the replay harness and the unit tests working
unchanged rather than forcing every caller to construct a registry it does not
need.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-025 | SR-002
Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.
AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.
AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.
AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.
Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.
Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-008, AR-024 | SR-002
Adds TRACES tags to code that already satisfies a Done requirement, so coverage
reflects what exists rather than starting from zero:
AR-001 face detection, AR-005 ArcFace alignment, AR-023 calibration fit,
DP-001/DP-002 the single analysis core behind the CLI, IR-001 truth-file
emission, IR-006 the Jellyfin round trip, GR-001/GR-002 gallery build and
incremental merge, VR-001 the embedding dump, VR-002 replay through the real
nodes, VR-003 per-second scoring.
Only Done requirements are tagged. A tag on Planned work would inflate coverage
with fiction that looks plausible — the same failure family as a gate that
cannot fail, and harder to spot.
GR-005 (gallery never leaves the instance) stays untagged deliberately: it is a
prohibition satisfied by the absence of an egress path, so there is no unit that
decides it. Same shape as PR-005 in the system spec, which has no software row
for the same reason. A goal held only by prohibitions cannot be verified by
pointing at code.
Coverage 5/63 to 14/63. The three VR tags are reported as tagged-but-unexecuted
and excluded from the numerator, since their tier cannot run on the CI host —
tagging deliberately cannot raise the number on its own.
Suite still 64 cases, 3199 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-001, AR-005, AR-023, DP-001, DP-002, IR-001, IR-006, GR-001, GR-002, VR-001, VR-002, VR-003
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.
Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.
Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
Register a KPN event handler in both scene_analyze and scene_preview:
- Overflow events accumulate per-node dropped-frame counts, printed on exit.
- A Closed event from any node other than result_sink at EOF means a stage
died; trip an atomic so the main loop bails out instead of hanging on
'done' forever, and exit non-zero.
Bumps external/KPN to the commit that exposes set_event_handler / NodeEvent.
dtourolle
