The registry closed a track when the *tracker's* timestamp passed
`track_extinction_sec`. But votes arrive from the matcher, which is a separate
KPN node behind a channel, and much the slower of the pair. Backpressure —
working exactly as AR-004 intends — turns that channel's depth into lag, so
the tracker's clock can be far ahead of the last frame anybody has voted on.
Tracks were therefore closed before their evidence arrived: the votes landed
on ids that no longer existed, were counted as dropped, and the track was
emitted unowned or not at all.
The symptom is the part worth remembering: **a deeper channel produced fewer
identifications, from identical input.** On the SuperHero fixture, 5 actors /
16 windows at depth 32 against 3 actors / 5 windows at depth 10322; through
the replay harness, capacity 32 gave 5 actors and 10322 gave 0. A throughput
knob was silently changing the answer, which makes every sweep tuned against
it suspect.
The fix is not to bound the channel against `track_extinction_sec` — that
makes an algorithm constant police a throughput knob and leaves the result a
function of scheduling. It is to reap on an evidence watermark: the matcher
advances it as it folds each frame in, and a track is only finished once
everything up to its extinction point has actually been voted on. Same device
`SceneBoundaries::scored_through()` uses for the AR-010 join — a consumer past
that point is asking about frames nobody has looked at yet, and the honest
answer is to wait rather than guess.
Association keeps the tracker's clock, and separating the two is the other
half. They answer different questions: "may this detection link to that
track?" is asked now, about a box seen `track_extinction_sec` ago; "is that
track finished?" cannot be answered until every vote is in. Deferring
association to the evidence clock — which deferring the erase alone did — left
retired tracks associable for as long as the matcher lagged, so a new face
re-associated onto a long-dead track and two people merged into one window.
The watermark is monotonic and only ever *delays* a reap, so no window is
extended by it: AR-013's "a window ends at the last sighting, never after" is
a property of `emit_locked`, which takes `last_seen` and never `now`.
`dropped_votes` is exposed and reported — by main at shutdown and through the
replay bindings — because this failed silently for as long as it did precisely
because nothing counted it. It warns rather than aborts: a dropped frame means
the output describes footage nobody analysed and is always wrong, while a
dropped vote degrades a claim without falsifying it, and there is no
measurement yet of how often it happens on real content.
replay.py's channel capacity stops being the whole film. It was sized that way
to dodge a PyNode overflow drop that AR-004 has since replaced with parking,
and removing backpressure that way is what made the defect above so extreme.
Tag separators in kpn_bindings.cpp corrected to pipes between requirement
types, which the traceability gate was reporting as diagnostics; the matrix is
regenerated and reports 0 orphan tags.
149/149.
TRACES: AR-004, AR-012, AR-013, AR-025 | VR-011 | SR-002 | PR-002
Each of these was recorded as done and was done in one place out of two.
AR-011 -- the TransNetV2 dedup window. The derived window
(dedup_window_sec, median observed interval halved) reached scenes.json
and nothing else. SceneBoundaries, the path that actually feeds
is_scene_boundary to the tracker, kept the literal 0.04 s under a
comment claiming it "matches the dedup scenes.json applies, so the two
views agree". They did not agree. 0.04 is one frame at 25 fps and wider
than a frame at 30, so two cuts on consecutive frames merged into one
and the loss was invisible: the pipeline simply saw fewer boundaries.
The detector now supplies the window it derived.
AR-019 -- ownership. The register says ownership "comes from the
registry, not a second local tally". Both existed: promotion fired on a
local accepted-frame count and fell back to a local per-actor plurality
when the registry had not yet claimed the track. That fallback was
reachable in the live pipeline, not just in tests -- three accepted
frames arrive well before a posterior crosses the ownership threshold --
so in practice the plurality usually decided, and it could not see the
AR-025 correlation discounting it was meant to defer to. The tally is
gone; promotion now requires the registry's verdict, with the accepted
-frame count demoted to an explicit evidence floor.
AR-017 -- the route. DeadTrack carried belief but no route, and the sink
wrote the literal string "live", so a field the schema publishes could
not distinguish anything. AR-017's own verification asks for "deferred
and pooled routes distinguishable". Route is now an enum on the claim.
Only `live` occurs today; `deferred` exists so AR-020's pass has
somewhere to write instead of a serialisation change to make.
Also: TrackGallery::forget had no callers, under a comment asserting the
matcher called it "on a cut or track disappearance". The cut half was
true by another route; the disappearance half was not, so a track that
died quietly kept its diversity buffer until the next cut cleared
everything. Replaced with prune_dead against the registry's own
liveness, the same shape as the tracker's prune_boxes -- a second
opinion about which tracks exist is a second thing that can be wrong.
Removes dead logistic/logit helpers and fixes five TRACES tags that used
a comma where a pipe separates requirement types, which the gate had
been reporting as diagnostics.
TRACES: AR-011, AR-017, AR-019 | IR-002 | SR-002, SR-005
AR-024's register row gives its verification tier as "Static check -- no
bare cosine outside a tagged EXCEPTION". No such check existed, so the
invariant was enforced by reading, and reading had missed a live
violation.
scripts/ci/check_raw_cosine.py is that check, wired into the
traceability workflow as a blocking step. It is honest about its reach:
it catches direct cosine_similarity() uses not routed through a
calibration, and it cannot follow a cosine through a variable across
statements. That limit is documented in the script rather than left for
someone to discover after trusting a pass.
What it caught, and what this commit removes with it:
The identity matcher's no-calibration fallback thresholded raw cosine
distance (match_threshold) plus a ratio test (match_ratio,
match_ratio_ceil). Worse than the invariant breach: it fed
max(0, cosine) into TrackRegistry::observe, whose contract reads
"posterior is a calibrated probability, never a raw cosine (AR-024) ...
so the accumulation cannot be fed an uncalibrated number by a careless
caller". It could, and did. And it disagreed with the rest of the
pipeline about what "the fit failed" means -- same_person_probability
answers that with the untuned default sigmoid and a loud warning, so
association stayed in probability space while matching alone left it.
One run, two policies, no announcement.
Now one rule: cal_.probability() always, with a warning when the fit is
not real. A worse answer than a fitted calibration, a better one than a
number whose units nothing else shares.
TrackGallery::set_calibration is mandatory for the same reason. Its
default was max(0, cosine), which made expand_band_lo = 0.90 mean
"cosine > 0.9" in a test and "P(same person) > 0.9" in production.
FaceTrackerFunc already threw without one; the expansion store now
matches.
One exception is recorded, in the calibration's own dedup. It is not a
close call: at 1 - 1e-7 it asks whether two vectors are the same vector,
and it runs on the fit's input, so a calibrated comparison there would
have to be calibrated by the fit it is feeding.
Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys
are read with a contains() check, so each one had been silently inert
since the field behind it was deleted -- a sweep varying one of them
measured nothing and reported an ordinary-looking F1.
TRACES: AR-024, AR-023 | SR-002
The per-film annex was folded in after the gallery multiply by a host-side
cosine loop over a vector of {embedding, actor} structs, justified in-comment
by "tens of embeddings". AR-018/AR-019 retired that assumption: every owned
track promotes, so the annex grows with cast size and film length.
TrackGallery now holds it as a contiguous row-major matrix with a parallel
actor index — the flat_emb_/flat_actor_ shape the baked gallery already uses —
and hands newly promoted rows to the matcher once per frame. The matcher pushes
them into the similarity engine's resident matrix through a new
ISimilarityEngine::append_rows, so one SGEMM covers baked and promoted
references alike and best-of-N is a single pass over one similarity column.
Capacity doubles on overflow, and the GPU backends grow device-to-device, so a
promotion never re-uploads the gallery across the bus.
Absorbing promotions runs once per frame, after every face has been scored.
Appending mid-frame would invalidate the similarity pointer the chunk loop is
still reading, and it also removes an incidental dependence on face order
within a frame — a promotion helps subsequent frames, never the one that
produced it, which is the semantics the expansion store already documented.
OpenBLAS becomes a requirement of the CPU GEMM backend rather than an
opportunistic upgrade. That path is what CI and the cpu builder image run, so
falling back to the scalar loop in silence meant AR-027 could be measured — or
believed — on a kernel no release uses. The loop survives as the correctness
oracle the BLAS backends are diffed against, behind SAE_ALLOW_SCALAR_GEMM.
Call site 3, the deferred TBI pass, is untouched: it does not exist until
AR-020, so AR-026 stays In Progress.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-026 | UT-004, UT-005 | SR-001
AR-019 — promotion may only borrow same-identity evidence from a span where
identity is certain, so every discontinuity signal now clears the buffers rather
than just the histogram cut.
is_scene_boundary was already named in the gate but never set by anything, so
that half of it was dead until AR-010 gave it a producer. It now does what the
spec always said. The third signal, an identity contradiction, needs no code
here: AR-015 closes a track whose belief swapped, so it can no longer promote.
Ownership now comes from the registry rather than a second tally. TrackGallery
was computing its own plurality vote over accepted frames, which meant two
different answers to "who is this track" could coexist in one run — and the
expansion one ignored the Bayesian accumulation entirely, weighting thirty
near-identical looks the same as thirty distinct ones. The local tally survives
only as a fallback for callers with no registry attached, which is the unit
tests and the replay harness.
Suite: 92 cases, 6133 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-019, AR-010, AR-015 | SR-005
AR-018 — an embedding joins a track's store only if its similarity to something
already there falls inside a band, rather than merely being far from the gallery.
Above the upper bound it is redundant: another look at a pose the store already
covers, teaching the annex nothing while costing a slot a novel view could have
used. Below the lower bound it is suspect: within one track every face is the
same person by construction, so an embedding unlike everything else on the track
is evidence that construction failed — a track-ID collision or a bad detection.
Admitting it is exactly how an actor's annex gets poisoned with someone else's
face.
The old gate had only the upper half of that idea, expressed as a raw cosine
against the gallery. Both bounds are now calibrated probabilities (AR-024), so
the same number means the same thing here as in association and evidence
weighting rather than three different things.
This catches track-ID collisions EARLIER than the spread gate did — at the door
rather than at promotion — so the buffer never becomes two-person in the first
place. The spread gate stays as a second line for a track that drifts gradually
instead of jumping. The existing test was asserting the mechanism rather than
the outcome, so it was rewritten to assert what actually matters: whichever gate
fires, the outsider must not reach the annex.
Rejections are counted. A store that admits nothing is as broken as one that
admits everything, and neither is visible otherwise.
Band defaults 0.90-0.95 are working values pending VR-007; the two bounds fail in
opposite directions and must be swept separately.
Suite: 92 cases, 6133 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-018, AR-024 | SR-005
AR-003 — max_faces defaults to 0, meaning no cap. A fixed cap discards the
SMALLEST faces first, which are exactly the background cast X-Ray still credits
with scene membership, so the pipeline was systematically losing the people it
is supposed to find in crowded scenes.
This is only safe now that AR-004 landed. Previously an uncapped frame would
have pushed more work into channels that dropped on overflow, trading a visible
cap for silent loss. With backpressure the producer slows instead, so per-frame
cost is contained rather than discarded.
The matcher's kMaxFaces used to throw above 32, which made it an accidental
second cap. It sizes the similarity engine's preallocated buffer, so it bounds
memory rather than face count — the frame is now scored in batches of that size.
Memory stays bounded; faces do not.
Largest-first ordering is kept even without the cap, and the comment now says
why: the Hungarian solver tie-breaks on index order, so that ordering is
load-bearing for the replay determinism test rather than a leftover of the cap.
Verified end to end on a real clip: identical output to the capped run (385
frames, 693 faces), which is expected since that footage peaks at 4 faces per
frame — the point is the absence of a regression. The committed fixtures remain
byte-identical and valid.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-003 | SR-002
Closes the link that made AR-012 inert: the tracker was maintaining registry
state, but nothing called observe(), so no belief accumulated, no track was ever
owned, and no presence claim could be emitted. Tracking worked and presence did
not.
The matcher now feeds every scored face to the registry as a calibrated
posterior plus its embedding. Deliberately every scored face, not only the ones
clearing prob_threshold: a run of near-misses for one actor is evidence, and
discarding it would leave ownership depending on the per-frame threshold this
redesign exists to stop relying on. The registry discounts for correlation and
decides ownership from the accumulated posterior (AR-025).
The registry is an optional dependency of the matcher. Without one it behaves
exactly as before, which keeps the replay harness and the unit tests working
unchanged rather than forcing every caller to construct a registry it does not
need.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-025 | SR-002
Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.
AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.
AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.
AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.
Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.
Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-008, AR-024 | SR-002
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.
Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.
Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
dtourolle
