AR-018 — an embedding joins a track's store only if its similarity to something
already there falls inside a band, rather than merely being far from the gallery.
Above the upper bound it is redundant: another look at a pose the store already
covers, teaching the annex nothing while costing a slot a novel view could have
used. Below the lower bound it is suspect: within one track every face is the
same person by construction, so an embedding unlike everything else on the track
is evidence that construction failed — a track-ID collision or a bad detection.
Admitting it is exactly how an actor's annex gets poisoned with someone else's
face.
The old gate had only the upper half of that idea, expressed as a raw cosine
against the gallery. Both bounds are now calibrated probabilities (AR-024), so
the same number means the same thing here as in association and evidence
weighting rather than three different things.
This catches track-ID collisions EARLIER than the spread gate did — at the door
rather than at promotion — so the buffer never becomes two-person in the first
place. The spread gate stays as a second line for a track that drifts gradually
instead of jumping. The existing test was asserting the mechanism rather than
the outcome, so it was rewritten to assert what actually matters: whichever gate
fires, the outsider must not reach the annex.
Rejections are counted. A store that admits nothing is as broken as one that
admits everything, and neither is visible otherwise.
Band defaults 0.90-0.95 are working values pending VR-007; the two bounds fail in
opposite directions and must be swept separately.
Suite: 92 cases, 6133 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-018, AR-024 | SR-005
The CPU path was a scalar triple loop. It is the correctness oracle for the GPU
backends, but it is also what CI runs — there is no GPU on the N100 host — and
since AR-003 removed the per-frame face cap, a crowded frame now scores many
faces against a library-scale gallery. Scoring one face against 5000 embeddings
is 2.6 MFLOP; in scalar that does not hold up (AR-027).
S(g,f) viewed as row-major [n_faces x n_gallery] is exactly query * gallery^T,
so the loop nest collapses into a single cblas_sgemm.
OpenBLAS is optional in the build: found via pkg-config, and the scalar path
remains when it is absent so no hard dependency is added and the two can be
diffed when a similarity looks wrong. The configure step warns rather than
failing, since a developer without it should still get a working tree.
The test target links it too. Without that the suite compiles the scalar
fallback while the builder image ships CBLAS, so CI would be verifying a kernel
that is not the one running in production — the same class of mistake as testing
a path the gate never executes.
Recorded as required (not optional) in the DP-007 image, for the same reason.
Suite: 92 cases, 6136 assertions, with CBLAS compiled in.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-026, AR-027, DP-007 | SR-001
AR-010 — is_scene_boundary had no producer: SceneDetectorFunc was a terminal
sink writing scenes.json and never annotating the frames flowing to face
detection. The flag was permanently false, so the boundary half of AR-007's
frame-dependent association was dead code that a test could still exercise
synthetically and appear to verify.
The topology already forks after decode — dense frames to TransNetV2, sampled
frames to face detection — so this is a fork-join. SceneBoundaries is the join:
the detector publishes each window's verdict with a watermark, and an annotator
on the sampled branch stamps the flag.
The watermark is the part that matters. TransNetV2 buffers 100 frames before it
can score any of them, so at any instant it has an opinion up to some time T and
none after. Without recording T a consumer cannot tell "no boundary" from "not
scored yet", and those demand opposite behaviour — treating unscored frames as
boundary-free is exactly what makes a downstream check pass while verifying
nothing.
Buffering alone does not work, which was my first attempt. Channel depth creates
lag only when the consumer is slower, and the face branch runs four orders of
magnitude faster per frame than TransNetV2 (0.01ms vs 400ms), so its channels
drain instantly and no lag accumulates. Measured: 106 of 364 frames outran the
detector. The annotator therefore waits on the watermark explicitly. The
detector signals completion so the tail cannot deadlock, and publishes from
flush_remaining too — without that the final frames arrive with no verdict.
Boundaries are deduped on publish, matching what scenes.json does at write time.
A run of adjacent high-scoring frames is one boundary, not several; leaving them
raw made this view report 357 where the file said 13. Now the two agree exactly.
Frames past the detector's last scored window remain unverified and are counted
as such rather than silently marked boundary-free.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-010 | SR-002
Making SceneDetectorFunc a pass-through does not work. TransNetV2 buffers 100
dense frames before it can score any of them and trusts only each window's
centre, so a boundary at time T is not known until roughly 3.3s after T at
30 fps. The face pipeline runs on a parallel branch and has long since passed T.
An association hint that arrives after the association is worthless.
Three options recorded with their costs: two-pass (correct, doubles the decode
that already dominates runtime), delaying the face branch (couples the two
branches' timing, which invites heisenbugs under backpressure), or leaving it
unwired.
Leaving it unwired costs less than it looks, which is what makes this a decision
rather than a defect. The redesign made cuts and boundaries do the same thing —
both say "spatial continuity is broken, associate on embedding" — so TransNetV2
adds nothing over the histogram except on transitions the histogram cannot see:
slow dissolves and fades. That gap is real but narrow.
Where TransNetV2 still earns its cost is AR-019, whose promotion gate wants a
span free of cuts and boundaries. A late answer is fine there, because promotion
happens on track confirmation rather than per frame — so it can be wired
offline against the collected boundary list, off the hot path entirely.
Recommendation: leave the association path on is_cut alone, wire boundaries into
AR-019, and revisit if dissolve-heavy material shows association failures.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-010, AR-019 | SR-002
AR-003 — max_faces defaults to 0, meaning no cap. A fixed cap discards the
SMALLEST faces first, which are exactly the background cast X-Ray still credits
with scene membership, so the pipeline was systematically losing the people it
is supposed to find in crowded scenes.
This is only safe now that AR-004 landed. Previously an uncapped frame would
have pushed more work into channels that dropped on overflow, trading a visible
cap for silent loss. With backpressure the producer slows instead, so per-frame
cost is contained rather than discarded.
The matcher's kMaxFaces used to throw above 32, which made it an accidental
second cap. It sizes the similarity engine's preallocated buffer, so it bounds
memory rather than face count — the frame is now scored in batches of that size.
Memory stays bounded; faces do not.
Largest-first ordering is kept even without the cap, and the comment now says
why: the Hungarian solver tie-breaks on index order, so that ordering is
load-bearing for the replay determinism test rather than a leftover of the cap.
Verified end to end on a real clip: identical output to the capped run (385
frames, 693 faces), which is expected since that footage peaks at 4 faces per
frame — the point is the absence of a regression. The committed fixtures remain
byte-identical and valid.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-003 | SR-002
The recorded pointer was be6e922 — the backpressure fix as originally committed,
before it was rebased onto KPN master. That commit exists on no pushed branch,
so a fresh clone of this branch could not fetch the submodule at all.
Now 6595e6e, the same change on KPN master.
Worth noting for next time: rebasing a submodule commit after the superproject
has already recorded it silently invalidates the pointer. Nothing in the
superproject's status shows it, because the submodule working tree is clean and
at a valid commit — just not the one recorded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004 | SR-002
Tier T2 — composition rather than units. The registry tests construct awkward
states directly; these feed the pieces real 480x360 footage with the cuts, gaps
and crowded frames that synthetic input does not produce.
Six cases:
- fixture integrity: exact frame and face counts, contiguous face_offset, and
the embedder identity each dump carries (GR-004). The counts are asserted
exactly rather than approximately, which was impossible before AR-004 — what
a lossy run dropped depended on timing.
- determinism: replaying a fixture twice gives identical track ids and windows.
This is the property the whole fixture strategy rests on; without it every
golden output derived from a fixture is unreliable and the CI replay tier is
worthless.
- every face is assigned a track, and flush leaves nothing open — a track still
live at EOF is a window that never reaches the output.
- windows are well-formed and inside the clip. A window ends at the last
sighting, so it can never extend past the footage that produced it.
- a longer extinction window yields fewer, longer tracks. On the sparse fixture
(140 faces over 385 frames) that is the difference the constant actually
makes: absorbing a gap versus splitting a window.
- the cut-heavy fixture still contains cuts. This guards the corpus, not the
code: a regeneration that produced cut-free fixtures would leave the
association tests passing while silently testing nothing.
Driving the functors directly rather than through a KPN network is deliberate —
no threads, no channels, no scheduling, so the same input gives the same output.
Suite: 86 cases, 6106 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004, AR-012, AR-013, VR-001, VR-002 | SR-002
A drop was reported to stderr and the process exited 0, so a run that discarded
320 frames "succeeded" and produced a truth file that looked complete. The
output in that case is a claim about footage that was never analysed, and
nothing in the file says so.
Now exits 2 and says why. Distinct from 1 (node crash) because the failures are
different: a crash produced no output, a drop produced output that cannot be
trusted.
This is also the regression test for AR-004 that otherwise did not exist. The
backpressure fix is one line in the KPN submodule — easy to lose in an update —
and with data pushes blocking, a drop can no longer occur on the data path. So
any drop now means either that fix regressed or a channel was disabled mid-run,
and both are worth stopping for.
Verified: a clean run still exits 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004 | SR-002
Five HDF5 embedding dumps from bali/ — Road to Bali (1952) — 3.6 MB total,
generated at 5 fps with a 32 px minimum face. CI never calls a model, so
inference happens on a GPU host and CI replays these as data; everything
downstream of embedding is cheap CPU maths.
Public domain is the reason this corpus rather than a convenient one: derived
fixtures can be committed, where anything cut from a copyrighted title could not
live in the repository at all.
The set covers distinct behaviours rather than being five of the same thing:
bali_28 has 9 cuts, so it exercises shot/reverse-shot association (AR-007);
bali_46 is sparse at 140 faces over 385 frames, so it exercises gaps and
extinction (AR-013); bali_13 is the busiest at 4 faces per frame; bali_31 is
short at 29s. All five recorded zero drops.
Both pinned parameters are consequences of measurements, not defaults: 5 fps
because 1 fps over a 77s clip is 77 frames, too thin for an extinction window
measured in tens of seconds; 32 px because that is the VR-005 floor, and the
corpus is 480x360 so a stricter value would reject most of what is there.
make_fixtures.sh regenerates them. Reproducibility is the requirement — a
fixture whose provenance is unknown is worse than none, because it will be
trusted. These are byte-reproducible only because of AR-004: before node
outputs blocked rather than dropped, the same command produced different dumps
run to run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: VR-001 | PR-002
Picks up the KPN fix: node data outputs block on a full channel rather than
dropping. Sentinels stay out-of-band, so EOF can always overtake a stalled data
path and the hold-and-wait deadlock that comment warns about is not reachable.
Verified on a 77s clip at 5 fps, which should yield 385 sampled frames:
before 65 written, 320 dropped, 29s, two runs differ
after 385 written, 0 dropped, 17s, two runs byte-identical
The determinism is the part that matters. Golden fixtures were impossible while
what got dropped depended on timing; VR-001 fixture generation is unblocked by
this, and so is the CI replay strategy that depends on it.
Faster rather than slower, which is worth recording because the intuition runs
the other way: a dropped frame has already cost its decode, and the overflow
exception cost more still.
AR-004 is not fully closed. Channel capacity remains a count of items, while a
face carries a 112x112 crop and a 512-float embedding — so a crowded frame
occupies far more memory per slot than a sparse one. Bounding by bytes in
flight is the remaining half, and it matters once max_faces is removed (AR-003).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004, VR-001 | SR-002
Backpressure cannot be implemented in this repository. Every node output in KPN
uses the dropping push() (pool_node.hpp:404 and :710, plus branch, fanout and
interrupt_node). A lossless push_blocking() already exists on both Channel and
OutputPort — "wait for the consumer to drain instead of dropping; the producer
just runs slower" — and nothing calls it. The fix is a per-channel policy or a
network default in KPN, and this pipeline should select lossless: a dropped
frame here does not degrade a result, it silently changes one.
Measured rather than inferred. One 77s clip at 5 fps should yield ~385 sampled
frames. On CPU it produced 49, ending at 51s, with 285 dropped at camera_pos
and 51 at face_aligner. Rebuilt with CUDA the same clip ran in 29s and reached
EOF correctly, and still dropped 320 at camera_pos, yielding 65. Faster
hardware moves where the queue backs up; it does not change what happens when
it does — which is why this is a correctness requirement rather than a
throughput one.
Raising channel capacity is therefore a stopgap: it lowers the probability of
overflow without changing the behaviour on overflow, and the failure it hides
is silent corruption of the output.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-003, AR-004, VR-001 | SR-002
Replaces the 66px working estimate with the sweep result. 258 probes degraded
to each size and matched against a native-resolution gallery: 16px 46.5% TPI,
24px 93.4%, 32px 98.1%, 40px 99.2%, flat to 112px. The knee is 24-32 and 32
sits within about a point of the ceiling.
The estimate was roughly twice too strict. At 66px a large share of usable
faces would have been discarded, and on 480x360 sources most of them — which
is exactly the resolution of the fixture corpus.
The more useful finding: false identification was 0.0 at every size, including
12px. Small faces fail by becoming unidentified, never by being attributed to
the wrong actor. That asymmetry is what makes a low threshold safe — the cost
of admitting a marginal face is a miss, not a false claim.
Caveat recorded rather than assumed: FPI grows with gallery size, so 258 actors
understates it against a full library. Treat 0.0 as an observation at this
scale, not a property.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-002, VR-005 | SR-002, PR-002
Records `bali/` — five ~77s clips of Road to Bali (1952) — as the fixture
source. Public domain, which is the point rather than a convenience: derived
fixtures can be committed, where anything cut from a copyrighted title could
not live in the repository at all.
Makes explicit what the tier table only implied: CI never calls a model. Not a
preference — the embedder measures ~930 ms/frame on the CPU provider, so a 77s
clip at 5 fps is six minutes of inference. Every model invocation happens
locally and CI consumes the result as data, which is what makes the T1/T2 split
load-bearing rather than stylistic.
Two properties of the corpus to design around: 480x360 puts many faces below
the AR-002 66px minimum, so generation must set and record --min-face-px; and
77s at 1 fps is too thin to exercise an extinction window measured in tens of
seconds, so fixtures want 5 fps.
The finding that matters: a trial dump produced 49 frames of an expected ~385,
stopping at 51s of 77s, with 285 frames dropped at camera_pos and 51 at
face_aligner on channel overflow. Channels drop rather than block, and what
drops depends on timing, so the same command twice can yield different dumps.
Golden fixtures cannot be built on that — AR-004 is a prerequisite for VR-001
fixtures, not just a throughput concern for crowd scenes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-004, VR-001 | PR-002
The sink no longer reconstructs presence from per-frame detections. A reaped
track already IS a window — [first_seen, last_seen] of a track an actor owned —
so it is pushed straight to the aggregator when it dies and written out as-is.
AR-012 completed end to end. The annealing pass is deleted, not disabled:
anneal_sec existed only to bridge gaps between isolated accepted frames, and a
track that survives its own gaps leaves it nothing to do. The field is REMOVED
from the output rather than zeroed — a field naming a mechanism the pipeline no
longer has is actively misleading to anyone reading a manifest, and would
outlive everyone who remembers why it reads 0.
IR-002 — schema_version 2, matching jRay/SPEC.md JR-002. Windows become objects
carrying `belief` and `route` rather than bare float pairs, so a consumer can
caveat or filter instead of treating every window as equally certain. The new
`extraction` block carries `extinction_sec` (the successor to anneal_sec, and
what a consumer actually needs to interpret a window) and `gallery_scope` —
global vs limited being the strongest single quality signal when two manifests
compete for one cut, since identical gallery_size can mean very different
recall.
AR-016 wired: a pre-write hook flushes the registry with the last timestamp
seen, so tracks still live at EOF are emitted. A film ends with faces on screen
and those tracks have not timed out; without this the closing scene's cast is
silently dropped, which reads as a recognition miss rather than a bookkeeping
bug.
IR-003 stays In Progress deliberately: the sink now writes after the flush, but
the deferred re-identification pass (AR-020) does not exist yet, so output is
still final at EOF rather than after it.
This is a BREAKING format change and part of the coordinated SR-003 bump — it
must ship together with the jRay reader and the server's acceptance of the new
shape, not ahead of them.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002, SR-003
Closes the link that made AR-012 inert: the tracker was maintaining registry
state, but nothing called observe(), so no belief accumulated, no track was ever
owned, and no presence claim could be emitted. Tracking worked and presence did
not.
The matcher now feeds every scored face to the registry as a calibrated
posterior plus its embedding. Deliberately every scored face, not only the ones
clearing prob_threshold: a run of near-misses for one actor is evidence, and
discarding it would leave ownership depending on the per-frame threshold this
redesign exists to stop relying on. The registry discounts for correlation and
decides ownership from the accumulated posterior (AR-025).
The registry is an optional dependency of the matcher. Without one it behaves
exactly as before, which keeps the replay harness and the unit tests working
unchanged rather than forcing every caller to construct a registry it does not
need.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-025 | SR-002
Merges feature/tracker-registry. See e9aea3f for the detail; in summary the
tracker no longer owns track state, association weights embedding over position
whenever position is uninformative, and every similarity comparison is a
calibrated probability.
Four constants retired: track_max_embed_dist, cut_revive_sim,
cut_inactive_max_frames, track_max_frames_missing.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-008, AR-024 | SR-002
Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.
AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.
AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.
AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.
Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.
Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.
Suite: 80 cases, 3250 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-007, AR-008, AR-024 | SR-002
A stale reference to 'the runtime/exact tiers' as the fallback for media too
short to carry an audio signature. The exact tier keyed on a file hash and was
withdrawn on legal grounds: it fingerprinted the individual copy a user holds
rather than the cut the timings describe.
The pipeline never emitted a video_hash, so nothing in the code changes — but a
spec that still names a withdrawn tier is what makes the withdrawal look like an
oversight to the next reader, which is exactly how it nearly got re-added.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Moves two responsibilities inside the registry that callers should never have
been trusted with.
AR-025 — per-frame evidence is discounted for correlation by the registry
itself, via EvidenceDiscounter. Log-odds accumulation is only valid for
independent observations, and consecutive frames of one track are anything but:
near-identical pose, lighting and expression. Accumulated naively, thirty frames
of the same face at the same angle drive the posterior to certainty on what is
effectively one measurement.
Each observation is weighted by how much it adds — a view already contributed
counts for ~nothing, a genuinely new pose counts in full. This reuses the
novelty judgement gallery expansion already makes rather than inventing a second
one. The discounter is a separate class the registry holds, so it stays testable
and swappable, but it is a constructor argument rather than an option: there is
no correct way to accumulate without it.
AR-024 — observe() takes a calibrated probability and converts to log-odds
internally. A caller can no longer hand it a raw cosine, which would have been
silently wrong rather than obviously so. Retiring the remaining raw-cosine
constants in the tracker is still open.
DeadTrack now reports effective_obs alongside observations: the raw count and
the evidence that actually counted. A large gap between them is a track the
camera stared at, and worth seeing.
Three tests, one of which is the point: two tracks given the same number of
observations at the same posterior, one repeating a single view and one seeing
eight distinct ones, must not end up equally confident. Without discounting they
would be identical.
Fixed a test that asserted a belief swap on tied evidence. A tie leaves
ownership where it is — a challenger must out-accumulate the incumbent, since
one contrary observation is noise. The original test passed only because it fed
raw log-odds directly.
Suite: 78 cases, 3245 assertions. Coverage 20/63 to 22/63.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-024, AR-025 | SR-002
The spine of the redesign. Presence is now the extent of a track an actor owns,
[first_seen, last_seen], rather than the subset of frames in which recognition
happened to succeed. An actor recognised only at the end of a long track is
present for all of it, which is what the scene-scoped ground truth actually
records.
AR-013 — `last_seen` as an optional carries the entire liveness state: unset
means on screen, set means went off at that timestamp and still revivable,
reaped means emitted and erased. No missing-frame counter, no expired flag. It
subsumes the tracker's existing two-pool split, so there is no separate revival
path — matching a dormant track is ordinary inter-frame association.
The asymmetry is the point: interior gaps are claimed, the trailing cool-down is
not. A face lost and re-associated within the timeout never closed its track, so
the gap is presence — someone briefly occluded has not left the scene. But a
track that dies ends at its last sighting, never at the death time. That is
precisely the over-claim the retired extinction_sec keep-alive produced, where
presence ran on into the closing credits.
AR-014 — a belief swap A→B closes the track and opens a successor at the swap
frame. Not a correction: two non-twins both clearing the threshold on one face
is not realistic, whereas a track_id carried across a viewpoint change onto a
different person is. Treating it as a swap-and-continue would emit one window
blending two people; treating it as a boundary yields two that are each right.
AR-015 — two live tracks owned by one actor means at least one is wrong, since a
person cannot be in two places at once. A reverse index catches it on the update
that causes it rather than by scanning. This makes identity a third cut
detector, independent of the histogram and TransNetV2 and firing where those
failed.
AR-016 — flush() closes tracks still live at EOF. Without it a film ending
mid-shot silently drops its closing cast, which presents as a recognition miss
rather than a bookkeeping bug.
Reaping hands the dead track to the aggregator and erases it, so the registry
holds only live tracks and its size is bounded by concurrent on-screen faces
rather than growing with the film.
Locking: a frame's association pass is atomic as a unit via FrameScope, since
per-call locking would let another thread observe a half-updated frame.
owner() reads tally and verdict under one lock — separately, a track could be
both unowned and owned within a single promotion decision. A vote for an
already-reaped track is dropped and counted, because a nonzero count means the
timeout is shorter than the matcher's lag.
11 unit tests, driven directly against the registry with no network and no
fixture — the awkward cases are constructed rather than hunted for. Suite: 75
cases, 3236 assertions. Coverage 14/63 to 20/63.
Not yet wired into FaceTrackerFunc; that is AR-007/AR-008.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-012, AR-013, AR-014, AR-015, AR-016, AR-017 | SR-002
Adds TRACES tags to code that already satisfies a Done requirement, so coverage
reflects what exists rather than starting from zero:
AR-001 face detection, AR-005 ArcFace alignment, AR-023 calibration fit,
DP-001/DP-002 the single analysis core behind the CLI, IR-001 truth-file
emission, IR-006 the Jellyfin round trip, GR-001/GR-002 gallery build and
incremental merge, VR-001 the embedding dump, VR-002 replay through the real
nodes, VR-003 per-second scoring.
Only Done requirements are tagged. A tag on Planned work would inflate coverage
with fiction that looks plausible — the same failure family as a gate that
cannot fail, and harder to spot.
GR-005 (gallery never leaves the instance) stays untagged deliberately: it is a
prohibition satisfied by the absence of an egress path, so there is no unit that
decides it. Same shape as PR-005 in the system spec, which has no software row
for the same reason. A goal held only by prohibitions cannot be verified by
pointing at code.
Coverage 5/63 to 14/63. The three VR tags are reported as tagged-but-unexecuted
and excluded from the numerator, since their tier cannot run on the CI host —
tagging deliberately cannot raise the number on its own.
Suite still 64 cases, 3199 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-001, AR-005, AR-023, DP-001, DP-002, IR-001, IR-006, GR-001, GR-002, VR-001, VR-002, VR-003
The tool's defaults already exclude `vendor`, which covers the submodule at
scripts/vendor/jray-project. Setting the key explicitly is a trap worth
documenting: it REPLACES the defaults rather than extending them, and matching
is on path components rather than prefixes — so ["scripts/vendor"] matches
nothing while silently dropping __pycache__, node_modules, build and the rest.
Verified: the submodule's source is not scanned, and the only vendored path in
the report is the system spec it reads for PR/SR orphan checking.
Coverage after the merges: 5/63, 0 orphans. Every tag names a real requirement,
and nothing claims a requirement that is still Planned.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
GR-004 — a gallery built with one embedding model is meaningless with another.
Cosine similarities across models are garbage but look entirely plausible, so
this fails silently and expensively; every measurement taken against a
mismatched pair would have been quietly wrong.
The stamp is the model basename plus a SHA-256 of its bytes, with embed_dim as
a cheap extra guard. The hash decides and the name explains, because neither
works alone: a name is a promise rather than a fact — models get re-exported in
place under an unchanged filename, which is exactly the case where the weights
differ and nothing else does — while a bare hash mismatch tells an operator
nothing actionable.
Mismatch is fatal in every mode with no bypass. Unstamped only warns, because
unstamped is unknown rather than known-bad, and an error firing on every legacy
gallery trains people to reach for the bypass reflexively. scripts/stamp_gallery.py
binds an existing gallery in place with no re-embedding, so the warning is a
migration step rather than a permanent state; --require-gallery-stamp promotes
it to an error once a site has migrated.
Two gaps found that would have defeated the requirement outright:
- Embedding dumps carried no stamp, so a replay — which has no live embedder —
had nothing to check the gallery against. Dumps now carry embedder_model and
embedder_sha256 as root attributes. Additive; schema_version stays 1. This is
the same gap the dump audit identified independently.
- --merge produced one file holding two embedding spaces, which no later check
can untangle. Merge paths now verify before writing.
The stamp also survives identity_matcher's calibration write-back, which would
otherwise have stripped it on the first analysis run — the check would have
worked exactly once.
Conflicts resolved additively: both branches appended a source to sae_gallery
and to the test target, and both edited the GR-004 register row.
Merged suite: 64 cases, 3199 assertions, passing on CPU with no GPU.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: GR-004, VR-001 | SR-001
Implements the server spec §3 construction: a 120 s window centred on the media
midpoint, downmixed and resampled to mono 11025 Hz, a 4096/1024 Hann STFT, 32
log-spaced bands over 300-3000 Hz, one byte per frame carrying a 5-bit peak band
and a 2-bit energy class, base64 with a v1: prefix.
IR-004 — the signature itself, in src/audio_signature.{hpp,cpp}. Decode reuses
the already-linked FFmpeg libraries; libswresample was missing from ffmpeg_libs
and is added. The FFT is written out rather than taken from a library: the
output must be bit-identical against a separate C# implementation, so a
dependency whose version can change the numerics is a liability.
IR-005 — a golden fixture at tests/fixtures/audio/, verified against an
independent Python implementation producing identical bytes. FLAC rather than
WAV because 120 s of 11025 Hz PCM is 2.6 MB and does not compress in git; both
decode to identical samples. The PCM checksum is asserted separately from the
signature so a codec-level divergence is distinguishable from a DSP one.
IR-007 — media under 120 s emits no signature at all, since the centred window
underflows. The rule must be identical in both producers or signatures never
match on exactly the short items most likely to be misidentified.
IR-008 — the v1: prefix is emitted and honoured, so a future change to the DSP
chain is detectable rather than silently non-matching.
Six parameters the spec left undefined had to be pinned to reproduce a byte
stream at all: periodic Hann, band value as the mean of linear magnitudes, ties
to the lowest band, the energy-class definition and its thresholds, byte layout,
and the base64 alphabet. These are now normative in the server spec — left only
in a C++ header, the C# side would have guessed and diverged.
Not yet emitted into the truth file; that is the coordinated schema_version bump
under IR-002/IR-003.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: IR-004, IR-005, IR-007, IR-008 | SR-003
Committed rather than ignored, matching house precedent: coverage becomes
visible to anyone browsing the repo, and its movement over time is real history
worth having in the log.
jray-project is added at scripts/vendor/jray-project and the extractor is used
from there. Only two files are repo-local: traceability.toml, which carries
everything repo-specific, and the CI workflow that invokes the vendored gate.
The extractor is deliberately NOT copied in. One implementation, parameterised
by config — a second copy would drift from the first, and the tool already
proves it works unchanged against all three registers.
Enables system_spec so PR/SR orphan checking runs: previously uncheckable,
because the system spec lived outside every component's checkout.
Gate is green at 0.0% of 63 requirements, which is correct — nothing is tagged
yet. Eleven are flagged unverifiable on this CI host and excluded from the
numerator rather than counted.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stamp is model basename + SHA-256 + embed_dim: the hash decides, the name
explains. A name alone is a promise rather than a fact — models get re-exported
in place under an unchanged filename, which is exactly the case where weights
differ and nothing else does. A hash alone is unactionable in an error message.
Mismatch is fatal in every mode with no bypass. Unstamped only warns, because
unstamped is unknown rather than known-bad, and an error that fires on every
legacy gallery trains people to reach for the bypass. A migration script binds
existing galleries in place with no re-embedding, so warn is not permanent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: GR-004 | SR-001
Adds the build/deploy story that was missing: containerised builder images for
cpu / cuda / rocm, and release jobs producing prebuilt binaries so a first
install need not compile.
States explicitly that this does not reverse DP-005. That requirement rejects
Docker as a *runtime* — GPU passthrough is fragile and exists only because of
the container. Using it as a *build* environment is the opposite case, and lets
one machine produce binaries for backends it cannot itself run. Build in a
container, run natively.
Two things deliberately cannot ship, and the installer must not imply otherwise:
TensorRT engines are GPU-architecture and TRT-version specific, so
build_trt_engines.sh still runs on the target; and models are ~725 MB in LFS,
orthogonal to the binary.
The base image is chosen by the OLDEST glibc to be supported, not by
convenience — a binary built in a container runs against the host's glibc, and
getting this wrong fails at load with GLIBC_2.xx not found. Accelerator runtimes
have the same shape of problem, so each image documents its compatible
CUDA/ROCm range and the installer checks it rather than discovering a mismatch
at first inference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: DP-005, DP-007, DP-008 | PR-004
Corrections from the dump audit and the completed agent work:
- AR-010 is not started, not in progress: is_scene_boundary has no producer
anywhere. SceneDetectorFunc is a terminal sink writing scenes.json and never
annotates the frame, so the field is permanently false and the dump column a
constant 0. A replay test of the frame-dependent track_alpha would pass
vacuously — the worst failure mode for a verification gate.
- T1 (functor-level) becomes the primary verification tier, not T2. KPN node
functors are plain callables constructed outside the network, so a node is
tested by calling operator() with hand-built inputs. That removes four
hazards at once: fixture provenance, replay-from-frame-0, cross-test state
leakage, and replay-harness nondeterminism. It also means a dead upstream
producer no longer blocks testing its consumer.
- VR-010 (dump provenance) and VR-011 (replay harness rewrite) added. A dump
made with LVFace is currently byte-indistinguishable from one made with
w600k-R50 — the GR-004 problem again, in the dump.
- Four requirements had no verification tier at all; the traceability gate
found them.
- DP-007: CI builder image, CPU-only, pinned by tag in the Gitea container
registry. Corpus fixtures go to the package registry rather than LFS: LFS is
pulled on clone and would tax every developer for data only CI reads.
- IR-004/005/007/008 marked done; the v1 DSP parameters they had to pin are
now normative in the server spec.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: AR-010, DP-007, IR-004, IR-005, IR-007, IR-008, VR-001, VR-010, VR-011 | SR-002, SR-003
Implements the content-derived spectral-peak signature from
JRay-public-server/SPEC.md §3 so a truth file is self-identifying: 120 s
window centred on the media midpoint, mono at 11025 Hz, 4096/1024 Hann
STFT, 32 log-spaced bins over 300-3000 Hz, one byte per frame (5-bit peak
band + 2-bit energy class), base64, `v1:` prefix.
Audio decode is a second stream from the FFmpeg libraries the pipeline
already links for video; libswresample is added to the existing
ffmpeg_libs interface target. The FFT is written out rather than pulled
from a library for the same reason the plugin vendors one: the output has
to be bit-identical across two languages, so a dependency whose version
could change the numerics is a liability.
The server spec fixes the geometry but not enough to reproduce a byte
stream — Hann periodicity, band aggregation, the energy-class definition,
tie-breaking and the base64 alphabet are all unconstrained by it. Those
are pinned in audio_signature.hpp and mirrored in the golden fixture, so
the plugin can be implemented from the fixture alone.
IR-005: tests/fixtures/audio/ carries a deterministic 120 s tone (FLAC —
lossless, so identical PCM to the WAV make_fixture.py emits, and 3.5x
smaller in git) plus the signature it must produce, the decoded-PCM
checksum and the full parameter contract. That directory is the artefact
shared with the plugin repo; the PCM checksum is separate from the
signature so a codec-level difference is distinguishable from a DSP one.
IR-007: media under 120 s emits no signature. Same for a file with no
audio stream or one that will not open — UR-9 is an enhancement and must
never be able to break a fetch.
Verified against an independent Python reference implementation: same
bytes. All 32 bands and all 4 energy classes appear in the golden vector,
and the window-centring test wraps the fixture in 90 s of silence either
side and requires the golden value back.
Not wired into the truth-file output yet — that is the schema_version
bump under IR-002/IR-003 and is deliberately out of scope here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The phase structure encoded ordering assumptions that stopped being true as the
design changed, and its Phase 2 still described retuning constants that are now
withdrawn. Ordering is now derived from per-requirement dependencies instead:
anything with no unmet dependency is startable.
Carries over the TrackRegistry design (now keyed to AR-012/AR-013) and records
what was withdrawn from the old plan, including the --presence-mode flag —
comparison against old behaviour uses recorded reference output rather than a
second live code path.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the requirements baseline for the pipeline redesign:
- SPEC.md — software requirements with Current/Gap deltas per item, so the
document doubles as a work list.
- requirements.md — stable flat IDs (AR/DP/IR/GR/VR) with parent traces,
priorities, statuses, and a per-requirement verification plan. Replaces the
thematic A1..E8 scheme, which had already produced an A1a and an out-of-order
E6; IDs are now permanent and never reused.
- IMPLEMENTATION-PLAN.md — phased work.
The central change is AR-012: presence follows track extent rather than
per-frame recognition, so a window starts when an actor appears rather than
when the recogniser first succeeded. anneal_sec and extinction_sec are
withdrawn rather than retuned — a track that survives its own gaps leaves them
nothing to do.
Verification is shaped by CI running on an N100 with no dGPU: the existing
HDF5 dump makes everything downstream of embedding replayable on CPU, which
covers the bulk of the redesign.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
trtexec rejects --minShapes/--optShapes/--maxShapes for a fully static model
("Static model does not take explicit shapes"). TransNetV2's input is fixed at
1x100x27x48x3, so the shape comes from the model itself.
Gallery build now over-fetches TMDB/Wikidata candidates by a configurable
factor: near-duplicate stills (the same photo at different crops or
resolutions) are discarded after embedding, so downloading exactly
images_per_actor left actors short of that many *distinct* embeddings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
build_trt_engines.sh hardcoded 'input.1' for the ArcFace and SCRFD shape
profiles, which only matches arcface_w600k_{r50,mbf}. Building engines for
any other embedder failed with:
Cannot find input tensor with name "input.1" in the network inputs!
Input names differ per model: LVFace-B_Glint360K uses 'data', arcface_r18
uses 'input', arcface_w600k_{r50,mbf} use 'input.1'. This matters now that
LVFace-B is the default embedder (src/config.hpp), so ARCFACE_MODEL=<LVFace>
is the expected path.
Read the name from each model via onnxruntime at build time.
OpenCV: distros (Arch/CachyOS) now ship OpenCV 5 as default. The config
package rejects a 5.x install when find_package requests 4, so probe for 5
first and fall back to 4. All components used here (core, imgproc, imgcodecs,
videoio, dnn, objdetect, highgui) exist in both.
TensorRT: nvinfer1::Dims5 was removed in TRT 10 (Dims2..Dims4 remain in
NvInferLegacyDims.h). Build the TransNetV2 rank-5 input shape via the generic
nvinfer1::Dims, which is valid on both 8.x and 10.x.
Her bbox is frozen at identical coordinates for t=2450 and t=2451; the
dump's own per-frame detections show only one real face at t=2451, and
it matches the Chloë Sevigny box (IoU 1.0), not hers. The frame is one
ghost overlapping one fresh misidentification, not two competing fresh
identities as previously written.
Replaces narrative claims with verified numbers across all report pages:
- Cross-model held-out validation (LVFace/mbf/r18, all 5 held-out
films): LVFace wins every film outright, not just "consistent with"
the training-set pick. r50 dropped from the detailed comparison
(gallery has ~30% fewer reference images per actor than the other
three models on identical source photos).
- Per-film training breakdown: LVFace does not win every training
film (mbf beats it on Lord of War); the 75.3% macro figure hides a
10.7pp spread.
- Gallery coverage computed per film (20.3%-78.6%) instead of one
flat 67%-missing average.
- Found and fixed a real scoring bug in optimize.py: a candidate
whose hardest film's replay timed out was averaged over survivors
instead of penalized, silently rewarding partial coverage. Affected
3 of 16 training combos; corrected throughout, and optimize.py now
scores an incomplete evaluation f1=0.0 instead of averaging over
whichever films happened to finish.
- Every FPI frame in the deep dive now comes from the proper montage
renderer (Onscreen/Offscreen panel, ghosts never drawn as boxes),
never the bare-box debug overlay used earlier.
- Every distinct out-of-cast name across all 9 films gets its own
frame at its first appearance (9 names, 4 films), not a
single-example spot check: 2 ground-truth gaps, 1 photograph
misread as a person, 6 genuine lookalike confusions.
- New methodology.md: the scene-level-vs-per-second scoring mismatch
that the rest of the report assumes, written out once.
- Cut the deadlock/gdb debugging narrative from the experiment log;
kept the one fact that matters (KPN's node/network split lets the
expensive GPU stage run once and the cheap stage replay against
cached embeddings).
- Plain declarative style throughout, no em dashes, no blog voice.
- switch report frames to the scene best/worst montage renderer
(Onscreen/Offscreen panels + TPI/FPI/FN legend): perfect-second hero,
wedding couple, funeral 19-of-20, polygraph bridging, crew-scene FN
ceiling, Robert Patrick ground-truth gap, rapid-cut double label,
Herbie Hancock on an in-fiction screen
- deep dive restructured: extinction bridging framed as designed
behavior with a measurable cost (debug overlay draws the boxes; the
shipped output is presence windows), plus the face-vs-presence
ceiling and two X-Ray-is-wrong exhibits
- Material polish: light/dark palette toggle, landing-page grid cards,
figure/caption CSS, how-to-read admonition; site_url set so 404 links
resolve under the Pages subpath
- README: perfect-second and screen-call frames committed (gitignore
exceptions), readme_example.jpg retired
- build_site.sh: stage_frame helper downscales montage frames to 1920px
and pulls any missing montage-frames packages
- experiment_charts.py generates 4 figures from experiments/ artifacts:
held-out per-film F1, 16-combo ranking, DE search landscape, and the
Downton detector-vs-tracker ghost timeline (replaces the blank
title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
placeholders; build_site.sh pins them to the HEAD commit's raw URLs
and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
and to session-memory names; mark artifact-registry paths with their
pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
- experiment_charts.py generates 4 figures from experiments/ artifacts:
held-out per-film F1, 16-combo ranking, DE search landscape, and the
Downton detector-vs-tracker ghost timeline (replaces the blank
title-card screenshot)
- new frames: 19-correct wedding shot (success case), Many Saints
ghost-vs-unknown frame (three error classes in one image)
- rename rep4-optimizer-results.md -> model-bakeoff.md; rep4 kept only
as the on-disk artifact prefix, explained once
- repo file references are now links via https://REPOLINK/<path>
placeholders; build_site.sh pins them to the HEAD commit's raw URLs
and fails the build if a linked path doesn't exist at HEAD
- drop references to removed scripts (scene_score.py, score_config.py)
and to session-memory names; mark artifact-registry paths with their
pull commands
- commit readme_example.jpg + pipeline_topology.svg so README renders
on the plain Gitea repo view
- deploy_pages.sh: push built site/ to the gitea-pages branch
Splits the rep4 write-up's key findings into their own linkable pages:
- best-model.md: calibration curves first (discriminative power, independent
of any threshold), then F1 on the benchmark — LVFace-B Glint360K wins both.
- gallery-scope.md: whole vs. cast-restricted gallery, isolated from model and
expansion choice — restriction wins on every axis, but isn't a shipped
runtime feature yet.
- pose-expansion.md: the training-set expand_gallery effect, and the held-out
replication attempt that found it doesn't reproduce (5 films, 2 models,
after catching and fixing a replay-timeout truncation bug and a bbox
first-match-instead-of-best-match bug in the comparison harness itself). An
honest null result, with the methodology errors documented since they're
exactly the kind that manufacture a false "it works!" finding.
- lvface-deep-dive.md: the winning model's held-out generalization gap, its
two failure modes (frozen-bbox ghost tracks), and a verified case (cross-
checked against Jellyfin's independent cast metadata) where LVFace
correctly identified an actor that X-Ray's ground truth failed to credit.
Adds a "report-highlights" artifact-registry package (scripts/artifacts/
push_artifacts.sh, pull_artifacts.sh) for hand-picked illustrative frames that
aren't reproducible via the automated best/worst montage selection, and wires
pulling it into scripts/docs/build_site.sh.
docs/rep4-optimizer-results.md is the main deliverable: the model bake-off +
threshold re-tune experiment log, including the ROCm teardown deadlock root
cause and fix, DE concurrency tuning, the 16-combo results table, held-out
validation against 5 films never seen by the optimizer (macro F1 67.4% vs.
75.3% training — a real generalization gap), the frozen-bbox "ghost track"
failure mode found via annotated frame evidence, calibration curves per model,
and an isolated-effects breakdown of gallery scope vs. pose expansion.
MkDocs site (mkdocs.yml, docs/index.md) renders docs/*.md; scripts/docs/
pulls referenced images from the artifact registry and generates the
calibration chart at build time (see the tooling commit) rather than
committing images to the repo.
experiments/ now keeps only scripts + README + SESSION_STATE.md in git — every
data artifact (galleries, dumps, X-Ray corpus, montage frames, trajectories,
manifests, results) moved to the Gitea package registry. film-lut.template.json
is the committed placeholder for the gitignored file-lut.json (real local
movie paths, never shared — some source filenames carry scene-release tags).
Adds models/transnetv2.onnx (via Git LFS, matching the other ONNX models) for
the new scene-detection path.
GPU-free, model-free tests for the pure logic: gallery HDF5 save/load
round-trips (actors, embeddings, embedded calibration) and legacy JSON
read back-compat; the calibration sigmoid fit, boundary inversion, and the
in-memory hash-keyed cache reuse/staleness; TrackGallery's diversity-buffer
eviction, novelty/spread safety gates, and promotion; FaceTracker's IoU/
embedding association and cross-cut track revival; and the GEMM similarity
backend (forced to CPU so the suite runs without a GPU).
Verified: all 39 test cases / 1640 assertions pass (cmake -DSAE_BUILD_TESTS=ON).
Optimizer (scripts/optimizer/): replay.py runs the real C++ tracker/matcher/
scene_tracker chain over a dumped-embeddings HDF5 via sae_kpn, so a threshold
sweep never re-decodes video or re-embeds faces. optimize.py drives scipy's
differential_evolution over the knob space, with DE-level parallelism
(multiple population candidates evaluated concurrently via a ThreadPoolExecutor)
on top of per-film replay parallelism. second_score.py is the per-second X-Ray
scoring metric (TPI/FPI/FN, out-of-cast misID weighted 10x, fair recall masked
to gallery-known cast) that superseded an earlier scene-union metric.
dump_error_frames.py / dump_scene_montage.py extract annotated video frames
(bounding boxes, TPI/FPI/FN captions, onscreen-vs-offscreen split) for visual
review of a replay against ground truth. Gallery utilities: cast_restrict.py,
gallery_membership.py, fetch_missing_actors.py, reembed_gallery.py.
scripts/validation/: X-Ray ground-truth loading and provider-agnostic identity
matching (identity.py's keys_for — an actor is the union of every id we can
derive, since pipeline output and ground truth don't share one id space).
scripts/artifacts/: push/pull scripts for the Gitea generic package registry —
galleries, montage frames, and experiment data (manifests/trajectories/results)
are pushed there instead of committed, since none are needed to run the app,
only benchmarks. Versioned by git short-SHA.
scripts/docs/: MkDocs site build (build_site.sh) and the calibration-curve
comparison chart (calibration_chart.py, matplotlib, reads each gallery's
embedded calibration).
Gallery-building scripts (make_jellyfin_gallery.py, make_gallery.py,
filter_gallery.py, run_from_jellyfin.py, movienet_eval.py, movienet_prep.py,
sae_gallery.py) updated to read/write HDF5 galleries exclusively, matching the
engine-side format switch. run_from_jellyfin.py and the optimizer no longer
carry movie source paths in shared manifests (some source filenames include
scene-release tags) — resolved locally via a gitignored file-lut.json instead.
New C++ sources:
- kpn_bindings.cpp (sae_kpn): assembles the real face_tracker/identity_matcher/
scene_tracker nodes inside a Python-driven KPN network via nanobind, for
offline threshold-sweep replay against dumped embeddings (scripts/optimizer/).
- track_gallery.hpp: per-film gallery expansion — promotes a confidently-
identified track's novel-pose reference views into an in-memory annex so
later frames/tracks of that actor at similar poses are recognised, without
touching the baked gallery.
- dump_embeddings.cpp: standalone exe that runs detect→embed only (no gallery,
no matching) and dumps per-frame face embeddings + metadata to HDF5, so a
parameter sweep can replay the expensive half once and vary tracking/matching
config freely downstream.
- scene_detector.hpp / scene_detector_node.hpp: TransNetV2-based shot-boundary
detection, opt-in alongside the always-on histogram cut detector.
- camera_position_change_detector_node.hpp, embedding_dump_node.hpp: supporting
nodes for the above.
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for
back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid
calibration (a, b, valid, hash) is now embedded directly in the gallery file
instead of a sidecar .calib_cache.json — identity_matcher reads it from the
loaded gallery and writes back only when the embeddings actually changed
(hash mismatch), skipping the O(n^2) refit otherwise.
Also includes: TensorRT inference backend support (ort_backend.cpp,
trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary
detection wired through frame_source/face_tracker/main, and CMake build
target updates for the new sources.
Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking
backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for
stateful functors) — needed by the optimizer's sae_kpn Python bindings.
scene_gap_hist.py scans scene_analyze output JSONs and, for every actor,
computes the gap (next_scene_start - prev_scene_end) between consecutive
scenes, emitting a text histogram of the distribution. Used to inform the
anneal_sec default.
movienet_eval: replace the per-element dot() with numpy — actor references are
loaded once as an ndarray and scored with a single matmul, keeping a
whole-library gallery fast.
movienet_prep: count and report frames referenced by annotations but absent
from Image.zip instead of skipping them silently.
LVFace-B_Glint360K.onnx shares ArcFace's I/O contract (112x112 aligned crop ->
L2-normalised 512-d) and input scaling, so it drops in via --arcface-model.
Note the caveat that galleries and calib caches must be rebuilt with the same
embedder used for analysis.
Add two cameo hunters that flag actors recognised in a title but absent from
its cast:
- cameo_jellyfin.py — pure-Jellyfin cast-membership check (no id cross-walk)
- cameo_hunt.py — TMDB filmography check (actor's combined_credits)
run_from_jellyfin.py now stamps the analysed title's Jellyfin item GUID into
the output JSON as top-level 'jellyfin_item_id' (scene_analyze can't know it),
which cameo_jellyfin.py uses to look up the cast in Jellyfin's own id space.
Document that field in the result-sink output schema header.
Register a KPN event handler in both scene_analyze and scene_preview:
- Overflow events accumulate per-node dropped-frame counts, printed on exit.
- A Closed event from any node other than result_sink at EOF means a stage
died; trip an atomic so the main loop bails out instead of hanging on
'done' forever, and exit non-zero.
Bumps external/KPN to the commit that exposes set_event_handler / NodeEvent.
Consolidate copy-pasted logic across the gallery/run scripts into shared
modules:
- sae_env.py — zero-dependency .env loader (populates os.environ)
- sae_tmdb.py — TMDB API helpers (tmdb_get, person images, id lookups)
- sae_jellyfin.py— Jellyfin API helpers (jf_get, id/URL normalisation)
- sae_gallery.py — image download + gallery.json writing
make_gallery, make_jellyfin_gallery and filter_gallery now import these
instead of carrying their own near-identical copies.
These are regenerable per-run outputs that were polluting the worktree:
calibration caches (*.calib_cache.csv/png), cameo detection run outputs
(cameo_progress.txt, cameo_report.txt), and scene-gap analysis plots.
dtourolleandClaude Opus 5