a2c699a8442233eab6d72a273a317c64cc5348ec
17
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  dtourolleandClaude Opus 5
|
a2c699a844 |
docs: AR-010 is blocked on a design decision, not an implementation gap
Making SceneDetectorFunc a pass-through does not work. TransNetV2 buffers 100 dense frames before it can score any of them and trusts only each window's centre, so a boundary at time T is not known until roughly 3.3s after T at 30 fps. The face pipeline runs on a parallel branch and has long since passed T. An association hint that arrives after the association is worthless. Three options recorded with their costs: two-pass (correct, doubles the decode that already dominates runtime), delaying the face branch (couples the two branches' timing, which invites heisenbugs under backpressure), or leaving it unwired. Leaving it unwired costs less than it looks, which is what makes this a decision rather than a defect. The redesign made cuts and boundaries do the same thing — both say "spatial continuity is broken, associate on embedding" — so TransNetV2 adds nothing over the histogram except on transitions the histogram cannot see: slow dissolves and fades. That gap is real but narrow. Where TransNetV2 still earns its cost is AR-019, whose promotion gate wants a span free of cuts and boundaries. A late answer is fine there, because promotion happens on track confirmation rather than per frame — so it can be wired offline against the collected boundary list, off the hot path entirely. Recommendation: leave the association path on is_cut alone, wire boundaries into AR-019, and revisit if dissolve-heavy material shows association failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-010, AR-019 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
dfb8f5801e |
feat: no fixed cap on faces per frame
AR-003 — max_faces defaults to 0, meaning no cap. A fixed cap discards the SMALLEST faces first, which are exactly the background cast X-Ray still credits with scene membership, so the pipeline was systematically losing the people it is supposed to find in crowded scenes. This is only safe now that AR-004 landed. Previously an uncapped frame would have pushed more work into channels that dropped on overflow, trading a visible cap for silent loss. With backpressure the producer slows instead, so per-frame cost is contained rather than discarded. The matcher's kMaxFaces used to throw above 32, which made it an accidental second cap. It sizes the similarity engine's preallocated buffer, so it bounds memory rather than face count — the frame is now scored in batches of that size. Memory stays bounded; faces do not. Largest-first ordering is kept even without the cap, and the comment now says why: the Hungarian solver tie-breaks on index order, so that ordering is load-bearing for the replay determinism test rather than a leftover of the cap. Verified end to end on a real clip: identical output to the capped run (385 frames, 693 faces), which is expected since that footage peaks at 4 faces per frame — the point is the absence of a regression. The committed fixtures remain byte-identical and valid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-003 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
cd62d6452d |
test: replay the real tracker and registry from committed fixtures
Tier T2 — composition rather than units. The registry tests construct awkward states directly; these feed the pieces real 480x360 footage with the cuts, gaps and crowded frames that synthetic input does not produce. Six cases: - fixture integrity: exact frame and face counts, contiguous face_offset, and the embedder identity each dump carries (GR-004). The counts are asserted exactly rather than approximately, which was impossible before AR-004 — what a lossy run dropped depended on timing. - determinism: replaying a fixture twice gives identical track ids and windows. This is the property the whole fixture strategy rests on; without it every golden output derived from a fixture is unreliable and the CI replay tier is worthless. - every face is assigned a track, and flush leaves nothing open — a track still live at EOF is a window that never reaches the output. - windows are well-formed and inside the clip. A window ends at the last sighting, so it can never extend past the footage that produced it. - a longer extinction window yields fewer, longer tracks. On the sparse fixture (140 faces over 385 frames) that is the difference the constant actually makes: absorbing a gap versus splitting a window. - the cut-heavy fixture still contains cuts. This guards the corpus, not the code: a regeneration that produced cut-free fixtures would leave the association tests passing while silently testing nothing. Driving the functors directly rather than through a KPN network is deliberate — no threads, no channels, no scheduling, so the same input gives the same output. Suite: 86 cases, 6106 assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-004, AR-012, AR-013, VR-001, VR-002 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
09a4650fd9 |
feat: backpressure — the pipeline slows instead of losing frames
Picks up the KPN fix: node data outputs block on a full channel rather than dropping. Sentinels stay out-of-band, so EOF can always overtake a stalled data path and the hold-and-wait deadlock that comment warns about is not reachable. Verified on a 77s clip at 5 fps, which should yield 385 sampled frames: before 65 written, 320 dropped, 29s, two runs differ after 385 written, 0 dropped, 17s, two runs byte-identical The determinism is the part that matters. Golden fixtures were impossible while what got dropped depended on timing; VR-001 fixture generation is unblocked by this, and so is the CI replay strategy that depends on it. Faster rather than slower, which is worth recording because the intuition runs the other way: a dropped frame has already cost its decode, and the overflow exception cost more still. AR-004 is not fully closed. Channel capacity remains a count of items, while a face carries a 112x112 crop and a 512-float embedding — so a crowded frame occupies far more memory per slot than a sparse one. Bounding by bytes in flight is the remaining half, and it matters once max_faces is removed (AR-003). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-004, VR-001 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
b98372bad8 |
docs: AR-004 is a KPN change, with the measurement behind it
Backpressure cannot be implemented in this repository. Every node output in KPN uses the dropping push() (pool_node.hpp:404 and :710, plus branch, fanout and interrupt_node). A lossless push_blocking() already exists on both Channel and OutputPort — "wait for the consumer to drain instead of dropping; the producer just runs slower" — and nothing calls it. The fix is a per-channel policy or a network default in KPN, and this pipeline should select lossless: a dropped frame here does not degrade a result, it silently changes one. Measured rather than inferred. One 77s clip at 5 fps should yield ~385 sampled frames. On CPU it produced 49, ending at 51s, with 285 dropped at camera_pos and 51 at face_aligner. Rebuilt with CUDA the same clip ran in 29s and reached EOF correctly, and still dropped 320 at camera_pos, yielding 65. Faster hardware moves where the queue backs up; it does not change what happens when it does — which is why this is a correctness requirement rather than a throughput one. Raising channel capacity is therefore a stopgap: it lowers the probability of overflow without changing the behaviour on overflow, and the failure it hides is silent corruption of the output. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-003, AR-004, VR-001 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
e0f9c95689 |
docs: minimum face size is 32px, measured (VR-005)
Replaces the 66px working estimate with the sweep result. 258 probes degraded to each size and matched against a native-resolution gallery: 16px 46.5% TPI, 24px 93.4%, 32px 98.1%, 40px 99.2%, flat to 112px. The knee is 24-32 and 32 sits within about a point of the ceiling. The estimate was roughly twice too strict. At 66px a large share of usable faces would have been discarded, and on 480x360 sources most of them — which is exactly the resolution of the fixture corpus. The more useful finding: false identification was 0.0 at every size, including 12px. Small faces fail by becoming unidentified, never by being attributed to the wrong actor. That asymmetry is what makes a low threshold safe — the cost of admitting a marginal face is a miss, not a false claim. Caveat recorded rather than assumed: FPI grows with gallery size, so 258 actors understates it against a full library. Treat 0.0 as an observation at this scale, not a property. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-002, VR-005 | SR-002, PR-002 |
||
|
dtourolleandClaude Opus 5
|
9e4cdc4efc |
docs: bali fixture corpus, and AR-004 blocks reproducible fixtures
Records `bali/` — five ~77s clips of Road to Bali (1952) — as the fixture source. Public domain, which is the point rather than a convenience: derived fixtures can be committed, where anything cut from a copyrighted title could not live in the repository at all. Makes explicit what the tier table only implied: CI never calls a model. Not a preference — the embedder measures ~930 ms/frame on the CPU provider, so a 77s clip at 5 fps is six minutes of inference. Every model invocation happens locally and CI consumes the result as data, which is what makes the T1/T2 split load-bearing rather than stylistic. Two properties of the corpus to design around: 480x360 puts many faces below the AR-002 66px minimum, so generation must set and record --min-face-px; and 77s at 1 fps is too thin to exercise an extinction window measured in tens of seconds, so fixtures want 5 fps. The finding that matters: a trial dump produced 49 frames of an expected ~385, stopping at 51s of 77s, with 285 frames dropped at camera_pos and 51 at face_aligner on channel overflow. Channels drop rather than block, and what drops depends on timing, so the same command twice can yield different dumps. Golden fixtures cannot be built on that — AR-004 is a prerequisite for VR-001 fixtures, not just a throughput concern for crowd scenes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-004, VR-001 | PR-002 |
||
|
dtourolleandClaude Opus 5
|
08941540cb |
feat: presence windows come from registry claims (schema_version 2)
The sink no longer reconstructs presence from per-frame detections. A reaped track already IS a window — [first_seen, last_seen] of a track an actor owned — so it is pushed straight to the aggregator when it dies and written out as-is. AR-012 completed end to end. The annealing pass is deleted, not disabled: anneal_sec existed only to bridge gaps between isolated accepted frames, and a track that survives its own gaps leaves it nothing to do. The field is REMOVED from the output rather than zeroed — a field naming a mechanism the pipeline no longer has is actively misleading to anyone reading a manifest, and would outlive everyone who remembers why it reads 0. IR-002 — schema_version 2, matching jRay/SPEC.md JR-002. Windows become objects carrying `belief` and `route` rather than bare float pairs, so a consumer can caveat or filter instead of treating every window as equally certain. The new `extraction` block carries `extinction_sec` (the successor to anneal_sec, and what a consumer actually needs to interpret a window) and `gallery_scope` — global vs limited being the strongest single quality signal when two manifests compete for one cut, since identical gallery_size can mean very different recall. AR-016 wired: a pre-write hook flushes the registry with the last timestamp seen, so tracks still live at EOF are emitted. A film ends with faces on screen and those tracks have not timed out; without this the closing scene's cast is silently dropped, which reads as a recognition miss rather than a bookkeeping bug. IR-003 stays In Progress deliberately: the sink now writes after the flush, but the deferred re-identification pass (AR-020) does not exist yet, so output is still final at EOF rather than after it. This is a BREAKING format change and part of the coordinated SR-003 bump — it must ship together with the jRay reader and the server's acceptance of the new shape, not ahead of them. Suite: 80 cases, 3250 assertions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-012, AR-016, IR-002, IR-003 | SR-002, SR-003 |
||
|
dtourolleandClaude Opus 5
|
be5f67fa96 |
feat: tracker on the registry — one pool, calibrated, frame-dependent
Merges feature/tracker-registry. See
|
||
|
dtourolleandClaude Opus 5
|
843852e19c |
feat: registry owns correlation discounting (AR-024, AR-025)
Moves two responsibilities inside the registry that callers should never have been trusted with. AR-025 — per-frame evidence is discounted for correlation by the registry itself, via EvidenceDiscounter. Log-odds accumulation is only valid for independent observations, and consecutive frames of one track are anything but: near-identical pose, lighting and expression. Accumulated naively, thirty frames of the same face at the same angle drive the posterior to certainty on what is effectively one measurement. Each observation is weighted by how much it adds — a view already contributed counts for ~nothing, a genuinely new pose counts in full. This reuses the novelty judgement gallery expansion already makes rather than inventing a second one. The discounter is a separate class the registry holds, so it stays testable and swappable, but it is a constructor argument rather than an option: there is no correct way to accumulate without it. AR-024 — observe() takes a calibrated probability and converts to log-odds internally. A caller can no longer hand it a raw cosine, which would have been silently wrong rather than obviously so. Retiring the remaining raw-cosine constants in the tracker is still open. DeadTrack now reports effective_obs alongside observations: the raw count and the evidence that actually counted. A large gap between them is a track the camera stared at, and worth seeing. Three tests, one of which is the point: two tracks given the same number of observations at the same posterior, one repeating a single view and one seeing eight distinct ones, must not end up equally confident. Without discounting they would be identical. Fixed a test that asserted a belief swap on tied evidence. A tie leaves ownership where it is — a challenger must out-accumulate the incumbent, since one contrary observation is noise. The original test passed only because it fed raw log-odds directly. Suite: 78 cases, 3245 assertions. Coverage 20/63 to 22/63. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-024, AR-025 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
f0c7126f80 |
feat: TrackRegistry — presence follows track extent
The spine of the redesign. Presence is now the extent of a track an actor owns, [first_seen, last_seen], rather than the subset of frames in which recognition happened to succeed. An actor recognised only at the end of a long track is present for all of it, which is what the scene-scoped ground truth actually records. AR-013 — `last_seen` as an optional carries the entire liveness state: unset means on screen, set means went off at that timestamp and still revivable, reaped means emitted and erased. No missing-frame counter, no expired flag. It subsumes the tracker's existing two-pool split, so there is no separate revival path — matching a dormant track is ordinary inter-frame association. The asymmetry is the point: interior gaps are claimed, the trailing cool-down is not. A face lost and re-associated within the timeout never closed its track, so the gap is presence — someone briefly occluded has not left the scene. But a track that dies ends at its last sighting, never at the death time. That is precisely the over-claim the retired extinction_sec keep-alive produced, where presence ran on into the closing credits. AR-014 — a belief swap A→B closes the track and opens a successor at the swap frame. Not a correction: two non-twins both clearing the threshold on one face is not realistic, whereas a track_id carried across a viewpoint change onto a different person is. Treating it as a swap-and-continue would emit one window blending two people; treating it as a boundary yields two that are each right. AR-015 — two live tracks owned by one actor means at least one is wrong, since a person cannot be in two places at once. A reverse index catches it on the update that causes it rather than by scanning. This makes identity a third cut detector, independent of the histogram and TransNetV2 and firing where those failed. AR-016 — flush() closes tracks still live at EOF. Without it a film ending mid-shot silently drops its closing cast, which presents as a recognition miss rather than a bookkeeping bug. Reaping hands the dead track to the aggregator and erases it, so the registry holds only live tracks and its size is bounded by concurrent on-screen faces rather than growing with the film. Locking: a frame's association pass is atomic as a unit via FrameScope, since per-call locking would let another thread observe a half-updated frame. owner() reads tally and verdict under one lock — separately, a track could be both unowned and owned within a single promotion decision. A vote for an already-reaped track is dropped and counted, because a nonzero count means the timeout is shorter than the matcher's lag. 11 unit tests, driven directly against the registry with no network and no fixture — the awkward cases are constructed rather than hunted for. Suite: 75 cases, 3236 assertions. Coverage 14/63 to 20/63. Not yet wired into FaceTrackerFunc; that is AR-007/AR-008. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-012, AR-013, AR-014, AR-015, AR-016, AR-017 | SR-002 |
||
|
dtourolleandClaude Opus 5
|
908d166173 |
feat: bind galleries to the embedder that built them
GR-004 — a gallery built with one embedding model is meaningless with another. Cosine similarities across models are garbage but look entirely plausible, so this fails silently and expensively; every measurement taken against a mismatched pair would have been quietly wrong. The stamp is the model basename plus a SHA-256 of its bytes, with embed_dim as a cheap extra guard. The hash decides and the name explains, because neither works alone: a name is a promise rather than a fact — models get re-exported in place under an unchanged filename, which is exactly the case where the weights differ and nothing else does — while a bare hash mismatch tells an operator nothing actionable. Mismatch is fatal in every mode with no bypass. Unstamped only warns, because unstamped is unknown rather than known-bad, and an error firing on every legacy gallery trains people to reach for the bypass reflexively. scripts/stamp_gallery.py binds an existing gallery in place with no re-embedding, so the warning is a migration step rather than a permanent state; --require-gallery-stamp promotes it to an error once a site has migrated. Two gaps found that would have defeated the requirement outright: - Embedding dumps carried no stamp, so a replay — which has no live embedder — had nothing to check the gallery against. Dumps now carry embedder_model and embedder_sha256 as root attributes. Additive; schema_version stays 1. This is the same gap the dump audit identified independently. - --merge produced one file holding two embedding spaces, which no later check can untangle. Merge paths now verify before writing. The stamp also survives identity_matcher's calibration write-back, which would otherwise have stripped it on the first analysis run — the check would have worked exactly once. Conflicts resolved additively: both branches appended a source to sae_gallery and to the test target, and both edited the GR-004 register row. Merged suite: 64 cases, 3199 assertions, passing on CPU with no GPU. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: GR-004, VR-001 | SR-001 |
||
|
dtourolleandClaude Opus 5
|
a2ebdc4cdd |
docs: GR-004 done — gallery/embedder binding
Stamp is model basename + SHA-256 + embed_dim: the hash decides, the name explains. A name alone is a promise rather than a fact — models get re-exported in place under an unchanged filename, which is exactly the case where weights differ and nothing else does. A hash alone is unactionable in an error message. Mismatch is fatal in every mode with no bypass. Unstamped only warns, because unstamped is unknown rather than known-bad, and an error that fires on every legacy gallery trains people to reach for the bypass. A migration script binds existing galleries in place with no re-embedding, so warn is not permanent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: GR-004 | SR-001 |
||
|
Claude
|
7db40f430d |
GR-004: bind galleries to the embedder that built them
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
dtourolleandClaude Opus 5
|
020306c94f |
docs: builder images and per-backend release binaries (DP-008)
Adds the build/deploy story that was missing: containerised builder images for cpu / cuda / rocm, and release jobs producing prebuilt binaries so a first install need not compile. States explicitly that this does not reverse DP-005. That requirement rejects Docker as a *runtime* — GPU passthrough is fragile and exists only because of the container. Using it as a *build* environment is the opposite case, and lets one machine produce binaries for backends it cannot itself run. Build in a container, run natively. Two things deliberately cannot ship, and the installer must not imply otherwise: TensorRT engines are GPU-architecture and TRT-version specific, so build_trt_engines.sh still runs on the target; and models are ~725 MB in LFS, orthogonal to the binary. The base image is chosen by the OLDEST glibc to be supported, not by convenience — a binary built in a container runs against the host's glibc, and getting this wrong fails at load with GLIBC_2.xx not found. Accelerator runtimes have the same shape of problem, so each image documents its compatible CUDA/ROCm range and the installer checks it rather than discovering a mismatch at first inference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: DP-005, DP-007, DP-008 | PR-004 |
||
|
dtourolleandClaude Opus 5
|
2919ed68d1 |
docs: audit findings, verification tiers, CI image, artifact storage
Corrections from the dump audit and the completed agent work: - AR-010 is not started, not in progress: is_scene_boundary has no producer anywhere. SceneDetectorFunc is a terminal sink writing scenes.json and never annotates the frame, so the field is permanently false and the dump column a constant 0. A replay test of the frame-dependent track_alpha would pass vacuously — the worst failure mode for a verification gate. - T1 (functor-level) becomes the primary verification tier, not T2. KPN node functors are plain callables constructed outside the network, so a node is tested by calling operator() with hand-built inputs. That removes four hazards at once: fixture provenance, replay-from-frame-0, cross-test state leakage, and replay-harness nondeterminism. It also means a dead upstream producer no longer blocks testing its consumer. - VR-010 (dump provenance) and VR-011 (replay harness rewrite) added. A dump made with LVFace is currently byte-indistinguishable from one made with w600k-R50 — the GR-004 problem again, in the dump. - Four requirements had no verification tier at all; the traceability gate found them. - DP-007: CI builder image, CPU-only, pinned by tag in the Gitea container registry. Corpus fixtures go to the package registry rather than LFS: LFS is pulled on clone and would tax every developer for data only CI reads. - IR-004/005/007/008 marked done; the v1 DSP parameters they had to pin are now normative in the server spec. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: AR-010, DP-007, IR-004, IR-005, IR-007, IR-008, VR-001, VR-010, VR-011 | SR-002, SR-003 |
||
|
dtourolleandClaude Opus 5
|
a5299daf6e |
docs: software spec, requirements register, and implementation plan
Adds the requirements baseline for the pipeline redesign: - SPEC.md — software requirements with Current/Gap deltas per item, so the document doubles as a work list. - requirements.md — stable flat IDs (AR/DP/IR/GR/VR) with parent traces, priorities, statuses, and a per-requirement verification plan. Replaces the thematic A1..E8 scheme, which had already produced an A1a and an out-of-order E6; IDs are now permanent and never reused. - IMPLEMENTATION-PLAN.md — phased work. The central change is AR-012: presence follows track extent rather than per-frame recognition, so a window starts when an actor appears rather than when the recogniser first succeeded. anneal_sec and extinction_sec are withdrawn rather than retuned — a track that survives its own gaps leaves them nothing to do. Verification is shaped by CI running on an N100 with no dGPU: the existing HDF5 dump makes everything downstream of embedding replayable on CPU, which covers the bulk of the redesign. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |