d113c83189fbe183af575a562ca1a51cf232e96b
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 dtourolle
|
90b44e0975 |
docs(register): make the status column describe the code
Eleven rows corrected, in both directions. Overstated: AR-011 (the derived dedup window reached scenes.json only), AR-017 (route was a literal), AR-019 (the local plurality tally was still deciding), AR-024 (its "static check" enforcement did not exist), DP-001 (scene_preview had forked and stopped compiling), VR-002 (the Python replay bindings have not compiled since the tracker redesign, and the fixtures it calls committed are gitignored registry artifacts). Understated: VR-010 was marked Planned while five VR-010 tags sat in the code implementing it. Rescoped: VR-007 now names the four AR-025 constants it was already being deferred to for, and which no sweep could reach until this pass. The Withdrawn note gets the longest correction, because it asserted a removal that had not happened and nothing could have caught that: the gate reads tags, and a withdrawn requirement has no tag to be orphaned. The general form is now written down there -- a status column is a claim, and the only claims this project checks automatically are the ones a test or a static check makes. Four of the rows above are the same pattern: recorded as done, and done in one place out of two. New: VR-016, a cadence study for cut_threshold. It is the one always-on signal with no recorded provenance, and its input rate depends on an unrelated flag -- with --scene-detect off, camera_pos compares frames a full second apart at the default sample_fps, and with it on, native-rate frames. Same constant, two meanings, and is_cut drives track_alpha to 0 and clears every expansion buffer. Also stops check_raw_cosine.py inflating its own metric: the extractor scans scripts/, so the tool's prose describing the exception tag was counted as four recorded exceptions. The count now reads 1, which is the number of real ones. TRACES: AR-011, AR-017, AR-019, AR-024, AR-025 | DP-001, DP-007 | IR-004 | VR-002, VR-007, VR-010, VR-016 |
||
|
dtourolle
|
06373817a2 |
refactor(config): give the tuned constants a real provenance, and make them reachable
Two problems, both of which made a number look more settled than it is. The provenance was a dead link. config.hpp cited docs/rep4-optimizer-results.md for prob_threshold, extinction_sec, anneal_sec and the expansion default. That file was renamed to model-bakeoff.md and then rewritten; the comments were never repointed, so the most consequential constant in the pipeline appeared to have no source at all. Following it up produced something worse than a broken link. prob_threshold=0.754 comes from the ORIGINAL rep4 document (still readable at `git show d340da7:docs/rep4-optimizer-results.md`). The rewrite that replaced it reports finding "a real scoring bug in optimize.py: a candidate whose hardest film's replay timed out was averaged over survivors instead of penalized, silently rewarding partial coverage. Affected 3 of 16 training combos". So 0.754 was fitted under scoring that was later found wrong, the corrected sweep converged elsewhere, and no corrected prob_threshold is recorded anywhere. The comment now says that, along with the surviving document's own verdict that the optimum "generalizes unevenly -- strong on 3 of 5 held-out films, badly broken on 2". Four constants were unreachable. ownership_logodds lived on TrackRegistry::Config, and max_views/admit_below/rho_max on EvidenceDiscounter::Config, which main built with the one-argument constructor -- so nothing short of a recompile could move any of them. rho_max's own comment defers to "the sweep (VR-007)" for where it belongs, and that sweep could not reach it. They now live in Config with CLI flags and are exposed to the replay harness. ownership_logodds is worth singling out: below it a track makes no presence claim at all, so it decides whether an actor is reported rather than how confidently -- arguably the most consequential constant after prob_threshold, and until now unswept and unsettable. No behaviour change: every default is the value that was compiled in. TRACES: AR-025, AR-017 | SR-002 |
||
|
dtourolle
|
7c7d4934ae |
refactor(presence): execute the extinction_sec/anneal_sec withdrawal
docs/SPEC.md specified this removal, listed its parts, and ended "grep
for both names and expect no survivors". There were about forty.
docs/requirements.md meanwhile recorded both constants as Withdrawn and
"deleted rather than retained at zero", on the grounds that a field
naming a mechanism the pipeline no longer has is actively misleading.
Neither statement was true of the code: Config still carried
extinction_sec 57.4 and anneal_sec 35.5, --extinction and --anneal still
parsed, and SceneTrackerFunc still ran its keep-alive in both shipped
pipelines, announcing its timeout at every startup.
SceneTrackerFunc is replaced by FrameAnnotationFunc, which is stateless:
same ports, same output type, no keep-alive. Presence belongs to
TrackRegistry (AR-012), where a window is the extent of a track an actor
owned and ends at the last sighting (AR-013). The keep-alive answered
that question a second time and answered it worse, by re-opening exactly
the trailing cool-down AR-013 refuses.
Visible change: --verbosity standard's frames[].identified listed every
actor inside the keep-alive, including ones absent from the frame. It
now lists what was matched in that frame. Minimal and xray output is
untouched -- both were already built from registry claims and never
consulted this node. No schema bump: the published extraction block
reports track_extinction_sec, a different knob that bounds
re-association and never extends a claim.
TrackRegistry::Config::extinction_sec is renamed track_extinction_sec to
match the Config field feeding it, so the grep SPEC.md asks for now
returns nothing rather than one confusing false positive.
Two targets turned out to have been silently dead, both since the
AR-007/AR-008 tracker redesign, and both for the same reason -- they
construct FaceTrackerFunc from a Config alone, a signature that stopped
existing when association moved into probability space:
- scene_preview is fixed here. It now mirrors main.cpp's construction
order exactly (matcher, then registry, then tracker) and wires the
registry's claims into the sink, which it was not doing. DP-001 says
modes are front-ends that must not fork pipeline logic; this one had
forked it and then rotted.
- sae_kpn is not fixed. Restructuring the seam so the tracker can reach
a calibration that only exists once the matcher is built is VR-011's
rewrite, not a patch, and presence claims do not cross the seam at all
today. It is now behind SAE_BUILD_KPN_BINDINGS=OFF with the reason
recorded, so `cmake --build` succeeds and the breakage is attributed
rather than rediscovered.
That second one is worth stating plainly: VR-002 ("replay drives the
real KPN nodes, not a reimplementation") is marked Done, and the module
that makes replay possible has not compiled for some time. The .so in a
stale build/ predates the change.
Python side: the two names are gone from optimize.py, replay.py and
run_holdout_all_models.py as Config keys. anneal_sec survives as
REPLAY_LOCAL_KEYS -- it still configures replay.py's own windowing,
which is a Python reimplementation that no longer matches the sink and
is documented as such. That divergence is VR-011's.
TRACES: AR-012, AR-013 | DP-001 | SR-002
|
||
|
dtourolle
|
e1de98e783 |
feat(ar-024): enforce the invariant statically, and delete the fallback it caught
AR-024's register row gives its verification tier as "Static check -- no bare cosine outside a tagged EXCEPTION". No such check existed, so the invariant was enforced by reading, and reading had missed a live violation. scripts/ci/check_raw_cosine.py is that check, wired into the traceability workflow as a blocking step. It is honest about its reach: it catches direct cosine_similarity() uses not routed through a calibration, and it cannot follow a cosine through a variable across statements. That limit is documented in the script rather than left for someone to discover after trusting a pass. What it caught, and what this commit removes with it: The identity matcher's no-calibration fallback thresholded raw cosine distance (match_threshold) plus a ratio test (match_ratio, match_ratio_ceil). Worse than the invariant breach: it fed max(0, cosine) into TrackRegistry::observe, whose contract reads "posterior is a calibrated probability, never a raw cosine (AR-024) ... so the accumulation cannot be fed an uncalibrated number by a careless caller". It could, and did. And it disagreed with the rest of the pipeline about what "the fit failed" means -- same_person_probability answers that with the untuned default sigmoid and a loud warning, so association stayed in probability space while matching alone left it. One run, two policies, no announcement. Now one rule: cal_.probability() always, with a warning when the fit is not real. A worse answer than a fitted calibration, a better one than a number whose units nothing else shares. TrackGallery::set_calibration is mandatory for the same reason. Its default was max(0, cosine), which made expand_band_lo = 0.90 mean "cosine > 0.9" in a test and "P(same person) > 0.9" in production. FaceTrackerFunc already threw without one; the expansion store now matches. One exception is recorded, in the calibration's own dedup. It is not a close call: at 1 - 1e-7 it asks whether two vectors are the same vector, and it runs on the fit's input, so a calibrated comparison there would have to be calibrated by the fit it is feeding. Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys are read with a contains() check, so each one had been silently inert since the field behind it was deleted -- a sweep varying one of them measured nothing and reported an ordinary-looking F1. TRACES: AR-024, AR-023 | SR-002 |
||
 dtourolleandClaude Opus 5
|
eff696b49a |
fix(expansion): finish AR-018, retiring the last two expansion cosines
AR-018 was marked Done while the promotion path still ran on the
constants it was meant to replace. track_gallery.hpp rejected a track
when buffer_spread (1 minus the minimum pairwise cosine) exceeded
expand_track_spread_max, and skipped a view when its raw gal_sim cleared
expand_novelty_sim. Both were bare cosines with no recorded EXCEPTION,
so both were defects under the AR-024 invariant rather than tagging gaps.
The calibrated band was real but unreachable. expand_band_lo/hi were
declared in Config and read nowhere, and set_band() had no callers, so
the gate always ran at the hardcoded 0.90/0.95 while --expand-novelty-sim
and --expand-spread-max stayed live flags.
The spread gate becomes store_coherence: the band's lower bound asked of
every pair in the store, in probability space, rather than a second
constant. admit() compares a newcomer only against its nearest existing
member, so a gradually drifting track chains A to B to C with every step
inside the band while A and C are strangers — the shape a track-ID
collision takes over a slow pan. The bound is re-asked pairwise before
anything reaches an actor's annex.
The novelty gate is deleted rather than converted. SPEC section AR-018
contrasts the band with expand_novelty_sim as the thing it replaces, and
AR-019 requires only that the band is satisfied. Novelty-seeking now
lives entirely in the eviction ordering, which ranks by similarity to the
actor's references instead of cutting at a constant, so there is nothing
left to tune but the two bounds.
BufEntry stored a raw cosine and the eviction loop compared two of them.
The map is monotonic so the ranking was never wrong, but it left a bare
cosine as a decision variable; it now stores the calibrated probability.
The [AR-018] Catch2 tag previously sat on the spread gate, reporting the
replaced mechanism as verification of its replacement. It now sits on the
band: both bounds asserted exactly, since they are inclusive and an
off-by-one there is invisible anywhere else; refusal counted on each
side; and the config bounds driven away from the shipped defaults so a
hardcoded fallback fails. The case that carries the invariant is "band
thresholds probability, not cosine" — under a calibration shifted by
0.10, cosine 0.84 is admitted and cosine 0.92 refused, the opposite of
their raw verdicts. A raw-cosine gate passes an identity-calibrated test
by accident and cannot pass that one. 15 cases, 38 assertions, passing.
scene_preview.cpp takes the flag rename because it would otherwise
reference deleted Config fields. It still does not compile, for reasons
predating this change: it also reads track_max_embed_dist and
track_max_frames_missing, retired by the earlier AR-024 tracker work, and
constructs FaceTrackerFunc with one argument where the registry and
calibration are now required.
Two notes for anyone reading the chain. The main.cpp flag rename and the
AR-018/AR-024 register rows landed in
|
||
|
Claude
|
7db40f430d |
GR-004: bind galleries to the embedder that built them
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
dtourolle
|
41a277bc19 |
feat(engine): HDF5-native galleries with embedded calibration; TensorRT backends; scene detection
Gallery format switches from JSON to HDF5 exclusively (JSON read-only kept for back-compat): save_gallery always writes HDF5, and the fitted Platt-sigmoid calibration (a, b, valid, hash) is now embedded directly in the gallery file instead of a sidecar .calib_cache.json — identity_matcher reads it from the loaded gallery and writes back only when the embeddings actually changed (hash mismatch), skipping the O(n^2) refit otherwise. Also includes: TensorRT inference backend support (ort_backend.cpp, trt_backend.cpp), gemm_backend improvements, TransNetV2-based scene-boundary detection wired through frame_source/face_tracker/main, and CMake build target updates for the new sources. Bumps the KPN submodule to feature/persistent-pipeline-reuse (push_blocking backpressure, node_ptr/node_stats introspection, ObjectVariantNodeWrapper for stateful functors) — needed by the optimizer's sae_kpn Python bindings. |
||
|
dtourolle
|
66298026e2 |
feat(pipeline): detect node crashes and tally dropped frames
Register a KPN event handler in both scene_analyze and scene_preview:
- Overflow events accumulate per-node dropped-frame counts, printed on exit.
- A Closed event from any node other than result_sink at EOF means a stage
died; trip an atomic so the main loop bails out instead of hanging on
'done' forever, and exit non-zero.
Bumps external/KPN to the commit that exposes set_event_handler / NodeEvent.
|
||
|
dtourolle
|
0ee131a692 | Add AMD support via ort alternative to trt | ||
|
dtourolle
|
a3ba53ddf7 | improved performance | ||
|
dtourolle
|
d753062c6c |
Initial commit: scene-actor-extraction pipeline
Source (KPN++ pipeline nodes, ArcFace embedders, SCRFD/YuNet detectors, gallery builder), build scripts, and eval artifacts. - external/KPN as a git submodule (gitea.tourolle.paris/dtourolle/KPN) - ONNX models tracked via Git LFS (models/*.onnx) - generated outputs, TensorRT engines, reference repos, and media ignored |