From 7c7d4934aea2f253edea979311e45a148b04602d Mon Sep 17 00:00:00 2001 From: Duncan Tourolle Date: Wed, 5 Aug 2026 16:21:15 +0200 Subject: [PATCH] refactor(presence): execute the extinction_sec/anneal_sec withdrawal docs/SPEC.md specified this removal, listed its parts, and ended "grep for both names and expect no survivors". There were about forty. docs/requirements.md meanwhile recorded both constants as Withdrawn and "deleted rather than retained at zero", on the grounds that a field naming a mechanism the pipeline no longer has is actively misleading. Neither statement was true of the code: Config still carried extinction_sec 57.4 and anneal_sec 35.5, --extinction and --anneal still parsed, and SceneTrackerFunc still ran its keep-alive in both shipped pipelines, announcing its timeout at every startup. SceneTrackerFunc is replaced by FrameAnnotationFunc, which is stateless: same ports, same output type, no keep-alive. Presence belongs to TrackRegistry (AR-012), where a window is the extent of a track an actor owned and ends at the last sighting (AR-013). The keep-alive answered that question a second time and answered it worse, by re-opening exactly the trailing cool-down AR-013 refuses. Visible change: --verbosity standard's frames[].identified listed every actor inside the keep-alive, including ones absent from the frame. It now lists what was matched in that frame. Minimal and xray output is untouched -- both were already built from registry claims and never consulted this node. No schema bump: the published extraction block reports track_extinction_sec, a different knob that bounds re-association and never extends a claim. TrackRegistry::Config::extinction_sec is renamed track_extinction_sec to match the Config field feeding it, so the grep SPEC.md asks for now returns nothing rather than one confusing false positive. Two targets turned out to have been silently dead, both since the AR-007/AR-008 tracker redesign, and both for the same reason -- they construct FaceTrackerFunc from a Config alone, a signature that stopped existing when association moved into probability space: - scene_preview is fixed here. It now mirrors main.cpp's construction order exactly (matcher, then registry, then tracker) and wires the registry's claims into the sink, which it was not doing. DP-001 says modes are front-ends that must not fork pipeline logic; this one had forked it and then rotted. - sae_kpn is not fixed. Restructuring the seam so the tracker can reach a calibration that only exists once the matcher is built is VR-011's rewrite, not a patch, and presence claims do not cross the seam at all today. It is now behind SAE_BUILD_KPN_BINDINGS=OFF with the reason recorded, so `cmake --build` succeeds and the breakage is attributed rather than rediscovered. That second one is worth stating plainly: VR-002 ("replay drives the real KPN nodes, not a reimplementation") is marked Done, and the module that makes replay possible has not compiled for some time. The .so in a stale build/ predates the change. Python side: the two names are gone from optimize.py, replay.py and run_holdout_all_models.py as Config keys. anneal_sec survives as REPLAY_LOCAL_KEYS -- it still configures replay.py's own windowing, which is a Python reimplementation that no longer matches the sink and is documented as such. That divergence is VR-011's. TRACES: AR-012, AR-013 | DP-001 | SR-002 --- CMakeLists.txt | 32 +++++++- scripts/docs/run_holdout_all_models.py | 6 +- scripts/optimizer/optimize.py | 4 +- scripts/optimizer/replay.py | 43 ++++++++--- scripts/optimizer/test_sae_kpn.py | 6 +- src/config.hpp | 29 ++++--- src/kpn_bindings.cpp | 36 ++++----- src/main.cpp | 15 ++-- src/nodes/frame_annotation_node.hpp | 43 +++++++++++ src/nodes/result_sink_node.hpp | 8 +- src/nodes/scene_tracker_node.hpp | 102 ------------------------- src/scene_preview.cpp | 41 +++++++--- src/track_registry.hpp | 13 +++- tests/test_face_tracker.cpp | 2 +- tests/test_replay_fixtures.cpp | 2 +- tests/test_track_registry.cpp | 2 +- 16 files changed, 203 insertions(+), 181 deletions(-) create mode 100644 src/nodes/frame_annotation_node.hpp delete mode 100644 src/nodes/scene_tracker_node.hpp diff --git a/CMakeLists.txt b/CMakeLists.txt index 2009689..97c88ca 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -322,11 +322,37 @@ nanobind_add_module(sae_embed src/python_bindings.cpp) target_link_libraries(sae_embed PRIVATE sae_gallery) # ── sae_kpn — Python module: run the real downstream nodes over dumped embeddings ─ -# Assembles face_tracker/identity_matcher/scene_tracker in a Python-driven KPN +# Assembles face_tracker/identity_matcher/frame_annotation in a Python-driven KPN # network (KPN_BUILD_PYTHON is enabled per-TU inside the .cpp). Powers the # threshold-sweep optimizer in scripts/optimizer/. -nanobind_add_module(sae_kpn src/kpn_bindings.cpp) -target_link_libraries(sae_kpn PRIVATE sae_gallery) +# +# OFF by default, and this is a statement of fact rather than a preference: the +# module HAS NOT COMPILED since the AR-007/AR-008 tracker redesign. FaceTrackerFunc +# now requires a TrackRegistry and a calibration at construction, and the binding +# still builds it from a Config alone. The .so in a stale build/ directory +# predates that change. +# +# Fixing it is VR-011's job, not a patch: the tracker needs the calibration, the +# calibration comes from the matcher, and the matcher is added to the network +# afterwards -- so the seam has to be restructured, exactly as main.cpp already +# is (matcher first, then registry, then tracker). Presence claims do not cross +# the seam at all today, which is the other half of the same rewrite. +# +# Recorded as a switch rather than left as a build error so that `cmake --build` +# succeeds and the breakage is attributed instead of rediscovered. Turning it on +# reproduces the failure immediately, which is the point. +# +# TRACES: VR-011 | PR-002 +option(SAE_BUILD_KPN_BINDINGS + "Build the sae_kpn Python module (BROKEN pending VR-011)" OFF) +if(SAE_BUILD_KPN_BINDINGS) + nanobind_add_module(sae_kpn src/kpn_bindings.cpp) + target_link_libraries(sae_kpn PRIVATE sae_gallery) +else() + message(STATUS + "sae_kpn: SKIPPED (SAE_BUILD_KPN_BINDINGS=OFF). The Python replay " + "bindings do not compile against the post-AR-012 tracker; see VR-011.") +endif() # ── sae_audio — Python module: the v1 audio signature (IR-004) ──────────────── # Compiles audio_signature.cpp directly and links only FFmpeg, rather than diff --git a/scripts/docs/run_holdout_all_models.py b/scripts/docs/run_holdout_all_models.py index af98817..3a75b28 100644 --- a/scripts/docs/run_holdout_all_models.py +++ b/scripts/docs/run_holdout_all_models.py @@ -79,8 +79,10 @@ def main(): "--dump", str(dump), "--gallery", str(gallery), "--out", str(pred_path), "--prob-threshold", str(cfg["prob_threshold"]), - "--anneal-sec", str(cfg["anneal_sec"]), - "--extinction-sec", str(cfg["extinction_sec"]), + # anneal_sec is replay-local now (it configures replay.py's + # own windowing, not the pipeline). extinction_sec is gone + # entirely with SceneTrackerFunc -- see AR-012/AR-013. + "--anneal-sec", str(cfg.get("anneal_sec", 10.0)), "--expand-gallery", ] print(f"RUN {model}/{film['slug']}...", file=sys.stderr) diff --git a/scripts/optimizer/optimize.py b/scripts/optimizer/optimize.py index 16abd67..aaab3b8 100644 --- a/scripts/optimizer/optimize.py +++ b/scripts/optimizer/optimize.py @@ -17,7 +17,7 @@ point from the trajectory (--trajectory). Usage: python scripts/optimizer/optimize.py --manifest films.json \ --gallery gallery_arcface_w600k_r50.json \ - --params prob_threshold:0.5:0.999 anneal_sec:1:30 extinction_sec:1:15 \ + --params prob_threshold:0.5:0.999 anneal_sec:1:30 track_alpha:0:1 \ --popsize 20 --maxiter 25 --trajectory traj.json """ from __future__ import annotations @@ -239,7 +239,7 @@ def main(): rec = {"eval": evals[0], "config": cfg, **m, "t": round(time.time() - t0, 1)} traj.append(rec) print(f"[opt] eval {evals[0]:3d} thr={cfg['prob_threshold']:.2f} " - f"ann={cfg['anneal_sec']:.0f} ext={cfg['extinction_sec']:.1f} → " + f"ann={cfg.get('anneal_sec', float('nan')):.0f} → " f"F1={m['f1']*100:.1f}% P={m['precision']*100:.1f}% R={m['recall']*100:.1f}% " f"agree={m.get('agreement', 0)*100:.1f}% misID={m.get('FPI_misid', 0)}", file=sys.stderr) diff --git a/scripts/optimizer/replay.py b/scripts/optimizer/replay.py index fcac538..14ac018 100644 --- a/scripts/optimizer/replay.py +++ b/scripts/optimizer/replay.py @@ -6,14 +6,14 @@ TRACES: VR-002 | PR-002 Reads an embedding dump (scripts/optimizer/SCHEMA.md), feeds each frame as an EmbeddedSceneFrame into a Python-assembled KPN network wiring the *real* C++ -face_tracker → identity_matcher → scene_tracker, and returns the same presence-window +face_tracker → identity_matcher → frame_annotation, and returns the same presence-window JSON that scene_analyze's result_sink produces (minimal schema). No decode, no GPU embedding — only the cheap downstream tail runs, so a sweep can vary Config knobs freely. See [[kpn-python-replay-optimizer]]. CLI: python scripts/optimizer/replay.py --dump film.h5 --gallery gallery.json \ - --out replayed.json [--prob-threshold 0.99] [--anneal 10] ... + --out replayed.json [--prob-threshold 0.99] [--anneal-sec 10] ... """ from __future__ import annotations @@ -164,7 +164,7 @@ def replay(dump_path: str, gallery: str, cfg: dict, build_dir: str, stop: bool = sae_kpn.add_face_tracker(net, "tracker", cfg, cap) sae_kpn.add_identity_matcher(net, "matcher", gallery, cfg, cap, stamp["model_name"], stamp["model_sha256"]) - sae_kpn.add_scene_tracker(net, "scene", cfg, cap) + sae_kpn.add_frame_annotation(net, "scene", cap) net.connect("replay", 0, "tracker", 0) net.connect("tracker", 0, "matcher", 0) net.connect("matcher", 0, "scene", 0) @@ -207,10 +207,26 @@ def replay(dump_path: str, gallery: str, cfg: dict, build_dir: str, stop: bool = def build_minimal(annotations, movie, fps, cfg) -> dict: - """Reproduce result_sink's minimal schema: per-actor annealed [start,end] windows. + """Per-actor [start,end] windows, built by annealing per-frame detections. - Mirrors ResultSinkFunc::build_actor_windows — merge each actor's detection - timestamps into windows, bridging gaps shorter than anneal_sec. + TRACES: VR-011 | PR-002 + + This NO LONGER mirrors ResultSinkFunc, and the docstring used to claim it + did. The sink builds a window from a TrackRegistry claim -- the extent + [first_seen, last_seen] of a track an actor owned (AR-012) -- so a window + starts when the actor appeared rather than when recognition first + succeeded, and interior gaps are absorbed by the track surviving them. + This function still bridges gaps between isolated accepted frames, which is + what anneal_sec did before AR-012/AR-013 withdrew it. + + So a replayed window and a pipeline window are answers to different + questions, and a sweep tuned against this one is not tuning the shipped + behaviour. That is VR-011's job -- "rewrite the replay harness for the + post-AR-012 output contract" -- and it is a rewrite, not an edit, because + the registry's claims do not cross the Python seam at all today. + + `anneal_sec` is therefore replay-local now: it configures THIS function and + is no longer forwarded to the C++ Config, which has no such field. """ anneal = float(cfg.get("anneal_sec", 10.0)) info = {} # actor_idx -> identity fields @@ -261,8 +277,12 @@ def build_minimal(annotations, movie, fps, cfg) -> dict: # a stale entry here silently inert rather than loudly wrong. CFG_KEYS = ["detector_conf", "prob_threshold", "match_prior", "track_alpha", "track_min_iou", "track_assoc_min_prob", - "track_extinction_sec", - "extinction_sec", "anneal_sec"] + "track_extinction_sec"] + +# Swept like a Config key but consumed entirely in Python, by build_minimal. +# Kept separate so nobody has to guess which of these the pipeline actually +# reads: everything in CFG_KEYS crosses the seam, and nothing here does. +REPLAY_LOCAL_KEYS = ["anneal_sec"] def main(): @@ -273,7 +293,7 @@ def main(): p.add_argument("--out", required=True, help="output presence JSON") p.add_argument("--raw-out", help="also write raw per-frame annotations (JSONL, with bboxes) here") p.add_argument("--build-dir", default=str(REPO / "build")) - for k in CFG_KEYS: + for k in CFG_KEYS + REPLAY_LOCAL_KEYS: p.add_argument(f"--{k.replace('_','-')}", type=float, default=None) # per-film gallery expansion: promotes pose-varied views of confidently-identified # actors into an in-memory annex, recovering ~+4 recall at no precision cost. @@ -285,7 +305,10 @@ def main(): p.add_argument("--require-gallery-stamp", action="store_true") args = p.parse_args() - cfg = {k: getattr(args, k) for k in CFG_KEYS if getattr(args, k) is not None} + # Both lists go into one dict: config_from_dict reads C++ keys with a + # contains() check and ignores the rest, and build_minimal reads its own. + cfg = {k: getattr(args, k) + for k in CFG_KEYS + REPLAY_LOCAL_KEYS if getattr(args, k) is not None} if args.expand_gallery: cfg["expand_gallery"] = True if args.require_gallery_stamp: diff --git a/scripts/optimizer/test_sae_kpn.py b/scripts/optimizer/test_sae_kpn.py index ca33455..71a3f3e 100644 --- a/scripts/optimizer/test_sae_kpn.py +++ b/scripts/optimizer/test_sae_kpn.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """ Smoke test for the sae_kpn module: assemble the real downstream pipeline nodes -(face_tracker → identity_matcher → scene_tracker) in a Python-driven KPN network, +(face_tracker → identity_matcher → frame_annotation) in a Python-driven KPN network, fed by a no-input Python source node, and verify SceneAnnotations flow out. Proves the KPN-native replay path works without any numpy port of node logic. @@ -31,7 +31,7 @@ def make_frame(t, n): def main(): net = sae_kpn.Network() sae_kpn._register_types(net) - cfg = {"prob_threshold": 0.99, "anneal_sec": 10.0, "extinction_sec": 5.0} + cfg = {"prob_threshold": 0.99, "track_extinction_sec": 5.0} frames = [make_frame(float(t), 1) for t in range(3)] frames.append({"timestamp_sec": 3.0, "eof": True}) @@ -48,7 +48,7 @@ def main(): sae_kpn.add_node_python(net, "replay", source, [], ["EmbeddedSceneFrame"], 8) sae_kpn.add_face_tracker(net, "tracker", cfg, 16) sae_kpn.add_identity_matcher(net, "matcher", GAL, cfg, 16) - sae_kpn.add_scene_tracker(net, "scene", cfg, 16) + sae_kpn.add_frame_annotation(net, "scene", 16) net.connect("replay", 0, "tracker", 0) net.connect("tracker", 0, "matcher", 0) net.connect("matcher", 0, "scene", 0) diff --git a/src/config.hpp b/src/config.hpp index 267cea8..5e45323 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -151,16 +151,25 @@ struct Config { double track_extinction_sec{5.0}; // ── Scene tracking ──────────────────────────────────────────────────────── - // extinction_sec re-tuned by DE against X-Ray per-second presence, 4-film rep4 - // matrix (docs/rep4-optimizer-results.md). Reverses the earlier "short is better" - // finding: with a stricter prob_threshold, a long extinction window bridges real - // presence gaps (occlusion, turned face) instead of just smearing FPs — every - // model's best config pushed to ~90%+ of the search ceiling (tried up to 60s). - // The ceiling kept getting hit, so treat 60 as "good enough", not a proven optimum. - double extinction_sec{57.4}; // keep actor active this many seconds after last detection - // anneal_sec: previously found INSENSITIVE at a 1–30s range; the wider rep4 sweep - // (1–60s) also pushed this to the ceiling alongside extinction_sec (see above). - double anneal_sec{35.5}; // merge actor windows separated by less than this into one epoch + // TRACES: AR-012, AR-013 | SR-002 + // extinction_sec (57.4) and anneal_sec (35.5) are GONE, along with + // SceneTrackerFunc, which is what read the first of them. docs/SPEC.md + // specified this removal and ended it "grep for both names and expect no + // survivors"; there were about forty, and the register meanwhile recorded + // both as Withdrawn and "deleted rather than retained at zero" on the + // grounds that a field naming a mechanism the pipeline no longer has is + // actively misleading. + // + // Both existed to bridge gaps between isolated accepted frames. A track + // that survives its own gaps leaves them nothing to do: AR-012 makes a + // window the extent of a track an actor owns, and AR-013 ends it at the + // last sighting. The keep-alive answered the same question again and + // answered it worse, by re-opening exactly the trailing cool-down AR-013 + // refuses. + // + // track_extinction_sec above is NOT the same knob under a new name. It + // bounds how long a lost track stays available for re-association, which is + // a tracking question; it never extends a presence claim. // ── Per-film gallery expansion ──────────────────────────────────────────── // Within one uncut track every face is the same physical person — a free diff --git a/src/kpn_bindings.cpp b/src/kpn_bindings.cpp index b486faf..6634116 100644 --- a/src/kpn_bindings.cpp +++ b/src/kpn_bindings.cpp @@ -1,5 +1,5 @@ // sae_kpn — run the real downstream pipeline nodes (face_tracker, identity_matcher, -// scene_tracker) inside a Python-assembled KPN network, fed by a Python HDF5 replay +// frame_annotation) inside a Python-assembled KPN network, fed by a Python HDF5 replay // source. Lets a parameter sweep re-run the exact C++ matching/tracking logic over // dumped embeddings — no video decode, no GPU — with different Config knobs each run. // @@ -19,7 +19,7 @@ #include "gallery/gallery_store.hpp" #include "nodes/face_tracker_node.hpp" #include "nodes/identity_matcher_node.hpp" -#include "nodes/scene_tracker_node.hpp" +#include "nodes/frame_annotation_node.hpp" #include #include @@ -171,13 +171,8 @@ static Config config_from_dict(nb::dict d) { // face tracker getf("track_alpha", cfg.track_alpha); getf("track_min_iou", cfg.track_min_iou); - getf("track_max_embed_dist", cfg.track_max_embed_dist); - geti("track_max_frames_missing", cfg.track_max_frames_missing); - getf("cut_revive_sim", cfg.cut_revive_sim); - geti("cut_inactive_max_frames", cfg.cut_inactive_max_frames); - // scene tracker - getd("extinction_sec", cfg.extinction_sec); - getd("anneal_sec", cfg.anneal_sec); + getf("track_assoc_min_prob", cfg.track_assoc_min_prob); + getd("track_extinction_sec", cfg.track_extinction_sec); // gallery expansion (usually off for sweeps; expose so it can be toggled) if (d.contains("expand_gallery")) cfg.expand_gallery = nb::cast(d["expand_gallery"]); /// TRACES: GR-004 | SR-001 @@ -189,7 +184,7 @@ static Config config_from_dict(nb::dict d) { using Net = kpn::python::PyNetwork; NB_MODULE(sae_kpn, m) { - m.doc() = "Real KPN downstream nodes (tracker/matcher/scene_tracker) for Python replay sweeps"; + m.doc() = "Real KPN downstream nodes (tracker/matcher/frame_annotation) for Python replay sweeps"; kpn::python::register_py_network(m, "Network"); @@ -270,30 +265,27 @@ NB_MODULE(sae_kpn, m) { }, "net"_a, "name"_a, "gallery"_a, "config"_a, "capacity"_a = 16, "embedder_model"_a = "", "embedder_sha256"_a = ""); - m.def("add_scene_tracker", [](Net& net, std::string name, nb::dict cfg_dict, std::size_t cap) { - Config cfg = config_from_dict(cfg_dict); + /// TRACES: AR-012, AR-013 | SR-002 + // Was add_scene_tracker, backed by the extinction-timer state machine. The + // node is gone (see frame_annotation_node.hpp) and so is the timer; this + // projects a matched frame into the same SceneAnnotation the Python sink + // already reads, so the seam's output type is unchanged. It takes no config + // because it has no state to configure -- which is the point. + m.def("add_frame_annotation", [](Net& net, std::string name, std::size_t cap) { auto node = std::make_shared, kpn::out<"annotation">>>(cap, cfg); + FrameAnnotationFunc, SaeVariant, kpn::in<"matched">, kpn::out<"annotation">>>(cap); net.add(std::move(name), std::move(node)); - }, "net"_a, "name"_a, "config"_a, "capacity"_a = 16); + }, "net"_a, "name"_a, "capacity"_a = 16); // ── Runtime setters (persistent-pipeline reuse across a threshold sweep) ───── // Build the network once, then change thresholds between replays — no rebuild, // no teardown (which is where the ROCm deadlock lives), no gallery reload. using MatcherWrap = kpn::ObjectVariantNodeWrapper< IdentityMatcherFunc, SaeVariant, kpn::in<"tracked">, kpn::out<"matched">>; - using SceneWrap = kpn::ObjectVariantNodeWrapper< - SceneTrackerFunc, SaeVariant, kpn::in<"matched">, kpn::out<"annotation">>; m.def("set_prob_threshold", [](Net& net, std::string name, float t) { auto* w = dynamic_cast(net.node_ptr(name)); if (!w) throw std::runtime_error("set_prob_threshold: '" + name + "' is not an identity_matcher"); w->functor().set_prob_threshold(t); }, "net"_a, "name"_a, "value"_a); - - m.def("set_extinction_sec", [](Net& net, std::string name, double s) { - auto* w = dynamic_cast(net.node_ptr(name)); - if (!w) throw std::runtime_error("set_extinction_sec: '" + name + "' is not a scene_tracker"); - w->functor().set_extinction_sec(s); - }, "net"_a, "name"_a, "value"_a); } diff --git a/src/main.cpp b/src/main.cpp index 5d47b1f..ed6f47d 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -8,11 +8,11 @@ // // [frame_source] ──Frame──► [face_detector] ──SceneFrame──► [face_aligner] // ──AlignedSceneFrame──► [embedder] ──EmbeddedSceneFrame──► -// [identity_matcher] ──MatchedSceneFrame──► [scene_tracker] +// [identity_matcher] ──MatchedSceneFrame──► [frame_annotation] // ──SceneAnnotation──► [result_sink] // // Debug build (SAE_DEBUG=1): -// [identity_matcher] output fans out to both [scene_tracker] AND [debug_renderer]. +// [identity_matcher] output fans out to both [frame_annotation] AND [debug_renderer]. // FanoutNode is auto-inserted by make_network(). // // Usage: @@ -23,7 +23,6 @@ // --fps sample rate in frames/sec (default: 1.0) // --verbosity <0|1|2> 0=minimal, 1=standard, 2=jellyfin-xray (default: 0) // --prob-threshold posterior P(match) to accept (default: 0.754) -// --extinction actor extinction window in seconds (default: 5.0) // --detector override SCRFD detector model path // --arcface override ArcFace model path // --scene-detect enable TransNetV2 shot-boundary detection (dense decode; @@ -64,7 +63,7 @@ #include "nodes/embedder_node.hpp" #include "nodes/face_tracker_node.hpp" #include "nodes/identity_matcher_node.hpp" -#include "nodes/scene_tracker_node.hpp" +#include "nodes/frame_annotation_node.hpp" #include "nodes/scene_detector_node.hpp" #include "scene_boundaries.hpp" #include "nodes/scene_boundary_annotator_node.hpp" @@ -155,7 +154,6 @@ static Config parse_args(int argc, char** argv) { else if (arg("--verbosity")) { int v = std::stoi(next()); cfg.verbosity = v == 2 ? Verbosity::xray : v == 1 ? Verbosity::standard : Verbosity::minimal; } else if (arg("--prior")) cfg.match_prior = std::stof(next()); else if (arg("--prob-threshold")) cfg.prob_threshold = std::stof(next()); - else if (arg("--extinction")) cfg.extinction_sec = std::stod(next()); else if (arg("--detector")) cfg.detector_model = next(); else if (arg("--detector-engine")) cfg.detector_engine = next(); else if (arg("--arcface")) cfg.arcface_model = next(); @@ -168,7 +166,6 @@ static Config parse_args(int argc, char** argv) { else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next()); else if (arg("--track-min-prob")) cfg.track_assoc_min_prob = std::stof(next()); else if (arg("--track-extinction")) cfg.track_extinction_sec = std::stod(next()); - else if (arg("--anneal")) cfg.anneal_sec = std::stod(next()); else if (arg("--expand-gallery")) cfg.expand_gallery = true; else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next()); else if (arg("--expand-band-lo")) cfg.expand_band_lo = std::stof(next()); @@ -255,7 +252,7 @@ int main(int argc, char** argv) { // and its final answer is only known when a track dies. auto same_person = same_person_probability(matcher_fn.calibration()); TrackRegistry::Config reg_cfg; - reg_cfg.extinction_sec = cfg.track_extinction_sec; + reg_cfg.track_extinction_sec = cfg.track_extinction_sec; auto registry = std::make_shared( reg_cfg, EvidenceDiscounter(same_person)); @@ -263,7 +260,7 @@ int main(int argc, char** argv) { FaceTrackerFunc ftracker_fn{cfg, registry, same_person}; - SceneTrackerFunc tracker_fn {cfg}; + FrameAnnotationFunc tracker_fn {}; ResultSinkFunc sink_fn {cfg, done}; /// TRACES: AR-012, AR-016 | IR-002, IR-003 | SR-002 @@ -299,7 +296,7 @@ int main(int argc, char** argv) { kpn::ObjectNode, kpn::out<"embedded">, "embedder", 0> embedder (embedder_fn, 32); kpn::ObjectNode, kpn::out<"tracked">, "face_tracker", 0> ftracker (ftracker_fn, 16); kpn::ObjectNode, kpn::out<"matched">, "identity_matcher", 0> matcher (matcher_fn, 16); - kpn::ObjectNode, kpn::out<"annotation">, "scene_tracker", 0> tracker (tracker_fn, 16); + kpn::ObjectNode, kpn::out<"annotation">, "frame_annotation", 0> tracker (tracker_fn, 16); kpn::ObjectNode,kpn::out<>, "result_sink", 0> sink (sink_fn, 16); // ── Pipeline observability + run loop (topology-agnostic) ────────────────── diff --git a/src/nodes/frame_annotation_node.hpp b/src/nodes/frame_annotation_node.hpp new file mode 100644 index 0000000..c381b46 --- /dev/null +++ b/src/nodes/frame_annotation_node.hpp @@ -0,0 +1,43 @@ +#pragma once +/// TRACES: AR-012, AR-013 | SR-002 +/// +/// FrameAnnotationFunc — project a matched frame into a per-frame annotation. +/// +/// Stateless, and that is the entire point of it. +/// +/// It replaces `SceneTrackerFunc`, which kept an extinction timer per actor and +/// reported an actor as visible for `extinction_sec` (57.4 s) after their last +/// detection. docs/SPEC.md specified that node's deletion -- "anneal_sec and +/// extinction_sec are deleted, not re-tuned ... SceneTrackerFunc goes with +/// them", with a removal list ending "grep for both names and expect no +/// survivors" -- and docs/requirements.md recorded both constants as Withdrawn, +/// deleted "rather than retained at zero", on the grounds that a field naming a +/// mechanism the pipeline no longer has is actively misleading. None of that +/// removal had happened. The node was still wired into both shipped pipelines +/// and still printed its timeout at every startup. +/// +/// **Presence is not this node's business.** AR-012 moved it to TrackRegistry, +/// where a window is `[first_seen, last_seen]` of a track an actor owns, and +/// AR-013 ends that window at the last sighting rather than after it. A +/// keep-alive here answered the same question a second time and answered it +/// worse: it re-opened the trailing cool-down the registry exists to refuse. +/// +/// What a consumer sees change: `--verbosity standard`'s `frames[].identified` +/// used to list every actor still inside the keep-alive, including ones absent +/// from the frame. It now lists what was actually matched in that frame. The +/// minimal and xray outputs are unaffected -- they were already built from +/// registry claims and never consulted this node. + +#include "types.hpp" + +#include +#include + +struct FrameAnnotationFunc { + static constexpr std::string_view label() { return "frame_annotation"; } + + SceneAnnotation operator()(MatchedSceneFrame mf) { + if (mf.source.eof) return {0.0, {}, /*eof=*/true}; + return {mf.source.timestamp_sec, std::move(mf.actors)}; + } +}; diff --git a/src/nodes/result_sink_node.hpp b/src/nodes/result_sink_node.hpp index 10f2026..fc1a059 100644 --- a/src/nodes/result_sink_node.hpp +++ b/src/nodes/result_sink_node.hpp @@ -23,7 +23,8 @@ using json = nlohmann::json; // // Verbosity::minimal — merges per-frame presence into contiguous time windows. // Output: { -// "schema_version": 1, "movie": "...", "sample_fps": ..., "anneal_sec": ..., +// "schema_version": 2, "movie": "...", +// "extraction": { "sample_fps": ..., "extinction_sec": ..., "gallery_scope": ... }, // "actors": [{ "name", "imdb_id", "tmdb_id", "jellyfin_id", "scenes": [[t0,t1], ...] }] // } // An optional top-level "jellyfin_item_id" (the analysed title's Jellyfin item @@ -123,8 +124,9 @@ private: /// schema_version 2, per jRay/SPEC.md JR-002. anneal_sec is REMOVED /// rather than zeroed: a field naming a mechanism the pipeline no /// longer has is actively misleading, and would outlive everyone who - /// remembers why it reads 0. extinction_sec succeeds it as the - /// parameter that actually shapes window extent. + /// remembers why it reads 0. The extraction block reports + /// track_extinction_sec, which bounds re-association -- not the + /// withdrawn actor keep-alive that shared its name. root["schema_version"] = kSchemaVersion; root["movie"] = cfg_.movie_path; root["extraction"] = { diff --git a/src/nodes/scene_tracker_node.hpp b/src/nodes/scene_tracker_node.hpp deleted file mode 100644 index 2de6897..0000000 --- a/src/nodes/scene_tracker_node.hpp +++ /dev/null @@ -1,102 +0,0 @@ -#pragma once -#include "types.hpp" -#include "config.hpp" - -#include -#include - -// ── SceneTrackerFunc ────────────────────────────────────────────────────────── -// KPN node: maintains an extinction-timer state machine per identified actor. -// -// On each MatchedSceneFrame: -// 1. Update last_seen for every matched known actor. -// 2. Expire actors whose last_seen is older than extinction_sec. -// 3. Emit SceneAnnotation with all currently active (non-expired) actors, -// including their most recently seen bbox and best similarity score. -// -// Unknown faces (actor_idx == -1) are passed through per-frame but are NOT -// tracked across frames — each frame reports its own unknowns independently. - -struct SceneTrackerFunc { - static constexpr std::string_view label() { return "scene_tracker"; } - - explicit SceneTrackerFunc(const Config& cfg) - : extinction_sec_(cfg.extinction_sec) - { - std::cerr << "[scene_tracker] extinction_sec=" << extinction_sec_ << "\n"; - } - - // Runtime setter for pipeline reuse across a sweep. Also clears the active-actor - // state so a re-run starts clean (no carry-over from the previous config's film). - void set_extinction_sec(double s) { extinction_sec_ = s; active_.clear(); } - - SceneAnnotation operator()(MatchedSceneFrame mf) { - if (mf.source.eof) return {0.0, {}, /*eof=*/true}; - - double now = mf.source.timestamp_sec; - - // Update known actors - for (const auto& ia : mf.actors) { - if (ia.actor_idx < 0) continue; // skip unknowns - - auto& slot = active_[ia.actor_idx]; - slot.last_seen = now; - slot.last_bbox = ia.bbox; - slot.last_crop = ia.crop; - slot.name = ia.name; - slot.imdb_id = ia.imdb_id; - slot.tmdb_id = ia.tmdb_id; - slot.jellyfin_id = ia.jellyfin_id; - // Keep the best (highest) similarity seen in this window - if (ia.similarity > slot.best_similarity) - slot.best_similarity = ia.similarity; - } - - // Expire stale actors - for (auto it = active_.begin(); it != active_.end(); ) { - if ((now - it->second.last_seen) > extinction_sec_) - it = active_.erase(it); - else - ++it; - } - - // Build annotation: active known actors - std::vector visible; - visible.reserve(active_.size() + mf.actors.size()); - - for (const auto& [actor_idx, slot] : active_) { - IdentifiedActor ia; - ia.actor_idx = actor_idx; - ia.name = slot.name; - ia.imdb_id = slot.imdb_id; - ia.tmdb_id = slot.tmdb_id; - ia.jellyfin_id = slot.jellyfin_id; - ia.similarity = slot.best_similarity; - ia.bbox = slot.last_bbox; - ia.crop = slot.last_crop; - visible.push_back(ia); - } - - // Append per-frame unknowns (actor_idx == -1) directly - for (const auto& ia : mf.actors) { - if (ia.actor_idx < 0) visible.push_back(ia); - } - - return {now, std::move(visible)}; - } - -private: - struct Slot { - double last_seen{0.0}; - float best_similarity{0.f}; - cv::Rect2f last_bbox; - cv::Mat last_crop; - std::string name; - std::string imdb_id; - std::string tmdb_id; - std::string jellyfin_id; - }; - - double extinction_sec_; - std::map active_; // actor_idx → state -}; diff --git a/src/scene_preview.cpp b/src/scene_preview.cpp index 81f64eb..d237be6 100644 --- a/src/scene_preview.cpp +++ b/src/scene_preview.cpp @@ -8,7 +8,7 @@ // // camera_pos (histogram cut detector) stamps Frame::cut_score / is_cut, which // ride through to the preview HUD's cut-score meter. -// ├──► [scene_tracker] ──► [result_sink] (background thread) +// ├──► [frame_annotation] ──► [result_sink] (background thread) // └──► [preview_node] (main thread) // // The main thread drives preview_node via preview.step(). When the movie ends @@ -30,7 +30,9 @@ #include "nodes/embedder_node.hpp" #include "nodes/face_tracker_node.hpp" #include "nodes/identity_matcher_node.hpp" -#include "nodes/scene_tracker_node.hpp" +#include "track_registry.hpp" +#include "evidence_discount.hpp" +#include "nodes/frame_annotation_node.hpp" #include "nodes/result_sink_node.hpp" #include "nodes/preview_node.hpp" @@ -71,7 +73,6 @@ static Config parse_args(int argc, char** argv) { else if (arg("--verbosity")) { int v = std::stoi(next()); cfg.verbosity = v == 2 ? Verbosity::xray : v == 1 ? Verbosity::standard : Verbosity::minimal; } else if (arg("--prior")) cfg.match_prior = std::stof(next()); else if (arg("--prob-threshold")) cfg.prob_threshold = std::stof(next()); - else if (arg("--extinction")) cfg.extinction_sec = std::stod(next()); else if (arg("--detector")) cfg.detector_model = next(); else if (arg("--detector-engine")) cfg.detector_engine = next(); else if (arg("--arcface")) cfg.arcface_model = next(); @@ -82,9 +83,8 @@ static Config parse_args(int argc, char** argv) { else if (arg("--min-face-px")) cfg.min_face_px = std::stof(next()); else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next()); else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next()); - else if (arg("--track-max-embed")) cfg.track_max_embed_dist = std::stof(next()); - else if (arg("--track-max-missing")) cfg.track_max_frames_missing = std::stoi(next()); - else if (arg("--anneal")) cfg.anneal_sec = std::stod(next()); + else if (arg("--track-min-prob")) cfg.track_assoc_min_prob = std::stof(next()); + else if (arg("--track-extinction")) cfg.track_extinction_sec = std::stod(next()); else if (arg("--trt-cache")) cfg.trt.cache_dir = next(); else if (arg("--trt-fp16")) cfg.trt.fp16 = true; else if (arg("--no-trt-fp16")) cfg.trt.fp16 = false; @@ -144,11 +144,32 @@ int main(int argc, char** argv) { FaceDetectorFunc detector_fn{cfg}; FaceAlignerFunc aligner_fn; EmbedderFunc embedder_fn{cfg}; - FaceTrackerFunc ftracker_fn{cfg}; + /// TRACES: DP-001, AR-007, AR-012, AR-024 | PR-004, SR-002 + // Construction order matters and is the same as main.cpp's, deliberately: + // the matcher fits (or loads) the calibration, the registry needs a + // discounter built from it, and the tracker needs both. DP-001 says modes + // are front-ends that must not fork pipeline logic -- this file had forked + // it and then rotted, constructing FaceTrackerFunc{cfg} against a signature + // that stopped existing with the AR-007/AR-008 redesign, so scene_preview + // has not compiled since. Keeping the order identical is what stops that + // recurring. IdentityMatcherFunc matcher_fn {gallery, cfg}; - SceneTrackerFunc tracker_fn {cfg}; + auto same_person = same_person_probability(matcher_fn.calibration()); + TrackRegistry::Config reg_cfg; + reg_cfg.track_extinction_sec = cfg.track_extinction_sec; + auto registry = std::make_shared( + reg_cfg, EvidenceDiscounter(same_person)); + matcher_fn.set_registry(registry); + + FaceTrackerFunc ftracker_fn{cfg, registry, same_person}; + FrameAnnotationFunc tracker_fn {}; ResultSinkFunc sink_fn {cfg, done}; + // AR-012/AR-016: windows come from registry claims, and tracks still live + // at EOF must be flushed or the closing scene's cast is never emitted. + registry->on_track_dead([&sink_fn](const DeadTrack& d) { sink_fn.add_claim(d); }); + sink_fn.set_pre_write_hook([registry](double last_ts) { registry->flush(last_ts); }); + // ── KPN ObjectNodes ─────────────────────────────────────────────────────── kpn::ObjectNode, kpn::out<"raw">, "frame_source", 0> source (source_fn, 32); kpn::ObjectNode, kpn::out<"frame">, "camera_pos", 0> campos (campos_fn, 32); @@ -157,13 +178,13 @@ int main(int argc, char** argv) { kpn::ObjectNode, kpn::out<"embedded">, "embedder", 0> embedder (embedder_fn, 32); kpn::ObjectNode, kpn::out<"tracked">, "face_tracker", 0> ftracker (ftracker_fn, 16); kpn::ObjectNode, kpn::out<"matched">, "identity_matcher", 0> matcher (matcher_fn, 16); - kpn::ObjectNode, kpn::out<"annotation">, "scene_tracker", 0> tracker (tracker_fn, 16); + kpn::ObjectNode, kpn::out<"annotation">, "frame_annotation", 0> tracker (tracker_fn, 16); kpn::ObjectNode,kpn::out<>, "result_sink", 0> sink (sink_fn, 16); // MainThreadNode — no thread spawned; driven by preview.step() below PreviewNode preview{cfg, 16}; - // matcher → FanoutNode → [scene_tracker, preview] (auto-inserted) + // matcher → FanoutNode → [frame_annotation, preview] (auto-inserted) auto net = kpn::make_network( kpn::edge(source.output<"raw">(), campos.input<"raw">()), kpn::edge(campos.output<"frame">(), detector.input<"frame">()), diff --git a/src/track_registry.hpp b/src/track_registry.hpp index eb23dc9..b0891e6 100644 --- a/src/track_registry.hpp +++ b/src/track_registry.hpp @@ -81,7 +81,16 @@ public: using DeadTrackFn = std::function; struct Config { - double extinction_sec{5.0}; ///< how long a lost track stays revivable + /// How long a lost track stays available for re-association. + /// + /// Named to match Config::track_extinction_sec, which feeds it, and + /// deliberately NOT `extinction_sec`: that name belonged to the + /// withdrawn actor keep-alive, and SPEC.md's removal list ends "grep + /// for both names and expect no survivors". A survivor here would be + /// the one false positive in that grep, on a field that means + /// something else entirely -- this one bounds re-association and never + /// extends a presence claim. + double track_extinction_sec{5.0}; float ownership_logodds{2.0f}; ///< belief needed to own a track (~0.88 posterior) }; @@ -216,7 +225,7 @@ private: void tick_locked(double now) { for (auto it = tracks_.begin(); it != tracks_.end(); ) { const auto& ls = it->second.last_seen; - if (ls && (now - *ls) > cfg_.extinction_sec) { + if (ls && (now - *ls) > cfg_.track_extinction_sec) { emit_locked(it->second, *ls); it = tracks_.erase(it); } else { diff --git a/tests/test_face_tracker.cpp b/tests/test_face_tracker.cpp index c0645bf..587a7b1 100644 --- a/tests/test_face_tracker.cpp +++ b/tests/test_face_tracker.cpp @@ -73,7 +73,7 @@ struct Rig { : reg(std::make_shared( [extinction] { TrackRegistry::Config c; - c.extinction_sec = extinction; + c.track_extinction_sec = extinction; return c; }(), EvidenceDiscounter([](float cos) { return std::max(0.f, cos); }))) diff --git a/tests/test_replay_fixtures.cpp b/tests/test_replay_fixtures.cpp index 518214f..1d755de 100644 --- a/tests/test_replay_fixtures.cpp +++ b/tests/test_replay_fixtures.cpp @@ -128,7 +128,7 @@ struct Replay { Replay run(const Dump& d, double extinction = 10.0) { Replay r; TrackRegistry::Config rc; - rc.extinction_sec = extinction; + rc.track_extinction_sec = extinction; auto cal = [](float cos) { return std::max(0.f, cos); }; auto reg = std::make_shared(rc, EvidenceDiscounter(cal)); diff --git a/tests/test_track_registry.cpp b/tests/test_track_registry.cpp index 980749a..1773281 100644 --- a/tests/test_track_registry.cpp +++ b/tests/test_track_registry.cpp @@ -45,7 +45,7 @@ EvidenceDiscounter disc() { TrackRegistry::Config cfg(double extinction = 5.0, float own = 2.0f) { TrackRegistry::Config c; - c.extinction_sec = extinction; + c.track_extinction_sec = extinction; c.ownership_logodds = own; return c; }