dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(replay): the whole replay chain is C++, including the sink

Browse Source 
The sae_kpn module has not compiled since the AR-007/AR-008 tracker redesign,
and was switched off at the build rather than patched because the fix is a
restructuring. Two failures, one cause.

It did not compile: `add_face_tracker` built FaceTrackerFunc from a Config
alone, and the tracker has required a TrackRegistry and a calibration since
association moved into probability space.

And presence was rebuilt in Python. `replay.py::build_minimal` merged
per-frame detections into windows by annealing gaps, which is what the
pipeline did before AR-012. The sink builds a window from a TrackRegistry
claim instead — the extent of a track an actor owned, starting when they
appeared rather than when recognition first succeeded. Those answer different
questions, so every sweep was tuning against a contract the shipped code had
stopped honouring.

Both follow from the seam being a factory per node. The chain has a
construction order — the matcher fits the calibration, the registry needs a
discounter built from it, the tracker needs both, and the sink needs the
registry's claims — and independent factories cannot express it, so the
tracker kept being built against a signature that no longer existed. One
`add_pipeline` mirrors main.cpp exactly and is now the only way to build the
chain, so the ordering cannot be got wrong again from Python. DP-001 is the
requirement behind it: a replay harness is a front-end, and its job is to
supply frames and read the result, not to re-derive presence.

Lifetimes needed a home. ResultSinkFunc holds `const Config&` and
`std::atomic<bool>&`, which under main() are locals in a frame outliving the
pipeline; there is no such frame when the network is built and torn down from
Python. ReplaySession owns both for the network's lifetime, keyed by network
and released explicitly — a sweep builds one network per replay and the sink
retains every annotation, so holding them forever would grow with films x
configs. Getting this wrong presented as an empty output_path: the sink
announced `[result_sink] writing ` and wrote nothing.

test_sae_kpn.py is ported rather than left behind. It called all three removed
factories and asserted on SceneAnnotations read back per frame; neither half
survives, so it now waits on pipeline_done and asserts on the file the sink
writes. Verified against gallery_lvface.h5: three frames through the real
chain, timestamps 0/1/2, truth file written. EOF is a control token the sink
flushes on and does not record, so three inputs give three frames, never four.

SAE_BUILD_KPN_BINDINGS goes back to ON.

TRACES: VR-011, VR-002 | DP-001 | PR-002
This commit is contained in:
dtourolle
2026-08-05 19:40:25 +02:00
parent 1141172b04
commit 48332d2041
6 changed files with 414 additions and 229 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/optimizer/test_sae_kpn.py
+74 -31
View File
@@ -1,17 +1,30 @@
#!/usr/bin/env python3
"""
Smoke test for the sae_kpn module: assemble the real downstream pipeline nodes
(face_tracker → identity_matcher → frame_annotation) in a Python-driven KPN network,
fed by a no-input Python source node, and verify SceneAnnotations flow out.
Smoke test for the sae_kpn module: assemble the real downstream pipeline
(tracker → matcher → annotation → sink) in a Python-driven KPN network, fed by a
no-input Python source node, and verify the sink writes a truth file.
TRACES: VR-011 | PR-002
Proves the KPN-native replay path works without any numpy port of node logic.
Rewritten for `add_pipeline`. It previously called three node factories and read
SceneAnnotations back through the seam, asserting on what came out per frame.
Neither half of that survives VR-011: the factories are gone because the chain
has a construction order Python could not express, and presence is now the C++
sink's answer, derived from TrackRegistry claims. Nothing is read per frame, so
the assertions are on the file the sink writes.
Run: python scripts/optimizer/test_sae_kpn.py [gallery.json] [build_dir]
"""
import json
import sys
import queue
import numpy as np
import tempfile
import time
from pathlib import Path
import numpy as np
REPO = Path(__file__).resolve().parent.parent.parent
GAL = sys.argv[1] if len(sys.argv) > 1 else str(REPO / "gallery_arcface_w600k_r50.json")
BUILD = sys.argv[2] if len(sys.argv) > 2 else str(REPO / "build")
@@ -31,7 +44,6 @@ def make_frame(t, n):
def main():
net = sae_kpn.Network()
sae_kpn._register_types(net)
cfg = {"prob_threshold": 0.99, "track_extinction_sec": 5.0}
frames = [make_frame(float(t), 1) for t in range(3)]
frames.append({"timestamp_sec": 3.0, "eof": True})
@@ -39,36 +51,67 @@ def main():
eof_frame = {"timestamp_sec": 3.0, "eof": True}
def source():
# Emit each frame once, then keep returning EOF (never block) so the node
# thread stays responsive to stop() after the sink has seen EOF.
# Emit each frame once, then keep returning EOF so the node thread stays
# responsive to stop(). The sleep matters: a no-input source is called in
# a tight loop, and hot-spinning EOFs pegs a core and floods the channel.
i = idx[0]
idx[0] += 1
return frames[i] if i < len(frames) else eof_frame
if i < len(frames):
return frames[i]
time.sleep(0.05)
return eof_frame
sae_kpn.add_node_python(net, "replay", source, [], ["EmbeddedSceneFrame"], 8)
sae_kpn.add_face_tracker(net, "tracker", cfg, 16)
sae_kpn.add_identity_matcher(net, "matcher", GAL, cfg, 16)
sae_kpn.add_frame_annotation(net, "scene", 16)
net.connect("replay", 0, "tracker", 0)
net.connect("tracker", 0, "matcher", 0)
net.connect("matcher", 0, "scene", 0)
net.build()
net.start()
with tempfile.TemporaryDirectory() as tmp:
out_path = str(Path(tmp) / "truth.json")
cfg = {
"prob_threshold": 0.99,
"track_extinction_sec": 5.0,
"output_path": out_path,
"movie_path": "sae_kpn smoke test",
"sample_fps": 1.0,
# Standard verbosity emits the per-frame array this test asserts on.
# At 0 the file carries only the actor epochs, and three random
# embeddings against a real gallery need not produce any.
"verbosity": 1,
}
got = []
for _ in range(4):
sa = net.read("scene", 0)
got.append(sa)
if sa.get("eof"):
break
net.stop()
sae_kpn.add_node_python(net, "replay", source, [], ["EmbeddedSceneFrame"], 16)
# No embedder stamp: these embeddings are random, not the output of any
# model, so there is nothing truthful to claim. That warns rather than
# failing, and would be fatal under SAE_REQUIRE_GALLERY_STAMP — which is
# correct, since an unverifiable binding is exactly what it guards.
sae_kpn.add_pipeline(net, GAL, cfg, 16)
non_eof = [g for g in got if not g.get("eof")]
assert len(non_eof) == 3, f"expected 3 annotations, got {len(non_eof)}"
assert got[-1].get("eof"), "expected trailing EOF"
assert [g["timestamp_sec"] for g in non_eof] == [0.0, 1.0, 2.0], "timestamps wrong"
assert all("visible_actors" in g for g in non_eof), "missing visible_actors"
print(f"OK: {len(non_eof)} annotations through the real KPN chain, EOF received")
net.connect("replay", 0, "tracker", 0)
net.connect("tracker", 0, "matcher", 0)
net.connect("matcher", 0, "annotation", 0)
net.connect("annotation", 0, "sink", 0)
net.build()
net.start()
# The sink writes on the EOF annotation. Wait for that rather than
# reading anything back: presence lives entirely on the C++ side.
deadline = time.time() + 30.0
while not sae_kpn.pipeline_done(net):
if time.time() > deadline:
sae_kpn.release_pipeline(net)
raise TimeoutError("sink never saw EOF within 30s")
time.sleep(0.02)
net.stop()
sae_kpn.release_pipeline(net)
with open(out_path) as f:
truth = json.load(f)
per_frame = truth.get("frames", [])
assert "actors" in truth, "truth file has no actors array"
assert len(per_frame) == 3, f"expected 3 frames, got {len(per_frame)}"
# EOF is a control token, not an observation: the sink flushes on it and does
# not record it, so three inputs give three frames and never four.
assert [f["t"] for f in per_frame] == [0.0, 1.0, 2.0], "timestamps wrong"
assert all("identified" in f for f in per_frame), "missing identified"
print(f"OK: {len(per_frame)} frames through the real KPN chain, sink wrote its truth file")
if __name__ == "__main__":