dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: tracker owns no state; association is frame-dependent and calibrated

Browse Source 
Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.

AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.

AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.

AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.

Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.

Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-008, AR-024 | SR-002
This commit is contained in:
dtourolle
2026-07-31 09:58:27 +02:00
co-authored by Claude Opus 5
parent 843852e19c
commit e9aea3fc41
6 changed files with 356 additions and 234 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_face_tracker.cpp
+117 -53
View File
@@ -13,8 +13,12 @@
#include "config.hpp"
#include "nodes/face_tracker_node.hpp"
#include "types.hpp"
#include "track_registry.hpp"
#include "evidence_discount.hpp"
#include <algorithm>
#include <cmath>
#include <memory>
namespace {
@@ -55,85 +59,145 @@ EmbeddedSceneFrame frame(double t, float x, float y, const Embedding& emb,
return ef;
}
Config tracker_cfg() {
Config cfg;
cfg.cut_revive_sim = 0.50f;
cfg.cut_inactive_max_frames = 5;
return cfg;
}
// Build a tracker over a fresh registry. The registry IS the tracker's state
// now (AR-008), so a test constructs both together and can inspect either.
struct Rig {
std::shared_ptr<TrackRegistry> reg;
FaceTrackerFunc ft;
explicit Rig(double extinction = 30.0, float assoc_min_prob = 0.5f)
: reg(std::make_shared<TrackRegistry>(
[extinction] {
TrackRegistry::Config c;
c.extinction_sec = extinction;
return c;
}(),
EvidenceDiscounter([](float cos) { return std::max(0.f, cos); })))
, ft([&] {
Config c;
c.track_assoc_min_prob = assoc_min_prob;
return c;
}(),
reg,
// Trivial calibration: cosine passed through as P(same). Real runs use
// the fit belonging to the active embedder (AR-023/AR-024).
[](float cos) { return std::max(0.f, cos); })
{}
int track_of(EmbeddedSceneFrame f) { return ft(std::move(f)).track_ids[0]; }
};
} // namespace
TEST_CASE("track id is stable across ordinary frames", "[face_tracker]") {
FaceTrackerFunc ft(tracker_cfg());
// ── AR-008 — one pool, ordinary association ──────────────────────────────────
TEST_CASE("track id is stable across ordinary frames", "[face_tracker][AR-008]") {
Rig r;
Embedding e = axis(0);
int id0 = ft(frame(0.0, 10, 10, e)).track_ids[0];
int id1 = ft(frame(1.0, 11, 10, e)).track_ids[0]; // overlaps → same track
int id0 = r.track_of(frame(0.0, 10, 10, e));
int id1 = r.track_of(frame(1.0, 11, 10, e)); // overlaps → same track
CHECK(id0 >= 0);
CHECK(id1 == id0);
}
TEST_CASE("cut revives the same track id for a matching identity", "[face_tracker]") {
FaceTrackerFunc ft(tracker_cfg());
TEST_CASE("a face lost across a cut and re-associated is the SAME track",
"[face_tracker][AR-008]") {
// Previously this was a distinct "revival" path guarded by a raw-cosine
// constant. There is no such path now: a dormant track is an ordinary
// association candidate, and continuity falls out of the embedding match.
Rig r;
// Pre-cut: establish a track for a person whose embedding is near-identical
// across the cut (sim well above cut_revive_sim), but whose box jumps so IoU
// is 0 — the ordinary spatial path cannot re-link it.
Embedding pre = at_sim(0, 1, 0.99f);
int id_pre = ft(frame(0.0, 10, 10, pre)).track_ids[0];
int id_pre = r.track_of(frame(0.0, 10, 10, pre));
REQUIRE(id_pre >= 0);
Embedding post = at_sim(0, 1, 0.98f); // cos(diff) ≈ 0.9997 > 0.50
auto out = ft(frame(1.0, 300, 300, post, /*is_cut=*/true));
CHECK(out.track_ids[0] == id_pre); // revived, not a fresh id
// Box jumps so IoU is zero — only the embedding can link it.
Embedding post = at_sim(0, 1, 0.98f);
CHECK(r.track_of(frame(1.0, 300, 300, post, /*is_cut=*/true)) == id_pre);
}
TEST_CASE("cut starts a fresh track when identity does not match", "[face_tracker]") {
FaceTrackerFunc ft(tracker_cfg());
int id_pre = ft(frame(0.0, 10, 10, axis(0))).track_ids[0];
TEST_CASE("a cut starts a fresh track when identity does not match",
"[face_tracker][AR-008]") {
Rig r;
int id_pre = r.track_of(frame(0.0, 10, 10, axis(0)));
REQUIRE(id_pre >= 0);
// Post-cut face is orthogonal (sim 0 < cut_revive_sim) and spatially disjoint
// → no revival, brand-new id.
auto out = ft(frame(1.0, 300, 300, axis(5), /*is_cut=*/true));
CHECK(out.track_ids[0] != id_pre);
CHECK(out.track_ids[0] >= 0);
// Orthogonal embedding and disjoint box: nothing links them.
int id_post = r.track_of(frame(1.0, 300, 300, axis(5), /*is_cut=*/true));
CHECK(id_post != id_pre);
CHECK(id_post >= 0);
}
TEST_CASE("parked track expires after cut_inactive_max_frames", "[face_tracker]") {
Config cfg = tracker_cfg();
cfg.cut_inactive_max_frames = 2;
FaceTrackerFunc ft(cfg);
// ── AR-007 — a cut makes association ignore position ─────────────────────────
TEST_CASE("on a cut, identity follows the embedding rather than the box",
"[face_tracker][AR-007]") {
// Two people swap screen positions across a cut while keeping their faces.
// If IoU still carried weight the ids would follow the boxes and swap; with
// alpha driven to embedding-only on a cut, they must follow the faces.
Rig r;
Embedding a = at_sim(0, 1, 0.99f);
Embedding b = at_sim(2, 3, 0.99f);
EmbeddedSceneFrame f0;
f0.source.timestamp_sec = 0.0;
f0.faces = {face_at(10, 10), face_at(300, 300)};
f0.crops = {cv::Mat(), cv::Mat()};
f0.embeddings = {a, b};
auto out0 = r.ft(std::move(f0));
const int id_a = out0.track_ids[0];
const int id_b = out0.track_ids[1];
REQUIRE(id_a >= 0);
REQUIRE(id_b >= 0);
REQUIRE(id_a != id_b);
// Same two people, positions exchanged, on a cut frame.
EmbeddedSceneFrame f1;
f1.source.timestamp_sec = 1.0;
f1.source.is_cut = true;
f1.faces = {face_at(300, 300), face_at(10, 10)};
f1.crops = {cv::Mat(), cv::Mat()};
f1.embeddings = {a, b};
auto out1 = r.ft(std::move(f1));
CHECK(out1.track_ids[0] == id_a); // A kept its id despite moving to B's box
CHECK(out1.track_ids[1] == id_b);
}
// ── AR-013 — extinction replaces the parked-pool frame counter ───────────────
TEST_CASE("a track past the extinction window is gone, not revived",
"[face_tracker][AR-013]") {
// The old design aged a parked pool in frames, which silently changed
// meaning with sample_fps. Extinction is in seconds and lives in the
// registry, so the tracker no longer counts anything.
Rig r(/*extinction=*/2.0);
Embedding person = at_sim(0, 1, 0.99f);
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0];
int id_pre = r.track_of(frame(0.0, 10, 10, person));
REQUIRE(id_pre >= 0);
// Cut with an unrelated face parks id_pre; then let the pool age past its
// limit with more unrelated, spatially-disjoint faces (each ages the pool by
// one). By the time the person returns, id_pre must be gone.
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park (age 1)
ft(frame(2.0, 300, 300, axis(7))); // age 2
ft(frame(3.0, 300, 300, axis(7))); // age 3 → id_pre dropped
// Unrelated faces elsewhere while the clock runs well past extinction.
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
r.track_of(frame(10.0, 300, 300, axis(7)));
auto out = ft(frame(4.0, 10, 10, person)); // same identity returns
CHECK(out.track_ids[0] != id_pre); // too late — fresh id
CHECK(r.track_of(frame(11.0, 10, 10, person)) != id_pre);
}
TEST_CASE("eof clears active and parked tracks", "[face_tracker]") {
FaceTrackerFunc ft(tracker_cfg());
Embedding person = at_sim(0, 1, 0.99f);
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0];
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park id_pre
TEST_CASE("a track within the extinction window is still a candidate",
"[face_tracker][AR-013]") {
Rig r(/*extinction=*/30.0);
Embedding person = at_sim(0, 1, 0.99f);
int id_pre = r.track_of(frame(0.0, 10, 10, person));
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
// Back inside the window: the same person continues the same track, so the
// gap is absorbed into one window rather than splitting it.
CHECK(r.track_of(frame(3.0, 10, 10, person)) == id_pre);
}
TEST_CASE("eof is forwarded", "[face_tracker]") {
Rig r;
EmbeddedSceneFrame eof;
eof.source.eof = true;
auto out = ft(std::move(eof));
CHECK(out.source.eof);
// After eof the pools are empty: the returning identity must get a fresh id,
// not the parked one.
auto out2 = ft(frame(2.0, 10, 10, person));
CHECK(out2.track_ids[0] != id_pre);
CHECK(r.ft(std::move(eof)).source.eof);
}