feat: tracker owns no state; association is frame-dependent and calibrated

Three requirements land together because they cannot be separated. The
cross-cut revival branch was the only user of cut_revive_sim, so retiring that
raw cosine forces the pool collapse, and collapsing the pool removes the only
caller of the constant. Splitting them would have produced an intermediate
commit whose only purpose was to be split.

AR-008 — FaceTrackerFunc no longer keeps its own tracks_/inactive_ maps; it
holds a shared_ptr<TrackRegistry> and operates on it directly. Two parallel
copies of track state could disagree, and every divergence would surface as a
wrong presence window with nothing to indicate it. There is now ONE candidate
pool: last_seen alone says whether IoU is meaningful. The park/revive path is
deleted outright — matching a dormant track is ordinary inter-frame
association, and continuity falls out of the embedding comparison the tracker
already did rather than being a mechanism of its own.

AR-007 — track_alpha becomes the base weight for ordinary frames only.
Association drops to embedding-only when position carries no information:
on is_cut or is_scene_boundary, because the viewpoint changed, and for a
dormant track, because time has passed since its box was last valid. The second
case matters as much as the first and had no equivalent before.

AR-024 — association cost is a calibrated probability, never a raw cosine. The
tracker takes the calibration belonging to the active embedder, the same
function object EvidenceDiscounter uses. track_max_embed_dist becomes
track_assoc_min_prob, which means the same thing for every model, gallery and
face size, where a bare cosine threshold did not.

Retired: track_max_embed_dist, cut_revive_sim, cut_inactive_max_frames, and
track_max_frames_missing — the last superseded by the registry's extinction
window. That one is worth naming: a frame count silently changed meaning with
sample_fps, so the same configuration behaved differently at 1 fps and 5 fps.
Extinction is in seconds and lives in one place.

Tests rewritten rather than deleted. The old cases asserted revival by raw
cosine; the same behaviours are now asserted through the registry — a face lost
across a cut and re-associated is the SAME track, one unbroken window, and a
face returning past the extinction window is not. Added the case AR-007 exists
for: two people swap screen positions across a cut while keeping their faces,
and identity must follow the embedding rather than the box.

Suite: 80 cases, 3250 assertions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-007, AR-008, AR-024 | SR-002
This commit is contained in:
2026-07-31 09:58:27 +02:00
co-authored by Claude Opus 5
parent 843852e19c
commit e9aea3fc41
6 changed files with 356 additions and 234 deletions
+18 -14
View File
@@ -94,21 +94,25 @@ struct Config {
float dense_scale{1.0f}; // dense-mode frame downscale (1 = off) float dense_scale{1.0f}; // dense-mode frame downscale (1 = off)
// ── Face tracking (frame-to-frame) ─────────────────────────────────────── // ── Face tracking (frame-to-frame) ───────────────────────────────────────
float track_alpha{0.4f}; // cost weight: 0=embedding only, 1=spatial only /// TRACES: AR-007, AR-008, AR-024 | SR-002
// track_alpha is the *base* weight, used on ordinary frames. It is
// frame-dependent (AR-007): on is_cut / is_scene_boundary, and for any track
// that is no longer on screen, it drops to 0 (embedding only), because
// position carries no information across a viewpoint change or a gap.
float track_alpha{0.4f}; // base cost weight: 0=embedding only, 1=spatial only
float track_min_iou{0.1f}; // IoU below which spatial link alone is rejected float track_min_iou{0.1f}; // IoU below which spatial link alone is rejected
float track_max_embed_dist{0.7f}; // cosine dist above which embedding link alone is rejected // Minimum P(same person) for an association to be admissible on appearance
int track_max_frames_missing{5}; // expire track after N consecutive missed frames // alone. This replaces track_max_embed_dist (a raw cosine distance, AR-024).
// 0.5 is not a tuned constant: it is the decision boundary. Below it the pair
// ── Cross-cut track re-association ──────────────────────────────────────── // is more likely two people than one, and no amount of IoU makes that a link
// A camera-angle change (Frame::is_cut) breaks spatial (IoU) continuity but // worth asserting on identity grounds.
// not identity: the same people are usually still on screen from a new angle. float track_assoc_min_prob{0.5f};
// Instead of destroying tracks on a cut, the tracker parks them in an // How long a track that has gone off screen stays available for association
// inactive pool. A post-cut detection whose raw cosine similarity to a parked // before the registry reaps it and emits its presence claim (AR-013).
// track's last-frame embedding is ≥ cut_revive_sim revives that track_id // Replaces track_max_frames_missing: a frame count silently changed meaning
// (identity continuity survives the cut); otherwise it starts a fresh track. // with sample_fps, and the same number had to be guessed twice (once for an
// Parked tracks that go unrevived for cut_inactive_max_frames are dropped. // ordinary miss, once for a cut). Seconds mean one thing at any sample rate.
float cut_revive_sim{0.50f}; // min raw cosine sim (last-frame emb) to revive across a cut double track_extinction_sec{5.0};
int cut_inactive_max_frames{5}; // drop a parked track after N frames without revival
// ── Scene tracking ──────────────────────────────────────────────────────── // ── Scene tracking ────────────────────────────────────────────────────────
// extinction_sec re-tuned by DE against X-Ray per-second presence, 4-film rep4 // extinction_sec re-tuned by DE against X-Ray per-second presence, 4-film rep4
+28
View File
@@ -9,6 +9,7 @@
#include <cstdlib> #include <cstdlib>
#include <cstring> #include <cstring>
#include <fstream> #include <fstream>
#include <functional>
#include <iostream> #include <iostream>
#include <stdexcept> #include <stdexcept>
#include <string> #include <string>
@@ -49,6 +50,33 @@ struct GalleryCalibration {
} }
}; };
/// TRACES: AR-023, AR-024 | SR-002
///
/// cosine → P(same person). The one probability space the pipeline reasons in.
///
/// Handed to every stage that has to decide whether two embeddings are the same
/// person — track association (AR-007), evidence discounting (AR-025), identity
/// matching — so a threshold of 0.5 means the same thing in all of them. A stage
/// that thresholded a raw cosine instead would be using a number that means
/// something different for every model, gallery and face size (AR-024).
///
/// **No prior term.** `log_prior_odds` adjusts for the gallery's base rate, which
/// is a question about *which of N actors*; association asks whether two faces
/// are one person, where the balanced fit is the right answer. Passing the
/// matcher's prior here would silently bias tracking by the size of the cast.
inline std::function<float(float)> same_person_probability(const GalleryCalibration& cal) {
if (!cal.valid) {
// Loud, because the failure mode is invisible: an untuned sigmoid still
// returns plausible probabilities, and every threshold downstream of it
// is then a guess wearing a calibrated number's clothes.
std::cerr << "[calibration] WARNING: no fitted calibration — association and "
"evidence weighting fall back to the untuned default sigmoid "
"(a=" << cal.a << ", b=" << cal.b << "). Probabilities are "
"not meaningful for this embedder.\n";
}
return [cal](float similarity) { return cal.probability(similarity); };
}
// Fit a logistic sigmoid to gallery pair similarities. // Fit a logistic sigmoid to gallery pair similarities.
// Positive pairs: same actor, different reference images. // Positive pairs: same actor, different reference images.
// Negative pairs: different actors (all cross-actor embedding pairs). // Negative pairs: different actors (all cross-actor embedding pairs).
+16 -5
View File
@@ -128,10 +128,8 @@ static Config parse_args(int argc, char** argv) {
else if (arg("--ratio-ceil")) cfg.match_ratio_ceil = std::stof(next()); else if (arg("--ratio-ceil")) cfg.match_ratio_ceil = std::stof(next());
else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next()); else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next());
else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next()); else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next());
else if (arg("--track-max-embed")) cfg.track_max_embed_dist = std::stof(next()); else if (arg("--track-min-prob")) cfg.track_assoc_min_prob = std::stof(next());
else if (arg("--track-max-missing")) cfg.track_max_frames_missing = std::stoi(next()); else if (arg("--track-extinction")) cfg.track_extinction_sec = std::stod(next());
else if (arg("--cut-revive-sim")) cfg.cut_revive_sim = std::stof(next());
else if (arg("--cut-inactive-max")) cfg.cut_inactive_max_frames = std::stoi(next());
else if (arg("--anneal")) cfg.anneal_sec = std::stod(next()); else if (arg("--anneal")) cfg.anneal_sec = std::stod(next());
else if (arg("--expand-gallery")) cfg.expand_gallery = true; else if (arg("--expand-gallery")) cfg.expand_gallery = true;
else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next()); else if (arg("--expand-buffer")) cfg.expand_buffer_size = std::stoi(next());
@@ -194,8 +192,21 @@ int main(int argc, char** argv) {
FaceDetectorFunc detector_fn{cfg}; FaceDetectorFunc detector_fn{cfg};
FaceAlignerFunc aligner_fn; FaceAlignerFunc aligner_fn;
EmbedderFunc embedder_fn{cfg}; EmbedderFunc embedder_fn{cfg};
FaceTrackerFunc ftracker_fn{cfg}; // Constructed before the tracker: it fits (or loads) the calibration, and
// the tracker must decide in that same probability space (AR-024).
IdentityMatcherFunc matcher_fn {gallery, cfg}; IdentityMatcherFunc matcher_fn {gallery, cfg};
/// TRACES: AR-007, AR-008, AR-012, AR-024 | SR-002
// The registry is created here and shared, not owned by a node: track state
// is not a stage in the stream, it is state several stages read and write,
// and its final answer is only known when a track dies.
auto same_person = same_person_probability(matcher_fn.calibration());
TrackRegistry::Config reg_cfg;
reg_cfg.extinction_sec = cfg.track_extinction_sec;
auto registry = std::make_shared<TrackRegistry>(
reg_cfg, EvidenceDiscounter(same_person));
FaceTrackerFunc ftracker_fn{cfg, registry, same_person};
SceneTrackerFunc tracker_fn {cfg}; SceneTrackerFunc tracker_fn {cfg};
ResultSinkFunc sink_fn {cfg, done}; ResultSinkFunc sink_fn {cfg, done};
#ifdef SAE_DEBUG #ifdef SAE_DEBUG
+169 -162
View File
@@ -1,102 +1,124 @@
#pragma once #pragma once
/// TRACES: AR-007, AR-008, AR-024 | SR-002
///
/// FaceTrackerFunc — KPN node that links face detections into tracks.
///
/// **The registry is the tracker's state.** The node owns no track map of its
/// own: it drives `TrackRegistry` through a `FrameScope` and reads the same
/// `Track` objects everything else reads. Two parallel copies could disagree,
/// and every divergence would surface as a wrong presence window rather than as
/// a crash — silently, and only in the output.
///
/// **One candidate pool** (AR-008). `last_seen` alone distinguishes a track that
/// is on screen from one that is dormant, and it only affects whether IoU means
/// anything. There is no parked pool and no revival branch: re-associating a
/// track whose face was lost — across a cut or not — is ordinary inter-frame
/// association, and it falls out of the embedding comparison already being done.
///
/// Assignment cost (track i, detection j):
///
/// p = P(same person | cosine(track mean, detection)) ← calibrated
/// alpha = base weight, or 0 when position carries no information
/// cost = alpha·(1 IoU) + (1 alpha)·(1 p)
///
/// gated to INF unless the pair is admissible on position *or* on identity.
///
/// **alpha is frame- and track-dependent** (AR-007). It falls to 0 —
/// embedding only — when either:
/// - the frame is flagged `is_cut` / `is_scene_boundary`: the viewpoint
/// changed, so the same person is at a new position; or
/// - the track is dormant (`last_seen` set): time has passed since its box was
/// last observed, so that box is stale regardless of cuts.
/// Both are the same statement — spatial continuity is broken — arrived at from
/// two directions, which is why they collapse into one rule rather than two
/// branches.
///
/// **Everything is thresholded in probability space** (AR-024). The cosine goes
/// through the calibration before it is compared to anything; the raw-cosine
/// constants `track_max_embed_dist` and `cut_revive_sim` are retired.
#include "types.hpp" #include "types.hpp"
#include "config.hpp" #include "config.hpp"
#include "track_registry.hpp"
#include <algorithm> #include <algorithm>
#include <cmath> #include <cmath>
#include <functional>
#include <iostream> #include <iostream>
#include <limits> #include <limits>
#include <map> #include <map>
#include <memory>
#include <stdexcept>
#include <string_view>
#include <vector> #include <vector>
// ── FaceTrackerFunc ───────────────────────────────────────────────────────────
// KPN node: links face detections across consecutive frames using the Hungarian
// algorithm on a combined spatial (IoU) + embedding (cosine distance) cost.
//
// Each track accumulates a running directional mean of its ArcFace embeddings
// (averaged then re-normalised to the unit sphere), used as the embedding side
// of the assignment cost below for more stable track continuity.
//
// Assignment cost (track i, detection j):
// cost = alpha * (1 - IoU) + (1-alpha) * min(cosine_dist/2, 1)
// Gated to INF when IoU < min_iou AND cosine_dist > max_embed_dist.
//
// Unmatched tracks have their frames_missing counter incremented; they are
// expired once frames_missing > max_frames_missing.
//
// Cross-cut re-association. A camera-angle change (Frame::is_cut, set by
// camera_position_change_detector) destroys spatial (IoU) continuity — the same
// person reappears at a new position — but not identity. On a cut the tracker
// does NOT discard its tracks; it parks them in an inactive pool keyed by their
// last-frame raw embedding. A post-cut detection whose raw cosine similarity to
// a parked track's last-frame embedding is ≥ cut_revive_sim revives that track:
// the original track_id, mean embedding and n_frames are restored (only the bbox
// jumps to the new detection), so identity continuity survives the cut. Parked
// tracks left unrevived for cut_inactive_max_frames are finally dropped.
struct FaceTrackerFunc { struct FaceTrackerFunc {
static constexpr std::string_view label() { return "face_tracker"; } static constexpr std::string_view label() { return "face_tracker"; }
struct TrackState { /// cosine similarity → P(same person). Supplied by the caller so the fit
cv::Rect2f bbox; /// belonging to the active embedder is used (AR-023/AR-024) — the same
Embedding mean_emb{}; /// pattern, and normally the same function object, as
Embedding last_emb{}; // raw embedding of the most recent matched frame /// `EvidenceDiscounter::Calibrate`.
int n_frames{0}; using Calibrate = std::function<float(float)>;
int frames_missing{0};
};
explicit FaceTrackerFunc(const Config& cfg) /// The registry is a constructor argument, not an option: a tracker without
: alpha_(cfg.track_alpha) /// one would have to keep its own tracks, which is the defect this replaces.
FaceTrackerFunc(const Config& cfg,
std::shared_ptr<TrackRegistry> registry,
Calibrate calibrate)
: registry_(std::move(registry))
, calibrate_(std::move(calibrate))
, alpha_base_(cfg.track_alpha)
, min_iou_(cfg.track_min_iou) , min_iou_(cfg.track_min_iou)
, max_embed_dist_(cfg.track_max_embed_dist) , min_assoc_prob_(cfg.track_assoc_min_prob)
, max_missing_(cfg.track_max_frames_missing)
, revive_sim_(cfg.cut_revive_sim)
, inactive_max_(cfg.cut_inactive_max_frames)
{ {
std::cerr << "[face_tracker] alpha=" << alpha_ if (!registry_)
throw std::invalid_argument("face_tracker: registry must not be null");
if (!calibrate_)
throw std::invalid_argument("face_tracker: a calibration is required — "
"association is decided in probability space");
std::cerr << "[face_tracker] alpha_base=" << alpha_base_
<< " min_iou=" << min_iou_ << " min_iou=" << min_iou_
<< " max_embed_dist=" << max_embed_dist_ << " min_assoc_prob=" << min_assoc_prob_ << "\n";
<< " max_missing=" << max_missing_
<< " cut_revive_sim=" << revive_sim_
<< " cut_inactive_max=" << inactive_max_ << "\n";
} }
TrackedSceneFrame operator()(EmbeddedSceneFrame ef) { TrackedSceneFrame operator()(EmbeddedSceneFrame ef) {
if (ef.source.eof) { if (ef.source.eof) {
tracks_.clear(); // Deliberately does *not* flush the registry. The identity matcher
inactive_.clear(); // runs downstream and its votes for the final frames are still in
// flight; reaping here would drop them (they would land on ids that
// no longer exist and show up as dropped_votes). AR-016's flush
// belongs at the pipeline's termination point, after the last vote.
boxes_.clear();
TrackedSceneFrame out; TrackedSceneFrame out;
out.source = std::move(ef.source); out.source = std::move(ef.source);
return out; return out;
} }
const int n_det = static_cast<int>(ef.embeddings.size()); const double t = ef.source.timestamp_sec;
const int n_det = static_cast<int>(ef.embeddings.size());
// Camera-angle change: park active tracks instead of destroying them so // Unconditional: the clock must advance on frames with no detections
// they can be revived by identity (raw last-frame embedding cosine) once // too, or a track only dies when some unrelated face happens to appear
// the same people reappear from the new angle. // and a film that ends mid-track never closes it (AR-013).
if (ef.source.is_cut && !tracks_.empty()) { auto scope = registry_->begin_frame(t);
std::cerr << "[face_tracker] cut — parking " << tracks_.size()
<< " track(s) into inactive pool\n";
for (auto& [tid, ts] : tracks_) {
ts.frames_missing = 0; // repurpose as time-since-parked counter
inactive_[tid] = std::move(ts);
}
tracks_.clear();
}
// Age the inactive pool every frame and drop tracks parked too long. // One pool (AR-008) — on-screen and dormant tracks compete together.
for (auto it = inactive_.begin(); it != inactive_.end(); ) { std::vector<Track*> cands = scope.candidates();
it->second.frames_missing++; const int n_trk = static_cast<int>(cands.size());
it = (it->second.frames_missing > inactive_max_)
? inactive_.erase(it) : std::next(it);
}
// Snapshot active track IDs so the map can be modified safely below prune_boxes(cands);
std::vector<int> tids; std::vector<Spatial*> sp(n_trk);
tids.reserve(tracks_.size()); for (int ti = 0; ti < n_trk; ++ti)
for (auto& [tid, _] : tracks_) tids.push_back(tid); sp[ti] = &boxes_.try_emplace(cands[ti]->id, Spatial{{}, t, false})
const int n_trk = static_cast<int>(tids.size()); .first->second;
// AR-007 — the frame half of the frame-dependent weighting. Both flags
// say the same thing to the tracker: whatever was at that position is
// not there any more.
const bool viewpoint_change =
ef.source.is_cut || ef.source.is_scene_boundary;
// ── Cost matrix [n_trk × n_det] ────────────────────────────────────── // ── Cost matrix [n_trk × n_det] ──────────────────────────────────────
constexpr float INF_COST = 1e6f; constexpr float INF_COST = 1e6f;
@@ -104,109 +126,108 @@ struct FaceTrackerFunc {
std::vector<float>(n_det, INF_COST)); std::vector<float>(n_det, INF_COST));
for (int ti = 0; ti < n_trk; ++ti) { for (int ti = 0; ti < n_trk; ++ti) {
const TrackState& ts = tracks_[tids[ti]]; // Spatial continuity holds only for a track that was on screen, whose
// box we have actually observed, on a frame that did not change the
// viewpoint. Otherwise the box is stale and IoU is noise.
const bool spatial_meaningful =
sp[ti]->observed && cands[ti]->on_screen() && !viewpoint_change;
const float alpha = spatial_meaningful ? alpha_base_ : 0.f;
for (int di = 0; di < n_det; ++di) { for (int di = 0; di < n_det; ++di) {
float iou_v = iou(ts.bbox, ef.faces[di].bbox); // AR-024 — the cosine is converted before it is used for
float emb_d = (ts.n_frames > 0) // anything, including the gate below.
? 1.f - cosine_similarity(ts.mean_emb, ef.embeddings[di]) const float p = calibrate_(
: 1.f; cosine_similarity(cands[ti]->mean, ef.embeddings[di]));
if (iou_v < min_iou_ && emb_d > max_embed_dist_) continue; const float iou_v = spatial_meaningful
float s = 1.f - iou_v; ? iou(sp[ti]->bbox, ef.faces[di].bbox) : 0.f;
float e = std::min(emb_d * 0.5f, 1.f);
cost[ti][di] = alpha_ * s + (1.f - alpha_) * e; // Either signal on its own can admit a link: a face that moved a
// little but whose embedding degraded (blur, profile turn) is
// still linkable on position, and a face that jumped across the
// frame is still linkable on identity. Neither ⇒ no link.
const bool spatial_ok = spatial_meaningful && iou_v >= min_iou_;
const bool identity_ok = p >= min_assoc_prob_;
if (!spatial_ok && !identity_ok) continue;
cost[ti][di] = alpha * (1.f - iou_v) + (1.f - alpha) * (1.f - p);
} }
} }
// ── Hungarian assignment ───────────────────────────────────────────── // ── Hungarian assignment ─────────────────────────────────────────────
std::vector<int> assign(n_trk, -1); std::vector<int> assign(n_trk, -1);
if (n_trk > 0 && n_det > 0) if (n_trk > 0 && n_det > 0)
assign = hungarian(cost, n_trk, n_det); assign = hungarian(cost, n_trk, n_det);
// ── Build output frame ─────────────────────────────────────────────── // ── Build output frame ───────────────────────────────────────────────
TrackedSceneFrame out; TrackedSceneFrame out;
out.source = ef.source; out.source = ef.source;
out.faces = ef.faces; out.faces = ef.faces;
out.crops = ef.crops; out.crops = ef.crops;
out.embeddings = ef.embeddings; out.embeddings = ef.embeddings;
out.track_ids.assign(n_det, -1); out.track_ids.assign(n_det, -1);
std::vector<bool> det_matched(n_det, false); std::vector<bool> det_matched(n_det, false);
// Update matched tracks
for (int ti = 0; ti < n_trk; ++ti) { for (int ti = 0; ti < n_trk; ++ti) {
int di = assign[ti]; const int di = assign[ti];
bool valid = (di >= 0 && di < n_det && cost[ti][di] < INF_COST * 0.5f); const int id = cands[ti]->id;
TrackState& ts = tracks_[tids[ti]]; const bool valid =
(di >= 0 && di < n_det && cost[ti][di] < INF_COST * 0.5f);
if (!valid) { if (!valid) {
ts.frames_missing++; // Only a track that *was* on screen can become lost, and it
// becomes lost as of its last sighting, never as of now — the
// gap after the final sighting is never claimed (AR-013). A
// track already dormant is left alone so its extinction clock
// keeps running from the right instant.
if (cands[ti]->on_screen()) scope.mark_lost(id, sp[ti]->last_ts);
continue; continue;
} }
update_mean(ts.mean_emb, ts.n_frames, ef.embeddings[di]);
ts.last_emb = ef.embeddings[di];
ts.bbox = ef.faces[di].bbox;
ts.n_frames++;
ts.frames_missing = 0;
det_matched[di] = true;
out.track_ids[di] = tids[ti]; scope.mark_seen(id, t, ef.embeddings[di]);
sp[ti]->bbox = ef.faces[di].bbox;
sp[ti]->last_ts = t;
sp[ti]->observed = true;
det_matched[di] = true;
out.track_ids[di] = id;
} }
// Handle unmatched detections: first try to revive a parked track by
// identity (raw last-frame embedding cosine), else start a fresh track.
for (int di = 0; di < n_det; ++di) { for (int di = 0; di < n_det; ++di) {
if (det_matched[di]) continue; if (det_matched[di]) continue;
const int id = scope.create(t, ef.embeddings[di]);
int tid = revive_from_inactive(ef.embeddings[di]); boxes_[id] = Spatial{ef.faces[di].bbox, t, true};
if (tid >= 0) { out.track_ids[di] = id;
// Restore the parked track: keep its identity statistics
// (mean_emb, n_frames), jump the bbox to the new detection.
TrackState ts = std::move(inactive_[tid]);
inactive_.erase(tid);
update_mean(ts.mean_emb, ts.n_frames, ef.embeddings[di]);
ts.last_emb = ef.embeddings[di];
ts.bbox = ef.faces[di].bbox;
ts.n_frames++;
ts.frames_missing = 0;
tracks_[tid] = std::move(ts);
out.track_ids[di] = tid;
std::cerr << "[face_tracker] revived track " << tid
<< " across cut\n";
continue;
}
tid = next_id_++;
TrackState ts;
ts.bbox = ef.faces[di].bbox;
ts.mean_emb = ef.embeddings[di];
ts.last_emb = ef.embeddings[di];
ts.n_frames = 1;
tracks_[tid] = ts;
out.track_ids[di] = tid;
}
// Expire stale tracks
for (auto it = tracks_.begin(); it != tracks_.end(); ) {
it = (it->second.frames_missing > max_missing_)
? tracks_.erase(it) : std::next(it);
} }
// No reaping here: begin_frame's tick owns the extinction sweep, so
// there is exactly one place a track can die.
return out; return out;
} }
private: private:
// Pick the parked track whose last-frame embedding is most similar to emb, // ── Spatial annotation ───────────────────────────────────────────────────
// returning its id if that raw cosine similarity clears revive_sim_, else -1. // The one piece of per-track state the registry does not hold, because it is
// The caller removes the returned track from the pool, so a later detection in // not about presence: where the face was, and when it was last seen there.
// the same frame cannot claim it again. // Keyed by registry track id and pruned against `candidates()` every frame,
int revive_from_inactive(const Embedding& emb) const { // so it cannot outlive or contradict the registry — it annotates the pool
int best_tid = -1; // rather than duplicating it.
float best_sim = revive_sim_; // threshold is the bar to beat (inclusive) struct Spatial {
for (const auto& [tid, ts] : inactive_) { cv::Rect2f bbox{};
float sim = cosine_similarity(ts.last_emb, emb); double last_ts{0.0}; ///< timestamp of the last frame this track matched
if (sim >= best_sim) { best_sim = sim; best_tid = tid; } bool observed{false}; ///< false until a detection has been assigned
// subsequent ties keep the later id; harmless, all clear the threshold };
// Drop boxes for ids the registry no longer has. `candidates()` is the
// authority on what exists; anything else is a leak (and, for a reused id,
// would be a stale box attached to a different person).
void prune_boxes(const std::vector<Track*>& cands) {
if (boxes_.size() == cands.size()) return; // common case: nothing died
std::map<int, Spatial> kept;
for (const Track* t : cands) {
auto it = boxes_.find(t->id);
if (it != boxes_.end()) kept.emplace(t->id, it->second);
} }
return best_tid; boxes_.swap(kept);
} }
// IoU of two axis-aligned bounding boxes // IoU of two axis-aligned bounding boxes
@@ -220,17 +241,6 @@ private:
return inter / (a.width * a.height + b.width * b.height - inter); return inter / (a.width * a.height + b.width * b.height - inter);
} }
// Online directional mean: average then re-normalise to unit sphere
static void update_mean(Embedding& mean, int n_prev, const Embedding& emb) {
float norm_sq = 0.f;
for (int k = 0; k < 512; ++k) {
mean[k] = (mean[k] * n_prev + emb[k]) / (n_prev + 1);
norm_sq += mean[k] * mean[k];
}
float inv = 1.f / std::sqrt(norm_sq);
for (int k = 0; k < 512; ++k) mean[k] *= inv;
}
// O(n³) potential-based Hungarian algorithm (Jonker-Volgenant / Kuhn-Munkres). // O(n³) potential-based Hungarian algorithm (Jonker-Volgenant / Kuhn-Munkres).
// Returns assign[row] = col (0-indexed), or -1 when row is matched to a // Returns assign[row] = col (0-indexed), or -1 when row is matched to a
// padded virtual column (i.e., unmatched). Rectangular matrices are padded // padded virtual column (i.e., unmatched). Rectangular matrices are padded
@@ -292,13 +302,10 @@ private:
return ans; return ans;
} }
std::map<int, TrackState> tracks_; std::shared_ptr<TrackRegistry> registry_;
std::map<int, TrackState> inactive_; // parked across a cut, keyed by track id Calibrate calibrate_;
int next_id_{0}; std::map<int, Spatial> boxes_; ///< track id → where it was, when
float alpha_; float alpha_base_;
float min_iou_; float min_iou_;
float max_embed_dist_; float min_assoc_prob_;
int max_missing_;
float revive_sim_;
int inactive_max_;
}; };
+8
View File
@@ -107,6 +107,14 @@ struct IdentityMatcherFunc {
sim_engine_ = make_similarity_engine(host_gallery.data(), n_gallery_, kMaxFaces); sim_engine_ = make_similarity_engine(host_gallery.data(), n_gallery_, kMaxFaces);
} }
/// TRACES: AR-023, AR-024 | SR-002
/// The fitted sigmoid. Exposed because the matcher is where it gets fitted
/// (and cached back to the gallery), but it is not the matcher's private
/// property: track association and evidence weighting must threshold in the
/// *same* probability space, or a "0.5" in one stage and a "0.5" in another
/// mean different things. See `same_person_probability`.
const GalleryCalibration& calibration() const { return cal_; }
// Runtime setter — lets a persistent pipeline be reused across a threshold sweep // Runtime setter — lets a persistent pipeline be reused across a threshold sweep
// without rebuilding the (expensive, gallery-resident) matcher. The gallery, // without rebuilding the (expensive, gallery-resident) matcher. The gallery,
// calibration and GPU sim-engine stay put; only the accept threshold changes. // calibration and GPU sim-engine stay put; only the accept threshold changes.
+117 -53
View File
@@ -13,8 +13,12 @@
#include "config.hpp" #include "config.hpp"
#include "nodes/face_tracker_node.hpp" #include "nodes/face_tracker_node.hpp"
#include "types.hpp" #include "types.hpp"
#include "track_registry.hpp"
#include "evidence_discount.hpp"
#include <algorithm>
#include <cmath> #include <cmath>
#include <memory>
namespace { namespace {
@@ -55,85 +59,145 @@ EmbeddedSceneFrame frame(double t, float x, float y, const Embedding& emb,
return ef; return ef;
} }
Config tracker_cfg() { // Build a tracker over a fresh registry. The registry IS the tracker's state
Config cfg; // now (AR-008), so a test constructs both together and can inspect either.
cfg.cut_revive_sim = 0.50f; struct Rig {
cfg.cut_inactive_max_frames = 5; std::shared_ptr<TrackRegistry> reg;
return cfg; FaceTrackerFunc ft;
}
explicit Rig(double extinction = 30.0, float assoc_min_prob = 0.5f)
: reg(std::make_shared<TrackRegistry>(
[extinction] {
TrackRegistry::Config c;
c.extinction_sec = extinction;
return c;
}(),
EvidenceDiscounter([](float cos) { return std::max(0.f, cos); })))
, ft([&] {
Config c;
c.track_assoc_min_prob = assoc_min_prob;
return c;
}(),
reg,
// Trivial calibration: cosine passed through as P(same). Real runs use
// the fit belonging to the active embedder (AR-023/AR-024).
[](float cos) { return std::max(0.f, cos); })
{}
int track_of(EmbeddedSceneFrame f) { return ft(std::move(f)).track_ids[0]; }
};
} // namespace } // namespace
TEST_CASE("track id is stable across ordinary frames", "[face_tracker]") { // ── AR-008 — one pool, ordinary association ──────────────────────────────────
FaceTrackerFunc ft(tracker_cfg()); TEST_CASE("track id is stable across ordinary frames", "[face_tracker][AR-008]") {
Rig r;
Embedding e = axis(0); Embedding e = axis(0);
int id0 = ft(frame(0.0, 10, 10, e)).track_ids[0]; int id0 = r.track_of(frame(0.0, 10, 10, e));
int id1 = ft(frame(1.0, 11, 10, e)).track_ids[0]; // overlaps → same track int id1 = r.track_of(frame(1.0, 11, 10, e)); // overlaps → same track
CHECK(id0 >= 0); CHECK(id0 >= 0);
CHECK(id1 == id0); CHECK(id1 == id0);
} }
TEST_CASE("cut revives the same track id for a matching identity", "[face_tracker]") { TEST_CASE("a face lost across a cut and re-associated is the SAME track",
FaceTrackerFunc ft(tracker_cfg()); "[face_tracker][AR-008]") {
// Previously this was a distinct "revival" path guarded by a raw-cosine
// constant. There is no such path now: a dormant track is an ordinary
// association candidate, and continuity falls out of the embedding match.
Rig r;
// Pre-cut: establish a track for a person whose embedding is near-identical
// across the cut (sim well above cut_revive_sim), but whose box jumps so IoU
// is 0 — the ordinary spatial path cannot re-link it.
Embedding pre = at_sim(0, 1, 0.99f); Embedding pre = at_sim(0, 1, 0.99f);
int id_pre = ft(frame(0.0, 10, 10, pre)).track_ids[0]; int id_pre = r.track_of(frame(0.0, 10, 10, pre));
REQUIRE(id_pre >= 0); REQUIRE(id_pre >= 0);
Embedding post = at_sim(0, 1, 0.98f); // cos(diff) ≈ 0.9997 > 0.50 // Box jumps so IoU is zero — only the embedding can link it.
auto out = ft(frame(1.0, 300, 300, post, /*is_cut=*/true)); Embedding post = at_sim(0, 1, 0.98f);
CHECK(out.track_ids[0] == id_pre); // revived, not a fresh id CHECK(r.track_of(frame(1.0, 300, 300, post, /*is_cut=*/true)) == id_pre);
} }
TEST_CASE("cut starts a fresh track when identity does not match", "[face_tracker]") { TEST_CASE("a cut starts a fresh track when identity does not match",
FaceTrackerFunc ft(tracker_cfg()); "[face_tracker][AR-008]") {
Rig r;
int id_pre = ft(frame(0.0, 10, 10, axis(0))).track_ids[0]; int id_pre = r.track_of(frame(0.0, 10, 10, axis(0)));
REQUIRE(id_pre >= 0); REQUIRE(id_pre >= 0);
// Post-cut face is orthogonal (sim 0 < cut_revive_sim) and spatially disjoint // Orthogonal embedding and disjoint box: nothing links them.
// → no revival, brand-new id. int id_post = r.track_of(frame(1.0, 300, 300, axis(5), /*is_cut=*/true));
auto out = ft(frame(1.0, 300, 300, axis(5), /*is_cut=*/true)); CHECK(id_post != id_pre);
CHECK(out.track_ids[0] != id_pre); CHECK(id_post >= 0);
CHECK(out.track_ids[0] >= 0);
} }
TEST_CASE("parked track expires after cut_inactive_max_frames", "[face_tracker]") { // ── AR-007 — a cut makes association ignore position ─────────────────────────
Config cfg = tracker_cfg(); TEST_CASE("on a cut, identity follows the embedding rather than the box",
cfg.cut_inactive_max_frames = 2; "[face_tracker][AR-007]") {
FaceTrackerFunc ft(cfg); // Two people swap screen positions across a cut while keeping their faces.
// If IoU still carried weight the ids would follow the boxes and swap; with
// alpha driven to embedding-only on a cut, they must follow the faces.
Rig r;
Embedding a = at_sim(0, 1, 0.99f);
Embedding b = at_sim(2, 3, 0.99f);
EmbeddedSceneFrame f0;
f0.source.timestamp_sec = 0.0;
f0.faces = {face_at(10, 10), face_at(300, 300)};
f0.crops = {cv::Mat(), cv::Mat()};
f0.embeddings = {a, b};
auto out0 = r.ft(std::move(f0));
const int id_a = out0.track_ids[0];
const int id_b = out0.track_ids[1];
REQUIRE(id_a >= 0);
REQUIRE(id_b >= 0);
REQUIRE(id_a != id_b);
// Same two people, positions exchanged, on a cut frame.
EmbeddedSceneFrame f1;
f1.source.timestamp_sec = 1.0;
f1.source.is_cut = true;
f1.faces = {face_at(300, 300), face_at(10, 10)};
f1.crops = {cv::Mat(), cv::Mat()};
f1.embeddings = {a, b};
auto out1 = r.ft(std::move(f1));
CHECK(out1.track_ids[0] == id_a); // A kept its id despite moving to B's box
CHECK(out1.track_ids[1] == id_b);
}
// ── AR-013 — extinction replaces the parked-pool frame counter ───────────────
TEST_CASE("a track past the extinction window is gone, not revived",
"[face_tracker][AR-013]") {
// The old design aged a parked pool in frames, which silently changed
// meaning with sample_fps. Extinction is in seconds and lives in the
// registry, so the tracker no longer counts anything.
Rig r(/*extinction=*/2.0);
Embedding person = at_sim(0, 1, 0.99f); Embedding person = at_sim(0, 1, 0.99f);
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0]; int id_pre = r.track_of(frame(0.0, 10, 10, person));
REQUIRE(id_pre >= 0); REQUIRE(id_pre >= 0);
// Cut with an unrelated face parks id_pre; then let the pool age past its // Unrelated faces elsewhere while the clock runs well past extinction.
// limit with more unrelated, spatially-disjoint faces (each ages the pool by r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
// one). By the time the person returns, id_pre must be gone. r.track_of(frame(10.0, 300, 300, axis(7)));
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park (age 1)
ft(frame(2.0, 300, 300, axis(7))); // age 2
ft(frame(3.0, 300, 300, axis(7))); // age 3 → id_pre dropped
auto out = ft(frame(4.0, 10, 10, person)); // same identity returns CHECK(r.track_of(frame(11.0, 10, 10, person)) != id_pre);
CHECK(out.track_ids[0] != id_pre); // too late — fresh id
} }
TEST_CASE("eof clears active and parked tracks", "[face_tracker]") { TEST_CASE("a track within the extinction window is still a candidate",
FaceTrackerFunc ft(tracker_cfg()); "[face_tracker][AR-013]") {
Embedding person = at_sim(0, 1, 0.99f); Rig r(/*extinction=*/30.0);
int id_pre = ft(frame(0.0, 10, 10, person)).track_ids[0];
ft(frame(1.0, 300, 300, axis(7), /*is_cut=*/true)); // park id_pre
Embedding person = at_sim(0, 1, 0.99f);
int id_pre = r.track_of(frame(0.0, 10, 10, person));
r.track_of(frame(1.0, 300, 300, axis(7), /*is_cut=*/true));
// Back inside the window: the same person continues the same track, so the
// gap is absorbed into one window rather than splitting it.
CHECK(r.track_of(frame(3.0, 10, 10, person)) == id_pre);
}
TEST_CASE("eof is forwarded", "[face_tracker]") {
Rig r;
EmbeddedSceneFrame eof; EmbeddedSceneFrame eof;
eof.source.eof = true; eof.source.eof = true;
auto out = ft(std::move(eof)); CHECK(r.ft(std::move(eof)).source.eof);
CHECK(out.source.eof);
// After eof the pools are empty: the returning identity must get a fresh id,
// not the parked one.
auto out2 = ft(frame(2.0, 10, 10, person));
CHECK(out2.track_ids[0] != id_pre);
} }