fix(AR-013): reap tracks on the evidence watermark, not the tracker's clock
The registry closed a track when the *tracker's* timestamp passed `track_extinction_sec`. But votes arrive from the matcher, which is a separate KPN node behind a channel, and much the slower of the pair. Backpressure — working exactly as AR-004 intends — turns that channel's depth into lag, so the tracker's clock can be far ahead of the last frame anybody has voted on. Tracks were therefore closed before their evidence arrived: the votes landed on ids that no longer existed, were counted as dropped, and the track was emitted unowned or not at all. The symptom is the part worth remembering: **a deeper channel produced fewer identifications, from identical input.** On the SuperHero fixture, 5 actors / 16 windows at depth 32 against 3 actors / 5 windows at depth 10322; through the replay harness, capacity 32 gave 5 actors and 10322 gave 0. A throughput knob was silently changing the answer, which makes every sweep tuned against it suspect. The fix is not to bound the channel against `track_extinction_sec` — that makes an algorithm constant police a throughput knob and leaves the result a function of scheduling. It is to reap on an evidence watermark: the matcher advances it as it folds each frame in, and a track is only finished once everything up to its extinction point has actually been voted on. Same device `SceneBoundaries::scored_through()` uses for the AR-010 join — a consumer past that point is asking about frames nobody has looked at yet, and the honest answer is to wait rather than guess. Association keeps the tracker's clock, and separating the two is the other half. They answer different questions: "may this detection link to that track?" is asked now, about a box seen `track_extinction_sec` ago; "is that track finished?" cannot be answered until every vote is in. Deferring association to the evidence clock — which deferring the erase alone did — left retired tracks associable for as long as the matcher lagged, so a new face re-associated onto a long-dead track and two people merged into one window. The watermark is monotonic and only ever *delays* a reap, so no window is extended by it: AR-013's "a window ends at the last sighting, never after" is a property of `emit_locked`, which takes `last_seen` and never `now`. `dropped_votes` is exposed and reported — by main at shutdown and through the replay bindings — because this failed silently for as long as it did precisely because nothing counted it. It warns rather than aborts: a dropped frame means the output describes footage nobody analysed and is always wrong, while a dropped vote degrades a claim without falsifying it, and there is no measurement yet of how often it happens on real content. replay.py's channel capacity stops being the whole film. It was sized that way to dodge a PyNode overflow drop that AR-004 has since replaced with parking, and removing backpressure that way is what made the defect above so extreme. Tag separators in kpn_bindings.cpp corrected to pipes between requirement types, which the traceability gate was reporting as diagnostics; the matrix is regenerated and reports 0 orphan tags. 149/149. TRACES: AR-004, AR-012, AR-013, AR-025 | VR-011 | SR-002 | PR-002
This commit is contained in:
+28
-1
@@ -258,7 +258,7 @@ static Config config_from_dict(nb::dict d) {
|
||||
if (d.contains("require_gallery_stamp"))
|
||||
cfg.require_gallery_stamp = nb::cast<bool>(d["require_gallery_stamp"]);
|
||||
|
||||
/// TRACES: VR-011, IR-001 | PR-002, SR-003
|
||||
/// TRACES: VR-011 | IR-001 | PR-002 | SR-003
|
||||
// The sink is a real node in this network now, so it needs the two things
|
||||
// that decide what it writes and where. Both used to be irrelevant here
|
||||
// because the replay never had a sink -- Python rebuilt presence instead,
|
||||
@@ -429,6 +429,33 @@ NB_MODULE(sae_kpn, m) {
|
||||
/// replay, which a long sweep will notice.
|
||||
m.def("release_pipeline", [](Net& net) { sessions().erase(&net); }, "net"_a);
|
||||
|
||||
/// TRACES: VR-011 | AR-025 | PR-002
|
||||
/// The registry's own count of how often it was wrong, exposed so a replay
|
||||
/// can fail on it instead of returning a plausible-looking empty answer.
|
||||
///
|
||||
/// `dropped_votes` is the one that matters here and it earned its keep
|
||||
/// immediately. A vote lands on a track the registry has already reaped when
|
||||
/// the matcher lags the tracker by more than track_extinction_sec of film.
|
||||
/// In scene_analyze that cannot happen -- channels are 16-64 deep, so
|
||||
/// backpressure pins the two nodes within a few frames of each other. This
|
||||
/// harness sized every channel to the whole film to avoid a PyNode overflow
|
||||
/// drop, which removed the backpressure entirely: the tracker ran the film
|
||||
/// to the end while the matcher was still in its first minute, every vote
|
||||
/// arrived after its track was gone, no track was ever owned, and the run
|
||||
/// produced zero presence windows while cheerfully reporting 1647 frames
|
||||
/// with an identified face.
|
||||
m.def("pipeline_diagnostics", [](Net& net) {
|
||||
nb::dict d;
|
||||
auto it = sessions().find(&net);
|
||||
if (it == sessions().end() || !it->second->registry) return d;
|
||||
const auto& r = *it->second->registry;
|
||||
d["dropped_votes"] = r.dropped_votes();
|
||||
d["belief_swaps"] = r.belief_swaps();
|
||||
d["actor_conflicts"] = r.actor_conflicts();
|
||||
d["live_tracks"] = static_cast<int>(r.live());
|
||||
return d;
|
||||
}, "net"_a);
|
||||
|
||||
/// True once the sink has written its output. The sink flushes on the EOF
|
||||
/// annotation, so a caller that reads the file before this is racing it.
|
||||
m.def("pipeline_done", [](Net& net) {
|
||||
|
||||
@@ -516,6 +516,45 @@ int main(int argc, char** argv) {
|
||||
std::cerr << "\n";
|
||||
}
|
||||
|
||||
/// TRACES: AR-025, AR-012 | SR-002
|
||||
// How often the registry was asked about a track it had already reaped.
|
||||
//
|
||||
// A vote is dropped when the matcher lags the tracker by more than
|
||||
// track_extinction_sec of FILM time. The two are adjacent nodes with a
|
||||
// 16-deep channel between them, and the matcher is much the slower of
|
||||
// the pair (a GEMM over the whole gallery against a Hungarian solve over
|
||||
// a handful of boxes), so that channel runs full and the lag is close to
|
||||
// its depth. In frames:
|
||||
//
|
||||
// lag_sec ~= channel_depth / sample_fps
|
||||
//
|
||||
// At the default sample_fps of 1.0 that is ~16 s against a 5 s window,
|
||||
// so votes CAN be dropped here, and each one is identity evidence that
|
||||
// never reached the track it belonged to -- presence under-reported, in
|
||||
// a way that reads as a recognition miss.
|
||||
//
|
||||
// Reported rather than fatal, deliberately, and the distinction from the
|
||||
// dropped-frame case below is real: a dropped frame means the output
|
||||
// describes footage nobody analysed, which is always wrong. A dropped
|
||||
// vote means one observation of a track went missing, which degrades a
|
||||
// claim without falsifying it. There is also no measurement yet of how
|
||||
// often it happens on real content -- so this prints the number that
|
||||
// would justify a harder line rather than presuming it. See VR-017.
|
||||
if (registry) {
|
||||
const int dv = registry->dropped_votes();
|
||||
if (dv > 0) {
|
||||
std::cerr << "[registry] WARNING: " << dv << " identity vote(s) "
|
||||
"arrived for already-reaped tracks. The matcher is "
|
||||
"lagging the tracker by more than track_extinction_sec ("
|
||||
<< cfg.track_extinction_sec << "s) of film; presence is "
|
||||
"under-reported. Raise --track-extinction or reduce the "
|
||||
"face_tracker/identity_matcher channel depth.\n";
|
||||
}
|
||||
std::cerr << "[registry] belief_swaps=" << registry->belief_swaps()
|
||||
<< " actor_conflicts=" << registry->actor_conflicts()
|
||||
<< " dropped_votes=" << dv << "\n";
|
||||
}
|
||||
|
||||
bool dropped = false;
|
||||
{
|
||||
std::lock_guard<std::mutex> lk(event_mtx);
|
||||
|
||||
@@ -152,7 +152,12 @@ struct IdentityMatcherFunc {
|
||||
/// Where per-frame identity evidence reaches the registry. Optional: with no
|
||||
/// registry attached the matcher behaves exactly as before, which keeps the
|
||||
/// replay harness and the unit tests working unchanged.
|
||||
void set_registry(std::shared_ptr<TrackRegistry> r) { registry_ = std::move(r); }
|
||||
void set_registry(std::shared_ptr<TrackRegistry> r) {
|
||||
registry_ = std::move(r);
|
||||
// This node is the evidence source, so the registry must not close a
|
||||
// track until this node's watermark has passed it (AR-013).
|
||||
if (registry_) registry_->expect_evidence();
|
||||
}
|
||||
|
||||
// Runtime setter — lets a persistent pipeline be reused across a threshold sweep
|
||||
// without rebuilding the (expensive, gallery-resident) matcher. The gallery,
|
||||
@@ -165,6 +170,22 @@ struct IdentityMatcherFunc {
|
||||
return {std::move(tf.source), {}};
|
||||
}
|
||||
|
||||
/// TRACES: AR-012, AR-013 | SR-002
|
||||
// Publish the evidence watermark BEFORE voting on this frame: every
|
||||
// observation strictly before it has now been folded in, so the registry
|
||||
// may reap against it. Unconditional -- a frame with no faces still
|
||||
// advances the watermark, or a long faceless stretch would stall reaping
|
||||
// and hold every dormant track open to the end of the film.
|
||||
//
|
||||
// This is what makes presence independent of node speed. The registry
|
||||
// used to reap on the TRACKER's clock, and backpressure (working as
|
||||
// AR-004 intends) means the tracker can be a whole channel's depth ahead
|
||||
// of this node -- so tracks were closed before their votes arrived, the
|
||||
// votes were dropped, and the run silently under-reported. Measured on
|
||||
// the SuperHero fixture before this change: channel depth 32 gave 5
|
||||
// actors, depth 10322 gave 0, from identical input.
|
||||
if (registry_) registry_->advance_evidence(tf.source.timestamp_sec);
|
||||
|
||||
// A hard cut changes the camera viewpoint. The face_tracker may revive a
|
||||
// track_id across the cut (identity continuity), but promotion must never
|
||||
// mix embeddings from two viewpoints under one buffer, so we still drop
|
||||
|
||||
+87
-5
@@ -138,13 +138,37 @@ public:
|
||||
FrameScope(TrackRegistry& reg, double now)
|
||||
: reg_(reg), lock_(reg.mu_) { reg_.tick_locked(now); }
|
||||
|
||||
/// All live tracks — **one pool**. `last_seen` tells the caller whether
|
||||
/// IoU is meaningful; a dormant track is matched on embedding alone.
|
||||
/// There is no separate revival path (AR-008).
|
||||
/// All ASSOCIABLE tracks — **one pool**. `last_seen` tells the caller
|
||||
/// whether IoU is meaningful; a dormant track is matched on embedding
|
||||
/// alone. There is no separate revival path (AR-008).
|
||||
///
|
||||
/// TRACES: AR-008, AR-013 | SR-002
|
||||
/// Filtered on the TRACKER's clock, deliberately, while reaping runs on
|
||||
/// the matcher's evidence watermark. The two answer different questions
|
||||
/// and must not share an answer:
|
||||
///
|
||||
/// "may this detection link to that track?" — a tracking question,
|
||||
/// asked now, about a box observed `track_extinction_sec` ago.
|
||||
/// "is that track finished, so its claim can be emitted?" — a presence
|
||||
/// question, which cannot be answered until every vote is in.
|
||||
///
|
||||
/// Conflating them makes the result depend on node speed in one
|
||||
/// direction or the other. Reaping on the tracker's clock closed tracks
|
||||
/// before their votes arrived. Deferring association to the evidence
|
||||
/// clock — which is what deferring the erase alone did — left retired
|
||||
/// tracks in the pool for as long as the matcher lagged, so a new face
|
||||
/// re-associated onto a long-dead track and two people merged into one
|
||||
/// window. Measured: 5 actors / 16 windows at channel depth 32 against
|
||||
/// 3 actors / 5 windows at depth 10322, from identical input.
|
||||
std::vector<Track*> candidates() {
|
||||
std::vector<Track*> out;
|
||||
out.reserve(reg_.tracks_.size());
|
||||
for (auto& [id, t] : reg_.tracks_) out.push_back(&t);
|
||||
for (auto& [id, t] : reg_.tracks_) {
|
||||
if (t.last_seen &&
|
||||
(reg_.now_ - *t.last_seen) > reg_.cfg_.track_extinction_sec)
|
||||
continue; // retired from association; still awaiting evidence
|
||||
out.push_back(&t);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
@@ -164,6 +188,50 @@ public:
|
||||
/// happens to appear, and a film ending mid-track never closes.
|
||||
void tick(double now) { std::lock_guard g(mu_); tick_locked(now); }
|
||||
|
||||
/// TRACES: AR-012, AR-013, AR-025 | SR-002
|
||||
/// The evidence watermark: every observation up to `t` has been folded in.
|
||||
///
|
||||
/// Reaping is driven by THIS, not by the tracker's clock, and the difference
|
||||
/// is what stops a correct answer from depending on how fast two nodes run.
|
||||
///
|
||||
/// The tracker and the matcher are separate KPN nodes with a channel between
|
||||
/// them, and the matcher is much the slower of the pair. Backpressure —
|
||||
/// working exactly as AR-004 intends — turns that channel's depth into lag,
|
||||
/// so the tracker's timestamp can be far ahead of the last frame anybody has
|
||||
/// actually voted on. Reaping on the tracker's clock therefore closed tracks
|
||||
/// before their evidence arrived: the votes landed on ids that no longer
|
||||
/// existed, were counted as dropped, and the track was emitted unowned or
|
||||
/// not at all. Deeper channel, fewer identifications, from identical input.
|
||||
///
|
||||
/// The fix is not to bound the channel against `track_extinction_sec`. That
|
||||
/// makes an algorithm constant police a throughput knob, and leaves the
|
||||
/// answer a function of scheduling. It is to reap on the watermark, which is
|
||||
/// the same device `SceneBoundaries::scored_through()` uses for the AR-010
|
||||
/// join: a consumer past that point is asking about frames nobody has looked
|
||||
/// at yet, and the honest response is to wait rather than to guess.
|
||||
///
|
||||
/// Monotonic, and only ever *delays* a reap, so no window can be extended by
|
||||
/// it — AR-013's "a window ends at the last sighting, never after" is a
|
||||
/// property of `emit_locked`, which takes `last_seen` and never `now`.
|
||||
void advance_evidence(double t) {
|
||||
std::lock_guard g(mu_);
|
||||
if (t > evidence_through_) evidence_through_ = t;
|
||||
reap_locked();
|
||||
}
|
||||
|
||||
/// TRACES: AR-013, AR-025 | SR-002
|
||||
/// Declare that some stage will publish an evidence watermark, so reaping
|
||||
/// must wait for it.
|
||||
///
|
||||
/// Explicit rather than inferred from "has anyone voted yet". Inferring it
|
||||
/// re-opens the bug exactly at startup: before the matcher's first frame no
|
||||
/// vote has been seen, so the registry would fall back to the tracker's
|
||||
/// clock during precisely the window in which the tracker is furthest
|
||||
/// ahead. `IdentityMatcherFunc::set_registry` calls this, so any pipeline
|
||||
/// with a matcher waits, and a test that drives the tracker alone keeps the
|
||||
/// simple behaviour instead of hanging on a watermark nobody will publish.
|
||||
void expect_evidence() { std::lock_guard g(mu_); awaits_evidence_ = true; }
|
||||
|
||||
// ── Evidence ─────────────────────────────────────────────────────────────
|
||||
/// Fold one observation into a track's belief (AR-025).
|
||||
///
|
||||
@@ -259,9 +327,20 @@ public:
|
||||
private:
|
||||
// ── Locked internals ─────────────────────────────────────────────────────
|
||||
void tick_locked(double now) {
|
||||
// The tracker's clock still bounds association (a dormant track is only
|
||||
// a candidate while it is alive), but it no longer decides death.
|
||||
now_ = now;
|
||||
reap_locked();
|
||||
}
|
||||
|
||||
/// Reap against the evidence watermark when a producer of one is attached
|
||||
/// (see expect_evidence); otherwise against the tracker's clock, which is
|
||||
/// the same thing when there is only one clock.
|
||||
void reap_locked() {
|
||||
const double clock = awaits_evidence_ ? evidence_through_ : now_;
|
||||
for (auto it = tracks_.begin(); it != tracks_.end(); ) {
|
||||
const auto& ls = it->second.last_seen;
|
||||
if (ls && (now - *ls) > cfg_.track_extinction_sec) {
|
||||
if (ls && (clock - *ls) > cfg_.track_extinction_sec) {
|
||||
emit_locked(it->second, *ls);
|
||||
it = tracks_.erase(it);
|
||||
} else {
|
||||
@@ -383,6 +462,9 @@ private:
|
||||
std::map<int, int> owner_index_; ///< actor_idx → live track_id (AR-015)
|
||||
DeadTrackFn on_dead_;
|
||||
int next_id_{0};
|
||||
double now_{0.0}; ///< tracker's clock (association)
|
||||
double evidence_through_{0.0}; ///< matcher's watermark (reaping)
|
||||
bool awaits_evidence_{false};
|
||||
int dropped_votes_{0};
|
||||
int belief_swaps_{0};
|
||||
int actor_conflicts_{0};
|
||||
|
||||
Reference in New Issue
Block a user