dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(ar-024): enforce the invariant statically, and delete the fallback it caught

Browse Source 
AR-024's register row gives its verification tier as "Static check -- no
bare cosine outside a tagged EXCEPTION". No such check existed, so the
invariant was enforced by reading, and reading had missed a live
violation.

scripts/ci/check_raw_cosine.py is that check, wired into the
traceability workflow as a blocking step. It is honest about its reach:
it catches direct cosine_similarity() uses not routed through a
calibration, and it cannot follow a cosine through a variable across
statements. That limit is documented in the script rather than left for
someone to discover after trusting a pass.

What it caught, and what this commit removes with it:

The identity matcher's no-calibration fallback thresholded raw cosine
distance (match_threshold) plus a ratio test (match_ratio,
match_ratio_ceil). Worse than the invariant breach: it fed
max(0, cosine) into TrackRegistry::observe, whose contract reads
"posterior is a calibrated probability, never a raw cosine (AR-024) ...
so the accumulation cannot be fed an uncalibrated number by a careless
caller". It could, and did. And it disagreed with the rest of the
pipeline about what "the fit failed" means -- same_person_probability
answers that with the untuned default sigmoid and a loud warning, so
association stayed in probability space while matching alone left it.
One run, two policies, no announcement.

Now one rule: cal_.probability() always, with a warning when the fit is
not real. A worse answer than a fitted calibration, a better one than a
number whose units nothing else shares.

TrackGallery::set_calibration is mandatory for the same reason. Its
default was max(0, cosine), which made expand_band_lo = 0.90 mean
"cosine > 0.9" in a test and "P(same person) > 0.9" in production.
FaceTrackerFunc already threw without one; the expansion store now
matches.

One exception is recorded, in the calibration's own dedup. It is not a
close call: at 1 - 1e-7 it asks whether two vectors are the same vector,
and it runs on the fit's input, so a calibrated comparison there would
have to be calibrated by the fit it is feeding.

Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys
are read with a contains() check, so each one had been silently inert
since the field behind it was deleted -- a sweep varying one of them
measured nothing and reported an ordinary-looking F1.

TRACES: AR-024, AR-023 | SR-002
This commit is contained in:
dtourolle
2026-08-05 15:46:33 +02:00
parent fd078c399f
commit e1de98e783
11 changed files with 375 additions and 89 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/nodes/identity_matcher_node.hpp
+62 -63
View File
@@ -19,19 +19,36 @@
// KPN node: compares each embedding against every reference embedding in the
// actor gallery using cosine similarity.
//
// Matching strategy — two modes selected at construction time:
// Matching strategy — one mode, always.
//
// Calibrated (preferred): gallery calibration fits a sigmoid
// P(match) = σ(a·similarity + b) from intra/inter-class pairs.
// A face is accepted if P(match | best_actor) > prob_threshold.
// Gallery calibration fits a sigmoid P(match) = σ(a·similarity + b) from
// intra/inter-class pairs. A face is accepted if P(match | best_actor) >
// prob_threshold. Per-actor best similarity is the closest reference
// embedding (best-of-N).
//
// Fallback (no calibration): dual-criterion accept —
// (a) best cosine distance < match_threshold, OR
// (b) ratio test: best_dist/second_best_dist < match_ratio
// AND best_dist < match_ratio_ceil.
/// TRACES: AR-024 | SR-002
// **There is no raw-cosine fallback.** There used to be: when the fit was
// invalid this node switched to a cosine-distance ceiling plus a ratio test
// (`match_threshold`, `match_ratio`, `match_ratio_ceil`). Three things were
// wrong with it, and the third is the one that mattered.
//
// In both modes, per-actor best similarity is determined by scanning
// reference embeddings and taking the closest (best-of-N).
// 1. It violated AR-024 outright, untagged — a bare cosine threshold means
// something different for every model, gallery and face size.
// 2. It disagreed with the rest of the pipeline about what "calibration
// failed" means. `same_person_probability` answers that question by
// falling back to the untuned default sigmoid and saying so loudly, so
// tracking and evidence weighting stayed in probability space while
// matching alone left it. One run, two policies.
// 3. Its accepted faces were still fed to `TrackRegistry::observe`, whose
// contract reads "posterior is a calibrated probability, never a raw
// cosine (AR-024) ... so the accumulation cannot be fed an uncalibrated
// number by a careless caller". It could. `max(0, cosine)` went straight
// into the log-odds accumulation as though it were a probability.
//
// An invalid fit now behaves exactly as everywhere else: the default sigmoid,
// with a warning that says the probabilities are not meaningful. That is a
// worse answer than a fitted calibration and a better one than a number whose
// units nothing else in the pipeline shares.
//
// Gallery scan: the full reference set (tens of thousands of 512-dim
// embeddings) is uploaded to the GPU once at construction time and stays
@@ -57,9 +74,6 @@ struct IdentityMatcherFunc {
: gallery_(gallery)
, prob_threshold_(cfg.prob_threshold)
, log_prior_odds_(std::log(cfg.match_prior / (1.f - cfg.match_prior)))
, threshold_(cfg.match_threshold)
, ratio_(cfg.match_ratio)
, ratio_ceil_(cfg.match_ratio_ceil)
, track_gallery_(cfg)
{
std::cerr << "[identity_matcher] flattening gallery embeddings...\n";
@@ -92,16 +106,21 @@ struct IdentityMatcherFunc {
<< cfg.gallery_path << "\n";
}
if (cal_.valid) {
std::cerr << "[identity_matcher] calibrated Bayesian matching"
<< " prior=" << cfg.match_prior
<< " P_threshold=" << prob_threshold_
<< " effective_sim_boundary="
<< cal_.boundary_at(prob_threshold_, log_prior_odds_) << "\n";
} else {
std::cerr << "[identity_matcher] threshold matching (calibration skipped)"
<< " threshold=" << threshold_
<< " ratio=" << ratio_ << " ratio_ceil=" << ratio_ceil_ << "\n";
/// TRACES: AR-024 | SR-002
// Same sentence either way, because it is the same decision rule; only
// the provenance of (a, b) differs. An unfitted sigmoid still returns
// plausible-looking probabilities, so the warning has to be the thing
// that distinguishes them — nothing downstream can.
std::cerr << "[identity_matcher] calibrated Bayesian matching"
<< " prior=" << cfg.match_prior
<< " P_threshold=" << prob_threshold_
<< " effective_sim_boundary="
<< cal_.boundary_at(prob_threshold_, log_prior_odds_) << "\n";
if (!cal_.valid) {
std::cerr << "[identity_matcher] WARNING: the calibration is NOT fitted "
"(a=" << cal_.a << ", b=" << cal_.b << ") — matching runs "
"on the untuned default sigmoid, so prob_threshold is not "
"comparable to a tuned run's.\n";
}
std::cerr << "[identity_matcher] gallery: "
<< gallery_.actors.size() << " actors, "
@@ -214,39 +233,27 @@ struct IdentityMatcherFunc {
if (sim > best_sim[ai]) best_sim[ai] = sim;
}
int best_actor = -1;
int second_actor = -1;
float best_s = -std::numeric_limits<float>::max();
float second_s = -std::numeric_limits<float>::max();
// Only the best matters now. The runner-up was tracked solely for
// the retired ratio test, which asked whether the best cosine stood
// out from the second — a question the calibrated posterior does
// not need, since it already says how likely the best match is to
// be right rather than how much it beat its neighbour by.
int best_actor = -1;
float best_s = -std::numeric_limits<float>::max();
for (int ai = 0; ai < static_cast<int>(best_sim.size()); ++ai) {
if (best_sim[ai] > best_s) {
second_s = best_s;
second_actor = best_actor;
best_s = best_sim[ai];
best_actor = ai;
} else if (best_sim[ai] > second_s) {
second_s = best_sim[ai];
second_actor = ai;
best_s = best_sim[ai];
best_actor = ai;
}
}
(void)second_actor;
bool accept = false;
if (best_actor >= 0) {
if (cal_.valid) {
accept = cal_.probability(best_s, log_prior_odds_) > prob_threshold_;
} else {
float best_d = 1.f - best_s;
float second_d = (second_s > -std::numeric_limits<float>::max())
? 1.f - second_s
: std::numeric_limits<float>::max();
bool absolute = best_d < threshold_;
bool ratio = (best_d < ratio_ceil_) &&
(second_d == std::numeric_limits<float>::max() ||
best_d / second_d < ratio_);
accept = absolute || ratio;
}
}
/// TRACES: AR-024 | SR-002
// One rule, whatever the fit's provenance. The cosine reaches a
// comparison only through cal_.probability().
const float best_p = best_actor >= 0
? cal_.probability(best_s, log_prior_odds_)
: 0.f;
const bool accept = best_actor >= 0 && best_p > prob_threshold_;
IdentifiedActor ia;
// Map bbox back to original video resolution when dense_scale
@@ -267,9 +274,7 @@ struct IdentityMatcherFunc {
ia.imdb_id = gallery_.actors[best_actor].imdb_id;
ia.tmdb_id = gallery_.actors[best_actor].tmdb_id;
ia.jellyfin_id = gallery_.actors[best_actor].jellyfin_id;
ia.similarity = cal_.valid
? cal_.probability(best_s, log_prior_odds_)
: best_s;
ia.similarity = best_p;
}
// Feed this face into per-film gallery expansion. best_actor/best_s
@@ -283,12 +288,9 @@ struct IdentityMatcherFunc {
// would make ownership depend on a per-frame threshold the redesign
// exists to stop relying on. The registry discounts for correlation
// and decides ownership from the accumulated posterior (AR-025).
if (registry_ && best_actor >= 0 && tf.track_ids[fi] >= 0) {
const float p = cal_.valid
? cal_.probability(best_s, log_prior_odds_)
: std::max(0.f, best_s);
registry_->observe(tf.track_ids[fi], best_actor, p, tf.embeddings[fi]);
}
if (registry_ && best_actor >= 0 && tf.track_ids[fi] >= 0)
registry_->observe(tf.track_ids[fi], best_actor, best_p,
tf.embeddings[fi]);
// TRACES: AR-019 | SR-005
// Ownership is the registry's, computed once. TrackGallery used to
@@ -339,9 +341,6 @@ private:
GalleryCalibration cal_;
float prob_threshold_;
float log_prior_odds_;
float threshold_;
float ratio_;
float ratio_ceil_;
/// flat_emb_ is the BAKED reference set only — it is the calibration fit's
/// input (AR-023) and is not touched again after construction. flat_actor_,
/// by contrast, is the actor mapping parallel to the *engine's* rows, so it