feat(ar-024): enforce the invariant statically, and delete the fallback it caught
AR-024's register row gives its verification tier as "Static check -- no bare cosine outside a tagged EXCEPTION". No such check existed, so the invariant was enforced by reading, and reading had missed a live violation. scripts/ci/check_raw_cosine.py is that check, wired into the traceability workflow as a blocking step. It is honest about its reach: it catches direct cosine_similarity() uses not routed through a calibration, and it cannot follow a cosine through a variable across statements. That limit is documented in the script rather than left for someone to discover after trusting a pass. What it caught, and what this commit removes with it: The identity matcher's no-calibration fallback thresholded raw cosine distance (match_threshold) plus a ratio test (match_ratio, match_ratio_ceil). Worse than the invariant breach: it fed max(0, cosine) into TrackRegistry::observe, whose contract reads "posterior is a calibrated probability, never a raw cosine (AR-024) ... so the accumulation cannot be fed an uncalibrated number by a careless caller". It could, and did. And it disagreed with the rest of the pipeline about what "the fit failed" means -- same_person_probability answers that with the untuned default sigmoid and a loud warning, so association stayed in probability space while matching alone left it. One run, two policies, no announcement. Now one rule: cal_.probability() always, with a warning when the fit is not real. A worse answer than a fitted calibration, a better one than a number whose units nothing else shares. TrackGallery::set_calibration is mandatory for the same reason. Its default was max(0, cosine), which made expand_band_lo = 0.90 mean "cosine > 0.9" in a test and "P(same person) > 0.9" in production. FaceTrackerFunc already threw without one; the expansion store now matches. One exception is recorded, in the calibration's own dedup. It is not a close call: at 1 - 1e-7 it asks whether two vectors are the same vector, and it runs on the fit's input, so a calibrated comparison there would have to be calibrated by the fit it is feeding. Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys are read with a contains() check, so each one had been silently inert since the field behind it was deleted -- a sweep varying one of them measured nothing and reported an ordinary-looking F1. TRACES: AR-024, AR-023 | SR-002
This commit is contained in:
+9
-3
@@ -78,9 +78,15 @@ struct Config {
|
||||
// earlier 9-film scene-union-metric tuning (0.76) — that metric is now known to
|
||||
// have hidden out-of-cast false positives (see docs/optimizer-experiments.md).
|
||||
float prob_threshold{0.754f}; // posterior P(match | sim, prior) threshold
|
||||
float match_threshold{0.45f}; // cosine distance hard ceiling fallback (no calibration)
|
||||
float match_ratio{0.80f}; // ratio test fallback: accept if best/second < ratio
|
||||
float match_ratio_ceil{0.65f}; // ratio test only fires below this absolute distance
|
||||
// TRACES: AR-024 | SR-002
|
||||
// match_threshold (0.45), match_ratio (0.80) and match_ratio_ceil (0.65) are
|
||||
// RETIRED, joining track_max_embed_dist, cut_revive_sim, expand_novelty_sim
|
||||
// and expand_track_spread_max. All were raw cosine distances, and they were
|
||||
// the accept rule whenever the calibration fit failed — so the one situation
|
||||
// in which the pipeline knew its probabilities were untrustworthy was the
|
||||
// one in which it stopped using them. An unfitted sigmoid is now the
|
||||
// fallback everywhere, which is at least the same wrong number in every
|
||||
// stage. See identity_matcher_node.hpp.
|
||||
|
||||
// ── Cut detection ────────────────────────────────────────────────────────
|
||||
float cut_threshold{0.70f}; // grayscale histogram correlation below this → hard cut
|
||||
|
||||
@@ -162,6 +162,19 @@ inline GalleryCalibration calibrate_gallery(
|
||||
for (const auto& e : by_actor[ai]) {
|
||||
bool dup = false;
|
||||
for (const auto& k : kept) {
|
||||
// EXCEPTION: AR-024 this asks whether two vectors are THE SAME
|
||||
// VECTOR, not whether two faces are the same person.
|
||||
//
|
||||
// Two independent reasons, either sufficient. First, at
|
||||
// 1 - 1e-7 the threshold is a floating-point identity test: it
|
||||
// catches one source image embedded twice, and no genuine pair
|
||||
// of distinct photographs lands there. Nothing about it is a
|
||||
// decision, so there is nothing for a probability to mean.
|
||||
//
|
||||
// Second, and structurally: this IS the calibration fit. The
|
||||
// dedup runs on its input, before (a, b) exist. A calibrated
|
||||
// comparison here would have to be calibrated by the fit it is
|
||||
// feeding, which is not a thing that can be arranged.
|
||||
if (cosine_similarity(e, k) > kDedupSimThreshold) { dup = true; break; }
|
||||
}
|
||||
if (!dup) kept.push_back(e);
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
#include <iostream>
|
||||
#include <limits>
|
||||
#include <map>
|
||||
#include <stdexcept>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
@@ -163,10 +164,20 @@ struct TrackGallery {
|
||||
}
|
||||
|
||||
/// TRACES: AR-024 | SR-005
|
||||
/// Supply the calibration belonging to the active embedder. Without it the
|
||||
/// band falls back to treating cosine as probability, which is wrong but
|
||||
/// bounded — and the default is loud in the header rather than silent.
|
||||
void set_calibration(std::function<float(float)> c) { calibrate_ = std::move(c); }
|
||||
/// Supply the calibration belonging to the active embedder.
|
||||
///
|
||||
/// Required, not optional. The default used to be `max(0, cosine)` — a raw
|
||||
/// cosine worn as a probability, which made `expand_band_lo = 0.90` mean
|
||||
/// "cosine above 0.9" in a test and "P(same person) above 0.9" in
|
||||
/// production. Those are wildly different gates, and nothing announced the
|
||||
/// switch. `FaceTrackerFunc` already refuses to construct without a
|
||||
/// calibration for the same reason; this now matches it.
|
||||
void set_calibration(std::function<float(float)> c) {
|
||||
if (!c) throw std::invalid_argument(
|
||||
"track_gallery: a calibration is required — the admission band is "
|
||||
"expressed in probability space (AR-024)");
|
||||
calibrate_ = std::move(c);
|
||||
}
|
||||
|
||||
/// Embeddings the band refused. A store that admits nothing is as wrong as
|
||||
/// one that admits everything, and neither is visible without this.
|
||||
@@ -341,8 +352,9 @@ private:
|
||||
}
|
||||
|
||||
/// cosine → P(same person). The one probability space the pipeline reasons
|
||||
/// in; see gallery_calibration.hpp's same_person_probability.
|
||||
std::function<float(float)> calibrate_{[](float c) { return std::max(0.f, c); }};
|
||||
/// in; see gallery_calibration.hpp's same_person_probability. Never default
|
||||
/// constructed to an identity-ish stand-in — see set_calibration.
|
||||
std::function<float(float)> calibrate_;
|
||||
std::size_t rejected_{0}; ///< admissions refused by the band
|
||||
|
||||
bool enabled_;
|
||||
|
||||
@@ -168,9 +168,6 @@ static Config config_from_dict(nb::dict d) {
|
||||
// identity matcher
|
||||
getf("match_prior", cfg.match_prior);
|
||||
getf("prob_threshold", cfg.prob_threshold);
|
||||
getf("match_threshold", cfg.match_threshold);
|
||||
getf("match_ratio", cfg.match_ratio);
|
||||
getf("match_ratio_ceil", cfg.match_ratio_ceil);
|
||||
// face tracker
|
||||
getf("track_alpha", cfg.track_alpha);
|
||||
getf("track_min_iou", cfg.track_min_iou);
|
||||
|
||||
+1
-4
@@ -22,7 +22,7 @@
|
||||
// --output <path> output JSON (default: annotations.json)
|
||||
// --fps <N> sample rate in frames/sec (default: 1.0)
|
||||
// --verbosity <0|1|2> 0=minimal, 1=standard, 2=jellyfin-xray (default: 0)
|
||||
// --match-threshold <f> cosine dist threshold (default: 0.45)
|
||||
// --prob-threshold <f> posterior P(match) to accept (default: 0.754)
|
||||
// --extinction <f> actor extinction window in seconds (default: 5.0)
|
||||
// --detector <path> override SCRFD detector model path
|
||||
// --arcface <path> override ArcFace model path
|
||||
@@ -155,7 +155,6 @@ static Config parse_args(int argc, char** argv) {
|
||||
else if (arg("--verbosity")) { int v = std::stoi(next()); cfg.verbosity = v == 2 ? Verbosity::xray : v == 1 ? Verbosity::standard : Verbosity::minimal; }
|
||||
else if (arg("--prior")) cfg.match_prior = std::stof(next());
|
||||
else if (arg("--prob-threshold")) cfg.prob_threshold = std::stof(next());
|
||||
else if (arg("--match-threshold")) cfg.match_threshold = std::stof(next());
|
||||
else if (arg("--extinction")) cfg.extinction_sec = std::stod(next());
|
||||
else if (arg("--detector")) cfg.detector_model = next();
|
||||
else if (arg("--detector-engine")) cfg.detector_engine = next();
|
||||
@@ -165,8 +164,6 @@ static Config parse_args(int argc, char** argv) {
|
||||
else if (arg("--conf")) cfg.detector_conf = std::stof(next());
|
||||
else if (arg("--max-faces")) cfg.max_faces = std::stoi(next());
|
||||
else if (arg("--min-face-px")) cfg.min_face_px = std::stof(next());
|
||||
else if (arg("--ratio")) cfg.match_ratio = std::stof(next());
|
||||
else if (arg("--ratio-ceil")) cfg.match_ratio_ceil = std::stof(next());
|
||||
else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next());
|
||||
else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next());
|
||||
else if (arg("--track-min-prob")) cfg.track_assoc_min_prob = std::stof(next());
|
||||
|
||||
@@ -19,19 +19,36 @@
|
||||
// KPN node: compares each embedding against every reference embedding in the
|
||||
// actor gallery using cosine similarity.
|
||||
//
|
||||
// Matching strategy — two modes selected at construction time:
|
||||
// Matching strategy — one mode, always.
|
||||
//
|
||||
// Calibrated (preferred): gallery calibration fits a sigmoid
|
||||
// P(match) = σ(a·similarity + b) from intra/inter-class pairs.
|
||||
// A face is accepted if P(match | best_actor) > prob_threshold.
|
||||
// Gallery calibration fits a sigmoid P(match) = σ(a·similarity + b) from
|
||||
// intra/inter-class pairs. A face is accepted if P(match | best_actor) >
|
||||
// prob_threshold. Per-actor best similarity is the closest reference
|
||||
// embedding (best-of-N).
|
||||
//
|
||||
// Fallback (no calibration): dual-criterion accept —
|
||||
// (a) best cosine distance < match_threshold, OR
|
||||
// (b) ratio test: best_dist/second_best_dist < match_ratio
|
||||
// AND best_dist < match_ratio_ceil.
|
||||
/// TRACES: AR-024 | SR-002
|
||||
// **There is no raw-cosine fallback.** There used to be: when the fit was
|
||||
// invalid this node switched to a cosine-distance ceiling plus a ratio test
|
||||
// (`match_threshold`, `match_ratio`, `match_ratio_ceil`). Three things were
|
||||
// wrong with it, and the third is the one that mattered.
|
||||
//
|
||||
// In both modes, per-actor best similarity is determined by scanning
|
||||
// reference embeddings and taking the closest (best-of-N).
|
||||
// 1. It violated AR-024 outright, untagged — a bare cosine threshold means
|
||||
// something different for every model, gallery and face size.
|
||||
// 2. It disagreed with the rest of the pipeline about what "calibration
|
||||
// failed" means. `same_person_probability` answers that question by
|
||||
// falling back to the untuned default sigmoid and saying so loudly, so
|
||||
// tracking and evidence weighting stayed in probability space while
|
||||
// matching alone left it. One run, two policies.
|
||||
// 3. Its accepted faces were still fed to `TrackRegistry::observe`, whose
|
||||
// contract reads "posterior is a calibrated probability, never a raw
|
||||
// cosine (AR-024) ... so the accumulation cannot be fed an uncalibrated
|
||||
// number by a careless caller". It could. `max(0, cosine)` went straight
|
||||
// into the log-odds accumulation as though it were a probability.
|
||||
//
|
||||
// An invalid fit now behaves exactly as everywhere else: the default sigmoid,
|
||||
// with a warning that says the probabilities are not meaningful. That is a
|
||||
// worse answer than a fitted calibration and a better one than a number whose
|
||||
// units nothing else in the pipeline shares.
|
||||
//
|
||||
// Gallery scan: the full reference set (tens of thousands of 512-dim
|
||||
// embeddings) is uploaded to the GPU once at construction time and stays
|
||||
@@ -57,9 +74,6 @@ struct IdentityMatcherFunc {
|
||||
: gallery_(gallery)
|
||||
, prob_threshold_(cfg.prob_threshold)
|
||||
, log_prior_odds_(std::log(cfg.match_prior / (1.f - cfg.match_prior)))
|
||||
, threshold_(cfg.match_threshold)
|
||||
, ratio_(cfg.match_ratio)
|
||||
, ratio_ceil_(cfg.match_ratio_ceil)
|
||||
, track_gallery_(cfg)
|
||||
{
|
||||
std::cerr << "[identity_matcher] flattening gallery embeddings...\n";
|
||||
@@ -92,16 +106,21 @@ struct IdentityMatcherFunc {
|
||||
<< cfg.gallery_path << "\n";
|
||||
}
|
||||
|
||||
if (cal_.valid) {
|
||||
std::cerr << "[identity_matcher] calibrated Bayesian matching"
|
||||
<< " prior=" << cfg.match_prior
|
||||
<< " P_threshold=" << prob_threshold_
|
||||
<< " effective_sim_boundary="
|
||||
<< cal_.boundary_at(prob_threshold_, log_prior_odds_) << "\n";
|
||||
} else {
|
||||
std::cerr << "[identity_matcher] threshold matching (calibration skipped)"
|
||||
<< " threshold=" << threshold_
|
||||
<< " ratio=" << ratio_ << " ratio_ceil=" << ratio_ceil_ << "\n";
|
||||
/// TRACES: AR-024 | SR-002
|
||||
// Same sentence either way, because it is the same decision rule; only
|
||||
// the provenance of (a, b) differs. An unfitted sigmoid still returns
|
||||
// plausible-looking probabilities, so the warning has to be the thing
|
||||
// that distinguishes them — nothing downstream can.
|
||||
std::cerr << "[identity_matcher] calibrated Bayesian matching"
|
||||
<< " prior=" << cfg.match_prior
|
||||
<< " P_threshold=" << prob_threshold_
|
||||
<< " effective_sim_boundary="
|
||||
<< cal_.boundary_at(prob_threshold_, log_prior_odds_) << "\n";
|
||||
if (!cal_.valid) {
|
||||
std::cerr << "[identity_matcher] WARNING: the calibration is NOT fitted "
|
||||
"(a=" << cal_.a << ", b=" << cal_.b << ") — matching runs "
|
||||
"on the untuned default sigmoid, so prob_threshold is not "
|
||||
"comparable to a tuned run's.\n";
|
||||
}
|
||||
std::cerr << "[identity_matcher] gallery: "
|
||||
<< gallery_.actors.size() << " actors, "
|
||||
@@ -214,39 +233,27 @@ struct IdentityMatcherFunc {
|
||||
if (sim > best_sim[ai]) best_sim[ai] = sim;
|
||||
}
|
||||
|
||||
int best_actor = -1;
|
||||
int second_actor = -1;
|
||||
float best_s = -std::numeric_limits<float>::max();
|
||||
float second_s = -std::numeric_limits<float>::max();
|
||||
// Only the best matters now. The runner-up was tracked solely for
|
||||
// the retired ratio test, which asked whether the best cosine stood
|
||||
// out from the second — a question the calibrated posterior does
|
||||
// not need, since it already says how likely the best match is to
|
||||
// be right rather than how much it beat its neighbour by.
|
||||
int best_actor = -1;
|
||||
float best_s = -std::numeric_limits<float>::max();
|
||||
for (int ai = 0; ai < static_cast<int>(best_sim.size()); ++ai) {
|
||||
if (best_sim[ai] > best_s) {
|
||||
second_s = best_s;
|
||||
second_actor = best_actor;
|
||||
best_s = best_sim[ai];
|
||||
best_actor = ai;
|
||||
} else if (best_sim[ai] > second_s) {
|
||||
second_s = best_sim[ai];
|
||||
second_actor = ai;
|
||||
best_s = best_sim[ai];
|
||||
best_actor = ai;
|
||||
}
|
||||
}
|
||||
(void)second_actor;
|
||||
|
||||
bool accept = false;
|
||||
if (best_actor >= 0) {
|
||||
if (cal_.valid) {
|
||||
accept = cal_.probability(best_s, log_prior_odds_) > prob_threshold_;
|
||||
} else {
|
||||
float best_d = 1.f - best_s;
|
||||
float second_d = (second_s > -std::numeric_limits<float>::max())
|
||||
? 1.f - second_s
|
||||
: std::numeric_limits<float>::max();
|
||||
bool absolute = best_d < threshold_;
|
||||
bool ratio = (best_d < ratio_ceil_) &&
|
||||
(second_d == std::numeric_limits<float>::max() ||
|
||||
best_d / second_d < ratio_);
|
||||
accept = absolute || ratio;
|
||||
}
|
||||
}
|
||||
/// TRACES: AR-024 | SR-002
|
||||
// One rule, whatever the fit's provenance. The cosine reaches a
|
||||
// comparison only through cal_.probability().
|
||||
const float best_p = best_actor >= 0
|
||||
? cal_.probability(best_s, log_prior_odds_)
|
||||
: 0.f;
|
||||
const bool accept = best_actor >= 0 && best_p > prob_threshold_;
|
||||
|
||||
IdentifiedActor ia;
|
||||
// Map bbox back to original video resolution when dense_scale
|
||||
@@ -267,9 +274,7 @@ struct IdentityMatcherFunc {
|
||||
ia.imdb_id = gallery_.actors[best_actor].imdb_id;
|
||||
ia.tmdb_id = gallery_.actors[best_actor].tmdb_id;
|
||||
ia.jellyfin_id = gallery_.actors[best_actor].jellyfin_id;
|
||||
ia.similarity = cal_.valid
|
||||
? cal_.probability(best_s, log_prior_odds_)
|
||||
: best_s;
|
||||
ia.similarity = best_p;
|
||||
}
|
||||
|
||||
// Feed this face into per-film gallery expansion. best_actor/best_s
|
||||
@@ -283,12 +288,9 @@ struct IdentityMatcherFunc {
|
||||
// would make ownership depend on a per-frame threshold the redesign
|
||||
// exists to stop relying on. The registry discounts for correlation
|
||||
// and decides ownership from the accumulated posterior (AR-025).
|
||||
if (registry_ && best_actor >= 0 && tf.track_ids[fi] >= 0) {
|
||||
const float p = cal_.valid
|
||||
? cal_.probability(best_s, log_prior_odds_)
|
||||
: std::max(0.f, best_s);
|
||||
registry_->observe(tf.track_ids[fi], best_actor, p, tf.embeddings[fi]);
|
||||
}
|
||||
if (registry_ && best_actor >= 0 && tf.track_ids[fi] >= 0)
|
||||
registry_->observe(tf.track_ids[fi], best_actor, best_p,
|
||||
tf.embeddings[fi]);
|
||||
|
||||
// TRACES: AR-019 | SR-005
|
||||
// Ownership is the registry's, computed once. TrackGallery used to
|
||||
@@ -339,9 +341,6 @@ private:
|
||||
GalleryCalibration cal_;
|
||||
float prob_threshold_;
|
||||
float log_prior_odds_;
|
||||
float threshold_;
|
||||
float ratio_;
|
||||
float ratio_ceil_;
|
||||
/// flat_emb_ is the BAKED reference set only — it is the calibration fit's
|
||||
/// input (AR-023) and is not touched again after construction. flat_actor_,
|
||||
/// by contrast, is the actor mapping parallel to the *engine's* rows, so it
|
||||
|
||||
@@ -71,7 +71,6 @@ static Config parse_args(int argc, char** argv) {
|
||||
else if (arg("--verbosity")) { int v = std::stoi(next()); cfg.verbosity = v == 2 ? Verbosity::xray : v == 1 ? Verbosity::standard : Verbosity::minimal; }
|
||||
else if (arg("--prior")) cfg.match_prior = std::stof(next());
|
||||
else if (arg("--prob-threshold")) cfg.prob_threshold = std::stof(next());
|
||||
else if (arg("--match-threshold")) cfg.match_threshold = std::stof(next());
|
||||
else if (arg("--extinction")) cfg.extinction_sec = std::stod(next());
|
||||
else if (arg("--detector")) cfg.detector_model = next();
|
||||
else if (arg("--detector-engine")) cfg.detector_engine = next();
|
||||
@@ -81,8 +80,6 @@ static Config parse_args(int argc, char** argv) {
|
||||
else if (arg("--conf")) cfg.detector_conf = std::stof(next());
|
||||
else if (arg("--max-faces")) cfg.max_faces = std::stoi(next());
|
||||
else if (arg("--min-face-px")) cfg.min_face_px = std::stof(next());
|
||||
else if (arg("--ratio")) cfg.match_ratio = std::stof(next());
|
||||
else if (arg("--ratio-ceil")) cfg.match_ratio_ceil = std::stof(next());
|
||||
else if (arg("--track-alpha")) cfg.track_alpha = std::stof(next());
|
||||
else if (arg("--track-min-iou")) cfg.track_min_iou = std::stof(next());
|
||||
else if (arg("--track-max-embed")) cfg.track_max_embed_dist = std::stof(next());
|
||||
|
||||
Reference in New Issue
Block a user