AR-024's register row gives its verification tier as "Static check -- no bare cosine outside a tagged EXCEPTION". No such check existed, so the invariant was enforced by reading, and reading had missed a live violation. scripts/ci/check_raw_cosine.py is that check, wired into the traceability workflow as a blocking step. It is honest about its reach: it catches direct cosine_similarity() uses not routed through a calibration, and it cannot follow a cosine through a variable across statements. That limit is documented in the script rather than left for someone to discover after trusting a pass. What it caught, and what this commit removes with it: The identity matcher's no-calibration fallback thresholded raw cosine distance (match_threshold) plus a ratio test (match_ratio, match_ratio_ceil). Worse than the invariant breach: it fed max(0, cosine) into TrackRegistry::observe, whose contract reads "posterior is a calibrated probability, never a raw cosine (AR-024) ... so the accumulation cannot be fed an uncalibrated number by a careless caller". It could, and did. And it disagreed with the rest of the pipeline about what "the fit failed" means -- same_person_probability answers that with the untuned default sigmoid and a loud warning, so association stayed in probability space while matching alone left it. One run, two policies, no announcement. Now one rule: cal_.probability() always, with a warning when the fit is not real. A worse answer than a fitted calibration, a better one than a number whose units nothing else shares. TrackGallery::set_calibration is mandatory for the same reason. Its default was max(0, cosine), which made expand_band_lo = 0.90 mean "cosine > 0.9" in a test and "P(same person) > 0.9" in production. FaceTrackerFunc already threw without one; the expansion store now matches. One exception is recorded, in the calibration's own dedup. It is not a close call: at 1 - 1e-7 it asks whether two vectors are the same vector, and it runs on the fit's input, so a calibrated comparison there would have to be calibrated by the fit it is feeding. Also drops seven dead keys from the optimizer's CFG_KEYS. Config keys are read with a contains() check, so each one had been silently inert since the field behind it was deleted -- a sweep varying one of them measured nothing and reported an ordinary-looking F1. TRACES: AR-024, AR-023 | SR-002
404 lines
17 KiB
C++
404 lines
17 KiB
C++
// TRACES: UT-005 | AR-018, AR-019, AR-024, AR-026 | SR-005, SR-001
|
|
//
|
|
// Unit tests for TrackGallery (gallery/track_gallery.hpp): per-film gallery
|
|
// expansion driven by track continuity. Pure, GPU-free, model-free — exercises
|
|
// the AR-018 banded admission at both bounds, the promotion-time coherence
|
|
// gate, the diversity-buffer eviction policy, plurality ownership, and
|
|
// idempotent promotion, all through the public interface.
|
|
//
|
|
// The band is defined in PROBABILITY space (AR-024), so every case below states
|
|
// its own cosine → probability map. It has to: set_calibration is now mandatory
|
|
// and there is no header default to inherit.
|
|
//
|
|
// There used to be one — `max(0, cosine)` — and it was the reason this comment
|
|
// was originally needed. Under it the two spaces coincided, so a test that
|
|
// forgot to name the mapping still passed, and a gate that silently reverted to
|
|
// raw cosine passed with it. The default is gone rather than merely discouraged,
|
|
// which is why identity_cal below is now an explicit choice a case makes and not
|
|
// a restatement of what would have happened anyway.
|
|
#include <catch2/catch_test_macros.hpp>
|
|
#include <catch2/matchers/catch_matchers_floating_point.hpp>
|
|
|
|
#include "config.hpp"
|
|
#include "gallery/track_gallery.hpp"
|
|
#include "types.hpp"
|
|
|
|
#include <algorithm>
|
|
#include <stdexcept>
|
|
#include <array>
|
|
#include <cmath>
|
|
#include <vector>
|
|
|
|
namespace {
|
|
|
|
constexpr float kPi = 3.14159265358979323846f;
|
|
|
|
// The band as the tests drive it. Kept in one place so a change to the shipped
|
|
// defaults does not silently invalidate the arithmetic in each case.
|
|
constexpr float kBandLo = 0.90f;
|
|
constexpr float kBandHi = 0.95f;
|
|
|
|
// Identity map: probability == cosine, so a case can place an embedding at an
|
|
// exact probability. cosine_similarity is a bare dot product over unit vectors
|
|
// (types.hpp), so the placements below are bit-exact, not approximate.
|
|
float identity_cal(float c) { return c; }
|
|
|
|
// Unit-norm embedding pointing along one axis. Cosine sim to another one-hot is
|
|
// 0, to itself 1.
|
|
Embedding one_hot(int slot) {
|
|
Embedding e{};
|
|
e[slot] = 1.0f;
|
|
return e;
|
|
}
|
|
|
|
// TRACES: AR-026 | SR-001
|
|
// The annex is a contiguous row-major matrix, not a vector of structs, so that
|
|
// the matcher can hand whole blocks of new rows to the GEMM path. Tests that
|
|
// want to compare one promoted view read it back through this.
|
|
Embedding annex_view(const TrackGallery& tg, int row) {
|
|
const float* p = tg.annex_row(row);
|
|
Embedding e{};
|
|
std::copy(p, p + 512, e.begin());
|
|
return e;
|
|
}
|
|
|
|
// Unit-norm embedding in the plane of axes i,j at cosine `cos_t` from axis i.
|
|
// Cosine sim to one_hot(i) is exactly cos_t.
|
|
Embedding at_sim(int i, int j, float cos_t) {
|
|
Embedding e{};
|
|
e[i] = cos_t;
|
|
e[j] = std::sqrt(std::max(0.f, 1.f - cos_t * cos_t));
|
|
return e;
|
|
}
|
|
|
|
// A spoke: shares axis 0 with every other spoke, and is otherwise unique. Any
|
|
// two DISTINCT spokes have cosine similarity exactly cos_t², so one constant
|
|
// places a whole mutually-in-band store. A spoke against itself is 1.0 — above
|
|
// the band's ceiling, i.e. redundant, which is the intended reading.
|
|
Embedding spoke(int k, float cos_t) { return at_sim(0, k, cos_t); }
|
|
|
|
// cos_t chosen so pairwise similarity between distinct spokes is 0.9197 —
|
|
// comfortably inside [0.90, 0.95], clear of both bounds.
|
|
constexpr float kSpokeCos = 0.959f;
|
|
|
|
// Two embeddings `deg` apart in the plane of axes 0,1. Cosine is cos(deg), so a
|
|
// chain of these can step through the band while its endpoints fall outside it.
|
|
Embedding on_circle(float deg) {
|
|
Embedding e{};
|
|
e[0] = std::cos(deg * kPi / 180.f);
|
|
e[1] = std::sin(deg * kPi / 180.f);
|
|
return e;
|
|
}
|
|
|
|
Config expand_cfg() {
|
|
Config cfg;
|
|
cfg.expand_gallery = true;
|
|
cfg.expand_buffer_size = 3;
|
|
cfg.expand_band_lo = kBandLo;
|
|
cfg.expand_band_hi = kBandHi;
|
|
cfg.expand_min_anchor_frames = 3;
|
|
return cfg;
|
|
}
|
|
|
|
const cv::Mat kNoCrop; // debug dumping off → crop unused
|
|
|
|
} // namespace
|
|
|
|
TEST_CASE("disabled: no annex growth when expand_gallery is off", "[track_gallery]") {
|
|
// expand_gallery defaults to true (see config.hpp) as of the rep4 bake-off —
|
|
// set it explicitly false here since this test exercises the disabled path,
|
|
// not whatever the struct's current default happens to be.
|
|
Config cfg;
|
|
cfg.expand_gallery = false;
|
|
TrackGallery tg(cfg);
|
|
REQUIRE_FALSE(tg.enabled());
|
|
for (int f = 0; f < 10; ++f)
|
|
tg.observe(1, one_hot(1), /*actor*/ 0, /*sim*/ 0.2f, /*accept*/ true, kNoCrop);
|
|
CHECK(tg.annex_size() == 0);
|
|
}
|
|
|
|
// ── AR-018: the band ─────────────────────────────────────────────────────────
|
|
|
|
TEST_CASE("band bounds come from config, not a hardcoded default", "[track_gallery][AR-018]") {
|
|
// The bounds were declared in Config and read nowhere, so the gate ran at
|
|
// whatever the header happened to initialise. Drive them somewhere the
|
|
// defaults are not and require the gate to follow.
|
|
Config cfg = expand_cfg();
|
|
cfg.expand_band_lo = 0.40f;
|
|
cfg.expand_band_hi = 0.60f;
|
|
TrackGallery tg(cfg);
|
|
tg.set_calibration(identity_cal);
|
|
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
// P = 0.50: inside the configured band, far below the shipped default lo.
|
|
tg.observe(1, at_sim(0, 1, 0.50f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 0);
|
|
|
|
// P = 0.92: inside the shipped default band, above the configured ceiling.
|
|
tg.observe(1, at_sim(0, 2, 0.92f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 1);
|
|
}
|
|
|
|
TEST_CASE("band admits at each bound exactly", "[track_gallery][AR-018]") {
|
|
// The verification plan asks for the bounds themselves, not a point safely
|
|
// inside them: an off-by-one in the comparison is invisible anywhere else.
|
|
// Both bounds are inclusive.
|
|
SECTION("lower bound exactly") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(1, at_sim(0, 1, kBandLo), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 0);
|
|
}
|
|
SECTION("upper bound exactly") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(1, at_sim(0, 1, kBandHi), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 0);
|
|
}
|
|
}
|
|
|
|
TEST_CASE("store never admits below the lower bound", "[track_gallery][AR-018]") {
|
|
// The lower bound is the poisoning guard: an embedding unlike everything
|
|
// already on the track is evidence the track is not one person.
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
|
|
tg.observe(1, at_sim(0, 1, kBandLo - 0.01f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 1);
|
|
|
|
tg.observe(1, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal: P = 0
|
|
CHECK(tg.band_rejected() == 2);
|
|
}
|
|
|
|
TEST_CASE("store never admits above the upper bound", "[track_gallery][AR-018]") {
|
|
// The upper bound is the redundancy guard: another look at a pose the store
|
|
// already covers teaches the annex nothing and costs a slot.
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
|
|
tg.observe(1, at_sim(0, 1, kBandHi + 0.01f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 1);
|
|
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop); // identical: P = 1
|
|
CHECK(tg.band_rejected() == 2);
|
|
}
|
|
|
|
TEST_CASE("band thresholds probability, not cosine", "[track_gallery][AR-018][AR-024]") {
|
|
// The invariant's actual claim, and the one a raw-cosine gate passes by
|
|
// accident under an identity calibration. With a calibration that shifts by
|
|
// +0.10, two embeddings get the OPPOSITE verdict from the one their bare
|
|
// cosines would earn — so admission here can only come from the calibrated
|
|
// value having been used.
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration([](float c) { return c + 0.10f; });
|
|
|
|
tg.observe(1, one_hot(0), 0, 0.30f, true, kNoCrop);
|
|
|
|
// cosine 0.84 (below lo, would be refused raw) → P = 0.94, inside the band.
|
|
tg.observe(1, at_sim(0, 1, 0.84f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 0);
|
|
|
|
// cosine 0.92 (inside the band, would be admitted raw) → P = 1.02, above it.
|
|
tg.observe(1, at_sim(0, 2, 0.92f), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.band_rejected() == 1);
|
|
}
|
|
|
|
TEST_CASE("a two-person track never poisons the annex", "[track_gallery][AR-018]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
// Two orthogonal identities under one track ID — a track-ID collision.
|
|
// The band refuses the outsider at the door, so the store never becomes
|
|
// two-person in the first place.
|
|
tg.observe(3, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(3, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(3, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal outlier
|
|
|
|
CHECK(tg.band_rejected() == 1); // refused at the door
|
|
REQUIRE(tg.annex_size() > 0); // the legitimate views still promote
|
|
for (int i = 0; i < tg.annex_size(); ++i)
|
|
CHECK(cosine_similarity(annex_view(tg, i), one_hot(400)) < 0.5f);
|
|
}
|
|
|
|
TEST_CASE("a track that drifts through the band is refused at promotion",
|
|
"[track_gallery][AR-018]") {
|
|
// `admit` compares a newcomer against its CLOSEST existing member, so a
|
|
// gradual drift chains past it: each step is in-band while the endpoints are
|
|
// strangers. This is the shape a collision takes over a slow pan, and the
|
|
// reason the lower bound is re-asked across every pair before promotion.
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
|
|
tg.observe(5, on_circle(0.f), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(5, on_circle(25.f), 0, 0.30f, true, kNoCrop); // P=0.906 vs 0° → in band
|
|
tg.observe(5, on_circle(50.f), 0, 0.30f, true, kNoCrop); // P=0.906 vs 25° → in band
|
|
|
|
CHECK(tg.band_rejected() == 0); // every step passed the door...
|
|
// ...but 0° and 50° are P=0.643 apart, below the floor: the whole track goes.
|
|
CHECK(tg.annex_size() == 0);
|
|
}
|
|
|
|
// ── AR-019: ownership and promotion ──────────────────────────────────────────
|
|
|
|
TEST_CASE("confirmed track promotes its store", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
REQUIRE(tg.enabled());
|
|
|
|
// A track owned by actor 0: every frame accepted, every view mutually
|
|
// in-band (P = 0.9197 between distinct spokes) and gallery-far (0.30).
|
|
for (int k = 1; k <= 3; ++k)
|
|
tg.observe(7, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
|
|
// 3 accepted frames == min_anchor_frames → confirmed and promoted.
|
|
CHECK(tg.annex_size() == 3);
|
|
for (int actor : tg.annex_actors()) CHECK(actor == 0);
|
|
}
|
|
|
|
TEST_CASE("registry ownership overrides the local tally", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
// Local accepted-frame plurality says actor 5; the registry's accumulated
|
|
// posterior says actor 9. The registry is authoritative.
|
|
tg.set_owner(11, 9);
|
|
for (int k = 1; k <= 3; ++k)
|
|
tg.observe(11, spoke(k, kSpokeCos), 5, 0.30f, true, kNoCrop);
|
|
REQUIRE(tg.annex_size() > 0);
|
|
for (int actor : tg.annex_actors()) CHECK(actor == 9);
|
|
}
|
|
|
|
TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
// Only 2 accepted frames < min_anchor_frames 3; the third fills the buffer
|
|
// but doesn't count toward ownership.
|
|
tg.observe(4, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(4, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(4, spoke(3, kSpokeCos), 0, 0.30f, false, kNoCrop);
|
|
CHECK(tg.annex_size() == 0);
|
|
}
|
|
|
|
TEST_CASE("plurality actor wins a mixed-vote track", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
// No registry attached (unit-test path): actor 5 accepted twice, actor 6
|
|
// once → plurality is 5.
|
|
tg.observe(8, spoke(1, kSpokeCos), 5, 0.30f, true, kNoCrop);
|
|
tg.observe(8, spoke(2, kSpokeCos), 5, 0.30f, true, kNoCrop);
|
|
tg.observe(8, spoke(3, kSpokeCos), 6, 0.30f, true, kNoCrop);
|
|
REQUIRE(tg.annex_size() > 0);
|
|
for (int actor : tg.annex_actors()) CHECK(actor == 5);
|
|
}
|
|
|
|
TEST_CASE("promotion is idempotent across a long track", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
for (int k = 1; k <= 3; ++k)
|
|
tg.observe(9, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
const int after_confirm = tg.annex_size();
|
|
REQUIRE(after_confirm > 0);
|
|
// Keep feeding the confirmed track: annex must not grow again.
|
|
for (int f = 0; f < 10; ++f)
|
|
tg.observe(9, spoke(4 + f, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.annex_size() == after_confirm);
|
|
}
|
|
|
|
TEST_CASE("clear_tracks drops buffers before confirmation", "[track_gallery][AR-019]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
// Two accepts, then a cut clears buffers; the third accept starts fresh and
|
|
// can't reach the anchor threshold on its own.
|
|
tg.observe(1, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.observe(1, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
tg.clear_tracks();
|
|
tg.observe(1, spoke(3, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
CHECK(tg.annex_size() == 0);
|
|
}
|
|
|
|
TEST_CASE("eviction keeps the gallery-far views", "[track_gallery][AR-018]") {
|
|
// Novelty is no longer a threshold — it is this ordering. With the buffer
|
|
// full, a more gallery-far newcomer must displace the best-recognised
|
|
// member, and a less novel one must be dropped rather than displace a
|
|
// better sample.
|
|
Config cfg = expand_cfg();
|
|
cfg.expand_buffer_size = 2;
|
|
cfg.expand_min_anchor_frames = 4;
|
|
TrackGallery tg(cfg);
|
|
tg.set_calibration(identity_cal);
|
|
|
|
tg.observe(6, spoke(1, kSpokeCos), 0, 0.80f, true, kNoCrop); // well recognised
|
|
tg.observe(6, spoke(2, kSpokeCos), 0, 0.40f, true, kNoCrop);
|
|
tg.observe(6, spoke(3, kSpokeCos), 0, 0.20f, true, kNoCrop); // novel: evicts the 0.80
|
|
tg.observe(6, spoke(4, kSpokeCos), 0, 0.90f, true, kNoCrop); // least novel: dropped
|
|
|
|
REQUIRE(tg.annex_size() == 2);
|
|
// Survivors are the two most gallery-far views: spokes 2 and 3.
|
|
for (int i = 0; i < tg.annex_size(); ++i) {
|
|
const Embedding view = annex_view(tg, i);
|
|
const bool is_2 = cosine_similarity(view, spoke(2, kSpokeCos)) > 0.99f;
|
|
const bool is_3 = cosine_similarity(view, spoke(3, kSpokeCos)) > 0.99f;
|
|
CHECK((is_2 || is_3));
|
|
}
|
|
}
|
|
|
|
// TRACES: UT-005 | AR-026 | SR-001
|
|
// The annex reaches the GEMM path by being drained, not re-read: the matcher
|
|
// pushes newly promoted rows into the similarity engine once per frame. Draining
|
|
// must therefore be exactly-once — a row handed over twice becomes a duplicate
|
|
// gallery entry that quietly doubles an actor's best-of-N chances, and a row
|
|
// never handed over is a promotion that silently does nothing.
|
|
TEST_CASE("promotions drain exactly once, in matrix order",
|
|
"[track_gallery][AR-026]") {
|
|
TrackGallery tg(expand_cfg());
|
|
tg.set_calibration(identity_cal);
|
|
|
|
std::vector<float> emb;
|
|
std::vector<int> actor;
|
|
|
|
CHECK(tg.drain_promotions(emb, actor) == 0); // nothing promoted yet
|
|
|
|
for (int k = 1; k <= 3; ++k)
|
|
tg.observe(7, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
|
|
REQUIRE(tg.annex_size() == 3);
|
|
|
|
const int drained = tg.drain_promotions(emb, actor);
|
|
CHECK(drained == 3);
|
|
CHECK(actor.size() == 3);
|
|
CHECK(emb.size() == 3 * 512);
|
|
for (int a : actor) CHECK(a == 0);
|
|
|
|
// Drained rows are the annex rows, in the same order — the engine's row i
|
|
// and flat_actor_[i] have to keep naming the same view.
|
|
for (int i = 0; i < drained; ++i)
|
|
for (int d = 0; d < 512; ++d)
|
|
CHECK(emb[static_cast<size_t>(i) * 512 + d] == tg.annex_row(i)[d]);
|
|
|
|
// Draining again yields nothing: the engine already holds these.
|
|
CHECK(tg.drain_promotions(emb, actor) == 0);
|
|
CHECK(actor.size() == 3);
|
|
|
|
// A second track promotes, and only its rows are handed over.
|
|
for (int k = 1; k <= 3; ++k)
|
|
tg.observe(8, spoke(k, kSpokeCos), 4, 0.30f, true, kNoCrop);
|
|
CHECK(tg.drain_promotions(emb, actor) == 3);
|
|
CHECK(actor.size() == 6);
|
|
CHECK(actor[5] == 4);
|
|
}
|
|
|
|
// ── AR-024: the calibration is not optional ─────────────────────────────────
|
|
|
|
/// TRACES: UT-005 | AR-024 | SR-005
|
|
TEST_CASE("a null calibration is refused, not silently replaced", "[track_gallery][AR-024]") {
|
|
// The class used to default calibrate_ to max(0, cosine). That made
|
|
// expand_band_lo = 0.90 mean "cosine above 0.9" here and "P(same person)
|
|
// above 0.9" in production — two very different gates, with nothing
|
|
// announcing which one was in force. FaceTrackerFunc already refused to
|
|
// construct without a calibration for exactly this reason; the expansion
|
|
// store now matches it.
|
|
TrackGallery tg(expand_cfg());
|
|
CHECK_THROWS_AS(tg.set_calibration(nullptr), std::invalid_argument);
|
|
}
|