dtourolle/scene-actor-extraction
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(gemm): the annex is a matrix, not a list — scored by the same GEMM

Browse Source 
The per-film annex was folded in after the gallery multiply by a host-side
cosine loop over a vector of {embedding, actor} structs, justified in-comment
by "tens of embeddings". AR-018/AR-019 retired that assumption: every owned
track promotes, so the annex grows with cast size and film length.

TrackGallery now holds it as a contiguous row-major matrix with a parallel
actor index — the flat_emb_/flat_actor_ shape the baked gallery already uses —
and hands newly promoted rows to the matcher once per frame. The matcher pushes
them into the similarity engine's resident matrix through a new
ISimilarityEngine::append_rows, so one SGEMM covers baked and promoted
references alike and best-of-N is a single pass over one similarity column.
Capacity doubles on overflow, and the GPU backends grow device-to-device, so a
promotion never re-uploads the gallery across the bus.

Absorbing promotions runs once per frame, after every face has been scored.
Appending mid-frame would invalidate the similarity pointer the chunk loop is
still reading, and it also removes an incidental dependence on face order
within a frame — a promotion helps subsequent frames, never the one that
produced it, which is the semantics the expansion store already documented.

OpenBLAS becomes a requirement of the CPU GEMM backend rather than an
opportunistic upgrade. That path is what CI and the cpu builder image run, so
falling back to the scalar loop in silence meant AR-027 could be measured — or
believed — on a kernel no release uses. The loop survives as the correctness
oracle the BLAS backends are diffed against, behind SAE_ALLOW_SCALAR_GEMM.

Call site 3, the deferred TBI pass, is untouched: it does not exist until
AR-020, so AR-026 stays In Progress.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: AR-026 | UT-004, UT-005 | SR-001
This commit is contained in:
dtourolle
2026-08-04 21:20:31 +02:00
co-authored by Claude Opus 5
parent f33403fff8
commit c1155cb607
11 changed files with 461 additions and 81 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/CMakeLists.txt
+8
View File
@@ -46,6 +46,14 @@ endif()
if(OPENBLAS_T_FOUND)
target_include_directories(sae_tests PRIVATE ${OPENBLAS_T_INCLUDE_DIRS})
target_link_libraries(sae_tests PRIVATE ${OPENBLAS_T_LINK_LIBRARIES})
elseif(NOT SAE_ALLOW_SCALAR_GEMM)
# Same rule as the CPU backend itself: testing the scalar loop while the
# shipped CPU path is OpenBLAS means the suite is not evidence about the
# kernel that runs.
message(FATAL_ERROR
"OpenBLAS not found, and the unit tests compile the CPU GEMM kernel "
"(AR-026). Install openblas-devel, or pass -DSAE_ALLOW_SCALAR_GEMM=ON "
"to test the scalar fallback deliberately.")
endif()
target_compile_definitions(sae_tests PRIVATE
tests/test_similarity.cpp
+102 -1
View File
@@ -1,4 +1,4 @@
// TRACES: AR-026 | SR-001
// TRACES: UT-004 | AR-026 | SR-001
//
// Unit tests for the CPU reference similarity engine (backends/gemm_backend.cpp,
// SAE_GEMM_CPU) and the l2_normalise helper. All pure, GPU-free, model-free.
@@ -82,6 +82,107 @@ TEST_CASE("CPU similarity engine matches hand-computed dot products", "[similari
CHECK_THAT(S[2 + 1 * n_gallery], WithinAbs(s, 1e-6f));
}
// TRACES: UT-004 | AR-026 | SR-001
// The annex half of AR-026: promoted rows are appended to the resident matrix
// and scored by the same GEMM as the baked references. What used to be a
// host-side cosine loop over TrackGallery::annex() is now these extra columns,
// so the equivalence that matters is "an appended row scores exactly what the
// reference dot product says", and "appending changes nothing about the rows
// already there".
TEST_CASE("appended rows are scored by the same GEMM as the baked gallery",
"[similarity][AR-026]") {
std::vector<float> gallery;
for (int slot : {0, 1}) {
auto e = one_hot(slot);
gallery.insert(gallery.end(), e.begin(), e.end());
}
auto engine = make_similarity_engine(gallery.data(), /*n_gallery=*/2, /*max_faces=*/2);
REQUIRE(engine->n_gallery() == 2);
// One query face at 45° between slots 1 and 2. Slot 2 is not in the baked
// gallery yet, so the face is currently "unrecognised at that pose".
const float s = std::sqrt(0.5f);
std::vector<float> query(static_cast<size_t>(2) * 512, 0.0f);
query[1] = s;
query[2] = s;
const float* before = engine->compute(query.data(), 1);
CHECK_THAT(before[0], WithinAbs(0.0f, 1e-6f)); // vs slot 0
CHECK_THAT(before[1], WithinAbs(s, 1e-6f)); // vs slot 1
// Promote the missing view — the annex row an owned track would contribute.
auto promoted = one_hot(2);
engine->append_rows(promoted.data(), 1);
REQUIRE(engine->n_gallery() == 3);
const float* after = engine->compute(query.data(), 1);
CHECK_THAT(after[0], WithinAbs(0.0f, 1e-6f)); // baked rows unchanged
CHECK_THAT(after[1], WithinAbs(s, 1e-6f));
CHECK_THAT(after[2], WithinAbs(s, 1e-6f)); // appended row, same multiply
}
TEST_CASE("appending many rows keeps every similarity exact", "[similarity][AR-026]") {
// Start from a one-row gallery and append past the initial capacity several
// times over — the growth path has to preserve what is already resident, and
// a film promotes far more rows than the gallery starts with.
auto seed = one_hot(0);
auto engine = make_similarity_engine(seed.data(), /*n_gallery=*/1, /*max_faces=*/1);
constexpr int kAppended = 40;
for (int i = 1; i <= kAppended; ++i) {
auto e = one_hot(i);
engine->append_rows(e.data(), 1);
}
REQUIRE(engine->n_gallery() == kAppended + 1);
// Query one-hot slot k: similarity is 1 against row k and 0 against all others.
for (int k : {0, 1, 17, kAppended}) {
auto q = one_hot(k);
const float* S = engine->compute(q.data(), 1);
for (int g = 0; g <= kAppended; ++g)
CHECK_THAT(S[g], WithinAbs(g == k ? 1.0f : 0.0f, 1e-6f));
}
}
TEST_CASE("appending a block of rows matches appending them one at a time",
"[similarity][AR-026]") {
// A promotion hands over a whole diversity buffer at once; that must be
// indistinguishable from the same rows arriving singly.
auto seed = one_hot(0);
std::vector<float> block;
for (int slot : {1, 2, 3}) {
auto e = one_hot(slot);
block.insert(block.end(), e.begin(), e.end());
}
auto bulk = make_similarity_engine(seed.data(), 1, 1);
bulk->append_rows(block.data(), 3);
auto singly = make_similarity_engine(seed.data(), 1, 1);
for (int i = 0; i < 3; ++i) singly->append_rows(block.data() + i * 512, 1);
REQUIRE(bulk->n_gallery() == singly->n_gallery());
const float t = std::sqrt(1.0f / 3.0f);
std::array<float, 512> q{};
q[1] = t; q[2] = t; q[3] = t;
const float* a = bulk->compute(q.data(), 1);
std::vector<float> a_copy(a, a + bulk->n_gallery());
const float* b = singly->compute(q.data(), 1);
for (int g = 0; g < bulk->n_gallery(); ++g)
CHECK_THAT(a_copy[g], WithinAbs(b[g], 1e-6f));
}
TEST_CASE("appending zero rows is a no-op", "[similarity][AR-026]") {
auto e = one_hot(0);
auto engine = make_similarity_engine(e.data(), 1, 1);
CHECK_NOTHROW(engine->append_rows(nullptr, 0));
CHECK(engine->n_gallery() == 1);
}
TEST_CASE("CPU similarity engine rejects too many faces", "[similarity]") {
auto e = one_hot(0);
auto engine = make_similarity_engine(e.data(), /*n_gallery=*/1, /*max_faces=*/1);
tests/test_track_gallery.cpp
+78 -20
View File
@@ -1,4 +1,4 @@
// TRACES: AR-018, AR-019, AR-024 | SR-005
// TRACES: UT-005 | AR-018, AR-019, AR-024, AR-026 | SR-005, SR-001
//
// Unit tests for TrackGallery (gallery/track_gallery.hpp): per-film gallery
// expansion driven by track continuity. Pure, GPU-free, model-free — exercises
@@ -18,8 +18,10 @@
#include "gallery/track_gallery.hpp"
#include "types.hpp"
#include <algorithm>
#include <array>
#include <cmath>
#include <vector>
namespace {
@@ -43,6 +45,17 @@ Embedding one_hot(int slot) {
return e;
}
// TRACES: AR-026 | SR-001
// The annex is a contiguous row-major matrix, not a vector of structs, so that
// the matcher can hand whole blocks of new rows to the GEMM path. Tests that
// want to compare one promoted view read it back through this.
Embedding annex_view(const TrackGallery& tg, int row) {
const float* p = tg.annex_row(row);
Embedding e{};
std::copy(p, p + 512, e.begin());
return e;
}
// Unit-norm embedding in the plane of axes i,j at cosine `cos_t` from axis i.
// Cosine sim to one_hot(i) is exactly cos_t.
Embedding at_sim(int i, int j, float cos_t) {
@@ -95,7 +108,7 @@ TEST_CASE("disabled: no annex growth when expand_gallery is off", "[track_galler
REQUIRE_FALSE(tg.enabled());
for (int f = 0; f < 10; ++f)
tg.observe(1, one_hot(1), /*actor*/ 0, /*sim*/ 0.2f, /*accept*/ true, kNoCrop);
CHECK(tg.annex().empty());
CHECK(tg.annex_size() == 0);
}
// ── AR-018: the band ─────────────────────────────────────────────────────────
@@ -199,9 +212,9 @@ TEST_CASE("a two-person track never poisons the annex", "[track_gallery][AR-018]
tg.observe(3, one_hot(400), 0, 0.30f, true, kNoCrop); // orthogonal outlier
CHECK(tg.band_rejected() == 1); // refused at the door
REQUIRE_FALSE(tg.annex().empty()); // the legitimate views still promote
for (const auto& e : tg.annex())
CHECK(cosine_similarity(e.emb, one_hot(400)) < 0.5f);
REQUIRE(tg.annex_size() > 0); // the legitimate views still promote
for (int i = 0; i < tg.annex_size(); ++i)
CHECK(cosine_similarity(annex_view(tg, i), one_hot(400)) < 0.5f);
}
TEST_CASE("a track that drifts through the band is refused at promotion",
@@ -219,7 +232,7 @@ TEST_CASE("a track that drifts through the band is refused at promotion",
CHECK(tg.band_rejected() == 0); // every step passed the door...
// ...but 0° and 50° are P=0.643 apart, below the floor: the whole track goes.
CHECK(tg.annex().empty());
CHECK(tg.annex_size() == 0);
}
// ── AR-019: ownership and promotion ──────────────────────────────────────────
@@ -235,8 +248,8 @@ TEST_CASE("confirmed track promotes its store", "[track_gallery][AR-019]") {
tg.observe(7, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
// 3 accepted frames == min_anchor_frames → confirmed and promoted.
CHECK(tg.annex().size() == 3);
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 0);
CHECK(tg.annex_size() == 3);
for (int actor : tg.annex_actors()) CHECK(actor == 0);
}
TEST_CASE("registry ownership overrides the local tally", "[track_gallery][AR-019]") {
@@ -247,8 +260,8 @@ TEST_CASE("registry ownership overrides the local tally", "[track_gallery][AR-01
tg.set_owner(11, 9);
for (int k = 1; k <= 3; ++k)
tg.observe(11, spoke(k, kSpokeCos), 5, 0.30f, true, kNoCrop);
REQUIRE_FALSE(tg.annex().empty());
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 9);
REQUIRE(tg.annex_size() > 0);
for (int actor : tg.annex_actors()) CHECK(actor == 9);
}
TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_gallery][AR-019]") {
@@ -259,7 +272,7 @@ TEST_CASE("unconfirmed track (too few accepts) does not promote", "[track_galler
tg.observe(4, spoke(1, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(4, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.observe(4, spoke(3, kSpokeCos), 0, 0.30f, false, kNoCrop);
CHECK(tg.annex().empty());
CHECK(tg.annex_size() == 0);
}
TEST_CASE("plurality actor wins a mixed-vote track", "[track_gallery][AR-019]") {
@@ -270,8 +283,8 @@ TEST_CASE("plurality actor wins a mixed-vote track", "[track_gallery][AR-019]")
tg.observe(8, spoke(1, kSpokeCos), 5, 0.30f, true, kNoCrop);
tg.observe(8, spoke(2, kSpokeCos), 5, 0.30f, true, kNoCrop);
tg.observe(8, spoke(3, kSpokeCos), 6, 0.30f, true, kNoCrop);
REQUIRE_FALSE(tg.annex().empty());
for (const auto& ae : tg.annex()) CHECK(ae.actor_idx == 5);
REQUIRE(tg.annex_size() > 0);
for (int actor : tg.annex_actors()) CHECK(actor == 5);
}
TEST_CASE("promotion is idempotent across a long track", "[track_gallery][AR-019]") {
@@ -279,12 +292,12 @@ TEST_CASE("promotion is idempotent across a long track", "[track_gallery][AR-019
tg.set_calibration(identity_cal);
for (int k = 1; k <= 3; ++k)
tg.observe(9, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
size_t after_confirm = tg.annex().size();
const int after_confirm = tg.annex_size();
REQUIRE(after_confirm > 0);
// Keep feeding the confirmed track: annex must not grow again.
for (int f = 0; f < 10; ++f)
tg.observe(9, spoke(4 + f, kSpokeCos), 0, 0.30f, true, kNoCrop);
CHECK(tg.annex().size() == after_confirm);
CHECK(tg.annex_size() == after_confirm);
}
TEST_CASE("clear_tracks drops buffers before confirmation", "[track_gallery][AR-019]") {
@@ -296,7 +309,7 @@ TEST_CASE("clear_tracks drops buffers before confirmation", "[track_gallery][AR-
tg.observe(1, spoke(2, kSpokeCos), 0, 0.30f, true, kNoCrop);
tg.clear_tracks();
tg.observe(1, spoke(3, kSpokeCos), 0, 0.30f, true, kNoCrop);
CHECK(tg.annex().empty());
CHECK(tg.annex_size() == 0);
}
TEST_CASE("eviction keeps the gallery-far views", "[track_gallery][AR-018]") {
@@ -315,11 +328,56 @@ TEST_CASE("eviction keeps the gallery-far views", "[track_gallery][AR-018]") {
tg.observe(6, spoke(3, kSpokeCos), 0, 0.20f, true, kNoCrop); // novel: evicts the 0.80
tg.observe(6, spoke(4, kSpokeCos), 0, 0.90f, true, kNoCrop); // least novel: dropped
REQUIRE(tg.annex().size() == 2);
REQUIRE(tg.annex_size() == 2);
// Survivors are the two most gallery-far views: spokes 2 and 3.
for (const auto& ae : tg.annex()) {
const bool is_2 = cosine_similarity(ae.emb, spoke(2, kSpokeCos)) > 0.99f;
const bool is_3 = cosine_similarity(ae.emb, spoke(3, kSpokeCos)) > 0.99f;
for (int i = 0; i < tg.annex_size(); ++i) {
const Embedding view = annex_view(tg, i);
const bool is_2 = cosine_similarity(view, spoke(2, kSpokeCos)) > 0.99f;
const bool is_3 = cosine_similarity(view, spoke(3, kSpokeCos)) > 0.99f;
CHECK((is_2 || is_3));
}
}
// TRACES: UT-005 | AR-026 | SR-001
// The annex reaches the GEMM path by being drained, not re-read: the matcher
// pushes newly promoted rows into the similarity engine once per frame. Draining
// must therefore be exactly-once — a row handed over twice becomes a duplicate
// gallery entry that quietly doubles an actor's best-of-N chances, and a row
// never handed over is a promotion that silently does nothing.
TEST_CASE("promotions drain exactly once, in matrix order",
"[track_gallery][AR-026]") {
TrackGallery tg(expand_cfg());
tg.set_calibration(identity_cal);
std::vector<float> emb;
std::vector<int> actor;
CHECK(tg.drain_promotions(emb, actor) == 0); // nothing promoted yet
for (int k = 1; k <= 3; ++k)
tg.observe(7, spoke(k, kSpokeCos), 0, 0.30f, true, kNoCrop);
REQUIRE(tg.annex_size() == 3);
const int drained = tg.drain_promotions(emb, actor);
CHECK(drained == 3);
CHECK(actor.size() == 3);
CHECK(emb.size() == 3 * 512);
for (int a : actor) CHECK(a == 0);
// Drained rows are the annex rows, in the same order — the engine's row i
// and flat_actor_[i] have to keep naming the same view.
for (int i = 0; i < drained; ++i)
for (int d = 0; d < 512; ++d)
CHECK(emb[static_cast<size_t>(i) * 512 + d] == tg.annex_row(i)[d]);
// Draining again yields nothing: the engine already holds these.
CHECK(tg.drain_promotions(emb, actor) == 0);
CHECK(actor.size() == 3);
// A second track promotes, and only its rows are handed over.
for (int k = 1; k <= 3; ++k)
tg.observe(8, spoke(k, kSpokeCos), 4, 0.30f, true, kNoCrop);
CHECK(tg.drain_promotions(emb, actor) == 3);
CHECK(actor.size() == 6);
CHECK(actor[5] == 4);
}