GR-004: bind galleries to the embedder that built them
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+28
-2
@@ -15,6 +15,7 @@
|
||||
|
||||
#include "types.hpp"
|
||||
#include "config.hpp"
|
||||
#include "gallery/embedder_stamp.hpp"
|
||||
#include "gallery/gallery_store.hpp"
|
||||
#include "nodes/face_tracker_node.hpp"
|
||||
#include "nodes/identity_matcher_node.hpp"
|
||||
@@ -163,6 +164,9 @@ static Config config_from_dict(nb::dict d) {
|
||||
getd("anneal_sec", cfg.anneal_sec);
|
||||
// gallery expansion (usually off for sweeps; expose so it can be toggled)
|
||||
if (d.contains("expand_gallery")) cfg.expand_gallery = nb::cast<bool>(d["expand_gallery"]);
|
||||
/// TRACES: GR-004 | SR-001
|
||||
if (d.contains("require_gallery_stamp"))
|
||||
cfg.require_gallery_stamp = nb::cast<bool>(d["require_gallery_stamp"]);
|
||||
return cfg;
|
||||
}
|
||||
|
||||
@@ -210,8 +214,17 @@ NB_MODULE(sae_kpn, m) {
|
||||
net.add(std::move(name), std::move(node));
|
||||
}, "net"_a, "name"_a, "config"_a, "capacity"_a = 16);
|
||||
|
||||
/// TRACES: GR-004 | SR-001
|
||||
// embedder_model / embedder_sha256 identify whatever produced the embeddings
|
||||
// that will be fed in. In a replay those come from the dump's own stamp (see
|
||||
// scripts/optimizer/SCHEMA.md), because there is no live embedder in the
|
||||
// network — the dump *is* the embedder as far as this gallery is concerned.
|
||||
// Passing neither leaves the binding unverifiable, which warns loudly and is
|
||||
// fatal under SAE_REQUIRE_GALLERY_STAMP.
|
||||
m.def("add_identity_matcher", [](Net& net, std::string name, std::string gallery_path,
|
||||
nb::dict cfg_dict, std::size_t cap) {
|
||||
nb::dict cfg_dict, std::size_t cap,
|
||||
std::string embedder_model,
|
||||
std::string embedder_sha256) {
|
||||
Config cfg = config_from_dict(cfg_dict);
|
||||
cfg.gallery_path = gallery_path; // needed to persist refreshed calibration back
|
||||
// Cache loaded galleries by path so a threshold sweep (many networks, same
|
||||
@@ -222,11 +235,24 @@ NB_MODULE(sae_kpn, m) {
|
||||
if (it == cache.end())
|
||||
it = cache.emplace(gallery_path,
|
||||
std::make_shared<ActorGallery>(load_gallery(gallery_path))).first;
|
||||
|
||||
// Checked on every construction, not only on the cache miss: the same
|
||||
// process may replay several dumps against one cached gallery.
|
||||
EmbedderStamp feeding;
|
||||
feeding.model_name = std::move(embedder_model);
|
||||
feeding.model_sha256 = std::move(embedder_sha256);
|
||||
enforce_embedder_stamp(it->second->embedder, feeding, gallery_path,
|
||||
feeding.model_name.empty()
|
||||
? "embeddings fed into this network"
|
||||
: feeding.model_name,
|
||||
cfg.require_gallery_stamp);
|
||||
|
||||
auto node = std::make_shared<kpn::ObjectVariantNodeWrapper<
|
||||
IdentityMatcherFunc, SaeVariant, kpn::in<"tracked">, kpn::out<"matched">>>(
|
||||
cap, *it->second, cfg);
|
||||
net.add(std::move(name), std::move(node));
|
||||
}, "net"_a, "name"_a, "gallery"_a, "config"_a, "capacity"_a = 16);
|
||||
}, "net"_a, "name"_a, "gallery"_a, "config"_a, "capacity"_a = 16,
|
||||
"embedder_model"_a = "", "embedder_sha256"_a = "");
|
||||
|
||||
m.def("add_scene_tracker", [](Net& net, std::string name, nb::dict cfg_dict, std::size_t cap) {
|
||||
Config cfg = config_from_dict(cfg_dict);
|
||||
|
||||
Reference in New Issue
Block a user