GR-004: bind galleries to the embedder that built them
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,115 @@
|
||||
#pragma once
|
||||
/// TRACES: GR-004 | SR-001
|
||||
//
|
||||
// Gallery ↔ embedder binding.
|
||||
//
|
||||
// A gallery is only valid for the embedder that built it. Cosine similarities
|
||||
// between embeddings from two different models are meaningless but *look*
|
||||
// plausible — nothing crashes, nothing is obviously wrong, and every number
|
||||
// measured downstream is quietly garbage. So the embedder's identity is stamped
|
||||
// into the gallery at build time and checked by every consumer at load time.
|
||||
//
|
||||
// ── What identifies an embedder ───────────────────────────────────────────────
|
||||
// Two fields, carried together:
|
||||
//
|
||||
// model_name basename of the model file, e.g. "LVFace-B_Glint360K.onnx"
|
||||
// model_sha256 hex SHA-256 of that file's bytes
|
||||
//
|
||||
// The hash is what *decides*; the name is what a human *reads*. Neither alone is
|
||||
// enough:
|
||||
//
|
||||
// • A name alone is a promise, not a fact. Models get re-exported, re-quantised
|
||||
// and overwritten in place under an unchanged filename — which is precisely
|
||||
// the case where the weights differ and nothing else does. A name-only stamp
|
||||
// is blind to exactly the failure it exists to catch.
|
||||
// • A hash alone is correct but unreadable: "expected 3f2a… got 9c1b…" tells an
|
||||
// operator nothing about what to do next.
|
||||
//
|
||||
// SHA-256 over the file bytes is derived from the artefact rather than asserted
|
||||
// about it, is stable across machines and filesystems, and needs no registry to
|
||||
// be kept up to date. Cost is ~0.1 s for a 250 MB ONNX, paid once per process
|
||||
// (results are memoised on path+mtime+size), which is noise next to model load.
|
||||
//
|
||||
// ── Degraded and legacy cases ─────────────────────────────────────────────────
|
||||
// A TRT-backend deployment may run from a prebuilt .engine with the source .onnx
|
||||
// absent, so the hash cannot be computed. Then the name is compared alone and the
|
||||
// result is reported as a *weak* match — believed, not proven.
|
||||
//
|
||||
// Galleries built before GR-004 carry no stamp at all. They warn loudly rather
|
||||
// than fail, because the state is unknown rather than known-bad, and because
|
||||
// hard-failing every pre-existing gallery would make the check something people
|
||||
// route around rather than trust. Set require_stamp (or SAE_REQUIRE_GALLERY_STAMP=1)
|
||||
// to promote "unknown" to a hard error — that is the mode measurement work runs in.
|
||||
//
|
||||
// A *mismatch* is always fatal, in every mode, with no bypass.
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
|
||||
struct EmbedderStamp {
|
||||
std::string model_name; // basename of the model file
|
||||
std::string model_sha256; // lowercase hex SHA-256 of the file's bytes ("" = unavailable)
|
||||
int32_t embed_dim{512};
|
||||
|
||||
bool empty() const { return model_name.empty() && model_sha256.empty(); }
|
||||
|
||||
// "LVFace-B_Glint360K.onnx (sha256 3f2a1c4d…)" — for error messages.
|
||||
std::string describe() const;
|
||||
};
|
||||
|
||||
// Identify the model at `model_path`. Missing/unreadable file → name filled from
|
||||
// the path, hash left empty (the weak-match path). Empty path → empty stamp.
|
||||
EmbedderStamp make_embedder_stamp(const std::string& model_path);
|
||||
|
||||
enum class StampVerdict {
|
||||
match, // hashes agree — binding proven
|
||||
weak_match, // names agree, no hash on one side — believed, unproven
|
||||
unstamped, // gallery predates GR-004 / was written without a stamp
|
||||
unknown_embedder, // gallery is stamped but the loaded embedder can't be identified
|
||||
mismatch, // proven different models — always fatal
|
||||
};
|
||||
|
||||
struct StampCheck {
|
||||
StampVerdict verdict{StampVerdict::match};
|
||||
std::string message; // human-readable, names BOTH sides
|
||||
|
||||
// A mismatch is fatal unconditionally. The three "cannot prove it" verdicts
|
||||
// are fatal only in strict mode.
|
||||
bool fatal(bool require_stamp) const {
|
||||
return verdict == StampVerdict::mismatch ||
|
||||
(require_stamp && verdict != StampVerdict::match);
|
||||
}
|
||||
};
|
||||
|
||||
// Pure comparison — no file I/O, no model loading. This is the unit under test.
|
||||
// `gallery_desc`/`embedder_desc` are only used to make the message locatable
|
||||
// (a gallery path, a dump path, "the embedder being loaded", …).
|
||||
StampCheck compare_embedder_stamps(const EmbedderStamp& built_with,
|
||||
const EmbedderStamp& loading_with,
|
||||
const std::string& gallery_desc = "gallery",
|
||||
const std::string& embedder_desc = "embedder");
|
||||
|
||||
// Apply the comparison: throw std::runtime_error on a fatal verdict, otherwise
|
||||
// log to stderr. `require_stamp` is OR-ed with SAE_REQUIRE_GALLERY_STAMP.
|
||||
void enforce_embedder_stamp(const EmbedderStamp& built_with,
|
||||
const EmbedderStamp& loading_with,
|
||||
const std::string& gallery_desc,
|
||||
const std::string& embedder_desc,
|
||||
bool require_stamp);
|
||||
|
||||
// Convenience for the common consumer shape: "I loaded this gallery and I am
|
||||
// about to embed with this model file." Hashes the model, then enforces.
|
||||
struct ActorGallery;
|
||||
void verify_gallery_embedder(const ActorGallery& gallery,
|
||||
const std::string& gallery_path,
|
||||
const std::string& arcface_model_path,
|
||||
bool require_stamp);
|
||||
|
||||
// SAE_REQUIRE_GALLERY_STAMP=1 → treat an unprovable binding as fatal.
|
||||
bool require_gallery_stamp_from_env();
|
||||
|
||||
// Lowercase hex SHA-256. Exposed so a test can pin the digest against the
|
||||
// published vectors, which is what guarantees the C++ and Python (hashlib)
|
||||
// stamps of the same file agree.
|
||||
std::string sha256_hex(const std::string& bytes);
|
||||
std::string sha256_file_hex(const std::string& path); // "" if unreadable
|
||||
Reference in New Issue
Block a user