GR-004: bind galleries to the embedder that built them
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
#!/usr/bin/env python3
|
||||
"""stamp_gallery.py — bind an existing gallery to the embedder that built it.
|
||||
|
||||
TRACES: GR-004 | SR-001
|
||||
|
||||
Galleries built before model binding carry no embedder stamp. They still load,
|
||||
but every consumer warns that it cannot tell whether the gallery and the embedder
|
||||
belong together — and under SAE_REQUIRE_GALLERY_STAMP=1 they refuse to run.
|
||||
|
||||
This is the migration path, and the reason the unstamped case is a warning rather
|
||||
than a hard failure: re-binding an existing gallery costs one command and no
|
||||
re-embedding, so nobody has to choose between a bricked setup and a check they
|
||||
route around.
|
||||
|
||||
python scripts/stamp_gallery.py --gallery gallery.h5 \\
|
||||
--arcface models/LVFace-B_Glint360K.onnx
|
||||
|
||||
The stamp is an ASSERTION: you are stating which model produced these vectors.
|
||||
Nothing can verify it from the vectors themselves, which is exactly why the stamp
|
||||
has to be written at build time going forward. Stamping the wrong model is worse
|
||||
than leaving it unstamped, because it converts a loud warning into a false
|
||||
all-clear — so --show it first if you are not certain.
|
||||
|
||||
python scripts/stamp_gallery.py --gallery gallery.h5 --show
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import h5py
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from sae_gallery import (describe_stamp, embedder_stamp, # noqa: E402
|
||||
read_gallery_stamp)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
p.add_argument("--gallery", required=True, help="gallery .h5 to stamp in place")
|
||||
p.add_argument("--arcface", help="the ONNX that built it (hashed into the stamp)")
|
||||
p.add_argument("--show", action="store_true", help="print the current stamp and exit")
|
||||
p.add_argument("--force", action="store_true",
|
||||
help="overwrite an existing stamp (refused otherwise)")
|
||||
args = p.parse_args()
|
||||
|
||||
path = Path(args.gallery)
|
||||
if path.suffix not in (".h5", ".hdf5"):
|
||||
return err(f"{path}: only HDF5 galleries can be stamped in place")
|
||||
|
||||
current = read_gallery_stamp(path)
|
||||
print(f"{path}: current stamp = {describe_stamp(current)}", file=sys.stderr)
|
||||
if args.show:
|
||||
return 0
|
||||
if not args.arcface:
|
||||
return err("--arcface is required (or use --show)")
|
||||
if current and not args.force:
|
||||
return err("gallery is already stamped — pass --force to overwrite, but be "
|
||||
"sure: a wrong stamp turns a warning into a false all-clear")
|
||||
|
||||
stamp = embedder_stamp(args.arcface)
|
||||
if not stamp["model_sha256"]:
|
||||
return err(f"cannot hash {args.arcface} — refusing to write a name-only "
|
||||
"stamp, which would claim more certainty than it has")
|
||||
|
||||
with h5py.File(path, "r+") as f:
|
||||
if "embedder" in f:
|
||||
del f["embedder"]
|
||||
g = f.create_group("embedder")
|
||||
g.attrs["model_name"] = stamp["model_name"]
|
||||
g.attrs["model_sha256"] = stamp["model_sha256"]
|
||||
g.attrs["embed_dim"] = stamp["embed_dim"]
|
||||
|
||||
print(f"{path}: stamped with {describe_stamp(stamp)}", file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
def err(msg: str) -> int:
|
||||
print(f"[stamp_gallery] {msg}", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in New Issue
Block a user