Claude
7db40f430d
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
62 lines
2.5 KiB
Python
62 lines
2.5 KiB
Python
"""Shared loader for the sae_embed nanobind module (SCRFD + ArcFace).
|
|
|
|
sae_embed.FaceEmbedder loads both ONNX sessions once and exposes an
|
|
embed(path) -> FaceResult method, avoiding the per-process model reload cost
|
|
of spawning the embed_faces CLI binary for every image.
|
|
|
|
resolve_arcface() exposes the same default-resolution logic load_embedder uses,
|
|
so a caller can stamp the gallery it is about to write with the model that
|
|
actually produced its embeddings (GR-004) — the resolved path, not the CLI
|
|
argument, which is often None.
|
|
"""
|
|
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
DEFAULT_ARCFACE = "arcface_w600k_r50.onnx"
|
|
|
|
|
|
def resolve_arcface(models_dir: str, arcface: str | None = None) -> str:
|
|
"""The ArcFace/LVFace ONNX path load_embedder would use for these arguments.
|
|
|
|
TRACES: GR-004 | SR-001 — single source of truth for "which model is this",
|
|
so the stamp written into a gallery can never drift from the model loaded."""
|
|
return arcface if arcface else str(Path(models_dir) / DEFAULT_ARCFACE)
|
|
|
|
|
|
def load_embedder(build_dir: str, models_dir: str, arcface: str | None = None,
|
|
conf: float = 0.5, nms: float = 0.4, max_side: int = 500):
|
|
"""Import sae_embed from build_dir and construct a FaceEmbedder.
|
|
|
|
Exits with a clear error if the module or models are missing — there is
|
|
no subprocess fallback.
|
|
"""
|
|
build_path = Path(build_dir).resolve()
|
|
sys.path.insert(0, str(build_path))
|
|
try:
|
|
import sae_embed
|
|
except ImportError as e:
|
|
sys.exit(
|
|
f"sae_embed module not found in {build_path}: {e}\n"
|
|
f"Build it first: cmake --build {build_dir} --target sae_embed"
|
|
)
|
|
|
|
models_path = Path(models_dir)
|
|
detector_path = str(models_path / "scrfd_500m_bnkps.onnx")
|
|
arcface_path = resolve_arcface(models_dir, arcface)
|
|
for model, name in [(detector_path, "SCRFD"), (arcface_path, "ArcFace")]:
|
|
if not Path(model).is_file():
|
|
sys.exit(f"{name} model not found: {model}\nRun: bash scripts/download_models.sh")
|
|
|
|
# A TRT-backend build cannot load .onnx; it needs pre-built engines from
|
|
# scripts/build_trt_engines.sh. Pass them when present (ignored by ORT).
|
|
trt = Path(models_path).parent / "trt_cache"
|
|
det_engine = trt / "scrfd.scrfd_500m_bnkps.640.fp16.engine"
|
|
arc_engine = trt / f"arcface.{Path(arcface_path).stem}.b4.fp16.engine"
|
|
|
|
return sae_embed.FaceEmbedder(
|
|
detector_path, arcface_path, conf, nms, max_side,
|
|
str(det_engine) if det_engine.is_file() else "",
|
|
str(arc_engine) if arc_engine.is_file() else "",
|
|
)
|