feat: bind galleries to the embedder that built them

GR-004 — a gallery built with one embedding model is meaningless with another.
Cosine similarities across models are garbage but look entirely plausible, so
this fails silently and expensively; every measurement taken against a
mismatched pair would have been quietly wrong.

The stamp is the model basename plus a SHA-256 of its bytes, with embed_dim as
a cheap extra guard. The hash decides and the name explains, because neither
works alone: a name is a promise rather than a fact — models get re-exported in
place under an unchanged filename, which is exactly the case where the weights
differ and nothing else does — while a bare hash mismatch tells an operator
nothing actionable.

Mismatch is fatal in every mode with no bypass. Unstamped only warns, because
unstamped is unknown rather than known-bad, and an error firing on every legacy
gallery trains people to reach for the bypass reflexively. scripts/stamp_gallery.py
binds an existing gallery in place with no re-embedding, so the warning is a
migration step rather than a permanent state; --require-gallery-stamp promotes
it to an error once a site has migrated.

Two gaps found that would have defeated the requirement outright:

- Embedding dumps carried no stamp, so a replay — which has no live embedder —
  had nothing to check the gallery against. Dumps now carry embedder_model and
  embedder_sha256 as root attributes. Additive; schema_version stays 1. This is
  the same gap the dump audit identified independently.
- --merge produced one file holding two embedding spaces, which no later check
  can untangle. Merge paths now verify before writing.

The stamp also survives identity_matcher's calibration write-back, which would
otherwise have stripped it on the first analysis run — the check would have
worked exactly once.

Conflicts resolved additively: both branches appended a source to sae_gallery
and to the test target, and both edited the GR-004 register row.

Merged suite: 64 cases, 3199 assertions, passing on CPU with no GPU.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

TRACES: GR-004, VR-001 | SR-001
This commit is contained in:
2026-07-30 19:04:07 +02:00
co-authored by Claude Opus 5
30 changed files with 1391 additions and 40 deletions
+51 -2
View File
@@ -1083,8 +1083,57 @@ surfaced as a build report.
different models are meaningless but *look* plausible — this fails silently and
expensively otherwise.
**Gap:** named as step 4 of the `service-conversion.md` implementation plan;
unbuilt. This is the highest-value small fix in the document.
### The stamp
Two fields, written together: the model file's **basename** and the **SHA-256 of
its bytes** (plus `embed_dim` as a cheap extra guard). Stored as the `/embedder`
group in the gallery HDF5, and as an optional top-level `"embedder"` object in
the legacy JSON format.
The hash *decides*; the name is what a human *reads*. Neither alone is enough. A
name is a promise rather than a fact — models get re-exported, re-quantised and
overwritten in place under an unchanged filename, which is exactly the case where
the weights differ and nothing else does, so a name-only stamp is blind to the
failure it exists to catch. A hash alone is correct but unactionable: *"expected
3f2a…, got 9c1b…"* tells an operator nothing about what to do next. SHA-256 over
the file is derived from the artefact rather than asserted about it, needs no
registry kept up to date, and costs ~0.1 s for a 250 MB ONNX once per process.
### Verdicts
| Verdict | When | Default | Under strict mode |
|---|---|---|---|
| `match` | hashes agree | proceed | proceed |
| `weak_match` | names agree, one side unhashable | **warn** | **error** |
| `unstamped` | gallery predates GR-004 | **warn** | **error** |
| `unknown_embedder` | gallery stamped, embedder unidentifiable | **warn** | **error** |
| `mismatch` | proven different models | **error** | **error** |
**A mismatch is fatal in every mode, with no bypass**, and the message names both
sides — what the gallery was built with and what is loaded.
The three "cannot prove it" verdicts warn loudly instead, because they describe an
*unknown* state rather than a *known-bad* one, and because every gallery built
before this requirement is unstamped. Hard-failing all of them would make the
check something people route around rather than trust. Strict mode
(`--require-gallery-stamp`, or `SAE_REQUIRE_GALLERY_STAMP=1`, which propagates to
subprocesses) promotes them to errors — that is the mode measurement work runs in.
`scripts/stamp_gallery.py` re-binds an existing gallery without re-embedding, so
migration costs one command; that is what makes "warn" a temporary state rather
than a permanent one.
### Scope of the check
Embedding **dumps** carry the same stamp (`embedder_model` / `embedder_sha256`
root attributes, `scripts/optimizer/SCHEMA.md`): a replay has no live embedder, so
the dump *is* the embedder as far as the gallery is concerned. Derived galleries
(filter, cast-restrict) inherit their source's stamp; `--merge` and the JSON
gallery merge check *before* writing, since a merged file holding two embedding
spaces cannot be untangled afterwards by any later check.
**Gap:** none. Stamped in `gallery_builder.cpp` and the Python builders; verified
in `scene_analyze`, `scene_preview`, the `sae_kpn` matcher binding, `replay.py`,
`optimize.py`, `movienet_eval.py` and the merge paths.
## GR-006 … GR-009 — Provenance tiers and poisoning guard
+15 -5
View File
@@ -260,13 +260,23 @@ Context crops opt-in behind `--dump-unidentified-crops`.
# Gallery
## GR-004 — Model binding
## GR-004 — Model binding — **DONE**
**Depends on:** nothing. **Startable immediately, highest value per line.**
**Depended on:** nothing. Landed before any measurement work, as intended.
Stamp embedder identity into the gallery at build; verify at load in
`scene_analyze`, `replay.py` and the optimizer. Mismatch is a hard error naming
both sides.
Stamp = model basename + SHA-256 of the ONNX, written as the `/embedder` group at
build time (`gallery_builder.cpp`, `sae_gallery.save_gallery_hdf5`) and verified
at load in `scene_analyze`, `scene_preview`, the `sae_kpn` matcher binding,
`replay.py`, `optimize.py` and `movienet_eval.py`. Mismatch is a hard error naming
both sides, with no bypass. Embedding dumps carry the same stamp, since a replay
has no live embedder to check against.
Unstamped legacy galleries **warn loudly and proceed** rather than failing:
unknown is not known-bad, and hard-failing every pre-existing gallery would turn
the check into something people disable. `--require-gallery-stamp` /
`SAE_REQUIRE_GALLERY_STAMP=1` promotes that to a hard error — measurement runs
should set it. `scripts/stamp_gallery.py` re-binds an existing gallery without
re-embedding, so the warning state is cheap to leave.
Cross-model similarities are meaningless but *look* plausible — this fails
silently and expensively, and it would corrupt every measurement taken during the
+1 -1
View File
@@ -294,7 +294,7 @@ because it will be trusted.
| IR-007 | **T1** | Media < 120 s emits no signature at all | Exactly 120 s; just under; zero-length audio. Must match the plugin's cutoff exactly — a caller-varying window length is what SR-004 forbids |
| IR-008 | T1 | `v1:` prefix emitted and honoured on read | Unknown prefix rejected, not guessed |
| GR-009 | T1 | Human-confirmed associations persist and are tier-tagged | Survives a gallery rebuild; distinguishable from baked and harvested |
| GR-004 | T1 | Mismatched embedder → hard startup error | Error names both sides |
| GR-004 | T1 | Mismatched embedder → hard startup error | Error names both sides; **unstamped warns, and errors under `SAE_REQUIRE_GALLERY_STAMP`**; same filename + different SHA-256 must still be a mismatch |
| GR-008 | T1 | Outlier flagged among an actor's references | Injected poisoned embedding detected |
| VR-009 | T1 | Posterior calibration holds | A 0.99 posterior is wrong ~1% of the time on held-out tracks |