Every embedding now carries the quality of the input it came from. Both
axes fall out of the AR-005 warp for free: crop_sharpness() is the
normalised Laplacian variance over the aligned 112x112, so contrast and
size cannot leak into it, and the alignment residual is the part of the
landmark deformation a similarity transform cannot explain, so in-plane
roll reads as zero and foreshortening does not.
Carried, not consumed. Nothing discounts or thresholds on either number
yet -- that is AR-030 and VR-012, and the knee has to be located against
recorded data before a gate is chosen. What this change buys is that the
data exists to locate it with.
No face is admitted unscored: the -1 sentinel is preserved rather than
clamped, and a degenerate landmark fit is counted rather than silently
dropped.
Takes the VR-001 dump to schema_version 2. The bump is not for readers,
which check for the datasets by name and replay a v1 dump unchanged; it
is so a consumer can tell "never scored" from "scored zero", which is
not recoverable from the arrays afterwards.
TRACES: AR-028, AR-029, AR-030 | VR-001 | SR-002
VR-010 — a dump made with one detector/embedder pair was byte-indistinguishable
from one made with another, except for the two attributes GR-004 added. Replayed
against a gallery from a different model, cosine similarities are meaningless
but look entirely plausible. The register states the principle directly: a
fixture whose provenance is unknown is worse than no fixture, because it will be
trusted.
Sixteen attributes now record everything that determines the dump's content:
detector model and thresholds, min_face_px, max_faces, cut_threshold,
dense_scale, bbox_upscale, start/end, track_assoc_min_prob, and scene_detect.
scene_detect is the one that matters most. is_scene_boundary is all-zero both
when the detector found nothing and when it never ran, and those mean completely
different things to a consumer — without the flag they are indistinguishable.
No schema_version bump: new root attributes are additive and replay.py already
reads attributes with a default, so older dumps stay readable and the committed
fixtures — which predate this — still load.
Also corrects SCHEMA.md, which claimed bbox was already mapped to original
resolution at dump time. It is not; the upscale is applied downstream in the
matcher, after the dump tap. Harmless while dense_scale is 1 and silently wrong
otherwise, so bbox_upscale is now recorded and the doc says what the code does.
Verified end to end: all sixteen attributes present and correct on a freshly
generated dump.
Suite: 92 cases, 6136 assertions.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: VR-010, VR-001 | PR-002
A gallery is only valid for the embedder that produced its vectors. Cosine
similarities across models are meaningless but *look* plausible, so the mistake
is silent and every measurement taken afterwards is suspect. Stamp the embedder
identity into the gallery at build; verify it at every load.
The stamp is the model file's basename plus the SHA-256 of its bytes (plus
embed_dim). The hash decides, the name explains. A name alone is a promise
rather than a fact — models get re-exported and overwritten in place under an
unchanged filename, which is exactly the case where the weights differ and
nothing else does. A hash alone is correct but unactionable in an error message.
SHA-256 is derived from the artefact, needs no registry kept current, and costs
~0.1s for a 250MB ONNX, memoised per process.
Mismatch is a hard error in every mode, with no bypass, naming both sides.
Unstamped legacy galleries warn loudly and proceed: unknown is not known-bad,
and hard-failing every pre-existing gallery would turn the check into something
people disable rather than trust. --require-gallery-stamp (or
SAE_REQUIRE_GALLERY_STAMP=1, which propagates to subprocesses) promotes that to
a hard error — the mode measurement work should run in. scripts/stamp_gallery.py
re-binds an existing gallery with no re-embedding, so "warn" is a cheap state to
leave rather than a permanent one.
Embedding dumps carry the same stamp: a replay has no live embedder, so the dump
is the embedder as far as the gallery is concerned. Derived galleries inherit
their source's stamp; --merge and the JSON gallery merge check before writing,
since one file holding two embedding spaces cannot be untangled afterwards.
Verified in: scene_analyze, scene_preview, the sae_kpn matcher binding,
replay.py, optimize.py (once per film at startup, before the first evaluation),
movienet_eval.py and both merge paths.
Stamp logic lives in src/gallery/embedder_stamp.{hpp,cpp} and its Python twin
scripts/sae_stamp.py, kept dependency-light so replay subprocesses do not pay
sae_gallery's requests/Pillow import to ask whether two models match.
Tests: 12 new cases in test_gallery_store.cpp covering the comparison logic,
both round trips, and the SHA-256 vectors that guarantee the C++ and hashlib
stamps agree. No ONNX or GPU required.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Optimizer (scripts/optimizer/): replay.py runs the real C++ tracker/matcher/
scene_tracker chain over a dumped-embeddings HDF5 via sae_kpn, so a threshold
sweep never re-decodes video or re-embeds faces. optimize.py drives scipy's
differential_evolution over the knob space, with DE-level parallelism
(multiple population candidates evaluated concurrently via a ThreadPoolExecutor)
on top of per-film replay parallelism. second_score.py is the per-second X-Ray
scoring metric (TPI/FPI/FN, out-of-cast misID weighted 10x, fair recall masked
to gallery-known cast) that superseded an earlier scene-union metric.
dump_error_frames.py / dump_scene_montage.py extract annotated video frames
(bounding boxes, TPI/FPI/FN captions, onscreen-vs-offscreen split) for visual
review of a replay against ground truth. Gallery utilities: cast_restrict.py,
gallery_membership.py, fetch_missing_actors.py, reembed_gallery.py.
scripts/validation/: X-Ray ground-truth loading and provider-agnostic identity
matching (identity.py's keys_for — an actor is the union of every id we can
derive, since pipeline output and ground truth don't share one id space).
scripts/artifacts/: push/pull scripts for the Gitea generic package registry —
galleries, montage frames, and experiment data (manifests/trajectories/results)
are pushed there instead of committed, since none are needed to run the app,
only benchmarks. Versioned by git short-SHA.
scripts/docs/: MkDocs site build (build_site.sh) and the calibration-curve
comparison chart (calibration_chart.py, matplotlib, reads each gallery's
embedded calibration).
Gallery-building scripts (make_jellyfin_gallery.py, make_gallery.py,
filter_gallery.py, run_from_jellyfin.py, movienet_eval.py, movienet_prep.py,
sae_gallery.py) updated to read/write HDF5 galleries exclusively, matching the
engine-side format switch. run_from_jellyfin.py and the optimizer no longer
carry movie source paths in shared manifests (some source filenames include
scene-release tags) — resolved locally via a gitignored file-lut.json instead.
dtourolle
