Which embedding model is best?

Three ArcFace variants (w600k-R50, R18, w600k-MBF) and LVFace-B (Glint360K, 455MB) were compared. r50 is excluded from the training/held-out comparison below; its gallery has roughly 30% fewer reference images per actor than the other three on the identical source photos, which confounds a direct score comparison (see the full experiment log for detail). It remains in the calibration comparison, which does not depend on the gallery image count.

First signal: calibration curves

Each gallery carries a fitted Platt sigmoid P(match | cosine similarity) = σ(a·sim + b), stored directly in the gallery HDF5 (src/gallery/gallery_calibration.hpp). This is a property of the embedding space alone, computed from intra- and inter-actor reference-image pairs with no tracking or scene logic involved, so it is a clean first read on discriminative power before running a benchmark.

model	a (steepness)	boundary at P=0.5
LVFace-B Glint360K	17.7	sim 0.228
ArcFace w600k-MBF	16.2	sim 0.267
ArcFace w600k-R50	15.4	sim 0.301
ArcFace R18	15.3	sim 0.309

LVFace has both the steepest transition and the lowest decision boundary, separating same-actor from different-actor reference pairs more confidently at a lower similarity than any ArcFace variant.

Second signal: held-out F1

Each model's own tuned full_exp config, replayed against the 5 films the optimizer never saw and scored the same way:

film	LVFace F1	mbf F1	r18 F1
Benny & Joon	83.0%	78.5%	77.1%
Lovelace	77.5%	73.7%	72.2%
Valerian and the City of a Thousand Planets	74.1%	70.2%	71.0%
Downton Abbey: A New Era	56.2%	55.0%	53.0%
The Many Saints of Newark	46.3%	44.5%	42.1%
macro average	67.4%	64.4%	63.1%

LVFace scores highest on all 5 held-out films; the ranking never flips between models. Total misID count across the 5 films: LVFace 1032, mbf 2197, r18 1224. LVFace has less than half mbf's misID total and still scores higher on every film.

Held-out results are stronger evidence than training results, because training numbers can reflect what the optimizer was tuned to fit rather than general performance. On training data, the ordering is not as clean:

film	LVFace F1	mbf F1	r18 F1	best
Café Society	68.1%	62.2%	60.1%	LVFace
Lord of War	75.6%	77.2%	75.6%	mbf
Scarface	71.5%	68.6%	64.1%	LVFace
Sound of Metal	78.8%	76.5%	71.6%	LVFace

mbf beats LVFace on Lord of War (77.2% vs 75.6%), the only film in either table where LVFace does not score highest. LVFace's training-set macro average (75.3%, see the full experiment log) is not a uniform win across every film it contributes to; the held-out result, where LVFace wins all 5 films outright, is the stronger claim.

This reverses an earlier, superseded benchmarking pass that used a scene-union metric and found the three models statistically indistinguishable (around 85% each), concluding LVFace was not worth its size. That metric masked out-of-cast false positives behind a gallery-intersect-cast recall filter; the per-second metric used here does not.

Full training-matrix picture

Best full-gallery combo per model (all three are full_exp), from the training matrix in the full experiment log:

model	F1	P	R	misID
LVFace-B Glint360K	75.3%	89.7%	65.4%	232
ArcFace w600k-MBF	72.0%	87.7%	61.4%	240
ArcFace R18	69.1%	87.6%	57.7%	242

LVFace leads within both the restricted and full gallery modes, visible directly in the chart above without reading the table. The three models' misID counts on the full gallery are nearly identical (232/240/242); LVFace's lead here is a precision-and-recall lead, not a misID one.

Operational note

Switching the default embedder is not a config change alone; the gallery is model-specific, since embeddings from different models are not comparable. Any existing gallery built against a different model must be rebuilt from source images before the new default takes effect. scripts/optimizer/reembed_gallery.py does this from a reference gallery's cached source images without re-downloading anything.