Deep dive: LVFace-B Glint360K

LVFace won the model bake-off (see best-model.md) and is the shipped default embedder. This page is the honest accounting of how it actually performs — including where it's wrong, and one case where the ground truth itself is wrong and LVFace is right.

Training vs. held-out: the generalization gap

The shipped config (prob_threshold=0.754, anneal_sec=35.54, extinction_sec=57.43, expand_gallery=true) was tuned against 4 films. Scored against the 5 films the optimizer never saw:

film	F1	P	R	TPI	FPI	misid	FN
Benny & Joon	83.0%	89.1%	77.7%	15125	1846	0	4337
Lovelace	77.5%	90.3%	67.9%	14990	1085	58	7085
Valerian and the City of a Thousand Planets	74.1%	97.1%	60.0%	18663	548	0	12467
Downton Abbey: A New Era	56.2%	97.8%	39.4%	52027	1173	0	80084
The Many Saints of Newark	46.3%	54.7%	40.1%	15922	4394	974	23791
macro average	67.4%	85.8%	57.0%

67.4% held-out vs. 75.3% on training — an ~8pp drop, and a 37pp spread between the best and worst held-out film. The config does not generalize uniformly; two films are outright failure cases, for two different reasons.

Failure mode 1: frozen-bbox "ghost tracks"

Both Many Saints of Newark (974 misIDs) and Downton Abbey (FN=80084, the worst recall of the five) trace to the same root cause, verified directly against the raw per-frame stream and the HDF5 dump's own detection counts — not inferred from the score alone.

At this second, three of the four labeled boxes ("Jon Bernthal", "Joey Diaz", "Billy Magnussen") sit over empty background — a blurred wall, hanging plates — with no face in them. The real face in frame carries a second, colliding label from another frozen box.

This is the starkest case: 15 actors named, all wrong, over a completely blank closing title card. Confirmed against the dump directly: face_count is 0 from this point onward (no detector output at all), yet the same 15 identities keep appearing with the exact same bounding box, unchanged to the pixel, for 57+ consecutive seconds.

This is SceneTrackerFunc::active_[actor_idx].last_bbox (src/nodes/scene_tracker_node.hpp) being re-emitted unchanged — the extinction state machine working exactly as coded, not a bug. The film cuts from a packed group shot straight into 40+ seconds of blank titles/credits, and extinction_sec=57.4 is comfortably long enough to bridge that entire gap without expiring, so the tracker faithfully reports "last known position" for a cast that is no longer on screen at all. extinction_sec was tuned toward long windows specifically because they bridge real gaps (occlusion, a turned face) in most training footage — this is the cost side of that trade, surfacing only when a film has a long enough faceless stretch to expose it.

Failure mode 2: a genuine misID (for contrast)

Not every held-out failure is a ghost. This is a real face, correctly detected, confidently misidentified:

(same many_saints_ghost_fpi.jpg frame above also shows Leslie Odom Jr.'s box carrying a second, colliding "Michael Gandolfini" label — two real tracks' frozen positions happening to overlap, not a detection error.)

Where LVFace beat X-Ray

Not every "misID" is actually wrong. second_score.py counts a name as a true out-of-cast misID whenever the named actor isn't in X-Ray's credited cast list for the film at all — but X-Ray's cast list is itself incomplete.

Germar Terrell Gardner — a real, clean, high-confidence detection — is counted as a misID here because he doesn't appear in X-Ray's people.csv for The Many Saints of Newark at all. But Jellyfin's independent cast metadata does credit him for this exact film (cross-checked via experiments/manifests/ jellyfin_casts.json, a completely separate data source from X-Ray). This isn't a lookalike error or a gallery mixup — it's the pipeline correctly recognising a real cast member that one ground-truth source happened to omit.

This doesn't mean every flagged misID is secretly correct — Many Saints' 974-count total is still overwhelmingly the frozen-bbox failure mode above, not uncredited-but-real cameos. But it's a reminder that the X-Ray corpus is a convenient, large-scale ground truth, not a perfect one, and the "misID" number in any of these tables has some irreducible noise floor from ground-truth gaps in the other direction too.

Summary

LVFace is the right default: it wins the model comparison outright, and its failures are traceable, understood, and mostly attributable to one tunable knob (extinction_sec) rather than the embedder itself. The held-out generalization gap (75.3% → 67.4%) is real and should be treated as the honest expected performance, not the training-set number.