# Deep dive: LVFace-B Glint360K

LVFace won the model bake-off (see `best-model.md`) and is the shipped default
embedder. This page is the honest accounting of how it actually performs —
including where it's wrong, and one case where the ground truth itself is
wrong and LVFace is right.

## Training vs. held-out: the generalization gap

The shipped config (`prob_threshold=0.754, anneal_sec=35.54,
extinction_sec=57.43, expand_gallery=true`) was tuned against 4 films. Scored
against the 5 films the optimizer never saw:

| film | F1 | P | R | TPI | FPI | misid | FN |
|---|---|---|---|---|---|---|---|
| Benny & Joon | 83.0% | 89.1% | 77.7% | 15125 | 1846 | 0 | 4337 |
| Lovelace | 77.5% | 90.3% | 67.9% | 14990 | 1085 | 58 | 7085 |
| Valerian and the City of a Thousand Planets | 74.1% | 97.1% | 60.0% | 18663 | 548 | 0 | 12467 |
| Downton Abbey: A New Era | 56.2% | 97.8% | 39.4% | 52027 | 1173 | 0 | 80084 |
| **The Many Saints of Newark** | **46.3%** | **54.7%** | 40.1% | 15922 | 4394 | **974** | 23791 |
| **macro average** | **67.4%** | 85.8% | 57.0% | | | | |

**67.4% held-out vs. 75.3% on training** — an ~8pp drop, and a **37pp spread
between the best and worst held-out film**. The config does not generalize
uniformly; two films are outright failure cases, for two different reasons.

## Failure mode 1: frozen-bbox "ghost tracks"

Both Many Saints of Newark (974 misIDs) and Downton Abbey (FN=80084, the worst
recall of the five) trace to the same root cause, verified directly against
the raw per-frame stream and the HDF5 dump's own detection counts — not
inferred from the score alone.

![Frozen ghost boxes over background, The Many Saints of Newark](assets/images/many_saints_ghost_fpi.jpg)

At this second, three of the four labeled boxes ("Jon Bernthal", "Joey Diaz",
"Billy Magnussen") sit over empty background — a blurred wall, hanging
plates — with no face in them. The real face in frame carries a second,
colliding label from another frozen box.

![15 ghost boxes over a blank title card, Downton Abbey: A New Era](assets/images/downton_abbey_ghost_fpi.jpg)

This is the starkest case: **15 actors named, all wrong, over a completely
blank closing title card.** Confirmed against the dump directly: `face_count`
is 0 from this point onward (no detector output at all), yet the same 15
identities keep appearing with the *exact same bounding box, unchanged to the
pixel*, for 57+ consecutive seconds.

This is `SceneTrackerFunc::active_[actor_idx].last_bbox`
(`src/nodes/scene_tracker_node.hpp`) being re-emitted unchanged — the
extinction state machine working exactly as coded, not a bug. The film cuts
from a packed group shot straight into 40+ seconds of blank titles/credits,
and `extinction_sec=57.4` is comfortably long enough to bridge that entire gap
without expiring, so the tracker faithfully reports "last known position" for
a cast that is no longer on screen at all. `extinction_sec` was tuned toward
long windows specifically because they bridge real gaps (occlusion, a turned
face) in most training footage — this is the cost side of that trade,
surfacing only when a film has a long enough faceless stretch to expose it.

## Failure mode 2: a genuine misID (for contrast)

Not every held-out failure is a ghost. This is a real face, correctly
detected, confidently misidentified:

*(same many_saints_ghost_fpi.jpg frame above also shows Leslie Odom Jr.'s box
carrying a second, colliding "Michael Gandolfini" label — two real tracks'
frozen positions happening to overlap, not a detection error.)*

## Where LVFace beat X-Ray

Not every "misID" is actually wrong. `second_score.py` counts a name as a true
out-of-cast misID whenever the named actor isn't in X-Ray's credited cast list
for the film at all — but X-Ray's cast list is itself incomplete.

![LVFace correctly identifies Germar Terrell Gardner, uncredited by X-Ray](assets/images/germar_beats_xray.jpg)

Germar Terrell Gardner — a real, clean, high-confidence detection — is counted
as a misID here because he doesn't appear in X-Ray's `people.csv` for The Many
Saints of Newark at all. But Jellyfin's independent cast metadata *does* credit
him for this exact film (cross-checked via `experiments/manifests/
jellyfin_casts.json`, a completely separate data source from X-Ray). This
isn't a lookalike error or a gallery mixup — it's the pipeline correctly
recognising a real cast member that one ground-truth source happened to omit.

This doesn't mean every flagged misID is secretly correct — Many Saints'
974-count total is still overwhelmingly the frozen-bbox failure mode above,
not uncredited-but-real cameos. But it's a reminder that the X-Ray corpus is a
convenient, large-scale ground truth, not a perfect one, and the "misID" number
in any of these tables has some irreducible noise floor from ground-truth gaps
in the other direction too.

## Summary

LVFace is the right default: it wins the model comparison outright, and its
failures are traceable, understood, and mostly attributable to one tunable
knob (`extinction_sec`) rather than the embedder itself. The held-out
generalization gap (75.3% → 67.4%) is real and should be treated as the honest
expected performance, not the training-set number.