Skip to content

Whole gallery vs. limited (cast-restricted) gallery

Two ways to run the matcher: full scores every detected face against the entire library gallery (2418 actors across the 9-film benchmark set); restricted pre-filters each film's gallery down to just its Jellyfin-credited cast (typically ~15 top-billed actors) before the matcher ever runs.

The result

Averaged across all 4 models and both expansion settings, on the 4 rep4 training films:

scope F1 P R total misID (8 evals)
full 71.2% 91.1% 59.0% 1073
restricted 74.5% 92.2% 62.9% 329

This is not a precision/recall trade — restriction wins on every axis at once: +3.3pp F1, +3.9pp recall, and less than a third the total misIDs. Fewer candidates in the matcher's search space means fewer opportunities for a look-alike false match (an actor who happens to share enough facial structure with someone in the film, but isn't actually in it), and the recall gain shows it isn't costing real detections to get there.

Per-model, every single model's best-scoring combo in the full 16-way matrix is a restricted variant — see the full table in rep4-optimizer-results.md. Two combos hit zero true out-of-cast misidentifications: arcface_w600k_mbf_restricted_exp (F1 76.5%) and, in full mode, LVFace-B_Glint360K_full_noexp (F1 72.4%) — restriction isn't the only way to reach misid=0, but it's the more reliable one.

Why this isn't the shipped default

Cast-restriction is implemented today only as an offline optimizer technique (scripts/optimizer/cast_restrict.py): it pre-builds a filtered gallery file per film, using Jellyfin's own cast list, before the benchmark ever calls the matcher. There's no runtime "restrict matching to this title's credited cast" switch in the shipped application — scene_analyze always matches against whatever single gallery file it's given.

Building that as a real feature would need, at minimum:

  • A live Jellyfin cast lookup at analysis time (the title is already known — run_from_jellyfin.py already does this same lookup for its own filter_gallery-based restriction path, just not wired into scene_analyze itself as a first-class option).
  • A decision on the fallback: what happens to a real, uncredited cameo (see the Germar Terrell Gardner case in the LVFace deep-dive) if the gallery never includes them at all?
  • Regenerating the restricted-gallery cache whenever the title's Jellyfin cast list changes.

This is why the shipped src/config.hpp defaults use the full-mode winner (LVFace-B_Glint360K_full_exp, F1 75.3% training / 67.4% held-out macro) rather than the higher-scoring restricted_exp (78.3%) — the 78.3% number describes a capability the app doesn't have yet, not what actually ships.