study(VR-005): minimum face size from downscaled gallery mugshots
Holds out one mugshot per actor, degrades that probe to each candidate
face size and matches it against a gallery held at native resolution,
reporting TPI/FPI per size. Replaces AR-002's 66x66 px working estimate
with a measurement. Needs no video and no ground truth beyond the
mugshot cache already on disk.
LVFace-B over 258 actors, 999 gallery embeddings, threshold 0.754:
px 12 16 20 24 32 40 48+
TPI 6.6% 46.5% 81.8% 93.4% 98.1% 99.2% 99.2%
FPI is 0.000 at every size — a face too small to identify degrades to
unidentified, never to a wrong name. rank-1 holds at >=99.6% from 24 px
up, so what fails first is the calibrated probability crossing
threshold, not the ranking.
Two limits on reading this. FPI grows with the number of actors
competing, so 258 understates it against a production library. And
detection and alignment run on the native image with only the resulting
112x112 crop degraded, so landmark error at small face sizes is excluded
by construction and the curve is an upper bound — VR-010 measures the
same question end to end, and lands well above these numbers.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TRACES: VR-005 | AR-002
This commit is contained in:
@@ -102,7 +102,7 @@ Status: `Done` · `In Progress` · `Planned` · `TBD` · `Withdrawn`
|
||||
| VR-002 | Replay drives the **real** KPN nodes, not a reimplementation | PR-002 | High | Done |
|
||||
| VR-003 | Scoring: micro-F1 against X-Ray, precision/recall logged at every evaluation | PR-002 | High | Done |
|
||||
| VR-004 | Reproducible validation corpus with ground truth | PR-002 | High | Done |
|
||||
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | Planned |
|
||||
| VR-005 | Minimum face size study — TPI/FPI vs probe size, gallery held at native res | PR-002 | Medium | Done |
|
||||
| VR-006 | Re-tune `scene_threshold` once native-rate decode lands | PR-002 | Low | Planned |
|
||||
| VR-007 | Expansion band, clustering threshold, and deferred-pass ablation | PR-002 | Medium | Planned |
|
||||
| VR-008 | Gallery scaling benchmark — throughput vs gallery size | PR-002 | Medium | Planned |
|
||||
|
||||
@@ -79,9 +79,30 @@ The table caches nulls (tmdb ids TMDB has no IMDb id for) and checkpoints, so a
|
||||
re-run only resolves new ids. TMDB is authoritative for this crosswalk — there is
|
||||
no clean free bulk `tmdb_person ↔ nm` file, so we query the API once and cache.
|
||||
|
||||
## Minimum face size (VR-005)
|
||||
|
||||
`min_face_size.py` is a separate, self-contained study: it needs no video and no
|
||||
ground truth, only the gallery mugshot cache. It holds out one image per actor,
|
||||
degrades that probe to each candidate face size and matches it against a gallery
|
||||
held at **native** resolution, reporting TPI/FPI per size — the measurement that
|
||||
replaces AR-002's 66×66 px estimate.
|
||||
|
||||
```bash
|
||||
python scripts/validation/min_face_size.py \
|
||||
--images images --gallery gallery_lvface.h5 \
|
||||
--arcface models/LVFace-B_Glint360K.onnx \
|
||||
--actors 100 --out experiments/results/vr005_min_face_size
|
||||
```
|
||||
|
||||
FPI grows with the number of actors competing, so a 100-actor run understates it
|
||||
against a library of thousands: read FPI as relative across sizes, not as an
|
||||
absolute rate. Re-run per `--arcface` model to see whether `min_face_px` should be
|
||||
one constant or scale with the embedder (GR-004).
|
||||
|
||||
## Files
|
||||
- `sample_eval.py` — CLI scorer.
|
||||
- `ground_truth.py` — `XRayGroundTruth`, `MovieNetGroundTruth` loaders.
|
||||
- `identity.py` — provider-agnostic match keys.
|
||||
- `tmdb_imdb_map.py` — build/consult the cached `tmdb→imdb` crosswalk.
|
||||
- `min_face_size.py` — VR-005 probe-size sweep (see above).
|
||||
- `test_sample_eval.py` — self-contained tests (`python scripts/validation/test_sample_eval.py`).
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user