# xsource — cross-source identification probe (VR-013)

Gallery from **one** recording, probes from **another**, swept over the probe's
input resolution. Complements VR-005, which asked the same question over gallery
mugshots: that one degrades an already-aligned 112×112 crop, holding alignment
perfect, so it isolates the embedder. This one downscales the **whole frame**
before the detector, so detection and landmark regression degrade with it.

Corpus: two Pexels clips of one shoot (4096×2160, 25 fps), four people, all four
present in both.

## Getting the data

Clips, frames and hand-sorted crops are gitignored; they live in the artifact
registry.

    scripts/artifacts/pull_artifacts.sh xsource     # clips + labelling, frames regenerated
    scripts/artifacts/push_artifacts.sh xsource     # after correcting labels

Pulling fetches the two clips and the hand-sorted crops, then regenerates the
frames with ffmpeg — ~320 MB of PNG that is deterministic from the clips, so it
is not worth shipping. Extraction settings are pinned in the pull script because
the manifests key on frame filenames *and* on detection order within each frame;
`verify_labels.py` runs at the end and will fail loudly if they drift.

Pull never overwrites an existing `labelling/`. That directory is human ground
truth — somebody looked at all 167 crops and put each one in a folder — and it
is the expensive part of this study, so push it once corrected.

Clips are Pexels-licensed: free to use, no attribution required, but not
CC or MIT. Fine as a frozen CI artifact on private infrastructure; do not
redistribute them as stock content.

## Scripts

| script | does |
|---|---|
| `dump_faces.py` | detect every face, write a context crop per detection + a manifest |
| `redraw_boxes.py` | redraw those crops with the detection boxed, in place |
| `propose_labels.py` | propose labels for one clip from another clip's hand-sorted folders |
| `make_review_site.py` | local `review.html` — current label, crop, better match, correct and export |
| `apply_corrections.py` | apply the exported `corrections.json` |
| `verify_labels.py` | integrity gate: index consistency, duplicates, separation. Exits non-zero on failure |
| `resolution_sweep.py` | the VR-013 measurement |
| `failure_analysis.py` | what explains the misses — pose, size, blur, detector confidence |
| `landmark_voting.py` | average SCRFD's overlapping detections instead of discarding them |
| `pose_label.py` | mesh-estimated head pose, for hand correction (feeds VR-012) |

Everything drives the shipped C++ through `sae_embed`; nothing reimplements
detection, alignment, the embedder or the calibration. Scoring goes through the
production gallery sigmoid — never a raw cosine (AR-024).

    LD_PRELOAD=/usr/lib/libcudnn_cnn.so.9 python3 resolution_sweep.py

The preload is needed while ORT's CUDA provider looks for
`cudnnGetConvolutionBackwardDataAlgorithm_v7`, which cuDNN 9 moved into
`libcudnn_cnn.so.9` behind a dispatch stub. Without it everything silently falls
back to CPU.

## What it found

**Resolution is not the binding constraint here.** TPI holds ~41–47% from 4096×2160
down to ~45 px faces, then falls: 23 px → 26%, 18 px → 12%, 14 px → 1.5%. Holding
90% of the plateau needs roughly 50 px end to end, against VR-005's ~22 px — the
gap is detection and landmark error, which VR-005 excludes by construction.

**FPI is 0.0% at every scale.** Resolution loss goes entirely to TBI: the pipeline
stops naming people rather than naming the wrong one.

**The ceiling is cross-view, not resolution.** Every person matches themselves
strongly *within* a recording (sim 0.55–0.85) and collapses *across* the two
(0.14–0.45, threshold 0.335). Only the person with frontal **gallery** references
identified reliably, whatever their probe pose — so the lever is gallery pose
coverage (`docs/pose-expansion.md`), not a better landmark model.

**Landmark voting helps.** SCRFD predicts each face from several anchors and NMS
discards all but one, throwing away a median of 3 landmark estimates per face.
Averaging them, weighted by confidence, lifts cross-clip TPI 41% → 49% for one
forward pass and no extra model. A MediaPipe mesh as landmark source went the
other way (41% → 16%): more stable within a recording, but a ring centroid is not
the annotated landmark ArcFace was trained on, and the embedder punishes the
off-distribution crop.

## Reading these numbers

Four identities, 70 probes, one shoot. The ~47% plateau is pose, not resolution —
half these faces are turned away and never clear threshold at any scale, so the
absolute rates say little and the *shape* is the result. Both clips contain all
four people, so there is no out-of-gallery class and the 10×-weighted out-of-cast
misID is **untested** here; holding one identity out of the gallery would fix
that. And the resolution curve is dominated by the single subject whose gallery
references are frontal.
