dtourolleandClaude Opus 5
01d7ead1e7
Gallery from one recording, probes from another, sweeping the probe's input resolution end to end. VR-005 asked the same question over gallery mugshots but degraded an already-aligned 112x112 crop with alignment held perfect, so it isolates the embedder. Here the whole frame is downscaled before the detector, so detection and landmark regression degrade with it — which is most of the difference. Corpus is two 4096x2160 clips of one shoot, four people, hand-sorted. Ground truth is sorted by hand and gated by verify_labels.py; labels carried down the scales geometrically by box position, never by embedding similarity, which would keep only the faces the embedder already gets right and drop the ones the sweep exists to find. Findings, all scored through the production gallery sigmoid at prob_threshold 0.754 — never a raw cosine: - Holding 90% of the plateau needs ~50 px end to end, against VR-005's ~22 px. min_face_px at 40 looks right; 32 would admit faces in the falling region. - FPI is 0.0% at every scale. Resolution loss goes entirely to TBI. - The ceiling is cross-view, not resolution: everyone matches themselves within a recording (0.55-0.85) and collapses across two (0.14-0.45, threshold 0.335). Only the subject with frontal *gallery* references identified reliably, whatever their probe pose — so the lever is gallery pose coverage, not a better landmark source. - Averaging SCRFD's overlapping detections instead of discarding them at NMS lifts cross-recording TPI 41% -> 49%, for one forward pass and no extra model. Four identities and one shoot, so the shape is the result and the absolute rates are not. Both clips contain all four people, so there is no out-of-gallery class and the 10x-weighted out-of-cast misID is untested here. Clips, frames, hand-sorted crops and results are gitignored and belong in the artifact registry — the sorting is human ground truth and expensive to redo. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> TRACES: VR-013 | AR-002, AR-005, AR-024
56 lines
2.4 KiB
Python
56 lines
2.4 KiB
Python
#!/usr/bin/env python3
|
|
"""Redraw every dumped crop with its detection box marked.
|
|
|
|
The original thumbnails padded by 0.5x the face on each side for
|
|
recognisability, which in a crowded frame pulls a neighbour into shot — often
|
|
more prominently than the subject. A label cannot be corrected from a picture
|
|
that does not say which face it refers to.
|
|
|
|
This rewrites each .jpg IN PLACE, wherever it currently sits, so any sorting
|
|
already done is preserved: only the pixels change, never the filename or the
|
|
folder. Re-run it after dump_faces.py, and re-check any sorting done before it.
|
|
"""
|
|
import glob, json, os, sys
|
|
import cv2
|
|
|
|
CLIPS = ["5157339", "5157344"]
|
|
OUT = 256
|
|
|
|
for clip in CLIPS:
|
|
man = {m["file"]: m for m in json.load(open(f"labelling/{clip}/manifest.json"))}
|
|
n = 0
|
|
for path in glob.glob(f"labelling/{clip}/**/*.jpg", recursive=True):
|
|
fname = os.path.basename(path)
|
|
m = man.get(fname)
|
|
if m is None:
|
|
continue
|
|
img = cv2.imread(f"frames/d{clip}_{m['frame']}.png")
|
|
if img is None:
|
|
sys.exit(f"missing frames/d{clip}_{m['frame']}.png")
|
|
|
|
x, y, w, h = (int(v) for v in m["bbox"])
|
|
pad = int(0.55 * max(w, h))
|
|
x0, y0 = max(0, x - pad), max(0, y - pad)
|
|
x1, y1 = min(img.shape[1], x + w + pad), min(img.shape[0], y + h + pad)
|
|
sub = img[y0:y1, x0:x1].copy()
|
|
|
|
# Box in the sub-image's coordinates, drawn before the resize so the
|
|
# line lands exactly on the face at any output size.
|
|
cv2.rectangle(sub, (x - x0, y - y0), (x - x0 + w, y - y0 + h), (0, 0, 255), 3)
|
|
# Dim everything outside the box so the subject is unmistakable even
|
|
# when a neighbour's face is larger or better lit.
|
|
mask = sub.copy()
|
|
mask[y - y0:y - y0 + h, x - x0:x - x0 + w] = 0
|
|
sub = cv2.addWeighted(sub, 1.0, mask, -0.35, 0)
|
|
|
|
scale = OUT / max(sub.shape[:2])
|
|
sub = cv2.resize(sub, (int(sub.shape[1] * scale), int(sub.shape[0] * scale)))
|
|
canvas = cv2.copyMakeBorder(
|
|
sub, 0, max(0, OUT - sub.shape[0]), 0, max(0, OUT - sub.shape[1]),
|
|
cv2.BORDER_CONSTANT, value=(20, 20, 20))[:OUT, :OUT]
|
|
cv2.putText(canvas, f"{int(m['px'])}px", (5, OUT - 8),
|
|
cv2.FONT_HERSHEY_SIMPLEX, 0.45, (0, 255, 255), 1)
|
|
cv2.imwrite(path, canvas)
|
|
n += 1
|
|
print(f"[{clip}] redrew {n} crops in place", file=sys.stderr)
|