Stop indexing faces too small or too blurred to be anyone
The library was storing faces at 52 source pixels and embedding whatever came
back. There was a size floor, but it was 40 pixels on the *bounding box*, and
there was no blur gate at all — so a subject walking through a half-second
exposure detected confidently, aligned cleanly, and produced a perfectly
ordinary-looking 512-vector. Nothing downstream can tell that apart from a real
face, and because blurs resemble each other more than they resemble the people
they were, they cluster together and weld unrelated identities into one group.
Two floors, both measured rather than guessed. `face_index --quality` runs the
detector over real proxies with both gates disabled and prints the distribution;
over 1,503 faces in 600 images of the reference library:
percentile crop px sharpness
1% 16 0.0006
25% 23 0.0025
50% 38 0.0071
75% 76 0.0284
99% 352 0.4282
The median face in a personal library is 38 pixels. Most of what the detector
finds is background: people across a square, a face on a poster, a stranger at
the next table. They are real detections and useless identifications.
**Size, on the crop rather than the box.** "At least 64x64" has to mean the
pixels the *embedder* sees, and the box is not that — the ArcFace template
reaches past it for forehead and chin, so the aligned crop spans roughly 1.3x
the box's shorter edge. The floor is therefore `min_source_px` on the aligned
crop, applied after the warp fixes the scale, and `min_face_px` drops to 48 as
what it always really was: a cheap pre-filter set low enough that it cannot
reject a face the real floor would have kept.
**Sharpness.** Variance of the Laplacian divided by the variance of the luma it
was taken over. The division is the part that matters: raw Laplacian variance
scales with contrast, so a threshold on it would quietly discard every backlit
portrait in the library. The ratio asks how much of the crop's variation is
edges rather than broad gradients, and is invariant to exposure.
What each pair removes, cumulatively, of everything the detector finds:
min crop min sharp size cut blur cut kept
64 0.000 70% 0% 30%
64 0.010 70% 3% 27%
64 0.020 70% 7% 23%
80 0.010 76% 2% 21%
64 and 0.020. The size floor does most of the work, and the blur floor removing
only 7% on top of it is the point rather than a disappointment: at 64 pixels
most faces are already sharp, and what it takes out is the large-but-soft one —
precisely the face that would otherwise contribute a confident, wrong embedding.
The two gates are not independent and the doc comments say so: a face under 112
pixels was upsampled to reach the embedder, and upsampling invents no edges, so
small faces score low on sharpness even when the original was crisp. That is why
`--quality` prints them together.
**This will re-index.** Around 70% of what the current settings store falls below
the new floors — faces between 20 and 40 pixels that nobody could identify. The
People screen gets shorter and every group in it gets better.
66 dr-face tests pass, including that a blurred crop scores below a sharp one,
that halving the contrast does not move the score, and that an upsampled face
scores below the same face at full size.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -39,11 +39,42 @@ pub struct DetectOptions {
|
||||
pub confidence: f32,
|
||||
/// Box IoU above which two detections are judged to be the same face.
|
||||
pub nms_iou: f32,
|
||||
/// Smallest face to keep, in source pixels on the shorter box edge.
|
||||
/// Cheap pre-filter: smallest box to keep, in source pixels on the shorter
|
||||
/// edge.
|
||||
///
|
||||
/// Below this there is not enough face left to align reliably — see
|
||||
/// docs/faces.md §7 for what the embedder actually receives at each size.
|
||||
/// **Not the real size floor** — [`DetectOptions::min_source_px`] is, and
|
||||
/// it is measured on the aligned crop rather than the box. This one exists
|
||||
/// only to throw away the obviously hopeless before paying for a warp, so
|
||||
/// it is deliberately set *below* what the real floor will accept: the
|
||||
/// aligned crop spans roughly 1.3x the box's shorter edge, so 48 here
|
||||
/// cannot reject a face that would have cleared 64 there.
|
||||
pub min_face_px: f32,
|
||||
/// Smallest face the embedder may be given, in **source pixels across the
|
||||
/// aligned crop** — `crop_px` in the catalog.
|
||||
///
|
||||
/// The honest statement of "a face must be at least 64x64", because this is
|
||||
/// the number of real pixels behind the 112x112 the model actually sees.
|
||||
/// The box's own size is not that: the ArcFace template reaches past the
|
||||
/// box for forehead and chin, so a 64-pixel box and a 64-pixel crop are
|
||||
/// different faces.
|
||||
///
|
||||
/// Below this the crop was upsampled to reach the embedder, and upsampling
|
||||
/// invents no detail — the embedding is of a soft, stretched face and is
|
||||
/// correspondingly untrustworthy.
|
||||
///
|
||||
/// Applied after alignment, so it lives with the sharpness floor rather
|
||||
/// than with the detector. See [`DetectOptions::min_sharpness`].
|
||||
pub min_source_px: f32,
|
||||
/// Least acceptable [`crate::align::Aligned112::sharpness`].
|
||||
///
|
||||
/// Applied after alignment rather than here, because it is a property of
|
||||
/// the warped crop the embedder receives and not of the box. The pipeline
|
||||
/// that enforces it is `dr_ui::faces::index_proxy`; it lives on this struct
|
||||
/// so that every quality decision about a face is configured in one place
|
||||
/// and a caller cannot enable one gate while forgetting the other.
|
||||
///
|
||||
/// Zero disables it, which is what a measurement run wants.
|
||||
pub min_sharpness: f32,
|
||||
}
|
||||
|
||||
impl Default for DetectOptions {
|
||||
@@ -51,7 +82,9 @@ impl Default for DetectOptions {
|
||||
Self {
|
||||
confidence: 0.5,
|
||||
nms_iou: 0.4,
|
||||
min_face_px: 40.0,
|
||||
min_face_px: 48.0,
|
||||
min_source_px: 64.0,
|
||||
min_sharpness: 0.020,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user