Cut the eye box from a landmark contour, and refuse eyes that cannot be read

SCRFD's eye point places a face, not an eye: on turned and smiling heads
the classifier's window had the eye in a corner, and two model-free ways
of re-centring it — the darkest blob, the most contrasty window — both
lost open eyes (19 → 15 and 19 → 9 of 25). Three landmark models were
then run over the same faces; Face Mesh V2 and InsightFace's 2d106det
tied at 22 of 25 and 2d106det ships, being the cheapest by far and under
the grant the detector and embedder already carry. The eye box is the
tight bounding box of its ten lid points, cut upright from the native
render, which is what the classifier was trained on.

The larger change is that the reading now carries, per eye, the source
pixels across the box and the sharpness of the patch — because the
commonest wrong answer on the reference library was a soft eye read as
closed, and a classifier shown a smear will always say something. An eye
under either floor, or narrower than six tenths of its partner (the far
eye of a turned head, whose contour collapses), is not asked; a face with
no readable eye is a fourth state, Unreadable, that no filter drops. On
twenty native renders the one real blink is caught, the laughing faces
are closed, the profiles are judged on the near eye, and the one thing
left beyond any floor is a face with a pot held over it.
This commit is contained in:
2026-09-19 14:04:08 +02:00
parent b908d861e0
commit f5956707e7
7 changed files with 690 additions and 311 deletions
+15 -5
View File
@@ -1,8 +1,10 @@
//! Faces and identity (S14, docs/faces.md).
//!
//! Two models, run over the proxy tier, producing per face a box, five
//! Two models, run over the native render, producing per face a box, five
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
//! arithmetic that turns embeddings into people (FR-CULL-9, FR-CULL-10).
//! arithmetic that turns embeddings into people (FR-CULL-9, FR-CULL-10). Two
//! more, optional, read each face's eyes and whether sunglasses hide them
//! (FR-CULL-13, [`classify`] and [`eyes`]).
//!
//! Like `dr-segment`, this crate is **device-free**: no GPU adapter, no
//! Slint, nothing that needs a display. Unlike `dr-segment`, it carries **no
@@ -20,7 +22,8 @@
//! runtime; this crate takes bytes and never fetches anything.
//!
//! docs/faces.md §2 is the full reading, including what would have to change
//! for that to stop being true.
//! for that to stop being true. The eye-state models are the exception: MIT,
//! weights and all, and shipped in `models/face/` (docs/faces.md §17).
//!
//! # Why the runtime is split behind a feature
//!
@@ -43,6 +46,8 @@ pub mod detect;
pub mod embed;
pub mod embedding;
pub mod eyes;
#[cfg(feature = "inference")]
pub mod landmarks;
pub mod naming;
pub mod neighbours;
@@ -68,7 +73,7 @@ pub mod neighbours;
pub const MIN_CROP_EDGE: u32 = 1025;
pub use align::{
eye_patches, head_views, warp, warp_pixels, Aligned112, EyePatch, EyePatches, HeadViews,
crop_box, eye_box, eye_patch, head_views, warp, warp_pixels, Aligned112, EyePatch, HeadViews,
Pixels, Similarity, ALIGNED_EDGE, ARCFACE_TEMPLATE,
};
pub use assign::{identity_shares, RIVAL_FLOOR, TOP_MATCHES};
@@ -85,7 +90,12 @@ pub use embed::{Embedded, Embedder};
pub use embedding::{
in_gallery, read_f16_bytes, Embedding, ModelId, EMBEDDING_DIM, MIN_GALLERY_QUALITY,
};
pub use eyes::{EyeReading, EyeState, EYES_OPEN_THRESHOLD, SUNGLASSES_THRESHOLD};
pub use eyes::{
Eye, EyeReading, EyeState, EYES_OPEN_THRESHOLD, HIDDEN_EYE_RATIO, MIN_EYE_PX,
MIN_EYE_SHARPNESS, SUNGLASSES_THRESHOLD,
};
#[cfg(feature = "inference")]
pub use landmarks::{Landmarker, Landmarks};
pub use naming::{name_for_instance, name_instances, NamedFace};
/// What can go wrong between an image and a face.