Cut the eye box from a landmark contour, and refuse eyes that cannot be read
SCRFD's eye point places a face, not an eye: on turned and smiling heads the classifier's window had the eye in a corner, and two model-free ways of re-centring it — the darkest blob, the most contrasty window — both lost open eyes (19 → 15 and 19 → 9 of 25). Three landmark models were then run over the same faces; Face Mesh V2 and InsightFace's 2d106det tied at 22 of 25 and 2d106det ships, being the cheapest by far and under the grant the detector and embedder already carry. The eye box is the tight bounding box of its ten lid points, cut upright from the native render, which is what the classifier was trained on. The larger change is that the reading now carries, per eye, the source pixels across the box and the sharpness of the patch — because the commonest wrong answer on the reference library was a soft eye read as closed, and a classifier shown a smear will always say something. An eye under either floor, or narrower than six tenths of its partner (the far eye of a turned head, whose contour collapses), is not asked; a face with no readable eye is a fourth state, Unreadable, that no filter drops. On twenty native renders the one real blink is caught, the laughing faces are closed, the profiles are judged on the near eye, and the one thing left beyond any floor is a face with a pot held over it.
This commit is contained in:
+15
-5
@@ -1,8 +1,10 @@
|
||||
//! Faces and identity (S14, docs/faces.md).
|
||||
//!
|
||||
//! Two models, run over the proxy tier, producing per face a box, five
|
||||
//! Two models, run over the native render, producing per face a box, five
|
||||
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
|
||||
//! arithmetic that turns embeddings into people (FR-CULL-9, FR-CULL-10).
|
||||
//! arithmetic that turns embeddings into people (FR-CULL-9, FR-CULL-10). Two
|
||||
//! more, optional, read each face's eyes and whether sunglasses hide them
|
||||
//! (FR-CULL-13, [`classify`] and [`eyes`]).
|
||||
//!
|
||||
//! Like `dr-segment`, this crate is **device-free**: no GPU adapter, no
|
||||
//! Slint, nothing that needs a display. Unlike `dr-segment`, it carries **no
|
||||
@@ -20,7 +22,8 @@
|
||||
//! runtime; this crate takes bytes and never fetches anything.
|
||||
//!
|
||||
//! docs/faces.md §2 is the full reading, including what would have to change
|
||||
//! for that to stop being true.
|
||||
//! for that to stop being true. The eye-state models are the exception: MIT,
|
||||
//! weights and all, and shipped in `models/face/` (docs/faces.md §17).
|
||||
//!
|
||||
//! # Why the runtime is split behind a feature
|
||||
//!
|
||||
@@ -43,6 +46,8 @@ pub mod detect;
|
||||
pub mod embed;
|
||||
pub mod embedding;
|
||||
pub mod eyes;
|
||||
#[cfg(feature = "inference")]
|
||||
pub mod landmarks;
|
||||
pub mod naming;
|
||||
pub mod neighbours;
|
||||
|
||||
@@ -68,7 +73,7 @@ pub mod neighbours;
|
||||
pub const MIN_CROP_EDGE: u32 = 1025;
|
||||
|
||||
pub use align::{
|
||||
eye_patches, head_views, warp, warp_pixels, Aligned112, EyePatch, EyePatches, HeadViews,
|
||||
crop_box, eye_box, eye_patch, head_views, warp, warp_pixels, Aligned112, EyePatch, HeadViews,
|
||||
Pixels, Similarity, ALIGNED_EDGE, ARCFACE_TEMPLATE,
|
||||
};
|
||||
pub use assign::{identity_shares, RIVAL_FLOOR, TOP_MATCHES};
|
||||
@@ -85,7 +90,12 @@ pub use embed::{Embedded, Embedder};
|
||||
pub use embedding::{
|
||||
in_gallery, read_f16_bytes, Embedding, ModelId, EMBEDDING_DIM, MIN_GALLERY_QUALITY,
|
||||
};
|
||||
pub use eyes::{EyeReading, EyeState, EYES_OPEN_THRESHOLD, SUNGLASSES_THRESHOLD};
|
||||
pub use eyes::{
|
||||
Eye, EyeReading, EyeState, EYES_OPEN_THRESHOLD, HIDDEN_EYE_RATIO, MIN_EYE_PX,
|
||||
MIN_EYE_SHARPNESS, SUNGLASSES_THRESHOLD,
|
||||
};
|
||||
#[cfg(feature = "inference")]
|
||||
pub use landmarks::{Landmarker, Landmarks};
|
||||
pub use naming::{name_for_instance, name_instances, NamedFace};
|
||||
|
||||
/// What can go wrong between an image and a face.
|
||||
|
||||
Reference in New Issue
Block a user