Read each face's eyes, and whether sunglasses hide them

Two MIT classifiers from the same author as the reference pipeline's
whole-body detector: OCEC answers P(open) for one 40×24 eye, SGC
P(sunglasses) for a 48×48 head. Both load in tract once their batch
dimension is pinned by tools/fix-face-model-shapes.sh, like the embedder.

The crops come through the same fitted similarity the aligned face does,
so an eye window is a constant in template units rather than a second
warp, and a tilted head yields an upright eye. Measured on 60 proxies
from the reference library: the eye window plateaus at 22×11, the S
variant beats M and L (which overfit their own domain), and for
sunglasses the aligned face beats a head framing but the higher of the
two catches 11 of 12 pairs against 9 for either alone.

The reading keeps both eyes and the sunglasses number apart, because a
wink averages to the least informative value and a lens of dark glass
draws a confident answer from the eye classifier — over a woman in
sunglasses it read the right eye 0.97 open. Sunglasses take precedence,
and a face behind them is neither open nor a blink.
This commit is contained in:
2026-09-19 14:03:31 +02:00
parent 2481904016
commit 6b51726322
7 changed files with 931 additions and 22 deletions
+8 -1
View File
@@ -34,12 +34,15 @@
pub mod align;
pub mod assign;
pub mod calibrate;
#[cfg(feature = "inference")]
pub mod classify;
pub mod cluster;
#[cfg(feature = "inference")]
pub mod detect;
#[cfg(feature = "inference")]
pub mod embed;
pub mod embedding;
pub mod eyes;
pub mod naming;
pub mod neighbours;
@@ -65,10 +68,13 @@ pub mod neighbours;
pub const MIN_CROP_EDGE: u32 = 1025;
pub use align::{
warp, warp_pixels, Aligned112, Pixels, Similarity, ALIGNED_EDGE, ARCFACE_TEMPLATE,
eye_patches, head_views, warp, warp_pixels, Aligned112, EyePatch, EyePatches, HeadViews,
Pixels, Similarity, ALIGNED_EDGE, ARCFACE_TEMPLATE,
};
pub use assign::{identity_shares, RIVAL_FLOOR, TOP_MATCHES};
pub use calibrate::{Calibration, Pairs, ReliabilityBand};
#[cfg(feature = "inference")]
pub use classify::{EyeClassifier, EyeModels, SunglassesClassifier};
pub use cluster::{
cluster, cluster_scored, split, Candidate, Cluster, Grouping, DEFAULT_MERGE_PROBABILITY,
};
@@ -79,6 +85,7 @@ pub use embed::{Embedded, Embedder};
pub use embedding::{
in_gallery, read_f16_bytes, Embedding, ModelId, EMBEDDING_DIM, MIN_GALLERY_QUALITY,
};
pub use eyes::{EyeReading, EyeState, EYES_OPEN_THRESHOLD, SUNGLASSES_THRESHOLD};
pub use naming::{name_for_instance, name_instances, NamedFace};
/// What can go wrong between an image and a face.