dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Filter the grid to a person with their eyes open

Browse Source 
An "Eyes open" chip beside the people chips, offered only while someone
is chosen and dropped when the last person goes, so no term narrows the
grid with nothing on the bar to say so. It compiles the rule in
dr_face::eyes into the person's face subquery — Anna, eyes open, whoever
else is blinking beside her — and drops a frame only on a closed eye that
could be read: sunglasses, eyes too small or soft to read, and faces never
read all pass, so an old library shows everything under the chip until
the measuring pass has run. A test drives the same readings through the
SQL and through the rule and requires them to agree.

The People screen badges a face "Eyes closed", "Sunglasses" or "Eyes
unclear" so the reason a frame is or is not in the grid can be read off
the face; the sweep loads the three models when they are beside the pair
and reads eyes on the indexing and measuring passes from the native
render; the coverage line counts unread faces as work to measure so an
already-indexed library keeps its Index button. The term travels with the
place.
This commit is contained in:
dtourolle
2026-09-19 14:04:35 +02:00
parent 6aae4c3eb0
commit 83f4253b6a
16 changed files with 984 additions and 183 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/faces.rs
+152 -7
View File
@@ -28,7 +28,10 @@ use std::sync::mpsc::{Receiver, Sender};
use dr_catalog::faces::{self, DetectedFace};
use dr_catalog::Catalog;
use dr_face::{align, Calibration, DetectOptions, Detection, Detector, Embedder, ModelId};
use dr_face::{
align, Calibration, DetectOptions, Detection, Detector, Embedder, EyeModels, EyeReading,
ModelId,
};
use dr_thumbs::{ThumbSize, ThumbStore};
use dr_types::settings::FaceSettings;
use dr_types::ImageId;
@@ -154,6 +157,14 @@ pub struct IndexAudit {
/// Outstanding with no proxy on disk. Costs a range request, same as the
/// ones above now do.
pub awaiting_proxy: u64,
/// TRACES: FR-CULL-13
/// Faces this model found that the measuring pass still has to read:
/// stored without their quality, or — on a device with the eye models —
/// without an eye reading. Work that is not visible in the coverage
/// figure, since every one of these images carries its run marker, and
/// that has to be counted here or the screen calls the library finished
/// and takes the button away that would finish it.
pub unmeasured: u64,
}
impl IndexAudit {
@@ -188,8 +199,19 @@ impl IndexAudit {
if outstanding > 0 {
s.push_str(&format!("; {outstanding} to index"));
}
if self.unmeasured > 0 {
s.push_str(&format!("; {} face(s) to measure", self.unmeasured));
}
s
}
/// Whether the sweep has nothing left to do — nothing to index *and*
/// nothing to measure. The screen hides the button on this, so it has to
/// be false while the measuring pass has work, or the eye readings of an
/// already-indexed library could never be filled in.
pub fn is_complete(&self) -> bool {
self.coverage.is_complete() && self.unmeasured == 0
}
}
/// Check every library image for a face-detection run marker.
@@ -197,13 +219,19 @@ impl IndexAudit {
/// The batch pass that answers "has face recognition been over all of this",
/// and the one to run before deciding whether to start a sweep. Cheap: two
/// counts and one indexed scan, no decoding and no inference.
///
/// `eyes` is whether this device has the eye models, and it decides whether
/// a face without an eye reading counts as work — see
/// `dr_catalog::faces::faces_unmeasured`.
pub fn audit(
catalog: &Catalog,
store: &ThumbStore,
model_id: &str,
eyes: bool,
) -> Result<IndexAudit, dr_catalog::CatalogError> {
let conn = catalog.connection();
let coverage = faces::coverage(conn, model_id)?;
let unmeasured = faces::faces_unmeasured(conn, model_id, eyes)?;
// Split the outstanding set by whether a proxy exists. This is the query
// `faces_outstanding` runs without the store filter, so the two cannot
@@ -240,9 +268,40 @@ pub fn audit(
coverage,
ready,
awaiting_proxy: awaiting,
unmeasured,
})
}
/// TRACES: FR-CULL-13
/// Read one face's eyes from the buffer its crop came from, if this device
/// can.
///
/// From the same pixels the embedder saw the face in, so what the models
/// see is the eye at the resolution the crop had — and `dr_face::eyes`'
/// readability floors are judged against real pixels rather than a proxy's
/// idea of them. `bbox` is the detector's `(x0, y0, x1, y1)` and `landmarks`
/// its five points, both in this buffer's pixels. `None` on a device without
/// the models, which is the ordinary state of one that has not been given
/// them, and `None` — logged — where the models refuse: a face the embedder
/// could use is not lost for the want of an eye reading.
fn read_eyes(
models: Option<&mut EyeModels>,
px: dr_face::Pixels<'_>,
width: usize,
height: usize,
bbox: (f32, f32, f32, f32),
landmarks: &[(f32, f32); 5],
) -> Option<EyeReading> {
let models = models?;
match models.read(px, width, height, bbox, landmarks) {
Ok(r) => r,
Err(e) => {
log::debug!("eye reading failed: {e}");
None
}
}
}
/// Detect and embed every face in one decoded proxy.
///
/// Coordinates come back **normalised to the long edge**, which is what the
@@ -256,6 +315,7 @@ pub fn audit(
pub fn index_proxy(
detector: &mut Detector,
embedder: &mut Embedder,
mut eye_models: Option<&mut EyeModels>,
rgb: &[f32],
width: usize,
height: usize,
@@ -310,6 +370,14 @@ pub fn index_proxy(
}
let embedded = embedder.embed(&aligned)?;
let eyes = read_eyes(
eye_models.as_deref_mut(),
dr_face::Pixels::RgbF32(rgb),
width,
height,
d.bbox,
&d.landmarks,
);
out.push(DetectedFace {
x: d.bbox.0 / long_edge,
@@ -321,6 +389,7 @@ pub fn index_proxy(
embedding: embedded.to_f16_bytes(),
crop_px: aligned.source_px(),
quality: Some(embedded.quality),
eyes,
model_id: embedder.model().as_str().to_string(),
// Cut here, while the buffer is still in hand. This is the only
// moment in the whole pipeline where the pixels are free.
@@ -370,6 +439,7 @@ const DETECT_EDGE: usize = 1600;
pub fn index_native(
detector: &mut Detector,
embedder: &mut Embedder,
mut eye_models: Option<&mut EyeModels>,
rgba: &[u8],
width: usize,
height: usize,
@@ -426,6 +496,17 @@ pub fn index_native(
let embedded = embedder.embed(&aligned)?;
let (bx, by) = (d.bbox.0 * sx, d.bbox.1 * sy);
let (bw, bh) = (d.width() * sx, d.height() * sy);
// From the native buffer, box and landmarks scaled like the crop's
// — the eye is a fortieth of the face, and it is here that the
// native render pays for itself twice.
let eyes = read_eyes(
eye_models.as_deref_mut(),
native,
width,
height,
(bx, by, bx + bw, by + bh),
&landmarks,
);
out.push(DetectedFace {
x: bx / long_edge,
@@ -437,6 +518,7 @@ pub fn index_native(
embedding: embedded.to_f16_bytes(),
crop_px: aligned.source_px(),
quality: Some(embedded.quality),
eyes,
model_id: embedder.model().as_str().to_string(),
crop: cut_crop_native(native, width, height, (bx, by, bw, bh)).unwrap_or_default(),
});
@@ -469,8 +551,14 @@ pub struct Measured {
/// established now is how much the model can make of each, which is the
/// quality itself, and a face that would have failed a gate is precisely one
/// that should come out short and stop vouching for anyone.
///
/// The eyes are read on the same pass where the device has the models, for
/// the faces that have no reading yet (schema V16): the pixels are in hand,
/// and the eye is a window on the same landmarks. A face that already has
/// one keeps it.
pub fn measure_native(
embedder: &mut Embedder,
mut eye_models: Option<&mut EyeModels>,
rgba: &[u8],
width: usize,
height: usize,
@@ -494,10 +582,29 @@ pub fn measure_native(
continue;
};
let embedded = embedder.embed(&aligned)?;
let eyes = if f.eyes.is_some() {
None
} else {
let bbox = (
f.x * long_edge,
f.y * long_edge,
(f.x + f.w) * long_edge,
(f.y + f.h) * long_edge,
);
read_eyes(
eye_models.as_deref_mut(),
native,
width,
height,
bbox,
&landmarks,
)
};
out.measured.push(faces::Measurement {
face: f.id,
embedding: embedded.to_f16_bytes(),
quality: embedded.quality,
eyes,
});
}
Ok(out)
@@ -658,8 +765,7 @@ fn normalise_landmarks(lm: &[(f32, f32); 5], long_edge: f32) -> [(f32, f32); 5]
pub fn spawn_store_face_sweep(
catalog_path: PathBuf,
store_dir: PathBuf,
detector_model: PathBuf,
embedder_model: PathBuf,
models: crate::FaceModelPaths,
model_id: String,
options: DetectOptions,
) -> Receiver<FaceSweepMessage> {
@@ -696,7 +802,7 @@ pub fn spawn_store_face_sweep(
// missing. This is also the path a library with face indexing enabled
// but no model downloaded takes (docs/faces.md §2.2), so it must be a
// quiet return rather than an error.
let mut detector = match Detector::from_path(&detector_model) {
let mut detector = match Detector::from_path(&models.detector) {
Ok(d) => d,
Err(e) => {
log::warn!("face sweep: cannot load the detector: {e}");
@@ -705,7 +811,7 @@ pub fn spawn_store_face_sweep(
}
};
let mut embedder =
match Embedder::from_path(&embedder_model, ModelId::new(model_id.clone())) {
match Embedder::from_path(&models.embedder, ModelId::new(model_id.clone())) {
Ok(e) => e,
Err(e) => {
log::warn!("face sweep: cannot load the embedder: {e}");
@@ -713,6 +819,7 @@ pub fn spawn_store_face_sweep(
return;
}
};
let mut eye_models = models.load_eyes();
// Nothing below this point can succeed, so do not pretend to try.
//
@@ -782,6 +889,7 @@ pub fn spawn_store_face_sweep(
let faces = match index_proxy(
&mut detector,
&mut embedder,
eye_models.as_mut(),
&rgb,
w as usize,
h as usize,
@@ -1360,6 +1468,7 @@ fn cut_crop(rgb: &[f32], width: usize, height: usize, d: &Detection) -> Option<V
pub fn index_preview(
detector: &mut Detector,
embedder: &mut Embedder,
eye_models: Option<&mut EyeModels>,
preview: &dr_decode::Preview,
options: &DetectOptions,
) -> Result<(Vec<DetectedFace>, u32), dr_face::FaceError> {
@@ -1367,6 +1476,7 @@ pub fn index_preview(
let faces = index_proxy(
detector,
embedder,
eye_models,
&rgb,
preview.width as usize,
preview.height as usize,
@@ -1499,6 +1609,7 @@ mod tests {
confidence: 0.9,
crop_px: 280.0,
quality: None,
eyes: None,
model_id: "w600k_mbf".into(),
person: None,
probability: 0.0,
@@ -1506,7 +1617,7 @@ mod tests {
};
let stored_copy = stored.clone();
let out = measure_native(&mut embedder, &rgba, w, h, &[stored]).expect("measure");
let out = measure_native(&mut embedder, None, &rgba, w, h, &[stored]).expect("measure");
assert!(out.dropped.is_empty());
assert_eq!(out.measured.len(), 1);
let m = &out.measured[0];
@@ -1526,7 +1637,7 @@ mod tests {
landmarks: [(0.3, 0.3); 5],
..stored_copy
};
let out = measure_native(&mut embedder, &rgba, w, h, &[junk]).expect("measure");
let out = measure_native(&mut embedder, None, &rgba, w, h, &[junk]).expect("measure");
assert!(out.measured.is_empty());
assert_eq!(out.dropped, vec![faces::FaceId(8)]);
}
@@ -1552,6 +1663,7 @@ mod tests {
},
ready: 30,
awaiting_proxy: 10,
unmeasured: 0,
};
let s = a.summary();
assert!(s.contains("60/100"), "{s}");
@@ -1577,11 +1689,40 @@ mod tests {
},
ready: 0,
awaiting_proxy: 0,
unmeasured: 0,
};
let s = a.summary();
assert!(!s.contains("ready"), "{s}");
assert!(!s.contains("awaiting"), "{s}");
assert!(!s.contains("measure"), "{s}");
assert!(s.contains("10/10"), "{s}");
assert!(a.is_complete());
}
/// TRACES: FR-CULL-13
/// A fully indexed library whose faces have not been read is not
/// finished: the screen hides the button on `is_complete`, and this is
/// the only way the readings of an old library ever get filled in.
#[test]
fn faces_left_to_measure_keep_the_audit_incomplete() {
let a = IndexAudit {
coverage: faces::Coverage {
images: 10,
indexed: 10,
without_faces: 7,
faces: 4,
},
ready: 0,
awaiting_proxy: 0,
unmeasured: 4,
};
assert!(a.coverage.is_complete());
assert!(!a.is_complete());
assert!(
a.summary().contains("4 face(s) to measure"),
"{}",
a.summary()
);
}
/// The figure the real library actually produced: 110 of 23,528 rounds to
@@ -1597,6 +1738,7 @@ mod tests {
},
ready: 69,
awaiting_proxy: 23_349,
unmeasured: 0,
};
let s = a.summary();
assert!(s.contains("0.5%"), "{s}");
@@ -1614,6 +1756,7 @@ mod tests {
},
ready: 99_999,
awaiting_proxy: 0,
unmeasured: 0,
};
assert!(a.summary().contains("<0.1%"), "{}", a.summary());
}
@@ -1663,6 +1806,7 @@ mod tests {
confidence: 0.9,
crop_px: 120.0,
quality: None,
eyes: None,
model_id: "w600k_mbf".into(),
person: None,
probability: 0.0,
@@ -1781,6 +1925,7 @@ mod tests {
embedding: embedding(identity, cosine),
crop_px: 150.0,
quality: None,
eyes: None,
model_id: TEST_MODEL.to_string(),
crop: Vec::new(),
};