Measure the faces already found rather than finding them again
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m13s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 54s
Build and test / Layer separation (push) Failing after 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 52s
Build and test / Android (aarch64) (push) Successful in 30m6s
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m13s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 54s
Build and test / Layer separation (push) Failing after 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 52s
Build and test / Android (aarch64) (push) Successful in 30m6s
Every face stored before its quality was kept holds a unit vector, and V14 forgot the run marker of each image holding one so that the next sweep would look again. Looking again meant detecting again: a whole re-detection per image, with every suggestion on it thrown away and the confirmations carried across by box overlap, to recover one number. The sweep now has a measuring pass between the proxy repair and the un-indexed images. It lists every image holding an unmeasured face, fetches the original once, warps each stored face from the landmarks it already has, embeds it, and writes the raw vector and its length over the old row. Ids, boxes and identities are untouched; the marker is re-written fresh so the sync exports the measured vectors. A face whose landmarks no longer make a warp is dropped, as detection would have refused to store it. `faces_unindexed` leaves those images to the measuring pass, so the V14 deletion no longer costs a second detection.
This commit is contained in:
@@ -442,6 +442,64 @@ pub fn index_native(
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// What re-embedding the faces already on one image produced.
|
||||
#[derive(Debug, Default, PartialEq)]
|
||||
pub struct Measured {
|
||||
pub measured: Vec<faces::Measurement>,
|
||||
/// Faces whose stored landmarks no longer make a warp. See
|
||||
/// `dr_catalog::faces::record_measurements` for what becomes of them.
|
||||
pub dropped: Vec<faces::FaceId>,
|
||||
}
|
||||
|
||||
/// TRACES: FR-CULL-8 | FR-CULL-9
|
||||
/// Embed the faces already found on one image again, from its native render.
|
||||
///
|
||||
/// The measuring half of the sweep, for faces stored before their quality was
|
||||
/// kept (schema V14). No detector: the boxes and landmarks in the catalog are
|
||||
/// taken as read, scaled back from the long edge they were normalised to, and
|
||||
/// each face is warped out of the native frame and embedded exactly as
|
||||
/// [`index_native`] would have done on the day. What comes back is the raw
|
||||
/// vector and its length, to be written over the old unit one.
|
||||
///
|
||||
/// The size and sharpness gates are deliberately not re-applied. They decide
|
||||
/// whether a face is worth *storing*, and these are stored; what is being
|
||||
/// established now is how much the model can make of each, which is the
|
||||
/// quality itself, and a face that would have failed a gate is precisely one
|
||||
/// that should come out short and stop vouching for anyone.
|
||||
pub fn measure_native(
|
||||
embedder: &mut Embedder,
|
||||
rgba: &[u8],
|
||||
width: usize,
|
||||
height: usize,
|
||||
faces: &[faces::Face],
|
||||
) -> Result<Measured, dr_face::FaceError> {
|
||||
let mut out = Measured::default();
|
||||
if !index_native_shape_ok(rgba, width, height) {
|
||||
return Ok(out);
|
||||
}
|
||||
let long_edge = width.max(height) as f32;
|
||||
let native = dr_face::Pixels::Rgba8(rgba);
|
||||
|
||||
for f in faces {
|
||||
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
|
||||
for (o, &(x, y)) in landmarks.iter_mut().zip(f.landmarks.iter()) {
|
||||
*o = (x * long_edge, y * long_edge);
|
||||
}
|
||||
let Some(aligned) = dr_face::warp_pixels(native, width, height, &landmarks) else {
|
||||
log::debug!("face {:?} has degenerate landmarks, dropped", f.id);
|
||||
out.dropped.push(f.id);
|
||||
continue;
|
||||
};
|
||||
let embedded = embedder.embed(&aligned)?;
|
||||
out.measured.push(faces::Measurement {
|
||||
face: f.id,
|
||||
embedding: embedded.to_f16_bytes(),
|
||||
quality: embedded.quality,
|
||||
});
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Whether a buffer is the native frame it claims to be.
|
||||
///
|
||||
/// Checked before anything expensive, and before the detector above all: a
|
||||
@@ -1388,6 +1446,88 @@ mod tests {
|
||||
assert_eq!(out[4], (360.0, 200.0));
|
||||
}
|
||||
|
||||
/// The measuring path end to end, with the real embedder where one is on
|
||||
/// this machine: landmarks come back from the long edge they were
|
||||
/// normalised to, the warp is built from them, and what is stored is the
|
||||
/// raw vector, whose length is the quality reported beside it.
|
||||
#[test]
|
||||
fn measuring_stores_the_raw_vector_at_the_quality_it_reports() {
|
||||
let dir = crate::library::shared_face_models_dir();
|
||||
let path = dir.join("arcface_mbf_b1.onnx");
|
||||
if !path.is_file() {
|
||||
eprintln!("no embedder at {}, skipping", path.display());
|
||||
return;
|
||||
}
|
||||
let model = ModelId::new("w600k_mbf");
|
||||
let mut embedder = Embedder::from_path(&path, model.clone()).expect("load embedder");
|
||||
|
||||
// A 600×400 frame with a gradient in it, and one face whose landmarks
|
||||
// are the ArcFace template scaled up and placed in the middle -- not
|
||||
// a face, but an unambiguous warp.
|
||||
let (w, h) = (600usize, 400usize);
|
||||
let mut rgba = vec![0u8; w * h * 4];
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let i = (y * w + x) * 4;
|
||||
rgba[i] = (x * 255 / w) as u8;
|
||||
rgba[i + 1] = (y * 255 / h) as u8;
|
||||
rgba[i + 2] = ((x + y) % 256) as u8;
|
||||
rgba[i + 3] = 255;
|
||||
}
|
||||
}
|
||||
let long_edge = w.max(h) as f32;
|
||||
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
|
||||
for (o, &(tx, ty)) in landmarks.iter_mut().zip(dr_face::ARCFACE_TEMPLATE.iter()) {
|
||||
// Two and a half times the template, offset into the frame, then
|
||||
// normalised to the long edge as the catalog stores it.
|
||||
*o = (
|
||||
(tx * 2.5 + 150.0) / long_edge,
|
||||
(ty * 2.5 + 60.0) / long_edge,
|
||||
);
|
||||
}
|
||||
let stored = faces::Face {
|
||||
id: faces::FaceId(7),
|
||||
image_id: ImageId(1),
|
||||
x: 0.25,
|
||||
y: 0.15,
|
||||
w: 0.5,
|
||||
h: 0.7,
|
||||
landmarks,
|
||||
confidence: 0.9,
|
||||
crop_px: 280.0,
|
||||
quality: None,
|
||||
model_id: "w600k_mbf".into(),
|
||||
person: None,
|
||||
probability: 0.0,
|
||||
confirmed: false,
|
||||
};
|
||||
|
||||
let stored_copy = stored.clone();
|
||||
let out = measure_native(&mut embedder, &rgba, w, h, &[stored]).expect("measure");
|
||||
assert!(out.dropped.is_empty());
|
||||
assert_eq!(out.measured.len(), 1);
|
||||
let m = &out.measured[0];
|
||||
assert_eq!(m.face, faces::FaceId(7));
|
||||
assert!(m.quality > 0.0);
|
||||
let (_, length) = dr_face::read_f16_bytes(model, &m.embedding).expect("decode");
|
||||
assert!(
|
||||
(length - m.quality).abs() < 0.05 * m.quality,
|
||||
"stored length {length} against reported quality {}",
|
||||
m.quality
|
||||
);
|
||||
|
||||
// Degenerate landmarks -- five points on one spot, which no
|
||||
// similarity can be fitted to -- are dropped, not embedded.
|
||||
let junk = faces::Face {
|
||||
id: faces::FaceId(8),
|
||||
landmarks: [(0.3, 0.3); 5],
|
||||
..stored_copy
|
||||
};
|
||||
let out = measure_native(&mut embedder, &rgba, w, h, &[junk]).expect("measure");
|
||||
assert!(out.measured.is_empty());
|
||||
assert_eq!(out.dropped, vec![faces::FaceId(8)]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_buffer_that_is_not_the_stated_size_indexes_nothing() {
|
||||
// No model is loaded here, so reaching the detector would panic. The
|
||||
|
||||
Reference in New Issue
Block a user