Keep the dense landmarks behind each eye reading, packed
The 106 points the eye boxes were cut from, stored beside the reading as 16-bit fixed point over the frame: 424 bytes a face, a seventh of a pixel on a 6000-pixel frame, where f16 at the same size would have been six. Derived data like the embedding, kept for the same reason — it cost a fetch and a model run, and the next per-face pass should run from the catalog. Shards carry it; a peer's shard from before it is still read.
This commit is contained in:
@@ -39,7 +39,7 @@ use crate::align::{
|
||||
SUNGLASSES_EDGE,
|
||||
};
|
||||
use crate::eyes::{Eye, EyeReading};
|
||||
use crate::landmarks::Landmarker;
|
||||
use crate::landmarks::{Landmarker, Landmarks};
|
||||
use crate::{install_backend, FaceError, Pixels};
|
||||
|
||||
/// A loaded OCEC graph.
|
||||
@@ -202,13 +202,18 @@ impl EyeModels {
|
||||
})
|
||||
}
|
||||
|
||||
/// Read one face's eyes.
|
||||
/// Read one face's eyes, and hand back the dense landmarks it read them
|
||||
/// from.
|
||||
///
|
||||
/// `bbox` is the detector's `(x0, y0, x1, y1)` and `landmarks5` its five
|
||||
/// points, both in source pixels; the buffer is the one the aligned
|
||||
/// crop was taken from, so an eye is read from the same pixels the
|
||||
/// embedder saw the face in. `None` where nothing could be cut — a
|
||||
/// degenerate box or landmarks — which the caller stores as "not read".
|
||||
///
|
||||
/// The landmarks come back because they cost a model run the caller will
|
||||
/// not want to pay twice: stored beside the reading, a later pass over
|
||||
/// faces — head pose, expression — has them without the original.
|
||||
pub fn read(
|
||||
&mut self,
|
||||
px: Pixels<'_>,
|
||||
@@ -216,7 +221,7 @@ impl EyeModels {
|
||||
height: usize,
|
||||
bbox: (f32, f32, f32, f32),
|
||||
landmarks5: &[(f32, f32); 5],
|
||||
) -> Result<Option<EyeReading>, FaceError> {
|
||||
) -> Result<Option<(EyeReading, Landmarks)>, FaceError> {
|
||||
let Some(lm) = self.landmarks.landmarks(px, width, height, bbox)? else {
|
||||
return Ok(None);
|
||||
};
|
||||
@@ -249,10 +254,11 @@ impl EyeModels {
|
||||
};
|
||||
let right = eye(&lm.right_eye())?;
|
||||
let left = eye(&lm.left_eye())?;
|
||||
Ok(Some(EyeReading {
|
||||
let reading = EyeReading {
|
||||
right,
|
||||
left,
|
||||
sunglasses: self.sunglasses.classify(&head)?,
|
||||
}))
|
||||
};
|
||||
Ok(Some((reading, lm)))
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user