Keep each face's quality, and never compare against a poor one
The embedder's raw output has a length, and the length is a reading of how recognisable the crop was: a blur, an occlusion or a hard profile comes out short. Normalising threw it away. A short vector sits near the middle of the sphere and matches a little of everyone, which is how one bad crop bridges two people in a grouping pass. So the length is kept — the store now holds the raw vector, re-normalised on load, with the length beside it as `faces.quality` — and a face under MIN_GALLERY_QUALITY (14) is a probe: measured against the gallery and placed where it fits, but never what another face is measured against. Two probes are never paired, and a probe is nobody's evidence for a confidence. The People screen shows the number as "Quality 17.3", dimmed below the floor. Faces indexed before this stored unit vectors and have no reading; they are admitted to the gallery, and schema V14 forgets the run marker of every image holding one so the next indexing pass measures them. A peer's unmeasured shard faces are not adopted, or a sync would write that marker back.
This commit is contained in:
@@ -16,6 +16,41 @@ use crate::align::{Aligned112, ALIGNED_EDGE};
|
||||
use crate::embedding::{normalise, Embedding, ModelId, EMBEDDING_DIM};
|
||||
use crate::{install_backend, FaceError};
|
||||
|
||||
/// What one pass of the embedder produces: the direction, and the length.
|
||||
///
|
||||
/// Two fields rather than a `quality` on [`Embedding`], because every other
|
||||
/// holder of an `Embedding` relies on it being unit length and compares by
|
||||
/// dot product; the length is a separate fact about the same face, and it is
|
||||
/// stored separately too.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Embedded {
|
||||
pub embedding: Embedding,
|
||||
/// L2 norm of the raw model output.
|
||||
///
|
||||
/// The model's own opinion of how recognisable the crop was — see
|
||||
/// [`crate::embedding::MIN_GALLERY_QUALITY`] for what it means and where
|
||||
/// it is used.
|
||||
pub quality: f32,
|
||||
}
|
||||
|
||||
impl Embedded {
|
||||
/// Storage form: the **raw** vector, `512 × f16`.
|
||||
///
|
||||
/// Not the unit vector. The length is the quality, and a store that held
|
||||
/// only the direction would have thrown it away at the one moment it could
|
||||
/// be known — which is what this crate used to do. Readers re-normalise
|
||||
/// ([`Embedding::from_f16_bytes`]), so every comparison is still a dot
|
||||
/// product, and [`crate::embedding::read_f16_bytes`] gives the length back
|
||||
/// to a reader that wants it.
|
||||
///
|
||||
/// f16 costs nothing extra at this scale: its precision is relative, so a
|
||||
/// component of a vector of length 20 is kept to the same three figures as
|
||||
/// the same component scaled to length 1.
|
||||
pub fn to_f16_bytes(&self) -> Vec<u8> {
|
||||
self.embedding.to_f16_bytes_scaled(self.quality)
|
||||
}
|
||||
}
|
||||
|
||||
/// A loaded ArcFace graph.
|
||||
pub struct Embedder {
|
||||
session: ort::session::Session,
|
||||
@@ -63,7 +98,7 @@ impl Embedder {
|
||||
}
|
||||
|
||||
/// Embed one aligned face.
|
||||
pub fn embed(&mut self, face: &Aligned112) -> Result<Embedding, FaceError> {
|
||||
pub fn embed(&mut self, face: &Aligned112) -> Result<Embedded, FaceError> {
|
||||
// `(x·255 − 127.5) / 128` — see the `/128` note in `detect::Letterbox`.
|
||||
let px = face.pixels();
|
||||
let mut input = Array4::<f32>::zeros((1, 3, ALIGNED_EDGE, ALIGNED_EDGE));
|
||||
@@ -95,11 +130,14 @@ impl Embedder {
|
||||
|
||||
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
|
||||
v.copy_from_slice(&data[..EMBEDDING_DIM]);
|
||||
normalise(&mut v);
|
||||
let quality = normalise(&mut v);
|
||||
|
||||
Ok(Embedding {
|
||||
model: self.model.clone(),
|
||||
v,
|
||||
Ok(Embedded {
|
||||
embedding: Embedding {
|
||||
model: self.model.clone(),
|
||||
v,
|
||||
},
|
||||
quality,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user