dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Keep each face's quality, and never compare against a poor one

Browse Source 
The embedder's raw output has a length, and the length is a reading of
how recognisable the crop was: a blur, an occlusion or a hard profile
comes out short. Normalising threw it away. A short vector sits near
the middle of the sphere and matches a little of everyone, which is how
one bad crop bridges two people in a grouping pass.

So the length is kept — the store now holds the raw vector, re-normalised
on load, with the length beside it as `faces.quality` — and a face under
MIN_GALLERY_QUALITY (14) is a probe: measured against the gallery and
placed where it fits, but never what another face is measured against.
Two probes are never paired, and a probe is nobody's evidence for a
confidence. The People screen shows the number as "Quality 17.3", dimmed
below the floor.

Faces indexed before this stored unit vectors and have no reading; they
are admitted to the gallery, and schema V14 forgets the run marker of
every image holding one so the next indexing pass measures them. A
peer's unmeasured shard faces are not adopted, or a sync would write
that marker back.
This commit is contained in:
dtourolle
2026-09-11 21:50:12 +02:00
parent a87139b838
commit 8b3abdb787
22 changed files with 1070 additions and 149 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-face/src/embedding.rs
+119 -18
View File
@@ -12,6 +12,43 @@
/// Embedding dimensionality. Fixed by the model family, not a parameter.
pub const EMBEDDING_DIM: usize = 512;
/// The shortest raw embedding a face may be *compared against*.
///
/// # What the length of the vector says
///
/// ArcFace is trained on the direction of its output and nothing else, and
/// the length it leaves behind turns out to be a free quality signal: the
/// magnitude grows with how recognisable the crop was to the model, and a
/// blurred, occluded, badly lit or hard-profile face comes out short. MagFace
/// (Meng et al., CVPR 2021) made that the training objective; the plain
/// ArcFace heads this crate runs already show it, weaker but usable, which is
/// why it is worth keeping the number the normalisation discards.
///
/// # Why it gates the gallery and not the face
///
/// A short vector is a bad *reference*: it sits nearer the centre of the
/// sphere than a real identity does and matches a little of everyone, which
/// is exactly the face that welds two people together in a clustering pass.
/// It is not a bad *probe* — the face is still real, still somebody, and
/// comparing it against good references is the only way it will ever be named.
/// So a face below this floor is compared against the gallery and never
/// becomes part of it: see `cluster::Candidate::in_gallery`.
///
/// 14 is the operating point for `w600k_mbf`, whose norms on the reference
/// library run from about 8 on a blur to the high 20s on a clean portrait. A
/// face whose quality was never recorded — indexed before the number was kept
/// — is not gated, because a rule that cannot be checked should admit, not
/// exclude.
pub const MIN_GALLERY_QUALITY: f32 = 14.0;
/// Whether an embedding of this quality may serve as a reference.
///
/// `None` is "not measured", and is admitted: the rule is about a number that
/// was read and found short, not about a number that is missing.
pub fn in_gallery(quality: Option<f32>) -> bool {
quality.is_none_or(|q| q >= MIN_GALLERY_QUALITY)
}
/// Which model produced an embedding.
///
/// Embeddings from different models are not comparable, and this is the one
@@ -58,10 +95,19 @@ impl Embedding {
}
/// Storage form: `512 × f16`, 1 KB per face (catalog.md §10.1).
///
/// This writes the unit vector. What the catalog stores is the raw one —
/// `embed::Embedded::to_f16_bytes` — because the length is the quality
/// and a unit vector has none left to read.
pub fn to_f16_bytes(&self) -> Vec<u8> {
self.to_f16_bytes_scaled(1.0)
}
/// The unit vector scaled by `length`, as `512 × f16`.
pub(crate) fn to_f16_bytes_scaled(&self, length: f32) -> Vec<u8> {
let mut out = Vec::with_capacity(EMBEDDING_DIM * 2);
for &x in self.v.iter() {
out.extend_from_slice(&f32_to_f16_bits(x).to_le_bytes());
out.extend_from_slice(&f32_to_f16_bits(x * length).to_le_bytes());
}
out
}
@@ -71,25 +117,42 @@ impl Embedding {
/// The f16 round-trip perturbs a unit vector by ~1e-3 in cosine — three
/// orders below the separation between a match and a non-match — but the
/// drift is free to remove and invisible if left, so it is removed here
/// rather than remembered at every call site.
/// rather than remembered at every call site. The same pass is what turns
/// a stored raw vector back into the unit one every comparison expects.
pub fn from_f16_bytes(model: ModelId, bytes: &[u8]) -> Option<Self> {
if bytes.len() != EMBEDDING_DIM * 2 {
return None;
}
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
for (i, chunk) in bytes.chunks_exact(2).enumerate() {
v[i] = f16_bits_to_f32(u16::from_le_bytes([chunk[0], chunk[1]]));
}
normalise(&mut v);
Some(Self { model, v })
read_f16_bytes(model, bytes).map(|(e, _)| e)
}
}
/// Read a stored vector back, with the length it was stored at.
///
/// The length is the quality where the blob is a raw one, and ~1 where it is
/// a unit vector from before raw vectors were stored — which is why the
/// catalog keeps the quality beside the blob rather than deriving it from
/// this: a unit vector reads as a quality of 1, not as "unmeasured".
pub fn read_f16_bytes(model: ModelId, bytes: &[u8]) -> Option<(Embedding, f32)> {
if bytes.len() != EMBEDDING_DIM * 2 {
return None;
}
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
for (i, chunk) in bytes.chunks_exact(2).enumerate() {
v[i] = f16_bits_to_f32(u16::from_le_bytes([chunk[0], chunk[1]]));
}
let length = normalise(&mut v);
Some((Embedding { model, v }, length))
}
fn dot(a: &[f32; EMBEDDING_DIM], b: &[f32; EMBEDDING_DIM]) -> f32 {
a.iter().zip(b.iter()).map(|(x, y)| x * y).sum()
}
pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) {
/// Scale `v` to unit length, and return the length it had.
///
/// The length is the one thing about the raw output that survives being
/// thrown away by everything downstream, and it is a quality signal
/// ([`MIN_GALLERY_QUALITY`]) — so it comes back out rather than being lost
/// here.
pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) -> f32 {
// Clamped rather than checked: a zero-norm embedding is a broken model,
// not a runtime condition worth an error path, and dividing by 1e-6 keeps
// the NaN out of the catalog.
@@ -97,6 +160,7 @@ pub(crate) fn normalise(v: &mut [f32; EMBEDDING_DIM]) {
for x in v.iter_mut() {
*x /= norm;
}
norm
}
// ── f16 ───────────────────────────────────────────────────────────────────
@@ -113,9 +177,10 @@ fn f32_to_f16_bits(x: f32) -> u16 {
let mant = bits & 0x007f_ffff;
if exp >= 0x1f {
// Overflow, inf, or NaN. Embeddings are unit-norm so this is the
// broken-model path; infinity is the honest answer, not a clamp that
// hides it.
// Overflow, inf, or NaN. No component of an embedding exceeds its
// length, and the lengths this model produces are in the tens, so
// this is the broken-model path; infinity is the honest answer, not a
// clamp that hides it.
return sign
| 0x7c00
| if mant != 0 && exp == 0x1f + 112 {
@@ -125,9 +190,9 @@ fn f32_to_f16_bits(x: f32) -> u16 {
};
}
if exp <= 0 {
// Subnormal or underflow. A component of a unit 512-vector is ~0.04,
// nowhere near here, so this branch exists for correctness rather than
// for traffic.
// Subnormal or underflow. A component of a unit 512-vector is ~0.04
// and a stored one is that times the length, nowhere near here, so
// this branch exists for correctness rather than for traffic.
if exp < -10 {
return sign;
}
@@ -221,6 +286,42 @@ mod tests {
}
}
#[test]
fn normalising_reports_the_length_it_removed() {
let mut v = Box::new([0.0_f32; EMBEDDING_DIM]);
v[0] = 3.0;
v[1] = 4.0;
let norm = normalise(&mut v);
assert!((norm - 5.0).abs() < 1e-6, "norm {norm}");
assert!((v[0] - 0.6).abs() < 1e-6 && (v[1] - 0.8).abs() < 1e-6);
}
/// The gate admits what it cannot measure: a face from before the number
/// was kept is not a face that was found wanting.
#[test]
fn an_unmeasured_quality_is_admitted_to_the_gallery() {
assert!(in_gallery(None));
assert!(in_gallery(Some(MIN_GALLERY_QUALITY)));
assert!(in_gallery(Some(27.5)));
assert!(!in_gallery(Some(MIN_GALLERY_QUALITY - 0.01)));
assert!(!in_gallery(Some(8.0)));
}
/// The storage form carries the length, and the length comes back out —
/// without touching the direction every comparison is made on.
#[test]
fn a_raw_vector_round_trips_with_its_length() {
let e = unit(3);
let raw = e.to_f16_bytes_scaled(21.5);
let (back, length) = read_f16_bytes(e.model.clone(), &raw).unwrap();
assert!((length - 21.5).abs() < 0.05, "length {length}");
assert!(e.cosine(&back).unwrap() > 0.9999);
// A unit vector from an older store reads as length 1, not as an
// error — see `read_f16_bytes` on why that is not "unmeasured".
let (_, one) = read_f16_bytes(e.model.clone(), &e.to_f16_bytes()).unwrap();
assert!((one - 1.0).abs() < 1e-2, "length {one}");
}
#[test]
fn f16_round_trip_rejects_a_wrong_length_blob() {
assert!(Embedding::from_f16_bytes(ModelId::new("m"), &[0u8; 100]).is_none());