dtourolle
8b3abdb787
The embedder's raw output has a length, and the length is a reading of how recognisable the crop was: a blur, an occlusion or a hard profile comes out short. Normalising threw it away. A short vector sits near the middle of the sphere and matches a little of everyone, which is how one bad crop bridges two people in a grouping pass. So the length is kept — the store now holds the raw vector, re-normalised on load, with the length beside it as `faces.quality` — and a face under MIN_GALLERY_QUALITY (14) is a probe: measured against the gallery and placed where it fits, but never what another face is measured against. Two probes are never paired, and a probe is nobody's evidence for a confidence. The People screen shows the number as "Quality 17.3", dimmed below the floor. Faces indexed before this stored unit vectors and have no reading; they are admitted to the gallery, and schema V14 forgets the run marker of every image holding one so the next indexing pass measures them. A peer's unmeasured shard faces are not adopted, or a sync would write that marker back.
400 lines
17 KiB
Rust
400 lines
17 KiB
Rust
//! TRACES: FR-CULL-9 | FR-CULL-10
|
||
//! How sure a *suggestion* is: an identity's share of the evidence for a face.
|
||
//!
|
||
//! [`crate::cluster`] decides which people exist; this decides what number to
|
||
//! put beside "we think this is Anna". They are not the same question, and the
|
||
//! answer to the second used to be a by-product of the first — the mean
|
||
//! calibrated probability between a face and *every* other member of its group.
|
||
//!
|
||
//! # Why a mean over the group is the wrong number
|
||
//!
|
||
//! It measures the wrong thing twice over.
|
||
//!
|
||
//! **It punishes large, well-photographed people.** Anna has two hundred faces
|
||
//! spanning fifteen years; a new photograph of her matches thirty of them
|
||
//! strongly and is near-orthogonal to the rest, because a face at 8 and a face
|
||
//! at 23 genuinely are. The mean lands around 0.2 and the interface reports a
|
||
//! correct suggestion as a doubtful one. The better a person is covered, the
|
||
//! worse their confidences get, which is exactly backwards.
|
||
//!
|
||
//! **It never asks who else it could be.** A face that matches Anna at 0.95 and
|
||
//! matches nobody else at all, and a face that matches Anna at 0.95 *and her
|
||
//! sister at 0.93*, are the same number under a within-group mean. The second
|
||
//! is the one the user actually needs to look at, and it was indistinguishable
|
||
//! from the first.
|
||
//!
|
||
//! # Coherence, times uniqueness
|
||
//!
|
||
//! Two questions, and the number is their product because they are genuinely
|
||
//! independent: *is this the same person at all*, and *of the people we know,
|
||
//! is it uniquely this one*.
|
||
//!
|
||
//! ```text
|
||
//! evidence(P) = Σ of the top n of { P(same | this face, f) : f ∈ P }
|
||
//! coherence = evidence(own) / (however many of the top n there were)
|
||
//! uniqueness = evidence(own) / (evidence(own) + Σ evidence(named rivals))
|
||
//! confidence = coherence × uniqueness
|
||
//! ```
|
||
//!
|
||
//! **Coherence** is a mean, like the old number, but over the face's best
|
||
//! [`TOP_MATCHES`] matches into the identity rather than over all of them. That single cap is what
|
||
//! stops a well-photographed person scoring worse than a thin one: the two
|
||
//! hundred faces a given photograph is legitimately orthogonal to no longer
|
||
//! count against it.
|
||
//!
|
||
//! **Uniqueness** is the competition. A sole strong match leaves it at 1 and
|
||
//! the confidence is the coherence; two identities matching equally well pull
|
||
//! it to 0.5 each, and the screen has told the user the truth, which is that
|
||
//! this face is a coin toss between two people.
|
||
//!
|
||
//! # Only the people the user has named compete
|
||
//!
|
||
//! Measured on a real 18,000-face library, normalising across *every* group
|
||
//! made the number useless: the median suggestion read 21% and four in five
|
||
//! read under half. The cause is not a bug in the arithmetic but a fact about
|
||
//! clustering — one person is spread across many groups, since the pairs that
|
||
//! would have joined them are the ones that fell short of the merge threshold.
|
||
//! Normalising over groups therefore makes a face compete against *itself*,
|
||
//! and the better covered the person, the more fragments there are to lose to.
|
||
//!
|
||
//! A fragment is not a rival. An identity the user has actually asserted is, so
|
||
//! the denominator counts only the people they have ruled on — a group carries
|
||
//! a [`Cluster::person`] when it holds a confirmation, a name, or an ignore —
|
||
//! and counts them **per person, not per group**, since one person is left in
|
||
//! several anchored groups for the same reason. Keying it by group had
|
||
//! Catherine competing with Catherine and put the median suggestion onto a
|
||
//! named person at 39%; keying it by person put it at 99.5%.
|
||
//!
|
||
//! Leave-one-out over that library's 2,702 confirmations across 54 named
|
||
//! people, this is the regime where the number is worth having: 99.3% of faces
|
||
//! are placed on the right person (the old mean managed 99.15%), the stated
|
||
//! percentage is monotone in being right, and it errs low — 100% correct
|
||
//! wherever it states 80% or more, 84% correct where it states under half.
|
||
//! Understating is the safe direction for a screen whose whole purpose is
|
||
//! deciding what to look at first, but it is *not* calibrated in the low bands
|
||
//! and should not be read as though it were.
|
||
//!
|
||
//! Two faces of one unnamed group cannot be told from two fragments of one
|
||
//! person by similarity alone; that is exactly why clustering stopped where it
|
||
//! did. So the module does not pretend to: where nobody is named, uniqueness is
|
||
//! 1 and the number falls back to plain coherence.
|
||
//!
|
||
//! # What it does not do
|
||
//!
|
||
//! It is not a merge threshold and must not become one. Clustering keeps
|
||
//! deciding on the pairwise calibrated probability: uniqueness is *relative*,
|
||
//! so a library with one named person in it would hand every stray face a
|
||
//! uniqueness of 1. The absolute question ("is this the same person at all")
|
||
//! and the comparative one ("of the people we know, which") are different, and
|
||
//! the product is what keeps both in the answer.
|
||
|
||
use std::collections::BTreeMap;
|
||
|
||
use crate::cluster::Cluster;
|
||
use crate::neighbours::Pair;
|
||
|
||
/// Who the evidence is for.
|
||
///
|
||
/// Ordered rather than hashed so the sums below are reproducible; the ordering
|
||
/// itself carries no meaning.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
|
||
enum Identity {
|
||
/// A person the user has confirmed a face onto. Every group anchored to
|
||
/// them is the same identity, however many of them the clusterer left.
|
||
Person(u64),
|
||
/// A group nobody has ruled on. It stands for itself and competes with
|
||
/// nothing.
|
||
Group(usize),
|
||
}
|
||
|
||
/// How many of an identity's best matches count as its evidence.
|
||
///
|
||
/// The cap is the whole reason the sum works: uncapped, evidence would grow
|
||
/// with a person's face count and the largest group in the library would win
|
||
/// every contest. Ten is enough that a person photographed from several angles
|
||
/// contributes more than one lucky frame, and small enough that the hundred
|
||
/// mediocre matches inside a well-covered identity cannot add up to a strong
|
||
/// one. It is a starting point, not a measured optimum — M7's corpus is where
|
||
/// it would be tuned.
|
||
pub const TOP_MATCHES: usize = 10;
|
||
|
||
/// The weakest match that counts as evidence for an identity.
|
||
///
|
||
/// Rivals are half the point of this module, so the evidence scan has to reach
|
||
/// *below* the merge threshold — a named person who matches at 0.6 will never
|
||
/// be merged into but is precisely the competition a suggestion should be
|
||
/// discounted for. Even odds is the natural floor: below it a pair is more
|
||
/// likely different people than the same, and it is not a small distinction —
|
||
/// summing the near-orthogonal pairs instead of dropping them lets fifty
|
||
/// identities' worth of noise, each contributing its *upper tail*, outweigh one
|
||
/// real match. Measured on a real library, that alone moved the median stated
|
||
/// confidence from 100% to 31%.
|
||
pub const RIVAL_FLOOR: f32 = 0.5;
|
||
|
||
/// How confident each face's placement is, indexed like the face slice the
|
||
/// clusters came from.
|
||
///
|
||
/// A face in no group, or one with no evidence for anybody, scores 0.
|
||
///
|
||
/// `gallery` is one flag per face — which faces may be evidence at all
|
||
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]). Its length is the face count.
|
||
/// A pair is evidence *about* either face but only *from* a gallery one: a
|
||
/// probe learns from the references it matched, and a reference learns nothing
|
||
/// from a probe that happened to match it, however well. Without that, the one
|
||
/// short vector in a group would be the strongest match every face in it had.
|
||
///
|
||
/// `pairs` must be the *evidence* list — scanned at [`RIVAL_FLOOR`], not at the
|
||
/// merge threshold. Passing the merge list still works but silently removes
|
||
/// every rival weaker than a merge, which is most of them, and every uniqueness
|
||
/// collapses to 1.
|
||
pub fn identity_shares(
|
||
gallery: &[bool],
|
||
clusters: &[Cluster],
|
||
pairs: &[Pair],
|
||
top: usize,
|
||
) -> Vec<f32> {
|
||
let faces = gallery.len();
|
||
// An identity is a *person*, not a group. One person routinely holds
|
||
// several anchored groups — the same reason they hold several unnamed ones
|
||
// — and keying this by group had Catherine competing with Catherine, which
|
||
// on the library it was measured against put the median suggestion onto a
|
||
// named person at 39%.
|
||
let key_of: Vec<Identity> = clusters
|
||
.iter()
|
||
.enumerate()
|
||
.map(|(g, c)| match c.person {
|
||
Some(p) => Identity::Person(p),
|
||
None => Identity::Group(g),
|
||
})
|
||
.collect();
|
||
|
||
let mut group_of = vec![usize::MAX; faces];
|
||
for (g, c) in clusters.iter().enumerate() {
|
||
for &m in &c.members {
|
||
if m < faces {
|
||
group_of[m] = g;
|
||
}
|
||
}
|
||
}
|
||
|
||
// Ordered, not hashed: the numbers are sums of floats over these buckets
|
||
// and this module inherits [`crate::cluster`]'s promise that the same input
|
||
// yields the same output, bit for bit.
|
||
let mut evidence: Vec<BTreeMap<Identity, Vec<f32>>> = vec![BTreeMap::new(); faces];
|
||
for p in pairs {
|
||
if p.i >= faces || p.j >= faces {
|
||
continue;
|
||
}
|
||
// A pair is evidence in both directions: j's identity hears about i,
|
||
// and i's identity hears about j. The pair list holds each unordered
|
||
// pair once, so both have to be recorded here — each only where the
|
||
// face doing the telling is in the gallery.
|
||
let (gi, gj) = (group_of[p.i], group_of[p.j]);
|
||
if gj != usize::MAX && gallery[p.j] {
|
||
evidence[p.i]
|
||
.entry(key_of[gj])
|
||
.or_default()
|
||
.push(p.probability);
|
||
}
|
||
if gi != usize::MAX && gallery[p.i] {
|
||
evidence[p.j]
|
||
.entry(key_of[gi])
|
||
.or_default()
|
||
.push(p.probability);
|
||
}
|
||
}
|
||
|
||
let mut out = vec![0.0; faces];
|
||
for (i, buckets) in evidence.iter_mut().enumerate() {
|
||
let mine = group_of[i];
|
||
if mine == usize::MAX {
|
||
continue;
|
||
}
|
||
let mine = key_of[mine];
|
||
let mut coherence = 0.0;
|
||
let mut ours = 0.0;
|
||
let mut rivals = 0.0;
|
||
for (&who, probabilities) in buckets.iter_mut() {
|
||
// Descending, and the ties broken by nothing: equal probabilities
|
||
// sum the same whichever order they land in.
|
||
probabilities.sort_by(|a, b| b.total_cmp(a));
|
||
let counted = probabilities.len().min(top);
|
||
let score: f32 = probabilities.iter().take(top).sum();
|
||
if who == mine {
|
||
ours = score;
|
||
coherence = score / counted as f32;
|
||
} else if matches!(who, Identity::Person(_)) {
|
||
// Only an identity the user has ruled on competes. A group
|
||
// nobody has ruled on and that matches this face is far more
|
||
// likely to be another fragment of the same person than a
|
||
// different one — see the module note, and the library it was
|
||
// measured on.
|
||
rivals += score;
|
||
}
|
||
}
|
||
let total = ours + rivals;
|
||
// No evidence at all: a face anchored into a group it has no measured
|
||
// similarity to. Nothing honest to report, so nothing is claimed.
|
||
out[i] = if total > 0.0 {
|
||
coherence * (ours / total)
|
||
} else {
|
||
0.0
|
||
};
|
||
}
|
||
out
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
/// An unnamed group: nobody has ruled on it, so it competes with nothing.
|
||
fn cluster(members: &[usize]) -> Cluster {
|
||
Cluster {
|
||
members: members.to_vec(),
|
||
person: None,
|
||
}
|
||
}
|
||
|
||
/// A group the user has confirmed a face onto — an identity, and therefore
|
||
/// a rival.
|
||
fn named(members: &[usize], person: u64) -> Cluster {
|
||
Cluster {
|
||
members: members.to_vec(),
|
||
person: Some(person),
|
||
}
|
||
}
|
||
|
||
fn pair(i: usize, j: usize, probability: f32) -> Pair {
|
||
Pair { i, j, probability }
|
||
}
|
||
|
||
/// `n` faces, every one of them fit to be compared against.
|
||
fn all(n: usize) -> Vec<bool> {
|
||
vec![true; n]
|
||
}
|
||
|
||
/// The failure the module exists to fix: face 0 matches its own group's
|
||
/// three members strongly, and the group has forty more it is unrelated to.
|
||
/// The old within-group mean reported ~0.07 for this.
|
||
#[test]
|
||
fn a_large_group_does_not_dilute_a_strong_match() {
|
||
let members: Vec<usize> = (0..44).collect();
|
||
let clusters = vec![cluster(&members)];
|
||
let pairs = vec![pair(0, 1, 0.99), pair(0, 2, 0.97), pair(0, 3, 0.95)];
|
||
|
||
let shares = identity_shares(&all(44), &clusters, &pairs, TOP_MATCHES);
|
||
assert!(
|
||
(shares[0] - 0.97).abs() < 1e-6,
|
||
"the mean of its three real matches, undiluted: {}",
|
||
shares[0]
|
||
);
|
||
}
|
||
|
||
/// Two named people matching equally well is a coin toss, and saying so is
|
||
/// the point — this is the sibling case FR-CULL-10 warns about.
|
||
#[test]
|
||
fn an_ambiguous_face_splits_its_confidence_between_the_rivals() {
|
||
let clusters = vec![named(&[0, 1, 2], 1), named(&[3, 4], 2)];
|
||
let pairs = vec![
|
||
pair(0, 1, 0.90),
|
||
pair(0, 2, 0.90),
|
||
pair(0, 3, 0.90),
|
||
pair(0, 4, 0.90),
|
||
];
|
||
|
||
let shares = identity_shares(&all(5), &clusters, &pairs, TOP_MATCHES);
|
||
// Coherent at 0.90, and only half of the evidence is its own.
|
||
assert!(
|
||
(shares[0] - 0.45).abs() < 1e-6,
|
||
"even evidence both ways: {}",
|
||
shares[0]
|
||
);
|
||
}
|
||
|
||
/// A rival below the merge threshold still has to count, which is why the
|
||
/// evidence scan reaches down to [`RIVAL_FLOOR`].
|
||
#[test]
|
||
fn a_rival_too_weak_to_merge_still_lowers_the_confidence() {
|
||
let clusters = vec![named(&[0, 1], 1), named(&[2, 3], 2)];
|
||
let sure = identity_shares(&all(4), &clusters, &[pair(0, 1, 0.95)], TOP_MATCHES);
|
||
let contested = identity_shares(
|
||
&all(4),
|
||
&clusters,
|
||
&[pair(0, 1, 0.95), pair(0, 2, 0.60)],
|
||
TOP_MATCHES,
|
||
);
|
||
|
||
assert_eq!(sure[0], 0.95, "nobody else to be: its coherence stands");
|
||
assert!(
|
||
contested[0] < 0.59 && contested[0] > 0.57,
|
||
"0.95 coherent, but 0.95 against 0.60: {}",
|
||
contested[0]
|
||
);
|
||
}
|
||
|
||
/// A fragment of the same person is not a rival. Measured on a real
|
||
/// library, counting unnamed groups as competition put four suggestions in
|
||
/// five under half — see the module note.
|
||
#[test]
|
||
fn an_unnamed_group_is_not_treated_as_competition() {
|
||
let clusters = vec![cluster(&[0, 1]), cluster(&[2, 3])];
|
||
let shares = identity_shares(
|
||
&all(4),
|
||
&clusters,
|
||
&[pair(0, 1, 0.95), pair(0, 2, 0.90)],
|
||
TOP_MATCHES,
|
||
);
|
||
assert_eq!(
|
||
shares[0], 0.95,
|
||
"an unnamed group took evidence off a suggestion"
|
||
);
|
||
}
|
||
|
||
/// The cap, doing its job: an identity with fifty mediocre matches must not
|
||
/// beat one with ten strong ones on volume alone.
|
||
#[test]
|
||
fn evidence_is_capped_so_the_biggest_group_cannot_win_on_volume() {
|
||
let small: Vec<usize> = (0..11).collect();
|
||
let large: Vec<usize> = (11..62).collect();
|
||
let clusters = vec![named(&small, 1), named(&large, 2)];
|
||
|
||
let mut pairs: Vec<Pair> = (1..11).map(|j| pair(0, j, 0.90)).collect();
|
||
pairs.extend((11..62).map(|j| pair(0, j, 0.55)));
|
||
|
||
let shares = identity_shares(&all(62), &clusters, &pairs, TOP_MATCHES);
|
||
// Ten at 0.90 against ten at 0.55 — not fifty-one at 0.55.
|
||
assert!(
|
||
(shares[0] - 0.90 * (9.0 / 14.5)).abs() < 1e-5,
|
||
"capped at ten either side: {}",
|
||
shares[0]
|
||
);
|
||
}
|
||
|
||
/// A probe learns from the references it matched; a reference learns
|
||
/// nothing from a probe. The pair is the same pair — what differs is who
|
||
/// is doing the telling.
|
||
#[test]
|
||
fn a_face_outside_the_gallery_is_nobody_s_evidence() {
|
||
let clusters = vec![named(&[0, 1, 2], 1)];
|
||
let gallery = vec![true, true, false];
|
||
let pairs = vec![pair(0, 1, 0.80), pair(0, 2, 0.99), pair(1, 2, 0.99)];
|
||
|
||
let shares = identity_shares(&gallery, &clusters, &pairs, TOP_MATCHES);
|
||
// Faces 0 and 1 hear only from each other: the 0.99 the probe offered
|
||
// them is not counted.
|
||
assert!((shares[0] - 0.80).abs() < 1e-6, "{}", shares[0]);
|
||
assert!((shares[1] - 0.80).abs() < 1e-6, "{}", shares[1]);
|
||
// The probe hears from both references.
|
||
assert!((shares[2] - 0.99).abs() < 1e-6, "{}", shares[2]);
|
||
}
|
||
|
||
/// A face nothing has any evidence about claims nothing.
|
||
#[test]
|
||
fn a_face_with_no_evidence_reports_no_confidence() {
|
||
let clusters = vec![cluster(&[0, 1])];
|
||
let shares = identity_shares(&all(2), &clusters, &[], TOP_MATCHES);
|
||
assert_eq!(shares, vec![0.0, 0.0]);
|
||
}
|
||
}
|