Keep the face, not just a way to find it again
A face was drawn by decoding the 1024px proxy it was found on and cutting the box out again, every time the People screen opened. That made the screen a derivative of the thumbnail cache: evict a proxy — which the cache may do at any moment — and the cell goes blank, with no way back short of re-fetching the original over the network and re-detecting it. It also cost a full JPEG decode per image, per visit, to show a 96px cell. So the crop is cut once, when the pixels are already in hand at detection time, and kept. A 160px JPEG is a few KB against the ~250 KB proxy it replaces reading. Where it lives is the interesting part. The catalog snapshot is uploaded *whole* on every sync and downloaded by every device, so a crop column there would put tens of MB on every round trip — the exact cost `face_shard`'s 25 MB cap exists to bound, and the reason bulk per-face data lives in shards already. Crops therefore travel in the face shards, beside the embeddings, and `snapshot_for_upload` strips them from the copy it writes. Nothing reads a crop out of a merged remote catalog — the merge touches collections and keywords only — so a receiving device loses nothing. A shard carrying crops holds around 3,500 faces rather than 22,000, which is the price of a second device showing People immediately instead of re-fetching every proxy. The column is nullable and the reader falls back to the proxy, so a face indexed before this still works and the next indexing pass fills it in. V10 also adds `people.ignored`, for a person the user has looked at and does not want to identify. Most clusters in a real library are strangers — passers-by, other people's guests, a face on a poster — and there is no way to tell "not yet looked at" from "looked at, don't care" without recording the second. It is a column rather than a deletion because a deleted cluster comes straight back on the next Regroup: the faces are still there and still similar, and nothing short of remembering the judgement survives re-clustering. Same argument `face_person_rejected` makes one level down. And `prune_empty_unnamed`, for what clustering leaves behind. Regroup creates a person per unanchored group and never removed the previous run's now-empty ones, so pressing it twice added a rail entry per group it no longer believed in. Named people are never touched however empty — a name is user data — nor is a merge tombstone, which must outlive its faces to keep redirecting. 298 tests pass, including that the snapshot carries no crops while the live catalog keeps them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -68,6 +68,18 @@ pub struct DetectedFace {
|
||||
/// Which model produced the embedding. Comparing across models is the one
|
||||
/// mistake that yields plausible garbage rather than an error.
|
||||
pub model_id: String,
|
||||
/// The face itself, cut out and encoded, ready to draw.
|
||||
///
|
||||
/// Cut at detection time because that is the one moment the pixels are
|
||||
/// already in memory. The alternative -- and what this replaced -- is
|
||||
/// re-decoding the whole proxy and cutting the box out again every time
|
||||
/// the People screen opens, which makes the screen a derivative of a cache
|
||||
/// that is entitled to evict anything at any time.
|
||||
///
|
||||
/// Empty is allowed and means "not cut": a caller with only a box and an
|
||||
/// embedding, such as a face adopted from a peer's shard that predates
|
||||
/// crops, stores nothing here and the reader falls back to the proxy.
|
||||
pub crop: Vec<u8>,
|
||||
}
|
||||
|
||||
/// A stored face, with whatever identity it has acquired.
|
||||
@@ -101,6 +113,13 @@ pub struct Person {
|
||||
pub confirmed_faces: u64,
|
||||
/// Faces the system suggests and the user has not ruled on.
|
||||
pub suggested_faces: u64,
|
||||
/// The user has looked at this group and does not want to identify it.
|
||||
///
|
||||
/// Distinct from an empty name, which means *not yet looked at*. Most
|
||||
/// clusters in a real library are strangers, and without somewhere to put
|
||||
/// that judgement the screen never gets shorter however much work the user
|
||||
/// does on it.
|
||||
pub ignored: bool,
|
||||
}
|
||||
|
||||
/// The FR-CULL-9 calibration, re-exported from where it is fitted.
|
||||
@@ -165,8 +184,8 @@ pub fn record_detections(
|
||||
tx.execute(
|
||||
"INSERT INTO faces
|
||||
(image_id, x, y, w, h, landmarks, detector_confidence,
|
||||
embedding, crop_px, model_id, detected_at)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11)",
|
||||
embedding, crop_px, model_id, detected_at, crop)
|
||||
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9, ?10, ?11, ?12)",
|
||||
rusqlite::params![
|
||||
image_id.0 as i64,
|
||||
f.x as f64,
|
||||
@@ -179,6 +198,10 @@ pub fn record_detections(
|
||||
f.crop_px as f64,
|
||||
f.model_id,
|
||||
now,
|
||||
// NULL rather than an empty blob, so "no crop" is one state in
|
||||
// the database rather than two the readers each have to know
|
||||
// about.
|
||||
(!f.crop.is_empty()).then_some(f.crop.as_slice()),
|
||||
],
|
||||
)?;
|
||||
let id = FaceId(tx.last_insert_rowid() as u64);
|
||||
@@ -415,7 +438,8 @@ pub fn people(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
|
||||
let mut q = conn.prepare(
|
||||
"SELECT p.id, p.uuid, p.name,
|
||||
COALESCE(SUM(fp.confirmed = 1), 0),
|
||||
COALESCE(SUM(fp.confirmed = 0), 0)
|
||||
COALESCE(SUM(fp.confirmed = 0), 0),
|
||||
p.ignored
|
||||
FROM people p
|
||||
LEFT JOIN face_person fp ON fp.person_id = p.id
|
||||
WHERE p.merged_into IS NULL
|
||||
@@ -429,6 +453,7 @@ pub fn people(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
|
||||
name: r.get(2)?,
|
||||
confirmed_faces: r.get::<_, i64>(3)? as u64,
|
||||
suggested_faces: r.get::<_, i64>(4)? as u64,
|
||||
ignored: r.get(5)?,
|
||||
})
|
||||
})?;
|
||||
rows.collect::<Result<_, _>>().map_err(Into::into)
|
||||
@@ -732,6 +757,114 @@ pub fn delete_all_face_data(conn: &Connection) -> Result<u64, CatalogError> {
|
||||
Ok(faces as u64)
|
||||
}
|
||||
|
||||
/// The stored crops for one person's faces.
|
||||
///
|
||||
/// Read separately from [`for_person`] rather than joined onto it, because the
|
||||
/// boxes are wanted in several places that do not draw anything — clustering
|
||||
/// anchors, the develop overlay, the segmentation naming pass — and dragging a
|
||||
/// few hundred KB of JPEG through those would be pure waste.
|
||||
///
|
||||
/// A face with no stored crop is simply absent from the map; the caller falls
|
||||
/// back to cutting one out of the proxy.
|
||||
pub fn crops_for_person(
|
||||
conn: &Connection,
|
||||
person: PersonId,
|
||||
include_suggested: bool,
|
||||
) -> Result<std::collections::HashMap<FaceId, Vec<u8>>, CatalogError> {
|
||||
let mut q = conn.prepare(
|
||||
"SELECT f.id, f.crop
|
||||
FROM faces f
|
||||
JOIN face_person fp ON fp.face_id = f.id
|
||||
WHERE fp.person_id = ?1 AND (?2 OR fp.confirmed = 1)
|
||||
AND f.crop IS NOT NULL",
|
||||
)?;
|
||||
let rows = q.query_map(rusqlite::params![person.0 as i64, include_suggested], |r| {
|
||||
Ok((FaceId(r.get::<_, i64>(0)? as u64), r.get::<_, Vec<u8>>(1)?))
|
||||
})?;
|
||||
rows.collect::<Result<_, _>>().map_err(Into::into)
|
||||
}
|
||||
|
||||
/// One face's stored crop, if it has one.
|
||||
pub fn crop(conn: &Connection, face: FaceId) -> Result<Option<Vec<u8>>, CatalogError> {
|
||||
conn.query_row(
|
||||
"SELECT crop FROM faces WHERE id = ?1",
|
||||
[face.0 as i64],
|
||||
|r| r.get::<_, Option<Vec<u8>>>(0),
|
||||
)
|
||||
.optional()
|
||||
.map(Option::flatten)
|
||||
.map_err(Into::into)
|
||||
}
|
||||
|
||||
/// How many of a model's faces still have no stored crop.
|
||||
///
|
||||
/// The measure of how much of the library is still drawing itself the slow way,
|
||||
/// and what a "re-index to fill these in" prompt would be counting.
|
||||
pub fn faces_without_crop(conn: &Connection, model_id: &str) -> Result<u64, CatalogError> {
|
||||
conn.query_row(
|
||||
"SELECT COUNT(*) FROM faces WHERE model_id = ?1 AND crop IS NULL",
|
||||
[model_id],
|
||||
|r| r.get::<_, i64>(0),
|
||||
)
|
||||
.map(|n| n as u64)
|
||||
.map_err(Into::into)
|
||||
}
|
||||
|
||||
/// The user does not want to identify this person.
|
||||
///
|
||||
/// Reversible, and deliberately not a deletion: deleting the group would only
|
||||
/// have it rebuilt by the next clustering pass, since the faces are still there
|
||||
/// and still similar to each other. See the V10 migration note.
|
||||
pub fn set_ignored(conn: &Connection, person: PersonId, ignored: bool) -> Result<(), CatalogError> {
|
||||
conn.execute(
|
||||
"UPDATE people
|
||||
SET ignored = ?2, revision = revision + 1, modified = ?3
|
||||
WHERE id = ?1",
|
||||
rusqlite::params![person.0 as i64, ignored, now_secs()],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Whether a person has been set aside.
|
||||
pub fn is_ignored(conn: &Connection, person: PersonId) -> Result<bool, CatalogError> {
|
||||
conn.query_row(
|
||||
"SELECT ignored FROM people WHERE id = ?1",
|
||||
[person.0 as i64],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.optional()
|
||||
.map(|v| v.unwrap_or(false))
|
||||
.map_err(Into::into)
|
||||
}
|
||||
|
||||
/// Delete unnamed people who hold no faces at all.
|
||||
///
|
||||
/// Clustering creates a person per unanchored group, and a later pass — a new
|
||||
/// threshold, more faces indexed, a split undone — can leave the previous run's
|
||||
/// group with nothing in it. Those are not tombstones and nothing refers to
|
||||
/// them; left alone they accumulate one rail entry per Regroup, which is what
|
||||
/// made pressing the button twice look like it had broken the screen.
|
||||
///
|
||||
/// **Named people are never touched, however empty.** A name is user data
|
||||
/// (FR-CULL-12), and a person the user named and then emptied by moving every
|
||||
/// face elsewhere is still a person they told us about. Nor is a person that
|
||||
/// something was merged into, which has to outlive its faces to keep
|
||||
/// redirecting.
|
||||
///
|
||||
/// Returns how many were removed.
|
||||
pub fn prune_empty_unnamed(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
let n = conn.execute(
|
||||
"DELETE FROM people
|
||||
WHERE name = ''
|
||||
AND merged_into IS NULL
|
||||
AND ignored = 0
|
||||
AND NOT EXISTS (SELECT 1 FROM face_person fp WHERE fp.person_id = people.id)
|
||||
AND NOT EXISTS (SELECT 1 FROM people o WHERE o.merged_into = people.id)",
|
||||
[],
|
||||
)?;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
// ── helpers ───────────────────────────────────────────────────────────────
|
||||
|
||||
fn read_face(r: &rusqlite::Row<'_>) -> rusqlite::Result<Face> {
|
||||
@@ -869,16 +1002,17 @@ mod tests {
|
||||
embedding: vec![seed; 1024],
|
||||
crop_px: 180.0,
|
||||
model_id: "w600k_mbf".into(),
|
||||
crop: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn migration_reaches_version_nine() {
|
||||
fn migration_reaches_the_current_version() {
|
||||
let c = db();
|
||||
let v: i64 = c
|
||||
.query_row("PRAGMA user_version", [], |r| r.get(0))
|
||||
.unwrap();
|
||||
assert_eq!(v, 9);
|
||||
assert_eq!(v, crate::schema::SCHEMA_VERSION);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -1282,4 +1416,168 @@ mod tests {
|
||||
assert_eq!(e[0].2[0], 7);
|
||||
assert!((e[0].3 - 180.0).abs() < 1e-3);
|
||||
}
|
||||
|
||||
// ── stored crops ──────────────────────────────────────────────────────
|
||||
|
||||
fn face_with_crop(seed: u8, crop: Vec<u8>) -> DetectedFace {
|
||||
DetectedFace { crop, ..face(seed) }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_crop_stored_with_a_face_comes_back_as_stored() {
|
||||
let c = db();
|
||||
let img = image(&c, 1);
|
||||
let ids = record_detections(
|
||||
&c,
|
||||
img,
|
||||
"w600k_mbf",
|
||||
1024,
|
||||
&[face_with_crop(3, vec![9; 128])],
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(crop(&c, ids[0]).unwrap(), Some(vec![9; 128]));
|
||||
}
|
||||
|
||||
/// A face indexed before crops existed has none, and that has to read back
|
||||
/// as an honest absence rather than an empty image.
|
||||
#[test]
|
||||
fn a_face_with_no_crop_reads_back_as_none() {
|
||||
let c = db();
|
||||
let img = image(&c, 1);
|
||||
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(3)]).unwrap();
|
||||
assert_eq!(crop(&c, ids[0]).unwrap(), None);
|
||||
assert_eq!(faces_without_crop(&c, "w600k_mbf").unwrap(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn crops_come_back_per_person_and_skip_the_ones_without() {
|
||||
let c = db();
|
||||
let a = image(&c, 1);
|
||||
let b = image(&c, 2);
|
||||
let with = record_detections(&c, a, "w600k_mbf", 1024, &[face_with_crop(1, vec![4; 16])])
|
||||
.unwrap()[0];
|
||||
let without = record_detections(&c, b, "w600k_mbf", 1024, &[face(2)]).unwrap()[0];
|
||||
|
||||
let p = create_person(&c, "Anna").unwrap();
|
||||
confirm(&c, with, p).unwrap();
|
||||
confirm(&c, without, p).unwrap();
|
||||
|
||||
let crops = crops_for_person(&c, p, true).unwrap();
|
||||
assert_eq!(crops.len(), 1, "a face with no crop should not appear");
|
||||
assert_eq!(crops.get(&with), Some(&vec![4; 16]));
|
||||
}
|
||||
|
||||
/// Re-indexing replaces the crop along with everything else, so a better
|
||||
/// pass over the same photograph updates what the screen draws.
|
||||
#[test]
|
||||
fn re_indexing_replaces_the_stored_crop() {
|
||||
let c = db();
|
||||
let img = image(&c, 1);
|
||||
record_detections(&c, img, "w600k_mbf", 1024, &[face_with_crop(1, vec![1; 8])]).unwrap();
|
||||
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face_with_crop(1, vec![2; 8])])
|
||||
.unwrap();
|
||||
assert_eq!(crop(&c, ids[0]).unwrap(), Some(vec![2; 8]));
|
||||
}
|
||||
|
||||
// ── setting a person aside ────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn a_person_can_be_ignored_and_un_ignored() {
|
||||
let c = db();
|
||||
let p = create_person(&c, "").unwrap();
|
||||
assert!(!is_ignored(&c, p).unwrap());
|
||||
|
||||
set_ignored(&c, p, true).unwrap();
|
||||
assert!(is_ignored(&c, p).unwrap());
|
||||
assert!(
|
||||
people(&c)
|
||||
.unwrap()
|
||||
.iter()
|
||||
.find(|q| q.id == p)
|
||||
.unwrap()
|
||||
.ignored
|
||||
);
|
||||
|
||||
set_ignored(&c, p, false).unwrap();
|
||||
assert!(!is_ignored(&c, p).unwrap());
|
||||
}
|
||||
|
||||
/// Ignoring is a judgement, so it bumps the revision the same way a rename
|
||||
/// does — a device that syncs has to see that something changed.
|
||||
#[test]
|
||||
fn ignoring_a_person_bumps_their_revision() {
|
||||
let c = db();
|
||||
let p = create_person(&c, "").unwrap();
|
||||
let before: i64 = c
|
||||
.query_row(
|
||||
"SELECT revision FROM people WHERE id = ?1",
|
||||
[p.0 as i64],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
set_ignored(&c, p, true).unwrap();
|
||||
let after: i64 = c
|
||||
.query_row(
|
||||
"SELECT revision FROM people WHERE id = ?1",
|
||||
[p.0 as i64],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
assert!(after > before);
|
||||
}
|
||||
|
||||
// ── pruning what clustering left behind ───────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn an_empty_unnamed_person_is_pruned() {
|
||||
let c = db();
|
||||
create_person(&c, "").unwrap();
|
||||
assert_eq!(prune_empty_unnamed(&c).unwrap(), 1);
|
||||
assert!(people(&c).unwrap().is_empty());
|
||||
}
|
||||
|
||||
/// The rule that matters: a name is user data and survives whatever else
|
||||
/// happens to the group.
|
||||
#[test]
|
||||
fn an_empty_named_person_is_kept() {
|
||||
let c = db();
|
||||
create_person(&c, "Anna").unwrap();
|
||||
assert_eq!(prune_empty_unnamed(&c).unwrap(), 0);
|
||||
assert_eq!(people(&c).unwrap().len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unnamed_person_with_faces_is_kept() {
|
||||
let c = db();
|
||||
let img = image(&c, 1);
|
||||
let f = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap()[0];
|
||||
let p = create_person(&c, "").unwrap();
|
||||
suggest(&c, f, p, 0.95).unwrap();
|
||||
assert_eq!(prune_empty_unnamed(&c).unwrap(), 0);
|
||||
}
|
||||
|
||||
/// An ignored group is empty of nothing — the user ruled on it, and
|
||||
/// deleting it would bring it straight back on the next Regroup.
|
||||
#[test]
|
||||
fn an_ignored_person_is_never_pruned() {
|
||||
let c = db();
|
||||
let p = create_person(&c, "").unwrap();
|
||||
set_ignored(&c, p, true).unwrap();
|
||||
assert_eq!(prune_empty_unnamed(&c).unwrap(), 0);
|
||||
}
|
||||
|
||||
/// A merge tombstone has to outlive its faces or the device on the other
|
||||
/// side of the sync resurrects the person it redirects.
|
||||
#[test]
|
||||
fn a_merge_target_is_never_pruned() {
|
||||
let c = db();
|
||||
let target = create_person(&c, "").unwrap();
|
||||
let source = create_person(&c, "").unwrap();
|
||||
c.execute(
|
||||
"UPDATE people SET merged_into = ?2 WHERE id = ?1",
|
||||
rusqlite::params![source.0 as i64, target.0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(prune_empty_unnamed(&c).unwrap(), 0);
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user