dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Record that face detection has run, not just what it found

Browse Source 
An image with no faces in it was indistinguishable from one that had
never been looked at, so every landscape, still life and document scan in
the library was re-detected on every pass, for ever. In a real library
that is most of it: on the 23,527-image test library, 64 of the first 110
images indexed contain no face at all.

Schema v9 adds face_index, a run marker per (image, model) carrying the
face count and the proxy edge it read. Keyed on the model, so a model
change puts every image back in the queue by itself.

That makes a coverage figure possible, which is the thing a user actually
wants to see. The audit also splits the outstanding set by whether a
proxy exists, because 23,417 awaiting a proxy and 110 ready to index are
different problems, and telling the user to run indexing again would not
fix the first.

The Identity screen gains Index faces, Stop, and the coverage line.
examples/face_index.rs is the same check and sweep without a window,
which is the right shape for an overnight pass.

Measured on the real library in release: 3.5 images/second, 110 images
and 125 faces in 30 seconds, and a second run correctly finds nothing
left to do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-26 22:13:41 +02:00
co-authored by Claude Opus 5
parent 3e607222c6
commit 26a1eb7e28
14 changed files with 971 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-catalog/src/faces.rs
+252 -22
View File
@@ -127,6 +127,8 @@ pub use dr_face::Calibration;
pub fn record_detections(
conn: &Connection,
image_id: ImageId,
model_id: &str,
source_edge: u32,
faces: &[DetectedFace],
) -> Result<Vec<FaceId>, CatalogError> {
let tx = conn.unchecked_transaction()?;
@@ -203,10 +205,124 @@ pub fn record_detections(
ids.push(id);
}
// The run marker, written whether or not anything was found. Zero faces is
// a real answer and recording it is what stops the next pass looking at
// this photograph again -- see the V9 migration.
tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, ?3, ?4, ?5)
ON CONFLICT(image_id, model_id) DO UPDATE SET
indexed_at = excluded.indexed_at,
faces_found = excluded.faces_found,
source_edge = excluded.source_edge",
rusqlite::params![
image_id.0 as i64,
model_id,
now,
faces.len() as i64,
source_edge as i64,
],
)?;
tx.commit()?;
Ok(ids)
}
/// How much of the library has been through face detection.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct Coverage {
/// Images that are candidates at all — present, not trashed.
pub images: u64,
/// Images this model has actually looked at.
pub indexed: u64,
/// Of those, how many had no face in them. Usually most of a library, and
/// worth showing so "0 faces" reads as a finding rather than a failure.
pub without_faces: u64,
/// Faces found across the whole library.
pub faces: u64,
}
impl Coverage {
/// Images still to look at.
pub fn outstanding(&self) -> u64 {
self.images.saturating_sub(self.indexed)
}
pub fn is_complete(&self) -> bool {
self.outstanding() == 0
}
/// Fraction indexed, `0.0..=1.0`. An empty library is complete, not zero:
/// there is nothing outstanding, and reporting 0% would read as a stall.
pub fn fraction(&self) -> f32 {
if self.images == 0 {
1.0
} else {
self.indexed as f32 / self.images as f32
}
}
}
/// Count what has and has not been indexed, for one model.
///
/// The question the batch pass asks before it starts and the UI asks to draw a
/// progress figure. Answerable only because [`record_detections`] writes a run
/// marker: counting `faces` rows would report how many faces exist, which is a
/// different number and never reaches the image count.
pub fn coverage(conn: &Connection, model_id: &str) -> Result<Coverage, CatalogError> {
let images: i64 = conn.query_row(
"SELECT COUNT(*) FROM images WHERE trashed_at IS NULL",
[],
|r| r.get(0),
)?;
let (indexed, without, faces): (i64, i64, i64) = conn.query_row(
"SELECT COUNT(*), COALESCE(SUM(faces_found = 0), 0), COALESCE(SUM(faces_found), 0)
FROM face_index fi
JOIN images i ON i.id = fi.image_id
WHERE fi.model_id = ?1 AND i.trashed_at IS NULL",
[model_id],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
)?;
Ok(Coverage {
images: images as u64,
indexed: indexed as u64,
without_faces: without as u64,
faces: faces as u64,
})
}
/// Whether one image has been through this model.
pub fn is_indexed(
conn: &Connection,
image_id: ImageId,
model_id: &str,
) -> Result<bool, CatalogError> {
conn.query_row(
"SELECT EXISTS(SELECT 1 FROM face_index WHERE image_id = ?1 AND model_id = ?2)",
rusqlite::params![image_id.0 as i64, model_id],
|r| r.get(0),
)
.map_err(Into::into)
}
/// Forget that an image was indexed, so the next pass looks again.
///
/// What a re-index asks for. Separate from deleting the faces because the two
/// are wanted at different times: clearing the marker alone re-detects and
/// replaces, which is the ordinary "try again with a better proxy" case.
pub fn clear_index_marker(
conn: &Connection,
image_id: ImageId,
model_id: &str,
) -> Result<(), CatalogError> {
conn.execute(
"DELETE FROM face_index WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id.0 as i64, model_id],
)?;
Ok(())
}
/// Every face on one image.
pub fn for_image(conn: &Connection, image_id: ImageId) -> Result<Vec<Face>, CatalogError> {
let mut q = conn.prepare(
@@ -621,6 +737,7 @@ pub fn delete_all_face_data(conn: &Connection) -> Result<u64, CatalogError> {
let faces: i64 = tx.query_row("SELECT COUNT(*) FROM faces", [], |r| r.get(0))?;
// Order matters only if foreign keys are off, which they may be on an old
// connection; deleting children first is correct either way.
tx.execute("DELETE FROM face_index", [])?;
tx.execute("DELETE FROM face_person_rejected", [])?;
tx.execute("DELETE FROM face_person", [])?;
tx.execute("DELETE FROM faces", [])?;
@@ -765,19 +882,19 @@ mod tests {
}
#[test]
fn migration_reaches_version_eight() {
fn migration_reaches_version_nine() {
let c = db();
let v: i64 = c
.query_row("PRAGMA user_version", [], |r| r.get(0))
.unwrap();
assert_eq!(v, 8);
assert_eq!(v, 9);
}
#[test]
fn detections_round_trip_with_their_landmarks() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, &[face(1), face(2)]).unwrap();
record_detections(&c, img, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
let got = for_image(&c, img).unwrap();
assert_eq!(got.len(), 2);
assert_eq!(got[0].model_id, "w600k_mbf");
@@ -792,8 +909,8 @@ mod tests {
fn re_detection_replaces_rather_than_appending() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, &[face(1), face(2)]).unwrap();
record_detections(&c, img, &[face(3)]).unwrap();
record_detections(&c, img, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
record_detections(&c, img, "w600k_mbf", 1024, &[face(3)]).unwrap();
assert_eq!(for_image(&c, img).unwrap().len(), 1);
}
@@ -803,7 +920,7 @@ mod tests {
fn re_detection_carries_a_confirmation_across() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, ids[0], anna).unwrap();
@@ -811,7 +928,7 @@ mod tests {
let mut moved = face(9);
moved.x = 0.11;
moved.y = 0.105;
record_detections(&c, img, &[moved]).unwrap();
record_detections(&c, img, "w600k_mbf", 1024, &[moved]).unwrap();
let got = for_image(&c, img).unwrap();
assert_eq!(got.len(), 1);
@@ -823,7 +940,7 @@ mod tests {
fn a_suggestion_never_overwrites_a_confirmation() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
let bob = create_person(&c, "Bob").unwrap();
@@ -837,7 +954,7 @@ mod tests {
fn a_rejection_stops_the_face_being_suggested_again() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
suggest(&c, ids[0], anna, 0.8).unwrap();
@@ -852,7 +969,7 @@ mod tests {
fn confirming_overrides_an_earlier_rejection() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
reject(&c, ids[0], anna).unwrap();
confirm(&c, ids[0], anna).unwrap();
@@ -864,8 +981,8 @@ mod tests {
let c = db();
let i1 = image(&c, 1);
let i2 = image(&c, 2);
let a = record_detections(&c, i1, &[face(1)]).unwrap();
let b = record_detections(&c, i2, &[face(2)]).unwrap();
let a = record_detections(&c, i1, "w600k_mbf", 1024, &[face(1)]).unwrap();
let b = record_detections(&c, i2, "w600k_mbf", 1024, &[face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
let annie = create_person(&c, "Annie").unwrap();
confirm(&c, a[0], anna).unwrap();
@@ -882,7 +999,7 @@ mod tests {
fn merging_does_not_duplicate_a_face_both_people_hold() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
let a = create_person(&c, "A").unwrap();
let b = create_person(&c, "B").unwrap();
confirm(&c, ids[0], a).unwrap();
@@ -899,7 +1016,7 @@ mod tests {
let c = db();
let img = image(&c, 1);
// Two faces of the same person in one frame — a mirror, or a collage.
let ids = record_detections(&c, img, &[face(1), face(2)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, ids[0], anna).unwrap();
confirm(&c, ids[1], anna).unwrap();
@@ -911,8 +1028,8 @@ mod tests {
let c = db();
let i1 = image(&c, 1);
let i2 = image(&c, 2);
let a = record_detections(&c, i1, &[face(1)]).unwrap();
let b = record_detections(&c, i2, &[face(2)]).unwrap();
let a = record_detections(&c, i1, "w600k_mbf", 1024, &[face(1)]).unwrap();
let b = record_detections(&c, i2, "w600k_mbf", 1024, &[face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, a[0], anna).unwrap();
suggest(&c, b[0], anna, 0.8).unwrap();
@@ -928,8 +1045,8 @@ mod tests {
let c = db();
let i1 = image(&c, 1);
let i2 = image(&c, 2);
let a = record_detections(&c, i1, &[face(1)]).unwrap();
let b = record_detections(&c, i2, &[face(2)]).unwrap();
let a = record_detections(&c, i1, "w600k_mbf", 1024, &[face(1)]).unwrap();
let b = record_detections(&c, i2, "w600k_mbf", 1024, &[face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, a[0], anna).unwrap();
suggest(&c, b[0], anna, 0.7).unwrap();
@@ -956,7 +1073,7 @@ mod tests {
fn deleting_face_data_leaves_the_photographs_alone() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, &[face(1), face(2)]).unwrap();
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, ids[0], anna).unwrap();
reject(&c, ids[1], anna).unwrap();
@@ -1033,8 +1150,8 @@ mod tests {
let c = db();
let i1 = image(&c, 1);
let i2 = image(&c, 2);
let a = record_detections(&c, i1, &[face(1)]).unwrap();
record_detections(&c, i2, &[face(2)]).unwrap();
let a = record_detections(&c, i1, "w600k_mbf", 1024, &[face(1)]).unwrap();
record_detections(&c, i2, "w600k_mbf", 1024, &[face(2)]).unwrap();
let anna = create_person(&c, "Anna").unwrap();
confirm(&c, a[0], anna).unwrap();
@@ -1042,11 +1159,124 @@ mod tests {
assert!(unassigned(&c, "some_other_model").unwrap().is_empty());
}
/// The bug this table exists for: a photograph with no faces must not look
/// like one that has never been indexed, or every landscape in the library
/// is re-examined on every pass, for ever.
#[test]
fn an_image_with_no_faces_still_counts_as_indexed() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 1024, &[]).unwrap();
assert!(is_indexed(&c, img, "w600k_mbf").unwrap());
assert!(for_image(&c, img).unwrap().is_empty());
let cov = coverage(&c, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1);
assert_eq!(cov.without_faces, 1);
assert_eq!(cov.faces, 0);
assert_eq!(cov.outstanding(), 0);
assert!(cov.is_complete());
}
#[test]
fn a_model_change_puts_every_image_back_in_the_queue() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
assert!(is_indexed(&c, img, "w600k_mbf").unwrap());
assert!(
!is_indexed(&c, img, "some_better_model").unwrap(),
"a new model must not inherit the old model's coverage"
);
assert_eq!(coverage(&c, "some_better_model").unwrap().outstanding(), 1);
}
#[test]
fn coverage_counts_images_not_faces() {
let c = db();
let i1 = image(&c, 1);
let i2 = image(&c, 2);
image(&c, 3);
// Three faces across two images; the third image is untouched.
record_detections(&c, i1, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
record_detections(&c, i2, "w600k_mbf", 1024, &[face(3)]).unwrap();
let cov = coverage(&c, "w600k_mbf").unwrap();
assert_eq!(cov.images, 3);
assert_eq!(cov.indexed, 2);
assert_eq!(cov.faces, 3);
assert_eq!(cov.outstanding(), 1);
assert!(!cov.is_complete());
assert!((cov.fraction() - 2.0 / 3.0).abs() < 1e-6);
}
#[test]
fn re_indexing_one_image_updates_its_marker_rather_than_adding_a_second() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 1024, &[face(1), face(2)]).unwrap();
record_detections(&c, img, "w600k_mbf", 2048, &[face(3)]).unwrap();
let cov = coverage(&c, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1, "a second marker row was written");
assert_eq!(cov.faces, 1);
let edge: i64 = c
.query_row(
"SELECT source_edge FROM face_index WHERE image_id = ?1",
[img.0 as i64],
|r| r.get(0),
)
.unwrap();
assert_eq!(edge, 2048, "the marker should record the newer proxy");
}
#[test]
fn clearing_a_marker_puts_that_image_back_in_the_queue() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
clear_index_marker(&c, img, "w600k_mbf").unwrap();
assert!(!is_indexed(&c, img, "w600k_mbf").unwrap());
assert_eq!(coverage(&c, "w600k_mbf").unwrap().outstanding(), 1);
// The faces themselves are untouched: clearing a marker asks for a
// re-detection, not a deletion.
assert_eq!(for_image(&c, img).unwrap().len(), 1);
}
#[test]
fn an_empty_library_is_complete_rather_than_zero_percent() {
let c = db();
let cov = coverage(&c, "w600k_mbf").unwrap();
assert!(cov.is_complete());
assert!((cov.fraction() - 1.0).abs() < 1e-6);
}
/// Deleting face data must clear the run markers too, or the library
/// reports itself fully indexed while holding no faces at all — and the
/// sweep then refuses to rebuild what the user just asked to remove.
#[test]
fn deleting_face_data_clears_the_run_markers() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
delete_all_face_data(&c).unwrap();
assert!(!is_indexed(&c, img, "w600k_mbf").unwrap());
let cov = coverage(&c, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 0);
assert_eq!(cov.outstanding(), 1);
}
#[test]
fn embeddings_come_back_as_stored() {
let c = db();
let img = image(&c, 1);
record_detections(&c, img, &[face(7)]).unwrap();
record_detections(&c, img, "w600k_mbf", 1024, &[face(7)]).unwrap();
let e = embeddings(&c, "w600k_mbf").unwrap();
assert_eq!(e.len(), 1);
assert_eq!(e[0].1, img);
core/dr-catalog/src/schema.rs
+48 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 8;
pub const SCHEMA_VERSION: i64 = 9;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -84,6 +84,12 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.pragma_update(None, "user_version", 8)?;
tx.commit()?;
}
if from < 9 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V9)?;
tx.pragma_update(None, "user_version", 9)?;
tx.commit()?;
}
Ok(from)
}
@@ -333,6 +339,47 @@ fn stem_of(path: &str) -> &str {
/// Both narrow the walk rather than reorder it, so the index still supplies the
/// ordering and SQLite tests the extra predicate per row. That is the cheap
/// direction: the expensive part was never the filtering, it was the sort.
const V9: &str = r#"
-- TRACES: FR-CULL-8
-- A record that face detection has *run* on an image, distinct from what it
-- found.
--
-- # Why the faces table cannot answer this
--
-- Without this, "has this image been indexed" is asked as "does it have any
-- faces", and those are not the same question. **A photograph with no faces in
-- it is indistinguishable from one that has never been looked at**, so every
-- indexing pass re-examines every landscape, every still life and every
-- document scan in the library, for ever. In a typical personal library that is
-- most of it: the pass never converges, and the cost is paid again on every
-- run rather than once.
--
-- It also makes a coverage figure possible, which is the thing a user actually
-- wants to see — "4,812 of 5,000 images indexed" — where counting face rows
-- can only ever report how many faces exist.
--
-- # Why it is keyed on the model
--
-- Embeddings from different models are not comparable, so a model change has
-- to re-index. Keying the marker on `(image_id, model_id)` makes that
-- automatic: new model, no marker, image comes back into the queue. The id
-- names the whole pipeline -- detector and embedder together -- because
-- changing either changes what is found.
CREATE TABLE face_index (
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
model_id TEXT NOT NULL,
indexed_at INTEGER NOT NULL,
-- Zero is a real and common answer, and recording it is the entire point.
faces_found INTEGER NOT NULL,
-- Long edge of the proxy this ran against. A face too small to detect at
-- 1024 may be findable at 2048, so a library whose proxies grow can
-- re-index the images that stand to gain instead of all of them.
source_edge INTEGER NOT NULL,
PRIMARY KEY (image_id, model_id)
);
CREATE INDEX face_index_model ON face_index(model_id);
"#;
const V8: &str = r#"
-- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
-- People and faces (docs/faces.md, docs/catalog.md §10).