dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

One completeness job over a registry of repairs, and a re-index button

Browse Source 
A library's records are never all complete at once. A face found before
its quality was kept has no quality; one found before the eye models
existed has no reading; one adopted from a peer's shard has no crop; an
image the fast detector examined on a 1024 px proxy has boxes the current
detector would not have drawn; an image the scan stat'ed has no capture
date. On the reference library that is 17,762 faces under the bare
w600k_mbf id with no quality, no reading and no dense landmarks, 4,144 of
them without a crop, beside 12,217 images the fast detector examined and
found nothing in. Every one of those gaps was its own pass — V14's
measuring pass, §17.5's eye pass, the sweep's proxy repair, the sweep's
detector upgrade — with its own work list, its own count and its own idea
of done, and adding a per-face field meant adding a pass. There was no
pass at all for the case the library is actually in: boxes and landmarks
drawn by a weaker detector on a proxy, which every later per-face pass
would have read from.

dr_ui::repairs replaces them with one job over a registry. A Repair names
one thing a record can lack — the predicate that says which images still
owe it, the input its handler needs (a header, the original, or a native
render), the handler, and what to record for an image that can never be
done. The job unions the predicates into one work list, fetches each
image once at the most any claimant asks for, renders it at most once,
and runs every handler whose predicate that image still matches, checked
again before each because a detection writes every field a per-face
handler would fill. The registry today: face-proxy, face-quality,
face-eyes, face-crop, face-detection, face-upgrade, metadata — the last
there to say that this is not a face job. Adding a field is one entry.

A repair's predicate is the only definition of its work: the count the
settings page shows, the list the job fetches and the check before its
handler run are one predicate, so the job converges. That is why the
registry is cut to what the device can do rather than listing what it
skips — an entry is a count and a set of originals to fetch — and why an
eye reading that cannot be cut is not a criterion.

The catalog side is generic to match: record_updates writes whichever
fields a FaceUpdate carries and re-marks the image so the shards export
it; faces_needing and count_needing answer a predicate the caller
supplies, replacing the measuring pass's three special cases.

Two buttons on the settings page run the job and differ in one
predicate. "Index faces" converges on coverage: has anything examined
this image. "Re-index every face" converges on provenance: face-detection
claims every image with no marker under the chosen detector, in either
of its forms (FaceDetector::model_ids, so a desktop in f32 and a tablet
on the Hexagon do not re-index each other's work), and a marker saying a
weaker one looked is not that. An original over the fetch budget is left
exactly as it was under the re-index, where the sweep marks it examined:
a re-detection with nothing found would delete the faces, and "cannot
fetch" is not "no faces".
This commit is contained in:
dtourolle
2026-09-19 18:52:13 +02:00
parent 2a4ac0ed3d
commit 5c00942b84
15 changed files with 2201 additions and 1450 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-catalog/src/faces.rs
+175 -130
View File
@@ -502,58 +502,77 @@ fn match_priors(prior: &[Prior], faces: &[DetectedFace]) -> Vec<Option<usize>> {
carried
}
/// A face embedded again from its stored landmarks: the new vector and its
/// length. What the measuring pass hands back per face.
/// What a per-face pass wants written over one stored face.
///
/// Each field is `Some` where the pass produced it and `None` where it is
/// to be left exactly as it was: a device without the eye models writing a
/// fresh vector must not blank a reading a peer had already made. One
/// struct for every pass rather than one writer per column, so a new
/// per-face field is a field here and a handler in `dr_ui::faces::repairs`,
/// and nothing else.
#[derive(Debug, Clone, PartialEq)]
pub struct Measurement {
pub struct FaceUpdate {
pub face: FaceId,
/// 512 × f16, raw — `dr_face::Embedded::to_f16_bytes`.
pub embedding: Vec<u8>,
pub quality: f32,
/// See [`DetectedFace::eyes`]. `None` where the measuring device has no
/// eye models, in which case the stored reading is left as it was.
pub eyes: Option<EyeReading>,
/// See [`DetectedFace::landmarks_dense`]; written with `eyes`.
pub landmarks_dense: Vec<u8>,
/// The raw vector (`dr_face::Embedded::to_f16_bytes`) and its length.
pub embedding: Option<(Vec<u8>, f32)>,
/// See [`DetectedFace::eyes`], with the dense landmarks it was read from
/// (see [`DetectedFace::landmarks_dense`]; empty stores NULL).
pub eyes: Option<(EyeReading, Vec<u8>)>,
/// See [`DetectedFace::crop`]. An empty crop is not written.
pub crop: Option<Vec<u8>>,
}
/// Write fresh embeddings over faces that were found before their quality was
/// kept, and re-mark the image as indexed.
impl FaceUpdate {
/// An update that changes nothing yet, for a handler to fill one field of.
pub fn for_face(face: FaceId) -> Self {
Self {
face,
embedding: None,
eyes: None,
crop: None,
}
}
}
/// Write what a per-face pass produced over the faces it read, and re-mark
/// the image as indexed.
///
/// The cheaper half of what `record_detections` does, for the case schema V14
/// created: the boxes and landmarks are right, the identities are the user's
/// work, and only the vector needs doing again. Updating in place is what
/// keeps `face_person` and the face ids exactly as they were -- a
/// re-detection carries identities across by matching old faces to new, and
/// a match is a judgement where an update in place is a fact.
/// The cheaper half of what `record_detections` does, for a face whose box
/// and landmarks are right and whose identity is the user's work, and which
/// lacks something a later pass can fill from the native render: its
/// quality (schema V14), its eye reading and dense landmarks (V16, V18), its
/// crop. Updating in place is what keeps `face_person` and the face ids
/// exactly as they were -- a re-detection carries identities across by
/// matching old faces to new, and a match is a judgement where an update
/// in place is a fact.
///
/// `dropped` are faces whose landmarks turned out to be degenerate -- the warp
/// could not be built from them. Deleted here, as detection would have refused
/// to store them (`dr_ui::faces::index_proxy`), and because a face left with
/// no reading would put its image back on the measuring pass's list on every
/// sweep, at the cost of an original each time.
/// `dropped` are faces whose landmarks turned out to be degenerate -- the
/// warp could not be built from them. Deleted here, as detection would have
/// refused to store them (`dr_ui::faces::index_native`), and because a face
/// that can never be filled would put its image back on the pass's list on
/// every sweep, at the cost of an original each time.
///
/// The run marker is re-written with a fresh time, and that is not
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how the measured vectors
/// reach the other devices.
pub fn record_measurements(
/// marker is newer than the store's copy, which is how what was written
/// here reaches the other devices.
pub fn record_updates(
conn: &Connection,
image_id: ImageId,
model_id: &str,
source_edge: u32,
measured: &[Measurement],
updates: &[FaceUpdate],
dropped: &[FaceId],
) -> Result<(), CatalogError> {
let tx = conn.unchecked_transaction()?;
for m in measured {
tx.execute(
"UPDATE faces SET embedding = ?2, quality = ?3 WHERE id = ?1",
rusqlite::params![m.face.0 as i64, m.embedding, f64::from(m.quality)],
)?;
// Written only when read: a device without the eye models measuring
// a face a peer had already read must not blank the reading.
if let Some(e) = m.eyes {
for u in updates {
if let Some((embedding, quality)) = &u.embedding {
tx.execute(
"UPDATE faces SET embedding = ?2, quality = ?3 WHERE id = ?1",
rusqlite::params![u.face.0 as i64, embedding, f64::from(*quality)],
)?;
}
if let Some((e, dense)) = &u.eyes {
tx.execute(
"UPDATE faces
SET eye_right = ?2, eye_right_px = ?3, eye_right_sharp = ?4,
@@ -561,7 +580,7 @@ pub fn record_measurements(
sunglasses = ?8, landmarks_dense = ?9
WHERE id = ?1",
rusqlite::params![
m.face.0 as i64,
u.face.0 as i64,
f64::from(e.right.open),
f64::from(e.right.px),
f64::from(e.right.sharpness),
@@ -569,10 +588,16 @@ pub fn record_measurements(
f64::from(e.left.px),
f64::from(e.left.sharpness),
f64::from(e.sunglasses),
(!m.landmarks_dense.is_empty()).then_some(m.landmarks_dense.as_slice()),
(!dense.is_empty()).then_some(dense.as_slice()),
],
)?;
}
if let Some(crop) = u.crop.as_ref().filter(|c| !c.is_empty()) {
tx.execute(
"UPDATE faces SET crop = ?2 WHERE id = ?1",
rusqlite::params![u.face.0 as i64, crop],
)?;
}
}
for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
@@ -604,45 +629,48 @@ pub fn record_measurements(
Ok(())
}
/// The faces on one image that have no quality reading yet — or, when the
/// device can read eyes, no eye reading either.
/// The faces on one image that a per-face pass still owes something to.
///
/// The measuring pass's per-image work: every face this model found whose
/// vector was stored as a unit one (schema V14), with the landmarks the
/// warp is rebuilt from; and, with `eyes`, every face never shown to the eye
/// models (schema V16). `eyes` is whether this device *has* those models —
/// a device without them must not list faces it cannot measure, or the pass
/// would fetch every original in the library to do nothing to it.
pub fn unmeasured_on_image(
/// `needs` is SQL over `faces` aliased as `f` -- `f.quality IS NULL`, say --
/// and it comes from the pass, not from here: which columns a face can lack
/// is the business of the handlers that fill them (`dr_ui::faces::repairs`),
/// and the catalog's part is to answer the question exactly as asked, so
/// that the count a screen shows, the list a sweep fetches and the faces a
/// handler is given are one predicate and the pass converges.
///
/// Keyed on the embedder half of `model_id`, like every other reader: a
/// face found by another detector in front of the same embedder is one of
/// this pipeline's faces.
pub fn faces_needing(
conn: &Connection,
image_id: ImageId,
model_id: &str,
eyes: bool,
needs: &str,
) -> Result<Vec<Face>, CatalogError> {
let mut q = conn.prepare(&format!(
"SELECT f.id FROM faces f
WHERE f.image_id = ?1 AND {} = ?2 AND ({needs})",
embedder_sql("f.model_id")
))?;
let owed: std::collections::HashSet<i64> = q
.query_map(
rusqlite::params![image_id.0 as i64, embedder_of(model_id)],
|r| r.get::<_, i64>(0),
)?
.collect::<Result<_, _>>()?;
Ok(for_image(conn, image_id)?
.into_iter()
.filter(|f| {
embedder_of(&f.model_id) == embedder_of(model_id)
&& (f.quality.is_none() || (eyes && f.eyes.is_none()))
})
.filter(|f| owed.contains(&(f.id.0 as i64)))
.collect())
}
/// How many of a model's faces have no quality reading, or — with `eyes` —
/// no eye reading.
///
/// What the measuring pass has left to do, for a screen that wants to say so.
/// `eyes` means what it means in [`unmeasured_on_image`].
pub fn faces_unmeasured(
conn: &Connection,
model_id: &str,
eyes: bool,
) -> Result<u64, CatalogError> {
/// How many of a model's faces a per-face pass still owes something to.
/// `needs` is what it is in [`faces_needing`].
pub fn count_needing(conn: &Connection, model_id: &str, needs: &str) -> Result<u64, CatalogError> {
conn.query_row(
&format!(
"SELECT COUNT(*) FROM faces WHERE {} = ?1 AND {}",
embedder_sql("model_id"),
unmeasured_sql("", eyes)
"SELECT COUNT(*) FROM faces f WHERE {} = ?1 AND ({needs})",
embedder_sql("f.model_id")
),
[embedder_of(model_id)],
|r| r.get::<_, i64>(0),
@@ -1381,22 +1409,6 @@ pub(crate) fn read_eyes(
}))
}
/// The predicate "this face still needs measuring", over `faces` aliased as
/// `prefix` (`"f."` or `""`).
///
/// One place for it because two queries ask — the count above and the
/// sweep's work list in `dr_ui::library` — and the two agreeing is what
/// makes the pass converge: a face the count reports is a face the list
/// fetches, and a face the list fetches is one whose reading the write
/// fills, so it leaves both.
pub fn unmeasured_sql(prefix: &str, eyes: bool) -> String {
if eyes {
format!("({prefix}quality IS NULL OR {prefix}eye_right IS NULL)")
} else {
format!("{prefix}quality IS NULL")
}
}
fn landmarks_to_blob(lm: &[(f32, f32); 5]) -> Vec<u8> {
let mut out = Vec::with_capacity(40);
for &(x, y) in lm {
@@ -1575,10 +1587,13 @@ mod tests {
);
}
const NEEDS_QUALITY: &str = "f.quality IS NULL";
const NEEDS_EYES: &str = "f.eye_right IS NULL";
/// The seven eye columns are one fact: a face with none of them reads as
/// unread, and the measuring pass is what fills them.
/// unread, and a per-face pass asking by predicate is what fills them.
#[test]
fn eyes_are_measured_only_where_the_device_can_read_them() {
fn eyes_are_filled_only_where_a_pass_produced_a_reading() {
let c = db();
let img = image(&c, 1);
let unread = DetectedFace {
@@ -1588,67 +1603,64 @@ mod tests {
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[unread, face(2)]).unwrap();
assert_eq!(for_image(&c, img).unwrap().len(), 2);
// Quality is present on both, so a device without the eye models has
// nothing to do here; one with them has one face to read.
assert_eq!(faces_unmeasured(&c, "w600k_mbf", false).unwrap(), 0);
assert_eq!(faces_unmeasured(&c, "w600k_mbf", true).unwrap(), 1);
// Quality is present on both, so a pass that only fills quality has
// nothing to do here; one that reads eyes has one face.
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_QUALITY).unwrap(), 0);
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_EYES).unwrap(), 1);
assert_eq!(
unmeasured_on_image(&c, img, "w600k_mbf", false)
faces_needing(&c, img, "w600k_mbf", NEEDS_QUALITY)
.unwrap()
.len(),
0
);
let todo = unmeasured_on_image(&c, img, "w600k_mbf", true).unwrap();
let todo = faces_needing(&c, img, "w600k_mbf", NEEDS_EYES).unwrap();
assert_eq!(todo.len(), 1);
assert_eq!(todo[0].id, ids[0]);
// A measurement with no reading leaves the columns alone …
record_measurements(
// An update with no reading leaves the columns alone …
record_updates(
&c,
img,
"w600k_mbf",
6000,
&[Measurement {
face: ids[0],
embedding: vec![9; 1024],
quality: 21.5,
eyes: None,
landmarks_dense: Vec::new(),
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
assert_eq!(faces_unmeasured(&c, "w600k_mbf", true).unwrap(), 1);
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_EYES).unwrap(), 1);
// … and one with a reading fills them.
record_measurements(
record_updates(
&c,
img,
"w600k_mbf",
6000,
&[Measurement {
face: ids[0],
embedding: vec![9; 1024],
quality: 21.5,
eyes: Some(EyeReading {
right: Eye {
open: 0.2,
px: 40.0,
sharpness: 0.2,
&[FaceUpdate {
eyes: Some((
EyeReading {
right: Eye {
open: 0.2,
px: 40.0,
sharpness: 0.2,
},
left: Eye {
open: 0.9,
px: 40.0,
sharpness: 0.2,
},
sunglasses: 0.0,
},
left: Eye {
open: 0.9,
px: 40.0,
sharpness: 0.2,
},
sunglasses: 0.0,
}),
landmarks_dense: vec![9; 424],
vec![9; 424],
)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
assert_eq!(faces_unmeasured(&c, "w600k_mbf", true).unwrap(), 0);
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_EYES).unwrap(), 0);
assert_eq!(
for_image(&c, img).unwrap()[0].landmarks_dense.len(),
424,
@@ -1662,10 +1674,46 @@ mod tests {
);
}
/// The measuring pass writes over the vector and nothing else: the face
/// keeps its id, its box and whoever the user said it was.
/// A crop is filled the same way, and an empty one is not written.
#[test]
fn measuring_replaces_the_vector_and_keeps_the_identity() {
fn a_crop_is_filled_in_place() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(&c, img, "w600k_mbf", 1024, &[face(1)]).unwrap();
assert_eq!(count_needing(&c, "w600k_mbf", "f.crop IS NULL").unwrap(), 1);
record_updates(
&c,
img,
"w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(Vec::new()),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
assert_eq!(count_needing(&c, "w600k_mbf", "f.crop IS NULL").unwrap(), 1);
record_updates(
&c,
img,
"w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(vec![1, 2, 3]),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
assert_eq!(count_needing(&c, "w600k_mbf", "f.crop IS NULL").unwrap(), 0);
assert_eq!(crop(&c, ids[0]).unwrap(), Some(vec![1, 2, 3]));
}
/// An update writes over the vector and nothing else: the face keeps its
/// id, its box and whoever the user said it was.
#[test]
fn an_update_replaces_the_vector_and_keeps_the_identity() {
let c = db();
let img = image(&c, 1);
let unmeasured = DetectedFace {
@@ -1682,9 +1730,9 @@ mod tests {
.unwrap();
let person = create_person(&c, "Anna").unwrap();
confirm(&c, ids[0], person).unwrap();
assert_eq!(faces_unmeasured(&c, "w600k_mbf", false).unwrap(), 2);
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_QUALITY).unwrap(), 2);
assert_eq!(
unmeasured_on_image(&c, img, "w600k_mbf", false)
faces_needing(&c, img, "w600k_mbf", NEEDS_QUALITY)
.unwrap()
.len(),
2
@@ -1695,17 +1743,14 @@ mod tests {
c.execute("UPDATE face_index SET indexed_at = indexed_at - 100", [])
.unwrap();
record_measurements(
record_updates(
&c,
img,
"w600k_mbf",
6000,
&[Measurement {
face: ids[0],
embedding: vec![9; 1024],
quality: 21.5,
eyes: None,
landmarks_dense: Vec::new(),
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[ids[1]],
)
@@ -1719,7 +1764,7 @@ mod tests {
assert!(got[0].confirmed);
let e = embeddings(&c, "w600k_mbf").unwrap();
assert_eq!(e[0].embedding[0], 9);
assert_eq!(faces_unmeasured(&c, "w600k_mbf", false).unwrap(), 0);
assert_eq!(count_needing(&c, "w600k_mbf", NEEDS_QUALITY).unwrap(), 0);
// The marker says one face at the native edge, and is fresh — which
// is what makes the sync export it again.
core/dr-catalog/src/lib.rs
+1 -1
View File
@@ -61,7 +61,7 @@ pub use collections::{Collection, CollectionKind, TreeRow};
pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash};
pub use error::CatalogError;
pub use face_shard::{FaceShardStore, SharedFace};
pub use faces::{Calibration, DetectedFace, Face, FaceId, Measurement, Person, PersonId};
pub use faces::{Calibration, DetectedFace, Face, FaceId, FaceUpdate, Person, PersonId};
pub use jobs::{Job, JobKind, Priority};
pub use keywords::{Coverage, Keyword, KeywordId, SelectionKeyword};
pub use merge::MergeReport;
core/dr-catalog/src/schema.rs
+10 -10
View File
@@ -701,20 +701,20 @@ const V14: &str = r#"
-- face this rule is not yet protecting anyone from, and the only way to
-- measure it is to embed it again.
--
-- The sweep's measuring pass is what does that: `dr_ui::library::
-- faces_unmeasured` lists every image holding a face with no reading, and
-- each face is embedded again from the native render with the landmarks it
-- already has, the raw vector written over the old one (`record_measurements`)
-- and nothing else touched -- not the id, not the box, not who the user said
-- it was. The faces keep drawing the People screen throughout.
-- The `face-quality` repair is what does that (`dr_ui::repairs`, once the
-- sweep's measuring pass): it lists every face with no reading, and each is
-- embedded again from the native render with the landmarks it already has,
-- the raw vector written over the old one (`record_updates`) and nothing
-- else touched -- not the id, not the box, not who the user said it was.
-- The faces keep drawing the People screen throughout.
--
-- The run markers of those images are forgotten too, exactly as V12 forgot
-- the runs made against too small a proxy. The build this shipped in had no
-- measuring pass yet, and a marker is the one thing that stops a face ever
-- being looked at again; with the pass in place `faces_unindexed` leaves
-- these images to it rather than detecting them from scratch, so the
-- deletion costs nothing -- and an image that was examined and found empty
-- keeps its marker, since there is nothing on it to measure.
-- being looked at again; with the repair in place, detection leaves an
-- image holding this embedder's faces to it rather than detecting from
-- scratch, so the deletion costs nothing -- and an image that was examined
-- and found empty keeps its marker, since there is nothing on it to measure.
--
-- The cost is a re-fetch of every image with a face on it, on the next pass
-- the user starts. That is a whole-library transfer (FR-NC-6), and it starts