dtourolle
84fade99ec
docs/ had 26 developer documents flat beside the manual, and the two audiences are very differently sized: most readers want the manual and the gesture reference, a few want the register, the designs and the measurements. The manual and gestures.md stay at the top; everything for someone changing the code moves to docs/dev/, and the two documents that name their own successors — the v0.1 milestone and the UI-refinement plan — go to docs/dev/archive/ rather than being deleted, since both are still cited. docs/README.md is the index, users first. Every reference follows: code comments, Cargo manifests, the workflows, the pre-commit hook, the bench and traceability tools (which locate the repo root by docs/dev/requirements.md now), packaging, the Docker READMEs, CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level deeper and is regenerated. Links out of the moved documents into the tree gain a level; a link checker over every Markdown file finds none broken.
2342 lines
90 KiB
Rust
2342 lines
90 KiB
Rust
//! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | NFR-ARCH-2 | NFR-SEC-5
|
||
//! Face indexing and clustering, as background passes over the library.
|
||
//!
|
||
//! `dr-face` knows how to find a face in a buffer and `dr-catalog` knows how to
|
||
//! store one. This is the pass that connects them: read the proxy the grid
|
||
//! already built, detect, align, embed, write, and — once detection has gone
|
||
//! quiet — group what was found into people.
|
||
//!
|
||
//! # Detection runs on proxies, never on originals
|
||
//!
|
||
//! FR-CULL-8 pins this to the FR-CULL-2 ladder, and the consequence is the
|
||
//! thing that makes the feature affordable: a library that has been browsed has
|
||
//! already paid for its proxies, so face indexing adds **no RAW decodes that
|
||
//! were not already happening**. The tier is `ThumbSize::Large` — 1024 on the
|
||
//! long edge — and docs/dev/faces.md §7 has the table of what the embedder actually
|
||
//! receives at that resolution.
|
||
//!
|
||
//! # Why clustering is a separate pass and not a job
|
||
//!
|
||
//! Detection is per image and parallel, so it is a job. Clustering is a
|
||
//! *whole-library* operation over the embeddings detection produced: it has no
|
||
//! natural `subject_id`, and running it per photograph would rebuild the world
|
||
//! on every one. It therefore runs debounced, when detection has been idle and
|
||
//! the face count has moved materially (catalog.md §10.2).
|
||
|
||
use std::path::PathBuf;
|
||
use std::sync::mpsc::{Receiver, Sender};
|
||
|
||
use dr_catalog::faces::{self, DetectedFace};
|
||
use dr_catalog::Catalog;
|
||
use dr_face::{
|
||
align, Calibration, DetectOptions, Detection, Detector, Embedder, EyeModels, EyeReading,
|
||
ModelId,
|
||
};
|
||
use dr_thumbs::{ThumbSize, ThumbStore};
|
||
use dr_types::settings::FaceSettings;
|
||
use dr_types::ImageId;
|
||
|
||
/// The tier a stored face crop is cut from. See the module note.
|
||
///
|
||
/// **No longer a tier faces are found or cropped at**, and the distinction is
|
||
/// the whole of `docs/dev/faces.md` §7: cutting an already-located face out of a
|
||
/// stored 1024px proxy so the People screen can draw a thumbnail of it is
|
||
/// fine, because that crop is only ever looked at. Sampling the *embedder's*
|
||
/// 112×112 from a buffer this size is what left 47% of the reference library's
|
||
/// faces upsampled, and FR-CULL-8 now requires that crop to come from the
|
||
/// native render.
|
||
pub const FACE_TIER: ThumbSize = ThumbSize::Large;
|
||
|
||
/// Progress from an indexing sweep.
|
||
#[derive(Debug, Clone, PartialEq)]
|
||
pub enum FaceSweepMessage {
|
||
/// How many images will be visited. Sent once, before any work.
|
||
Total(usize),
|
||
/// One image finished, with the faces found in it.
|
||
Indexed { image: ImageId, faces: usize },
|
||
/// Images the pass gave up on, counted since the last message.
|
||
///
|
||
/// **Progress has to advance on these or it does not advance at all.**
|
||
/// The count moved only on `Indexed`, so a run where every image failed
|
||
/// sat at 0/169 from the first tick to the last and then reported
|
||
/// success: the receiver was told the total, told nothing, and told the
|
||
/// pass had ended. That is exactly what happened on the reference library
|
||
/// for a day, and it is why the sweep was thought to be hanging when it
|
||
/// was in fact finishing in fourteen seconds.
|
||
///
|
||
/// A batch rather than one message per image, because failures come back
|
||
/// lane-sized and the interesting number is how many. The reason for each
|
||
/// is logged where it happens.
|
||
Failed { images: usize },
|
||
/// The pass ended.
|
||
Finished {
|
||
images: usize,
|
||
faces: usize,
|
||
failed: usize,
|
||
},
|
||
}
|
||
|
||
/// An image waiting to be indexed.
|
||
#[derive(Debug, Clone, PartialEq)]
|
||
pub struct FaceRequest {
|
||
pub image_id: ImageId,
|
||
/// The thumbnail store's key — `oc:fileid`, stable across a server-side
|
||
/// move and the same id every other client sees (FR-NC-5).
|
||
pub file_id: u64,
|
||
}
|
||
|
||
/// Images that have a usable proxy and have not been through this model.
|
||
///
|
||
/// Asks `face_index` — the *run* marker — rather than asking whether the image
|
||
/// has any faces. Those are different questions, and confusing them is the
|
||
/// difference between a pass that converges and one that does not: a
|
||
/// photograph with no face in it would otherwise look identical to one never
|
||
/// examined, so every landscape in the library would be re-detected on every
|
||
/// run, for ever. See the V9 migration.
|
||
///
|
||
/// Keyed on the embedder, so an embedder upgrade re-indexes rather than
|
||
/// leaving the library half-described by weights that are no longer
|
||
/// comparable — and a detector change, which keeps the embedder, does not.
|
||
pub fn faces_outstanding(
|
||
catalog: &Catalog,
|
||
store: &ThumbStore,
|
||
model_id: &str,
|
||
) -> Result<Vec<FaceRequest>, dr_catalog::CatalogError> {
|
||
let mut stmt = catalog.connection().prepare(&format!(
|
||
"SELECT i.id, r.file_id
|
||
FROM images i
|
||
JOIN remote r ON r.image_id = i.id
|
||
WHERE r.file_id IS NOT NULL
|
||
AND i.trashed_at IS NULL
|
||
AND i.shadowed_by IS NULL
|
||
AND NOT EXISTS (
|
||
SELECT 1 FROM face_index fi
|
||
WHERE fi.image_id = i.id AND {} = ?1
|
||
)
|
||
ORDER BY i.id",
|
||
faces::embedder_sql("fi.model_id")
|
||
))?;
|
||
// The store's index in one read rather than a probe per image; see
|
||
// `audit`, which splits the same list the same way.
|
||
let held = store.held(FACE_TIER).unwrap_or_else(|e| {
|
||
log::warn!("faces: reading the thumbnail index: {e}");
|
||
Default::default()
|
||
});
|
||
let rows = stmt
|
||
.query_map([faces::embedder_of(model_id)], |r| {
|
||
Ok(FaceRequest {
|
||
image_id: ImageId(r.get::<_, i64>(0)? as u64),
|
||
file_id: r.get::<_, i64>(1)? as u64,
|
||
})
|
||
})?
|
||
.filter_map(Result::ok)
|
||
// This pass reads the store and only the store, so an image without a
|
||
// proxy is not work it can do.
|
||
//
|
||
// **Not because FR-CULL-8 forbids it.** That requirement keeps indexing
|
||
// off the *full decode* and says the opposite about proxies — "where no
|
||
// proxy exists, the job requests one at background priority". An
|
||
// earlier comment here read it the other way round, and the result was
|
||
// a whole-library button that could only reach photographs the user had
|
||
// personally zoomed into. `repairs::spawn` is that
|
||
// requirement implemented; this one is the local-only variant.
|
||
.filter(|req| held.contains(&req.file_id))
|
||
.collect();
|
||
Ok(rows)
|
||
}
|
||
|
||
/// What a coverage check found.
|
||
///
|
||
/// The catalog can say how many images have been through the model; only this
|
||
/// layer can say *why* the rest have not, because the reason usually lives in
|
||
/// the thumbnail store rather than the catalog.
|
||
#[derive(Debug, Clone, PartialEq, Eq, Default)]
|
||
pub struct IndexAudit {
|
||
pub coverage: faces::Coverage,
|
||
/// Outstanding, with a proxy already on disk.
|
||
///
|
||
/// **Not "ready to index".** It was, when a pass existed that detected on
|
||
/// the stored proxy; that proxy is 1024 and detection now refuses it
|
||
/// (§7c), so these need a fetch exactly like the others do. Kept as a
|
||
/// separate number only because it says something true about the cache.
|
||
pub ready: u64,
|
||
/// Outstanding with no proxy on disk. Costs a range request, same as the
|
||
/// ones above now do.
|
||
pub awaiting_proxy: u64,
|
||
/// TRACES: FR-CULL-8a
|
||
/// What each repair in the registry still lists, by its label
|
||
/// (`crate::repairs::counts`) — the faces stored without their quality,
|
||
/// without an eye reading on a device with the eye models, without a
|
||
/// crop; the images a weaker detector indexed. Work that is not visible
|
||
/// in the coverage figure, since every one of these images carries its
|
||
/// run marker, and that has to be counted here or the screen calls the
|
||
/// library finished and takes the button away that would finish it.
|
||
///
|
||
/// Detection's own entry is left out: it is the outstanding figure
|
||
/// above, split by proxy.
|
||
pub owed: Vec<(&'static str, u64)>,
|
||
}
|
||
|
||
impl IndexAudit {
|
||
/// One line, for a log or a status strip.
|
||
pub fn summary(&self) -> String {
|
||
let c = &self.coverage;
|
||
if c.images == 0 {
|
||
return "no images in the library".into();
|
||
}
|
||
// Whole numbers read fine at 40% and lie at 0.47%, which rounds to
|
||
// "0%" beside a count of 110 — a figure that says the feature is
|
||
// broken when it is merely early. One decimal below ten percent, and
|
||
// a floor so real progress never displays as none.
|
||
let pct = c.fraction() * 100.0;
|
||
let shown = if c.indexed > 0 && pct < 0.1 {
|
||
"<0.1%".to_string()
|
||
} else if pct < 10.0 {
|
||
format!("{pct:.1}%")
|
||
} else {
|
||
format!("{pct:.0}%")
|
||
};
|
||
let mut s = format!(
|
||
"{}/{} images indexed ({shown}), {} face(s), {} image(s) with none",
|
||
c.indexed, c.images, c.faces, c.without_faces,
|
||
);
|
||
// One number, not two. The split used to matter because one of the
|
||
// two passes could only do the images that already had a proxy; now
|
||
// that detection refuses that proxy's size, both halves cost the same
|
||
// fetch and telling the user "169 ready, 0 to fetch" only implies a
|
||
// distinction that no longer decides anything.
|
||
let outstanding = self.ready + self.awaiting_proxy;
|
||
if outstanding > 0 {
|
||
s.push_str(&format!("; {outstanding} to index"));
|
||
}
|
||
for (label, n) in &self.owed {
|
||
if *n > 0 {
|
||
s.push_str(&format!("; {n} {label}"));
|
||
}
|
||
}
|
||
s
|
||
}
|
||
|
||
/// Whether anything but detection is left: the state an already-indexed
|
||
/// library is in the day the eye models arrive, where the button reads
|
||
/// "Read eye state" rather than promising to index.
|
||
pub fn has_repairs(&self) -> bool {
|
||
self.owed.iter().any(|(_, n)| *n > 0)
|
||
}
|
||
|
||
/// Whether the sweep has nothing left to do — nothing to index *and*
|
||
/// nothing to measure. The screen hides the button on this, so it has to
|
||
/// be false while the measuring pass has work, or the eye readings of an
|
||
/// already-indexed library could never be filled in.
|
||
pub fn is_complete(&self) -> bool {
|
||
self.coverage.is_complete() && !self.has_repairs()
|
||
}
|
||
}
|
||
|
||
/// Check every library image for a face-detection run marker.
|
||
///
|
||
/// The batch pass that answers "has face recognition been over all of this",
|
||
/// and the one to run before deciding whether to start a sweep. Cheap: two
|
||
/// counts and one indexed scan, no decoding and no inference.
|
||
///
|
||
/// `repairs` is the registry this device would run (`crate::repairs::
|
||
/// registry`), which decides what counts as work: a device without the eye
|
||
/// models has no eye repair and so lists no faces to read.
|
||
pub fn audit(
|
||
catalog: &Catalog,
|
||
store: &ThumbStore,
|
||
model_id: &str,
|
||
repairs: &[crate::repairs::Repair],
|
||
) -> Result<IndexAudit, dr_catalog::CatalogError> {
|
||
let conn = catalog.connection();
|
||
let coverage = faces::coverage(conn, model_id)?;
|
||
let owed = crate::repairs::counts(catalog, store, repairs)?
|
||
.into_iter()
|
||
.zip(repairs.iter())
|
||
.filter(|(_, r)| r.name != "face-detection")
|
||
.map(|(c, _)| c)
|
||
.collect();
|
||
|
||
// Split the outstanding set by whether a proxy exists. This is the query
|
||
// `faces_outstanding` runs without the store filter, so the two cannot
|
||
// disagree about what is outstanding. Shadowed images are excluded here
|
||
// for the same reason `faces::coverage` excludes them: they are the JPEG
|
||
// half of a pair, no sweep will ever index one, and counting them makes
|
||
// the outstanding figure a number that cannot reach zero.
|
||
let mut stmt = conn.prepare(&format!(
|
||
"SELECT r.file_id
|
||
FROM images i
|
||
JOIN remote r ON r.image_id = i.id
|
||
WHERE r.file_id IS NOT NULL
|
||
AND i.trashed_at IS NULL
|
||
AND i.shadowed_by IS NULL
|
||
AND NOT EXISTS (
|
||
SELECT 1 FROM face_index fi
|
||
WHERE fi.image_id = i.id AND {} = ?1
|
||
)",
|
||
faces::embedder_sql("fi.model_id")
|
||
))?;
|
||
// One read of the store's index, not one probe per outstanding image:
|
||
// `contains` answers the same question, and asked four thousand times
|
||
// it cost more than every query above put together. A store whose index
|
||
// cannot be read is treated as holding nothing, which is what `contains`
|
||
// reports for it too.
|
||
let held = store.held(FACE_TIER).unwrap_or_else(|e| {
|
||
log::warn!("faces: reading the thumbnail index: {e}");
|
||
Default::default()
|
||
});
|
||
let (mut ready, mut awaiting) = (0u64, 0u64);
|
||
for file_id in stmt
|
||
.query_map([faces::embedder_of(model_id)], |r| r.get::<_, i64>(0))?
|
||
.filter_map(Result::ok)
|
||
{
|
||
if held.contains(&(file_id as u64)) {
|
||
ready += 1;
|
||
} else {
|
||
awaiting += 1;
|
||
}
|
||
}
|
||
|
||
Ok(IndexAudit {
|
||
coverage,
|
||
ready,
|
||
awaiting_proxy: awaiting,
|
||
owed,
|
||
})
|
||
}
|
||
|
||
/// TRACES: FR-CULL-8a
|
||
/// Read one face's eyes from the buffer its crop came from, if this device
|
||
/// can.
|
||
///
|
||
/// From the same pixels the embedder saw the face in, so what the models
|
||
/// see is the eye at the resolution the crop had — and `dr_face::eyes`'
|
||
/// readability floors are judged against real pixels rather than a proxy's
|
||
/// idea of them. `bbox` is the detector's `(x0, y0, x1, y1)` and `landmarks`
|
||
/// its five points, both in this buffer's pixels. `None` on a device without
|
||
/// the models, which is the ordinary state of one that has not been given
|
||
/// them, and `None` — logged — where the models refuse: a face the embedder
|
||
/// could use is not lost for the want of an eye reading.
|
||
///
|
||
/// Returns the reading and the dense landmarks it was read from, the
|
||
/// latter already packed for the catalog (`landmarks_dense`, normalised by
|
||
/// `long_edge` like the five points).
|
||
fn read_eyes(
|
||
models: Option<&mut EyeModels>,
|
||
px: dr_face::Pixels<'_>,
|
||
width: usize,
|
||
height: usize,
|
||
bbox: (f32, f32, f32, f32),
|
||
landmarks: &[(f32, f32); 5],
|
||
long_edge: f32,
|
||
) -> (Option<EyeReading>, Vec<u8>) {
|
||
let Some(models) = models else {
|
||
return (None, Vec::new());
|
||
};
|
||
match models.read(px, width, height, bbox, landmarks) {
|
||
Ok(Some((reading, dense))) => (Some(reading), dense.to_packed_bytes(long_edge)),
|
||
Ok(None) => (None, Vec::new()),
|
||
Err(e) => {
|
||
log::debug!("eye reading failed: {e}");
|
||
(None, Vec::new())
|
||
}
|
||
}
|
||
}
|
||
|
||
/// Detect and embed every face in one decoded proxy.
|
||
///
|
||
/// Coordinates come back **normalised to the long edge**, which is what the
|
||
/// catalog stores: a face must survive the proxy it was found on being evicted
|
||
/// and regenerated at a different size.
|
||
///
|
||
/// A face whose landmarks are degenerate is dropped rather than stored with a
|
||
/// junk embedding. That happens — a detector firing on a motion-blurred profile
|
||
/// can put all five landmarks on a line — and one junk embedding in the
|
||
/// clustering graph can bridge two real people.
|
||
pub fn index_proxy(
|
||
detector: &mut Detector,
|
||
embedder: &mut Embedder,
|
||
mut eye_models: Option<&mut EyeModels>,
|
||
rgb: &[f32],
|
||
width: usize,
|
||
height: usize,
|
||
options: &DetectOptions,
|
||
) -> Result<Vec<DetectedFace>, dr_face::FaceError> {
|
||
let long_edge = width.max(height) as f32;
|
||
if long_edge <= 0.0 {
|
||
return Ok(Vec::new());
|
||
}
|
||
|
||
let dets = detector.detect(rgb, width, height, options)?;
|
||
let mut out = Vec::with_capacity(dets.len());
|
||
|
||
for d in &dets {
|
||
let Some(aligned) = align::warp(rgb, width, height, &d.landmarks) else {
|
||
log::debug!("face with degenerate landmarks skipped");
|
||
continue;
|
||
};
|
||
|
||
// The size floor, on the crop rather than the box. `min_face_px` has
|
||
// already thrown away the hopeless; this is the real gate, and it is
|
||
// here because `source_px` is only known once the warp has fixed the
|
||
// scale. A face below it was upsampled to reach the embedder, and no
|
||
// amount of upsampling puts back detail the sensor never recorded.
|
||
if aligned.source_px() < options.min_source_px {
|
||
log::debug!(
|
||
"face skipped: {:.0} source px below {:.0}",
|
||
aligned.source_px(),
|
||
options.min_source_px
|
||
);
|
||
continue;
|
||
}
|
||
|
||
// The blur gate, and the reason it is here rather than in the
|
||
// detector: sharpness is a property of the *aligned* crop, so it
|
||
// cannot be known until the warp has run.
|
||
//
|
||
// A motion-blurred face detects confidently, aligns cleanly and embeds
|
||
// to a perfectly ordinary-looking vector. Nothing downstream can tell
|
||
// it apart from a real one — and because blurs resemble each other
|
||
// more than they resemble the people they were, they cluster together
|
||
// and weld unrelated identities into one group. Dropping it costs a
|
||
// face the user could not have identified anyway.
|
||
let sharpness = aligned.sharpness();
|
||
if sharpness < options.min_sharpness {
|
||
log::debug!(
|
||
"face at {:.0}px skipped: sharpness {sharpness:.4} below {:.4}",
|
||
aligned.source_px(),
|
||
options.min_sharpness
|
||
);
|
||
continue;
|
||
}
|
||
|
||
let embedded = embedder.embed(&aligned)?;
|
||
let (eyes, landmarks_dense) = read_eyes(
|
||
eye_models.as_deref_mut(),
|
||
dr_face::Pixels::RgbF32(rgb),
|
||
width,
|
||
height,
|
||
d.bbox,
|
||
&d.landmarks,
|
||
long_edge,
|
||
);
|
||
|
||
out.push(DetectedFace {
|
||
x: d.bbox.0 / long_edge,
|
||
y: d.bbox.1 / long_edge,
|
||
w: d.width() / long_edge,
|
||
h: d.height() / long_edge,
|
||
landmarks: normalise_landmarks(&d.landmarks, long_edge),
|
||
confidence: d.confidence,
|
||
embedding: embedded.to_f16_bytes(),
|
||
crop_px: aligned.source_px(),
|
||
quality: Some(embedded.quality),
|
||
eyes,
|
||
landmarks_dense,
|
||
model_id: embedder.model().as_str().to_string(),
|
||
// Cut here, while the buffer is still in hand. This is the only
|
||
// moment in the whole pipeline where the pixels are free.
|
||
crop: cut_crop(rgb, width, height, d).unwrap_or_default(),
|
||
});
|
||
}
|
||
|
||
Ok(out)
|
||
}
|
||
|
||
/// TRACES: FR-CULL-8
|
||
/// Long edge the detector's input is reduced to, in pixels.
|
||
///
|
||
/// Detection is indifferent above this — `dr_face::detect` letterboxes into a
|
||
/// fixed 640×640 whatever it receives, so a face is the same size to the model
|
||
/// from a 1600px buffer as from a 6000px one (faces.md §4.1). What the
|
||
/// reduction buys is the aliasing that a single bilinear step from native to
|
||
/// 640 would introduce: a 9× decimation samples one pixel in nine and drops
|
||
/// small faces into the gaps between samples. Box-filtering to 1600 first, and
|
||
/// letting the letterbox take the remaining 2.5×, keeps every source pixel in
|
||
/// the average.
|
||
///
|
||
/// It also bounds the buffer that has to exist as `f32`: 1600×1067 is 20 MB
|
||
/// against 288 MB for a native 24 MP frame.
|
||
const DETECT_EDGE: usize = 1600;
|
||
|
||
/// TRACES: FR-CULL-8 | NFR-RES-2
|
||
/// Detect and embed every face in one **native-resolution** render.
|
||
///
|
||
/// The pass FR-CULL-8 specifies, and the two resolutions in it are the whole
|
||
/// point:
|
||
///
|
||
/// - the **detector** is handed a box-filtered reduction, because it discards
|
||
/// anything above its own 640px input anyway;
|
||
/// - the **crop** is warped out of the native buffer, because that is the one
|
||
/// place source resolution becomes embedding quality.
|
||
///
|
||
/// Boxes and landmarks come back in the reduction's coordinates and are scaled
|
||
/// to native before a single crop pixel is read. Getting that scaling wrong
|
||
/// does not fail loudly — it yields faces in plausible-looking places with
|
||
/// crops taken from beside them — so it is one multiply applied in one place
|
||
/// rather than at each use.
|
||
///
|
||
/// `rgba` is the native render, tightly packed 8-bit RGBA. It is never
|
||
/// converted wholesale: [`dr_face::Pixels`] samples it where the warp and the
|
||
/// stored crop actually touch it.
|
||
pub fn index_native(
|
||
detector: &mut Detector,
|
||
embedder: &mut Embedder,
|
||
mut eye_models: Option<&mut EyeModels>,
|
||
rgba: &[u8],
|
||
width: usize,
|
||
height: usize,
|
||
options: &DetectOptions,
|
||
) -> Result<Vec<DetectedFace>, dr_face::FaceError> {
|
||
if !index_native_shape_ok(rgba, width, height) {
|
||
return Ok(Vec::new());
|
||
}
|
||
let long_edge = width.max(height) as f32;
|
||
let native = dr_face::Pixels::Rgba8(rgba);
|
||
|
||
// One reduction, reused for every face on the image.
|
||
let (dw, dh, small) = reduce_for_detection(rgba, width, height);
|
||
if dw == 0 || dh == 0 {
|
||
return Ok(Vec::new());
|
||
}
|
||
let dets = detector.detect(&small, dw, dh, options)?;
|
||
|
||
// Detector coordinates to native. Separate factors rather than one, because
|
||
// the reduction rounds each axis independently and assuming they match puts
|
||
// every landmark a fraction of a face off on the shorter one.
|
||
let (sx, sy) = (width as f32 / dw as f32, height as f32 / dh as f32);
|
||
|
||
let mut out = Vec::with_capacity(dets.len());
|
||
for d in &dets {
|
||
let landmarks = scale_landmarks(&d.landmarks, sx, sy);
|
||
let Some(aligned) = dr_face::warp_pixels(native, width, height, &landmarks) else {
|
||
log::debug!("face with degenerate landmarks skipped");
|
||
continue;
|
||
};
|
||
|
||
// Both gates read the aligned crop, so they mean what they say only now
|
||
// that the crop comes from native pixels — `source_px` is the real
|
||
// count, not the proxy's idea of it. See `index_proxy` for why each
|
||
// one is here rather than in the detector.
|
||
if aligned.source_px() < options.min_source_px {
|
||
log::debug!(
|
||
"face skipped: {:.0} source px below {:.0}",
|
||
aligned.source_px(),
|
||
options.min_source_px
|
||
);
|
||
continue;
|
||
}
|
||
let sharpness = aligned.sharpness();
|
||
if sharpness < options.min_sharpness {
|
||
log::debug!(
|
||
"face at {:.0}px skipped: sharpness {sharpness:.4} below {:.4}",
|
||
aligned.source_px(),
|
||
options.min_sharpness
|
||
);
|
||
continue;
|
||
}
|
||
|
||
let embedded = embedder.embed(&aligned)?;
|
||
let (bx, by) = (d.bbox.0 * sx, d.bbox.1 * sy);
|
||
let (bw, bh) = (d.width() * sx, d.height() * sy);
|
||
// From the native buffer, box and landmarks scaled like the crop's
|
||
// — the eye is a fortieth of the face, and it is here that the
|
||
// native render pays for itself twice.
|
||
let (eyes, landmarks_dense) = read_eyes(
|
||
eye_models.as_deref_mut(),
|
||
native,
|
||
width,
|
||
height,
|
||
(bx, by, bx + bw, by + bh),
|
||
&landmarks,
|
||
long_edge,
|
||
);
|
||
|
||
out.push(DetectedFace {
|
||
x: bx / long_edge,
|
||
y: by / long_edge,
|
||
w: bw / long_edge,
|
||
h: bh / long_edge,
|
||
landmarks: normalise_landmarks(&landmarks, long_edge),
|
||
confidence: d.confidence,
|
||
embedding: embedded.to_f16_bytes(),
|
||
crop_px: aligned.source_px(),
|
||
quality: Some(embedded.quality),
|
||
eyes,
|
||
landmarks_dense,
|
||
model_id: embedder.model().as_str().to_string(),
|
||
crop: cut_crop_native(native, width, height, (bx, by, bw, bh)).unwrap_or_default(),
|
||
});
|
||
}
|
||
|
||
Ok(out)
|
||
}
|
||
|
||
/// What re-embedding the faces already on one image produced.
|
||
#[derive(Debug, Default, PartialEq)]
|
||
pub struct Measured {
|
||
pub measured: Vec<faces::FaceUpdate>,
|
||
/// Faces whose stored landmarks no longer make a warp. See
|
||
/// `dr_catalog::faces::record_updates` for what becomes of them.
|
||
pub dropped: Vec<faces::FaceId>,
|
||
}
|
||
|
||
/// TRACES: FR-CULL-8 | FR-CULL-9
|
||
/// Embed the faces already found on one image again, from its native render.
|
||
///
|
||
/// The `face-quality` repair (`crate::repairs`), for faces stored before their
|
||
/// quality was kept (schema V14). No detector: the boxes and landmarks in the catalog are
|
||
/// taken as read, scaled back from the long edge they were normalised to, and
|
||
/// each face is warped out of the native frame and embedded exactly as
|
||
/// [`index_native`] would have done on the day. What comes back is the raw
|
||
/// vector and its length, to be written over the old unit one.
|
||
///
|
||
/// The size and sharpness gates are deliberately not re-applied. They decide
|
||
/// whether a face is worth *storing*, and these are stored; what is being
|
||
/// established now is how much the model can make of each, which is the
|
||
/// quality itself, and a face that would have failed a gate is precisely one
|
||
/// that should come out short and stop vouching for anyone.
|
||
///
|
||
/// The eyes are read on the same pass where the device has the models, for
|
||
/// the faces that have no reading yet (schema V16): the pixels are in hand,
|
||
/// and the eye is a window on the same landmarks. A face that already has
|
||
/// one keeps it.
|
||
pub fn measure_native(
|
||
embedder: &mut Embedder,
|
||
mut eye_models: Option<&mut EyeModels>,
|
||
rgba: &[u8],
|
||
width: usize,
|
||
height: usize,
|
||
faces: &[faces::Face],
|
||
) -> Result<Measured, dr_face::FaceError> {
|
||
let mut out = Measured::default();
|
||
if !index_native_shape_ok(rgba, width, height) {
|
||
return Ok(out);
|
||
}
|
||
let long_edge = width.max(height) as f32;
|
||
let native = dr_face::Pixels::Rgba8(rgba);
|
||
|
||
for f in faces {
|
||
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
|
||
for (o, &(x, y)) in landmarks.iter_mut().zip(f.landmarks.iter()) {
|
||
*o = (x * long_edge, y * long_edge);
|
||
}
|
||
let Some(aligned) = dr_face::warp_pixels(native, width, height, &landmarks) else {
|
||
log::debug!("face {:?} has degenerate landmarks, dropped", f.id);
|
||
out.dropped.push(f.id);
|
||
continue;
|
||
};
|
||
let embedded = embedder.embed(&aligned)?;
|
||
let (eyes, landmarks_dense) = if f.eyes.is_some() {
|
||
(None, Vec::new())
|
||
} else {
|
||
let bbox = (
|
||
f.x * long_edge,
|
||
f.y * long_edge,
|
||
(f.x + f.w) * long_edge,
|
||
(f.y + f.h) * long_edge,
|
||
);
|
||
read_eyes(
|
||
eye_models.as_deref_mut(),
|
||
native,
|
||
width,
|
||
height,
|
||
bbox,
|
||
&landmarks,
|
||
long_edge,
|
||
)
|
||
};
|
||
out.measured.push(faces::FaceUpdate {
|
||
embedding: Some((embedded.to_f16_bytes(), embedded.quality)),
|
||
eyes: eyes.map(|e| (e, landmarks_dense)),
|
||
..faces::FaceUpdate::for_face(f.id)
|
||
});
|
||
}
|
||
Ok(out)
|
||
}
|
||
|
||
/// Whether a buffer is the native frame it claims to be.
|
||
///
|
||
/// Checked before anything expensive, and before the detector above all: a
|
||
/// mismatched buffer would otherwise be read past its end by the reduction.
|
||
fn index_native_shape_ok(rgba: &[u8], width: usize, height: usize) -> bool {
|
||
width > 0 && height > 0 && rgba.len() == width * height * 4
|
||
}
|
||
|
||
/// Landmarks from the detector's reduction into native coordinates.
|
||
fn scale_landmarks(lm: &[(f32, f32); 5], sx: f32, sy: f32) -> [(f32, f32); 5] {
|
||
let mut out = [(0.0_f32, 0.0_f32); 5];
|
||
for (o, &(x, y)) in out.iter_mut().zip(lm.iter()) {
|
||
*o = (x * sx, y * sy);
|
||
}
|
||
out
|
||
}
|
||
|
||
/// Box-filter a native RGBA render down to [`DETECT_EDGE`], as packed `f32` RGB.
|
||
///
|
||
/// Averaging rather than sampling. The detector's job is to find small faces,
|
||
/// and a point-sampled reduction is exactly the operation that removes them:
|
||
/// at a 4× decimation fifteen of every sixteen pixels are discarded, and a
|
||
/// 40px face survives or not depending on where it happens to sit relative to
|
||
/// the sample grid.
|
||
fn reduce_for_detection(rgba: &[u8], width: usize, height: usize) -> (usize, usize, Vec<f32>) {
|
||
reduce_to(rgba, width, height, DETECT_EDGE)
|
||
}
|
||
|
||
/// [`reduce_for_detection`], to a stated long edge.
|
||
///
|
||
/// Split out so the averaging can be tested at a size a test can reason about
|
||
/// by hand, rather than by building a 1600px fixture.
|
||
fn reduce_to(rgba: &[u8], width: usize, height: usize, target: usize) -> (usize, usize, Vec<f32>) {
|
||
let long = width.max(height);
|
||
if long <= target {
|
||
// Already small enough; convert without resampling rather than round
|
||
// -tripping through a scale of 1.
|
||
let mut out = vec![0.0_f32; width * height * 3];
|
||
for (i, px) in rgba.chunks_exact(4).enumerate() {
|
||
for c in 0..3 {
|
||
out[i * 3 + c] = px[c] as f32 / 255.0;
|
||
}
|
||
}
|
||
return (width, height, out);
|
||
}
|
||
|
||
let scale = target as f32 / long as f32;
|
||
let (dw, dh) = (
|
||
((width as f32 * scale).round() as usize).max(1),
|
||
((height as f32 * scale).round() as usize).max(1),
|
||
);
|
||
let mut out = vec![0.0_f32; dw * dh * 3];
|
||
for oy in 0..dh {
|
||
// Source span of this destination row, as a half-open range, so
|
||
// adjacent rows tile the source exactly and no row is counted twice.
|
||
let y0 = oy * height / dh;
|
||
let y1 = (((oy + 1) * height) / dh).max(y0 + 1).min(height);
|
||
for ox in 0..dw {
|
||
let x0 = ox * width / dw;
|
||
let x1 = (((ox + 1) * width) / dw).max(x0 + 1).min(width);
|
||
|
||
let mut acc = [0.0_f32; 3];
|
||
let mut n = 0.0_f32;
|
||
for y in y0..y1 {
|
||
for x in x0..x1 {
|
||
let i = (y * width + x) * 4;
|
||
for (c, a) in acc.iter_mut().enumerate() {
|
||
*a += rgba[i + c] as f32;
|
||
}
|
||
n += 1.0;
|
||
}
|
||
}
|
||
let o = (oy * dw + ox) * 3;
|
||
for c in 0..3 {
|
||
out[o + c] = acc[c] / n / 255.0;
|
||
}
|
||
}
|
||
}
|
||
(dw, dh, out)
|
||
}
|
||
|
||
/// The stored face thumbnail, cut from the native buffer.
|
||
///
|
||
/// Same framing as [`cut_crop`] and deliberately a separate function rather
|
||
/// than a generalisation of it: this one takes a box already in native
|
||
/// coordinates, and blurring that distinction is how a crop ends up sampled
|
||
/// from the wrong scale.
|
||
pub(crate) fn cut_crop_native(
|
||
px: dr_face::Pixels<'_>,
|
||
width: usize,
|
||
height: usize,
|
||
bbox: (f32, f32, f32, f32),
|
||
) -> Option<Vec<u8>> {
|
||
if width == 0 || height == 0 {
|
||
return None;
|
||
}
|
||
let (bx, by, bw, bh) = bbox;
|
||
let cx = bx + bw * 0.5;
|
||
let cy = by + bh * 0.5;
|
||
let half = bw.max(bh) * 0.5 * (1.0 + CROP_MARGIN);
|
||
if !(half.is_finite() && half > 0.5 && cx.is_finite() && cy.is_finite()) {
|
||
return None;
|
||
}
|
||
|
||
let edge = STORED_CROP_EDGE;
|
||
let mut out = vec![0u8; (edge * edge * 4) as usize];
|
||
let step = (half * 2.0) / edge as f32;
|
||
for oy in 0..edge {
|
||
let sy = cy - half + (oy as f32 + 0.5) * step;
|
||
for ox in 0..edge {
|
||
let sx = cx - half + (ox as f32 + 0.5) * step;
|
||
let o = ((oy * edge + ox) * 4) as usize;
|
||
for c in 0..3 {
|
||
let v = px.channel(width, height, sx as isize, sy as isize, c);
|
||
out[o + c] = (v.clamp(0.0, 1.0) * 255.0).round() as u8;
|
||
}
|
||
out[o + 3] = 255;
|
||
}
|
||
}
|
||
|
||
match dr_thumbs::encode_rgba(edge, edge, &out) {
|
||
Ok(bytes) => Some(bytes),
|
||
Err(e) => {
|
||
log::debug!("encoding a face crop: {e}");
|
||
None
|
||
}
|
||
}
|
||
}
|
||
|
||
fn normalise_landmarks(lm: &[(f32, f32); 5], long_edge: f32) -> [(f32, f32); 5] {
|
||
let mut out = [(0.0_f32, 0.0_f32); 5];
|
||
for (o, &(x, y)) in out.iter_mut().zip(lm.iter()) {
|
||
*o = (x / long_edge, y / long_edge);
|
||
}
|
||
out
|
||
}
|
||
|
||
/// Index every image whose proxy is **already on this disk**, in the background.
|
||
///
|
||
/// Not the whole-library pass — that is `repairs::spawn`, which
|
||
/// fetches what it has not got. This one never touches the network, which makes
|
||
/// it the right shape for a tool run against a local store (see
|
||
/// `examples/face_index.rs`) and the wrong shape for a user pressing "index my
|
||
/// library", because for an unbrowsed library the work list is nearly empty.
|
||
///
|
||
/// Strictly background work: it competes with thumbnailing, not with rendering
|
||
/// (NFR-ARCH-2), and it is interruptible simply by dropping the receiver — the
|
||
/// next run resumes from what is already in the catalog, because
|
||
/// [`faces_outstanding`] asks the catalog what is missing rather than keeping a
|
||
/// cursor. That is what makes it survive process death (FR-PLAT-AND-3) with no
|
||
/// repeated work beyond the in-flight image.
|
||
#[allow(clippy::too_many_arguments)]
|
||
pub fn spawn_store_face_sweep(
|
||
catalog_path: PathBuf,
|
||
store_dir: PathBuf,
|
||
models: crate::FaceModelPaths,
|
||
model_id: String,
|
||
options: DetectOptions,
|
||
) -> Receiver<FaceSweepMessage> {
|
||
let (tx, rx) = std::sync::mpsc::channel();
|
||
|
||
std::thread::spawn(move || {
|
||
let finish_empty = |tx: &Sender<FaceSweepMessage>| {
|
||
let _ = tx.send(FaceSweepMessage::Finished {
|
||
images: 0,
|
||
faces: 0,
|
||
failed: 0,
|
||
});
|
||
};
|
||
|
||
let catalog = match Catalog::open(&catalog_path) {
|
||
Ok(c) => c,
|
||
Err(e) => {
|
||
log::warn!("face sweep: cannot open catalog: {e}");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
};
|
||
let store = match ThumbStore::open(&store_dir) {
|
||
Ok(s) => s,
|
||
Err(e) => {
|
||
log::warn!("face sweep: cannot open the thumbnail store: {e}");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
};
|
||
|
||
// Models first: they are the expensive failure, and there is no point
|
||
// listing ten thousand images before discovering the weights are
|
||
// missing. This is also the path a library with face indexing enabled
|
||
// but no model downloaded takes (docs/dev/faces.md §2.2), so it must be a
|
||
// quiet return rather than an error.
|
||
let mut detector = match Detector::from_path(&models.detector) {
|
||
Ok(d) => d,
|
||
Err(e) => {
|
||
log::warn!("face sweep: cannot load the detector: {e}");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
};
|
||
let mut embedder =
|
||
match Embedder::from_path(&models.embedder, ModelId::new(model_id.clone())) {
|
||
Ok(e) => e,
|
||
Err(e) => {
|
||
log::warn!("face sweep: cannot load the embedder: {e}");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
};
|
||
let mut eye_models = models.load_eyes();
|
||
|
||
// Nothing below this point can succeed, so do not pretend to try.
|
||
//
|
||
// This pass detects on what the store holds, and the store's largest
|
||
// tier is `FACE_TIER` — 1024, which is below the floor. Left to run it
|
||
// would fetch nothing, decode every proxy it has, and report every
|
||
// single image as failed: twenty thousand refusals that all say the
|
||
// same thing. The pass is not repairable here either, because there is
|
||
// no larger tier for it to read; the pixels it needs have to come off
|
||
// the server, which is `repairs::spawn`'s job.
|
||
if FACE_TIER.edge() < dr_face::MIN_CROP_EDGE {
|
||
log::warn!(
|
||
"face sweep: the local store's largest tier is {}px, below the {}px \
|
||
crop floor — use the library pass, which renders at native resolution",
|
||
FACE_TIER.edge(),
|
||
dr_face::MIN_CROP_EDGE,
|
||
);
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
|
||
let wanted = match faces_outstanding(&catalog, &store, &model_id) {
|
||
Ok(w) => w,
|
||
Err(e) => {
|
||
log::warn!("face sweep: {e}");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
};
|
||
|
||
let total = wanted.len();
|
||
if total == 0 {
|
||
log::info!("face sweep: every image with a proxy is already indexed");
|
||
finish_empty(&tx);
|
||
return;
|
||
}
|
||
log::info!("face sweep: {total} image(s) to index");
|
||
if tx.send(FaceSweepMessage::Total(total)).is_err() {
|
||
return;
|
||
}
|
||
|
||
let (mut images, mut found, mut failed) = (0usize, 0usize, 0usize);
|
||
|
||
for req in wanted {
|
||
let thumb = match store.get(req.file_id, FACE_TIER) {
|
||
Ok(Some(t)) => t,
|
||
// Evicted between the listing and now. Not a failure: the next
|
||
// pass will find it, or the thumbnail sweep will rebuild it.
|
||
Ok(None) => continue,
|
||
Err(e) => {
|
||
log::debug!("face sweep: reading proxy for {:?}: {e}", req.image_id);
|
||
failed += 1;
|
||
continue;
|
||
}
|
||
};
|
||
|
||
let (w, h, rgba) = match dr_thumbs::codec::decode_rgba(&thumb.bytes) {
|
||
Ok(v) => v,
|
||
Err(e) => {
|
||
log::debug!("face sweep: decoding proxy for {:?}: {e}", req.image_id);
|
||
failed += 1;
|
||
continue;
|
||
}
|
||
};
|
||
let rgb = rgba_to_rgb_f32(&rgba);
|
||
|
||
let faces = match index_proxy(
|
||
&mut detector,
|
||
&mut embedder,
|
||
eye_models.as_mut(),
|
||
&rgb,
|
||
w as usize,
|
||
h as usize,
|
||
&options,
|
||
) {
|
||
Ok(f) => f,
|
||
Err(e) => {
|
||
log::debug!("face sweep: indexing {:?}: {e}", req.image_id);
|
||
failed += 1;
|
||
continue;
|
||
}
|
||
};
|
||
|
||
if let Err(e) = faces::record_detections(
|
||
catalog.connection(),
|
||
req.image_id,
|
||
&model_id,
|
||
w.max(h),
|
||
&faces,
|
||
) {
|
||
log::warn!("face sweep: storing faces for {:?}: {e}", req.image_id);
|
||
failed += 1;
|
||
continue;
|
||
}
|
||
|
||
images += 1;
|
||
found += faces.len();
|
||
if tx
|
||
.send(FaceSweepMessage::Indexed {
|
||
image: req.image_id,
|
||
faces: faces.len(),
|
||
})
|
||
.is_err()
|
||
{
|
||
// Receiver dropped: the window closed, or the user turned face
|
||
// indexing off. Stop, leaving everything written so far.
|
||
log::info!("face sweep: cancelled after {images} image(s)");
|
||
return;
|
||
}
|
||
}
|
||
|
||
log::info!("face sweep: {found} face(s) across {images} image(s), {failed} failed");
|
||
let _ = tx.send(FaceSweepMessage::Finished {
|
||
images,
|
||
faces: found,
|
||
failed,
|
||
});
|
||
});
|
||
|
||
rx
|
||
}
|
||
|
||
/// Everything a grouping pass reads before it decides anything.
|
||
///
|
||
/// Split out because two callers need exactly this and must agree on it: the
|
||
/// pass that writes, and the preview that reports what the pass *would* do. A
|
||
/// preview built from a second, subtly different reading — anchors omitted,
|
||
/// say — would answer a question about a library nobody has.
|
||
struct Population {
|
||
cal: Calibration,
|
||
candidates: Vec<dr_face::Candidate>,
|
||
/// Catalog ids, parallel to `candidates`.
|
||
ids: Vec<faces::FaceId>,
|
||
/// Faces the user has ruled on, and who they are. See [`Population::read`].
|
||
anchors: std::collections::HashMap<faces::FaceId, faces::PersonId>,
|
||
}
|
||
|
||
impl Population {
|
||
/// Takes the catalog and not its connection: `rusqlite` is `dr-catalog`'s
|
||
/// dependency and not this crate's, and reaching for the connection type by
|
||
/// name here would drag it across a layer that has kept clear of it.
|
||
fn read(catalog: &Catalog, model_id: &str) -> Result<Self, dr_catalog::CatalogError> {
|
||
let conn = catalog.connection();
|
||
// No valid calibration is not a reason to refuse to cluster — it is a
|
||
// reason not to *display* a confidence (FR-CULL-9). `Calibration::
|
||
// default` is the reference implementation's fitted curve with `valid`
|
||
// false, which is a documented operating point rather than an invented
|
||
// one.
|
||
let cal = faces::calibration(conn, model_id)?
|
||
.map(|(c, _)| c)
|
||
.unwrap_or_else(Calibration::default);
|
||
|
||
// Which faces the user has already ruled on, so they enter as anchors.
|
||
//
|
||
// Anything the user has ruled on anchors, and there are three ways of
|
||
// ruling — only the first of which is obvious.
|
||
//
|
||
// A **confirmation** is the plain case. **Setting a group aside** is one
|
||
// too, and the faces it covers are only ever suggestions, so anchoring
|
||
// confirmations alone let every ignored group scatter into fresh unnamed
|
||
// groups that were not ignored, and the strangers came straight back.
|
||
//
|
||
// And so is **giving a group a name**. That was the omission that did
|
||
// the most damage, because it is silent. Naming a cluster does not
|
||
// confirm its faces — they stay suggestions — so the next Regroup cut
|
||
// them loose, regrouped them into a brand new person, and left the named
|
||
// one holding nothing. `prune_empty_unnamed` will not remove it, because
|
||
// it has a name. Name the new group the same thing and it happens again.
|
||
// That is how one library came to hold sixteen people called Catherine,
|
||
// fourteen of them empty, with her faces split across the two that were
|
||
// not.
|
||
//
|
||
// A name is a judgement about *this group* (FR-CULL-12), exactly as an
|
||
// ignore is. Anchoring them all also does one better: a newly indexed
|
||
// face that matches a named person now merges *into* them rather than
|
||
// arriving as a stranger.
|
||
let mut anchors = std::collections::HashMap::new();
|
||
for p in faces::people(conn)? {
|
||
let ruled_on = p.ignored || !p.name.trim().is_empty();
|
||
for f in faces::for_person(conn, p.id, ruled_on)? {
|
||
anchors.insert(f.id, p.id);
|
||
}
|
||
}
|
||
|
||
let stored = faces::embeddings(conn, model_id)?;
|
||
let model = ModelId::new(model_id.to_string());
|
||
let mut candidates = Vec::with_capacity(stored.len());
|
||
let mut ids = Vec::with_capacity(stored.len());
|
||
for f in stored {
|
||
let Some(emb) = dr_face::Embedding::from_f16_bytes(model.clone(), &f.embedding) else {
|
||
log::warn!("face {:?} has a malformed embedding, skipped", f.face);
|
||
continue;
|
||
};
|
||
candidates.push(dr_face::Candidate {
|
||
face: f.face.0,
|
||
image: f.image.0,
|
||
embedding: emb.v.to_vec(),
|
||
crop_px: f.crop_px,
|
||
quality: f.quality,
|
||
confirmed_person: anchors.get(&f.face).map(|p| p.0),
|
||
});
|
||
ids.push(f.face);
|
||
}
|
||
|
||
// Not every anchored face enters the scan: each person is stood for
|
||
// by their references (`dr_face::references`), and the rest of their
|
||
// faces stay out of the comparison. They keep their place -- they are
|
||
// in `anchors`, so the pass never releases them -- and they are the
|
||
// cost that would otherwise grow with every name the user gives.
|
||
let mut by_person: std::collections::BTreeMap<u64, Vec<usize>> =
|
||
std::collections::BTreeMap::new();
|
||
for (i, c) in candidates.iter().enumerate() {
|
||
if let Some(p) = c.confirmed_person {
|
||
by_person.entry(p).or_default().push(i);
|
||
}
|
||
}
|
||
let mut keep = vec![true; candidates.len()];
|
||
let mut anchored = 0usize;
|
||
for members in by_person.values() {
|
||
anchored += members.len();
|
||
// Only a person over the cap loses anyone: under it, `select`
|
||
// returns every eligible face, and the ineligible are kept too,
|
||
// since a short vector was a probe before and still is.
|
||
if members.len() <= dr_face::references::MAX_REFERENCES {
|
||
continue;
|
||
}
|
||
let embeddings: Vec<&[f32]> = members
|
||
.iter()
|
||
.map(|&i| candidates[i].embedding.as_slice())
|
||
.collect();
|
||
let quality: Vec<Option<f32>> =
|
||
members.iter().map(|&i| candidates[i].quality).collect();
|
||
let chosen = dr_face::references::select(
|
||
&embeddings,
|
||
&quality,
|
||
dr_face::references::MAX_REFERENCES,
|
||
);
|
||
for &i in members {
|
||
keep[i] = false;
|
||
}
|
||
for &k in &chosen {
|
||
keep[members[k]] = true;
|
||
}
|
||
// A person none of whose faces is long enough to vouch is still
|
||
// a person, and a pass they had no anchor in would file their
|
||
// next face as a stranger. The longest stand in.
|
||
if chosen.is_empty() {
|
||
let mut by_quality = members.clone();
|
||
by_quality.sort_by(|&a, &b| {
|
||
let qa = candidates[a].quality.unwrap_or(0.0);
|
||
let qb = candidates[b].quality.unwrap_or(0.0);
|
||
qb.total_cmp(&qa).then(a.cmp(&b))
|
||
});
|
||
for &i in by_quality.iter().take(dr_face::references::MAX_REFERENCES) {
|
||
keep[i] = true;
|
||
}
|
||
}
|
||
}
|
||
let left_out = keep.iter().filter(|k| !**k).count();
|
||
if left_out > 0 {
|
||
let mut i = 0;
|
||
candidates.retain(|_| {
|
||
i += 1;
|
||
keep[i - 1]
|
||
});
|
||
let mut i = 0;
|
||
ids.retain(|_| {
|
||
i += 1;
|
||
keep[i - 1]
|
||
});
|
||
log::info!(
|
||
"references: {} of {anchored} anchored face(s) stand for {} people; {left_out} left out of the scan",
|
||
anchored - left_out,
|
||
by_person.len(),
|
||
);
|
||
}
|
||
|
||
Ok(Self {
|
||
cal,
|
||
candidates,
|
||
ids,
|
||
anchors,
|
||
})
|
||
}
|
||
}
|
||
|
||
/// What a grouping pass would produce, without producing it.
|
||
///
|
||
/// The numbers `dr_face::cluster`'s own tuning table is built from, for one
|
||
/// setting rather than ten — because the question a photographer is actually
|
||
/// asking of the dials is "what does *my* library look like at this value", and
|
||
/// the doc-comment table answers it for a library that is not theirs.
|
||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||
pub struct GroupingPreview {
|
||
/// Faces that went in.
|
||
pub faces: usize,
|
||
/// Groups that would survive the smallest-group rule.
|
||
pub groups: usize,
|
||
/// Faces those groups would hold.
|
||
pub grouped: usize,
|
||
/// The biggest group. The tell for over-merging: it is the number that runs
|
||
/// away when the confidence is set too low.
|
||
pub largest: usize,
|
||
}
|
||
|
||
/// Report what a grouping pass would do, writing nothing.
|
||
///
|
||
/// Read-only by construction — it never opens a transaction — which is what
|
||
/// makes it safe to run repeatedly while the user moves a slider. Comparing two
|
||
/// settings by *applying* both would leave the second one's answer polluted by
|
||
/// the first one's suggestions.
|
||
pub fn preview_grouping(
|
||
catalog: &Catalog,
|
||
model_id: &str,
|
||
grouping: &FaceSettings,
|
||
) -> Result<GroupingPreview, dr_catalog::CatalogError> {
|
||
let min_group = grouping.min_group_size.max(1) as usize;
|
||
let pop = Population::read(catalog, model_id)?;
|
||
if pop.candidates.is_empty() {
|
||
return Ok(GroupingPreview::default());
|
||
}
|
||
|
||
let clusters = dr_face::cluster(&pop.candidates, &pop.cal, grouping.merge_probability);
|
||
// The same rule the writing pass applies, so the preview and the result
|
||
// cannot disagree about what counts as a person.
|
||
let kept: Vec<_> = clusters
|
||
.iter()
|
||
.filter(|c| c.members.len() >= min_group || c.person.is_some())
|
||
.collect();
|
||
|
||
Ok(GroupingPreview {
|
||
faces: pop.candidates.len(),
|
||
groups: kept.len(),
|
||
grouped: kept.iter().map(|c| c.members.len()).sum(),
|
||
largest: kept.iter().map(|c| c.members.len()).max().unwrap_or(0),
|
||
})
|
||
}
|
||
|
||
/// Group the library's faces into people, writing suggestions.
|
||
///
|
||
/// Confirmations are never touched: they enter the clusterer as anchors and
|
||
/// come back out unchanged, which is the invariant FR-CULL-10 turns on. What
|
||
/// this writes is the *suggested* half, and it may be re-run at any time.
|
||
///
|
||
/// Returns how many suggestions were written and how many new unnamed people
|
||
/// were created.
|
||
pub fn recluster(
|
||
catalog: &Catalog,
|
||
model_id: &str,
|
||
grouping: &FaceSettings,
|
||
) -> Result<(usize, usize), dr_catalog::CatalogError> {
|
||
let min_probability = grouping.merge_probability;
|
||
let min_group = grouping.min_group_size.max(1) as usize;
|
||
let conn = catalog.connection();
|
||
|
||
let Population {
|
||
cal,
|
||
candidates,
|
||
ids,
|
||
anchors: confirmed,
|
||
} = Population::read(catalog, model_id)?;
|
||
if candidates.is_empty() {
|
||
return Ok((0, 0));
|
||
}
|
||
|
||
let dr_face::Grouping {
|
||
clusters,
|
||
confidence,
|
||
} = dr_face::cluster_scored(&candidates, &cal, min_probability);
|
||
|
||
let mut suggested = 0usize;
|
||
let mut created = 0usize;
|
||
// Every face this pass actually placed. What is *not* in here at the end is
|
||
// a face the previous pass had an opinion about and this one does not, and
|
||
// it has to be let go — see below.
|
||
let mut placed = std::collections::HashSet::with_capacity(ids.len());
|
||
for c in &clusters {
|
||
// A group of one is not a person, and the user says how much bigger
|
||
// than one it has to be (`FaceSettings::min_group_size`). Naming every
|
||
// stray face would fill the People view with noise they then have to
|
||
// dismiss one entry at a time.
|
||
//
|
||
// Only ever applied to a group the system invented. A group with a
|
||
// `person` is one the user has already confirmed, named or set aside,
|
||
// and a display preference does not overrule a judgement (FR-CULL-12).
|
||
if c.members.len() < min_group && c.person.is_none() {
|
||
continue;
|
||
}
|
||
|
||
let person = match c.person {
|
||
Some(p) => faces::PersonId(p),
|
||
None => {
|
||
created += 1;
|
||
// Unnamed: FR-CULL-10 has the user name a group, and a group
|
||
// the system named would be a guess wearing a fact's clothes.
|
||
faces::create_person(conn, "")?
|
||
}
|
||
};
|
||
|
||
for &m in &c.members {
|
||
let face = ids[m];
|
||
placed.insert(face);
|
||
if confirmed.contains_key(&face) {
|
||
continue;
|
||
}
|
||
// The probability the user is shown is this identity's share of
|
||
// the evidence for the face, against every other identity that
|
||
// could plausibly claim it (dr_face::assign) — not the single best
|
||
// edge, which cannot tell a sole match from a coin toss between
|
||
// two siblings.
|
||
let p = confidence[m];
|
||
if faces::suggest(conn, face, person, p)? {
|
||
suggested += 1;
|
||
}
|
||
}
|
||
}
|
||
|
||
// Faces the previous pass placed and this one did not.
|
||
//
|
||
// Without this the parameters above are only half connected to the screen.
|
||
// Raise the smallest group to three and the pass stops *creating* groups of
|
||
// two — but last pass's group of two still holds its two suggestions, so it
|
||
// is not empty, so the prune below leaves it, and the rail does not change.
|
||
// The setting would appear to do nothing until the library was reindexed.
|
||
//
|
||
// Suggestions are this pass's own output (FR-CULL-10), so withdrawing one it
|
||
// no longer stands behind is exactly what it is entitled to do. Anchors are
|
||
// skipped by construction: a confirmed, named or ignored face is in
|
||
// `confirmed`, enters as an anchor, and comes back out inside a group that
|
||
// is never dropped.
|
||
let mut released = 0usize;
|
||
for face in &ids {
|
||
if placed.contains(face) || confirmed.contains_key(face) {
|
||
continue;
|
||
}
|
||
faces::unassign(conn, *face)?;
|
||
released += 1;
|
||
}
|
||
if released > 0 {
|
||
log::info!("reclustering released {released} face(s) it no longer groups");
|
||
}
|
||
|
||
// Groups the *previous* pass created that this one left empty. Without
|
||
// this, every press of Regroup adds a rail entry per group it no longer
|
||
// believes in, and the screen fills with "Unnamed (0 faces)" — which is
|
||
// what made pressing the button twice look like it had broken something.
|
||
match faces::prune_empty_unnamed(conn) {
|
||
Ok(0) => {}
|
||
Ok(n) => log::info!("reclustering removed {n} empty group(s) from the previous pass"),
|
||
Err(e) => log::warn!("pruning empty groups: {e}"),
|
||
}
|
||
|
||
log::info!(
|
||
"reclustered {} face(s) into {} group(s): {suggested} suggestion(s), {created} new",
|
||
candidates.len(),
|
||
clusters.len()
|
||
);
|
||
Ok((suggested, created))
|
||
}
|
||
|
||
/// The answer from a preview pass.
|
||
#[derive(Debug, Clone, PartialEq)]
|
||
pub enum PreviewMessage {
|
||
Ready(GroupingPreview),
|
||
Failed(String),
|
||
}
|
||
|
||
/// Report what a grouping pass would do, **on a worker thread**.
|
||
///
|
||
/// Same reasoning as [`spawn_recluster`], and the same cost: a preview is a
|
||
/// clustering pass that throws its answer away, so it is exactly as unbounded
|
||
/// as the pass it is previewing and exactly as unwelcome on the UI thread.
|
||
///
|
||
/// Cancellation is dropping the receiver — so moving the slider again while one
|
||
/// is in flight abandons it, which is the behaviour a dial with a preview
|
||
/// button needs.
|
||
pub fn spawn_grouping_preview(
|
||
catalog_path: PathBuf,
|
||
model_id: String,
|
||
grouping: FaceSettings,
|
||
) -> Receiver<PreviewMessage> {
|
||
let (tx, rx) = std::sync::mpsc::channel();
|
||
|
||
std::thread::spawn(move || {
|
||
let msg = match Catalog::open(&catalog_path) {
|
||
Ok(catalog) => match preview_grouping(&catalog, &model_id, &grouping) {
|
||
Ok(p) => PreviewMessage::Ready(p),
|
||
Err(e) => PreviewMessage::Failed(e.to_string()),
|
||
},
|
||
Err(e) => PreviewMessage::Failed(format!("cannot open catalog: {e}")),
|
||
};
|
||
let _ = tx.send(msg);
|
||
});
|
||
|
||
rx
|
||
}
|
||
|
||
/// Progress from a regrouping pass.
|
||
#[derive(Debug, Clone, PartialEq)]
|
||
pub enum ReclusterMessage {
|
||
/// How many faces went in. Sent once, before the arithmetic starts.
|
||
Started { faces: usize },
|
||
/// It finished.
|
||
Finished { suggested: usize, created: usize },
|
||
/// It did not.
|
||
Failed(String),
|
||
}
|
||
|
||
/// Group the library's faces into people, **on a worker thread**.
|
||
///
|
||
/// The reason this exists rather than callers just invoking [`recluster`]: it
|
||
/// used to run inside the Slint callback, on the UI thread, and clustering a
|
||
/// real library is not something a callback can do. The window froze for as
|
||
/// long as it took, with no progress, no cancel and no repaint — the button
|
||
/// looked broken because from the outside it was indistinguishable from broken.
|
||
///
|
||
/// It is much faster now (see [`dr_face::cluster`]), but *fast* is not the same
|
||
/// as *bounded*: the work grows with the library and the one thing that must
|
||
/// not grow with the library is how long the window stops answering. So it runs
|
||
/// where every other long pass in this module runs.
|
||
///
|
||
/// Cancellation is dropping the receiver, exactly as with the indexing sweep.
|
||
/// Nothing is left half-written: [`recluster`] does its work in the catalog's
|
||
/// own transactions, and a pass abandoned partway simply leaves the previous
|
||
/// grouping in place to be redone.
|
||
pub fn spawn_recluster(
|
||
catalog_path: PathBuf,
|
||
model_id: String,
|
||
grouping: FaceSettings,
|
||
) -> Receiver<ReclusterMessage> {
|
||
let (tx, rx) = std::sync::mpsc::channel();
|
||
|
||
std::thread::spawn(move || {
|
||
let catalog = match Catalog::open(&catalog_path) {
|
||
Ok(c) => c,
|
||
Err(e) => {
|
||
let _ = tx.send(ReclusterMessage::Failed(format!(
|
||
"cannot open catalog: {e}"
|
||
)));
|
||
return;
|
||
}
|
||
};
|
||
|
||
// Announced before the work so the screen can say what it is chewing
|
||
// on. Cheap: it is a count, not the embeddings themselves.
|
||
let count = faces::embeddings(catalog.connection(), &model_id)
|
||
.map(|e| e.len())
|
||
.unwrap_or(0);
|
||
if tx.send(ReclusterMessage::Started { faces: count }).is_err() {
|
||
return;
|
||
}
|
||
|
||
let msg = match recluster(&catalog, &model_id, &grouping) {
|
||
Ok((suggested, created)) => ReclusterMessage::Finished { suggested, created },
|
||
Err(e) => ReclusterMessage::Failed(e.to_string()),
|
||
};
|
||
let _ = tx.send(msg);
|
||
});
|
||
|
||
rx
|
||
}
|
||
|
||
/// A face cut out of its photograph, ready to draw.
|
||
#[derive(Debug, Clone, PartialEq)]
|
||
pub struct FaceCrop {
|
||
pub width: u32,
|
||
pub height: u32,
|
||
/// Tightly packed RGBA.
|
||
pub rgba: Vec<u8>,
|
||
}
|
||
|
||
/// How much of the surrounding frame a face crop keeps, per side.
|
||
///
|
||
/// A face cut exactly to its detection box reads as a mugshot: no hair, no
|
||
/// chin, no context, and a row of them is genuinely hard to tell apart — which
|
||
/// matters, because telling them apart is the entire task the People screen
|
||
/// asks of the user. A third on each side gives back the head.
|
||
const CROP_MARGIN: f32 = 0.35;
|
||
|
||
/// Cut one face out of its proxy.
|
||
///
|
||
/// The box is normalised to the long edge (the catalog's convention), so this
|
||
/// works whatever size the proxy happens to be now — the property that made
|
||
/// normalising worth the trouble. The thumbnail cache is entitled to evict a
|
||
/// proxy and regenerate it at another resolution, and a face stored in pixels
|
||
/// would then point at the wrong part of the picture.
|
||
pub fn crop_face(
|
||
rgba: &[u8],
|
||
width: u32,
|
||
height: u32,
|
||
face: &faces::Face,
|
||
out_edge: u32,
|
||
) -> Option<FaceCrop> {
|
||
if width == 0 || height == 0 || out_edge == 0 {
|
||
return None;
|
||
}
|
||
let long_edge = width.max(height) as f32;
|
||
|
||
// Square, centred on the face: the grid draws square cells, and cropping to
|
||
// a square here rather than letterboxing there means the face fills the
|
||
// cell instead of floating in it.
|
||
let cx = (face.x + face.w * 0.5) * long_edge;
|
||
let cy = (face.y + face.h * 0.5) * long_edge;
|
||
let half = (face.w.max(face.h) * long_edge * 0.5) * (1.0 + CROP_MARGIN);
|
||
if !(half.is_finite() && half > 0.5) {
|
||
return None;
|
||
}
|
||
|
||
let mut out = vec![0u8; (out_edge * out_edge * 4) as usize];
|
||
let step = (half * 2.0) / out_edge as f32;
|
||
|
||
for oy in 0..out_edge {
|
||
let sy = cy - half + (oy as f32 + 0.5) * step;
|
||
for ox in 0..out_edge {
|
||
let sx = cx - half + (ox as f32 + 0.5) * step;
|
||
let o = ((oy * out_edge + ox) * 4) as usize;
|
||
// Nearest neighbour: this is a downscale of an already-small proxy
|
||
// shown at ~96 px, and a bilinear tap would cost four reads per
|
||
// pixel for a difference nobody can see at that size. Outside the
|
||
// frame stays transparent, so a face at the very edge of the
|
||
// picture is drawn short rather than smeared.
|
||
if sx < 0.0 || sy < 0.0 || sx >= width as f32 || sy >= height as f32 {
|
||
continue;
|
||
}
|
||
let i = ((sy as u32 * width + sx as u32) * 4) as usize;
|
||
if i + 4 <= rgba.len() {
|
||
out[o..o + 4].copy_from_slice(&rgba[i..i + 4]);
|
||
}
|
||
}
|
||
}
|
||
|
||
Some(FaceCrop {
|
||
width: out_edge,
|
||
height: out_edge,
|
||
rgba: out,
|
||
})
|
||
}
|
||
|
||
/// Edge of the crop stored with a face.
|
||
///
|
||
/// Above both [`crate::identity::FACE_CROP_EDGE`] (128) and
|
||
/// `COVER_CROP_EDGE` (80), so the stored image is downsampled to draw and never
|
||
/// upsampled — a stored crop the same size as the grid cell would go soft the
|
||
/// moment either constant grew. 160 px of JPEG is a few KB, which is nothing
|
||
/// beside the 250 KB proxy it saves decoding.
|
||
pub const STORED_CROP_EDGE: u32 = 160;
|
||
|
||
/// Cut one detected face out of the buffer it was found in, as a JPEG.
|
||
///
|
||
/// The same square [`crop_face`] would cut — centred on the box, widened by
|
||
/// [`CROP_MARGIN`] — so a face drawn from its stored crop and one drawn the old
|
||
/// way from the proxy are the same picture. Working in pixels rather than
|
||
/// normalised coordinates because at this point in the pipeline that is what
|
||
/// there is; the normalising happens afterwards.
|
||
///
|
||
/// **Out-of-frame samples clamp to the edge rather than going transparent.**
|
||
/// [`crop_face`] leaves them clear, which is right when the caller can composite
|
||
/// them; JPEG has no alpha, so the same choice here would bake a black bar into
|
||
/// every face near the edge of its photograph — and then drag that face's mean
|
||
/// luma down far enough for `load_cover` to reject it as too dark.
|
||
///
|
||
/// `None` where the geometry is degenerate, which is the caller's cue to store
|
||
/// nothing and fall back to the proxy.
|
||
fn cut_crop(rgb: &[f32], width: usize, height: usize, d: &Detection) -> Option<Vec<u8>> {
|
||
if width == 0 || height == 0 {
|
||
return None;
|
||
}
|
||
let cx = d.bbox.0 + d.width() * 0.5;
|
||
let cy = d.bbox.1 + d.height() * 0.5;
|
||
let half = d.width().max(d.height()) * 0.5 * (1.0 + CROP_MARGIN);
|
||
if !(half.is_finite() && half > 0.5 && cx.is_finite() && cy.is_finite()) {
|
||
return None;
|
||
}
|
||
|
||
let edge = STORED_CROP_EDGE;
|
||
let mut out = vec![0u8; (edge * edge * 4) as usize];
|
||
let step = (half * 2.0) / edge as f32;
|
||
|
||
for oy in 0..edge {
|
||
let sy = cy - half + (oy as f32 + 0.5) * step;
|
||
let sy = (sy.max(0.0) as usize).min(height - 1);
|
||
for ox in 0..edge {
|
||
let sx = cx - half + (ox as f32 + 0.5) * step;
|
||
let sx = (sx.max(0.0) as usize).min(width - 1);
|
||
|
||
let i = (sy * width + sx) * 3;
|
||
let o = ((oy * edge + ox) * 4) as usize;
|
||
if i + 3 > rgb.len() {
|
||
continue;
|
||
}
|
||
for c in 0..3 {
|
||
out[o + c] = (rgb[i + c].clamp(0.0, 1.0) * 255.0).round() as u8;
|
||
}
|
||
out[o + 3] = 255;
|
||
}
|
||
}
|
||
|
||
match dr_thumbs::encode_rgba(edge, edge, &out) {
|
||
Ok(bytes) => Some(bytes),
|
||
Err(e) => {
|
||
log::debug!("encoding a face crop: {e}");
|
||
None
|
||
}
|
||
}
|
||
}
|
||
|
||
/// Detect and embed every face in a decoded preview.
|
||
///
|
||
/// The counterpart to [`index_proxy`] for the fetching sweep, which holds a
|
||
/// `dr_decode::Preview` rather than a thumbnail out of the store. The preview
|
||
/// arrives **already turned the right way up** — `fetch_preview` applies the
|
||
/// orientation before returning — so the coordinates this produces are in the
|
||
/// photograph's space, which is the space the catalog stores and the develop
|
||
/// overlay draws in. Nothing here has to know about the sensor.
|
||
///
|
||
/// Returns the faces and the long edge they were normalised against, which is
|
||
/// what `record_detections` stores so a proxy regenerated at another size does
|
||
/// not move them.
|
||
pub fn index_preview(
|
||
detector: &mut Detector,
|
||
embedder: &mut Embedder,
|
||
eye_models: Option<&mut EyeModels>,
|
||
preview: &dr_decode::Preview,
|
||
options: &DetectOptions,
|
||
) -> Result<(Vec<DetectedFace>, u32), dr_face::FaceError> {
|
||
let rgb = rgba_to_rgb_f32(&preview.rgba);
|
||
let faces = index_proxy(
|
||
detector,
|
||
embedder,
|
||
eye_models,
|
||
&rgb,
|
||
preview.width as usize,
|
||
preview.height as usize,
|
||
options,
|
||
)?;
|
||
Ok((faces, preview.width.max(preview.height)))
|
||
}
|
||
|
||
/// `dr-thumbs` decodes to RGBA; `dr-face` reads packed `f32` RGB.
|
||
fn rgba_to_rgb_f32(rgba: &[u8]) -> Vec<f32> {
|
||
let mut out = Vec::with_capacity(rgba.len() / 4 * 3);
|
||
for px in rgba.chunks_exact(4) {
|
||
out.push(px[0] as f32 / 255.0);
|
||
out.push(px[1] as f32 / 255.0);
|
||
out.push(px[2] as f32 / 255.0);
|
||
}
|
||
out
|
||
}
|
||
|
||
#[cfg(test)]
|
||
mod tests {
|
||
use super::*;
|
||
|
||
#[test]
|
||
fn the_detector_reduction_averages_rather_than_samples() {
|
||
// Four source pixels per destination pixel, with one bright pixel in
|
||
// each group. A point sampler returns either 255 or 0 depending on
|
||
// which corner it lands on; the average is the same every time, and
|
||
// that stability is what stops a small face vanishing on a grid
|
||
// alignment it has no control over.
|
||
let (w, h) = (4usize, 4usize);
|
||
let mut rgba = vec![0u8; w * h * 4];
|
||
for y in 0..h {
|
||
for x in 0..w {
|
||
let i = (y * w + x) * 4;
|
||
let bright = x % 2 == 0 && y % 2 == 0;
|
||
for c in 0..3 {
|
||
rgba[i + c] = if bright { 255 } else { 0 };
|
||
}
|
||
rgba[i + 3] = 255;
|
||
}
|
||
}
|
||
// Force a 2x reduction regardless of DETECT_EDGE by reducing by hand
|
||
// through the same helper on a source larger than the target.
|
||
let (dw, dh, out) = reduce_to(&rgba, w, h, 2);
|
||
assert_eq!((dw, dh), (2, 2));
|
||
for v in out.chunks_exact(3) {
|
||
assert!(
|
||
(v[0] - 0.25).abs() < 1e-6,
|
||
"each destination pixel averages one bright of four, got {}",
|
||
v[0]
|
||
);
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn the_reduction_keeps_the_aspect_ratio_it_was_given() {
|
||
let (w, h) = (400usize, 100usize);
|
||
let rgba = vec![128u8; w * h * 4];
|
||
let (dw, dh, out) = reduce_to(&rgba, w, h, 100);
|
||
assert_eq!(dw, 100);
|
||
assert_eq!(dh, 25);
|
||
assert_eq!(out.len(), dw * dh * 3);
|
||
}
|
||
|
||
#[test]
|
||
fn landmarks_scale_back_to_the_native_frame() {
|
||
// The failure this guards is silent: landmarks left in the detector's
|
||
// coordinates put every crop near the top-left corner of the frame,
|
||
// which produces faces that look like faces of something else.
|
||
let lm = [
|
||
(10.0, 20.0),
|
||
(30.0, 40.0),
|
||
(50.0, 60.0),
|
||
(70.0, 80.0),
|
||
(90.0, 100.0),
|
||
];
|
||
let out = scale_landmarks(&lm, 4.0, 2.0);
|
||
assert_eq!(out[0], (40.0, 40.0));
|
||
assert_eq!(out[4], (360.0, 200.0));
|
||
}
|
||
|
||
/// The measuring path end to end, with the real embedder where one is on
|
||
/// this machine: landmarks come back from the long edge they were
|
||
/// normalised to, the warp is built from them, and what is stored is the
|
||
/// raw vector, whose length is the quality reported beside it.
|
||
#[test]
|
||
fn measuring_stores_the_raw_vector_at_the_quality_it_reports() {
|
||
let dir = crate::library::shared_face_models_dir();
|
||
let path = dir.join("arcface_mbf_b1.onnx");
|
||
if !path.is_file() {
|
||
eprintln!("no embedder at {}, skipping", path.display());
|
||
return;
|
||
}
|
||
let model = ModelId::new("w600k_mbf");
|
||
let mut embedder = Embedder::from_path(&path, model.clone()).expect("load embedder");
|
||
|
||
// A 600×400 frame with a gradient in it, and one face whose landmarks
|
||
// are the ArcFace template scaled up and placed in the middle -- not
|
||
// a face, but an unambiguous warp.
|
||
let (w, h) = (600usize, 400usize);
|
||
let mut rgba = vec![0u8; w * h * 4];
|
||
for y in 0..h {
|
||
for x in 0..w {
|
||
let i = (y * w + x) * 4;
|
||
rgba[i] = (x * 255 / w) as u8;
|
||
rgba[i + 1] = (y * 255 / h) as u8;
|
||
rgba[i + 2] = ((x + y) % 256) as u8;
|
||
rgba[i + 3] = 255;
|
||
}
|
||
}
|
||
let long_edge = w.max(h) as f32;
|
||
let mut landmarks = [(0.0_f32, 0.0_f32); 5];
|
||
for (o, &(tx, ty)) in landmarks.iter_mut().zip(dr_face::ARCFACE_TEMPLATE.iter()) {
|
||
// Two and a half times the template, offset into the frame, then
|
||
// normalised to the long edge as the catalog stores it.
|
||
*o = (
|
||
(tx * 2.5 + 150.0) / long_edge,
|
||
(ty * 2.5 + 60.0) / long_edge,
|
||
);
|
||
}
|
||
let stored = faces::Face {
|
||
id: faces::FaceId(7),
|
||
image_id: ImageId(1),
|
||
x: 0.25,
|
||
y: 0.15,
|
||
w: 0.5,
|
||
h: 0.7,
|
||
landmarks,
|
||
confidence: 0.9,
|
||
crop_px: 280.0,
|
||
quality: None,
|
||
eyes: None,
|
||
landmarks_dense: Vec::new(),
|
||
model_id: "w600k_mbf".into(),
|
||
person: None,
|
||
probability: 0.0,
|
||
confirmed: false,
|
||
};
|
||
|
||
let stored_copy = stored.clone();
|
||
let out = measure_native(&mut embedder, None, &rgba, w, h, &[stored]).expect("measure");
|
||
assert!(out.dropped.is_empty());
|
||
assert_eq!(out.measured.len(), 1);
|
||
let m = &out.measured[0];
|
||
assert_eq!(m.face, faces::FaceId(7));
|
||
let (embedding, quality) = m.embedding.as_ref().expect("a vector");
|
||
assert!(*quality > 0.0);
|
||
let (_, length) = dr_face::read_f16_bytes(model, embedding).expect("decode");
|
||
assert!(
|
||
(length - quality).abs() < 0.05 * quality,
|
||
"stored length {length} against reported quality {quality}"
|
||
);
|
||
|
||
// Degenerate landmarks -- five points on one spot, which no
|
||
// similarity can be fitted to -- are dropped, not embedded.
|
||
let junk = faces::Face {
|
||
id: faces::FaceId(8),
|
||
landmarks: [(0.3, 0.3); 5],
|
||
..stored_copy
|
||
};
|
||
let out = measure_native(&mut embedder, None, &rgba, w, h, &[junk]).expect("measure");
|
||
assert!(out.measured.is_empty());
|
||
assert_eq!(out.dropped, vec![faces::FaceId(8)]);
|
||
}
|
||
|
||
#[test]
|
||
fn a_buffer_that_is_not_the_stated_size_indexes_nothing() {
|
||
// No model is loaded here, so reaching the detector would panic. The
|
||
// point is that it does not: the shape check comes first.
|
||
let rgba = vec![0u8; 10];
|
||
assert!(!index_native_shape_ok(&rgba, 100, 100));
|
||
assert!(!index_native_shape_ok(&rgba, 0, 0));
|
||
assert!(index_native_shape_ok(&vec![0u8; 4 * 100 * 100], 100, 100));
|
||
}
|
||
|
||
#[test]
|
||
fn an_audit_summary_counts_the_outstanding_as_one_number() {
|
||
let a = IndexAudit {
|
||
coverage: faces::Coverage {
|
||
images: 100,
|
||
indexed: 60,
|
||
without_faces: 40,
|
||
faces: 35,
|
||
},
|
||
ready: 30,
|
||
awaiting_proxy: 10,
|
||
owed: Vec::new(),
|
||
};
|
||
let s = a.summary();
|
||
assert!(s.contains("60/100"), "{s}");
|
||
assert!(s.contains("60%"), "{s}");
|
||
assert!(!s.contains("60.0%"), "whole numbers above ten percent: {s}");
|
||
// One number, not two. The split described a pass that could index
|
||
// whatever already had a proxy; detection now refuses that proxy's
|
||
// size (§7), so both halves cost the same fetch and "30 ready, 10 to
|
||
// fetch" would imply a distinction that decides nothing.
|
||
assert!(s.contains("40 to index"), "{s}");
|
||
assert!(!s.contains("ready"), "the split should be gone: {s}");
|
||
assert!(!s.contains("to fetch"), "the split should be gone: {s}");
|
||
}
|
||
|
||
#[test]
|
||
fn a_complete_audit_mentions_neither() {
|
||
let a = IndexAudit {
|
||
coverage: faces::Coverage {
|
||
images: 10,
|
||
indexed: 10,
|
||
without_faces: 7,
|
||
faces: 4,
|
||
},
|
||
ready: 0,
|
||
awaiting_proxy: 0,
|
||
owed: Vec::new(),
|
||
};
|
||
let s = a.summary();
|
||
assert!(!s.contains("ready"), "{s}");
|
||
assert!(!s.contains("awaiting"), "{s}");
|
||
assert!(!s.contains("measure"), "{s}");
|
||
assert!(s.contains("10/10"), "{s}");
|
||
assert!(a.is_complete());
|
||
}
|
||
|
||
/// TRACES: FR-CULL-8a
|
||
/// A fully indexed library whose faces have not been read is not
|
||
/// finished: the screen hides the button on `is_complete`, and this is
|
||
/// the only way the readings of an old library ever get filled in.
|
||
#[test]
|
||
fn faces_left_to_measure_keep_the_audit_incomplete() {
|
||
let a = IndexAudit {
|
||
coverage: faces::Coverage {
|
||
images: 10,
|
||
indexed: 10,
|
||
without_faces: 7,
|
||
faces: 4,
|
||
},
|
||
ready: 0,
|
||
awaiting_proxy: 0,
|
||
owed: vec![
|
||
("images with faces to read for quality", 4),
|
||
("images with faces to read for eye state", 0),
|
||
],
|
||
};
|
||
assert!(a.coverage.is_complete());
|
||
assert!(!a.is_complete());
|
||
assert!(
|
||
a.summary()
|
||
.ends_with("; 4 images with faces to read for quality"),
|
||
"{}",
|
||
a.summary()
|
||
);
|
||
}
|
||
|
||
/// The figure the real library actually produced: 110 of 23,528 rounds to
|
||
/// "0%" at whole-number precision, which reads as nothing having happened.
|
||
#[test]
|
||
fn early_progress_does_not_display_as_zero() {
|
||
let a = IndexAudit {
|
||
coverage: faces::Coverage {
|
||
images: 23_528,
|
||
indexed: 110,
|
||
without_faces: 64,
|
||
faces: 125,
|
||
},
|
||
ready: 69,
|
||
awaiting_proxy: 23_349,
|
||
owed: Vec::new(),
|
||
};
|
||
let s = a.summary();
|
||
assert!(s.contains("0.5%"), "{s}");
|
||
assert!(!s.contains("(0%)"), "{s}");
|
||
}
|
||
|
||
#[test]
|
||
fn a_single_image_in_a_huge_library_still_shows_something() {
|
||
let a = IndexAudit {
|
||
coverage: faces::Coverage {
|
||
images: 100_000,
|
||
indexed: 1,
|
||
without_faces: 1,
|
||
faces: 0,
|
||
},
|
||
ready: 99_999,
|
||
awaiting_proxy: 0,
|
||
owed: Vec::new(),
|
||
};
|
||
assert!(a.summary().contains("<0.1%"), "{}", a.summary());
|
||
}
|
||
|
||
#[test]
|
||
fn an_empty_library_says_so_rather_than_reporting_zero_of_zero() {
|
||
assert_eq!(IndexAudit::default().summary(), "no images in the library");
|
||
}
|
||
|
||
#[test]
|
||
fn landmarks_normalise_against_the_long_edge() {
|
||
let lm = [
|
||
(512.0, 256.0),
|
||
(0.0, 0.0),
|
||
(1024.0, 512.0),
|
||
(10.0, 20.0),
|
||
(5.0, 5.0),
|
||
];
|
||
let n = normalise_landmarks(&lm, 1024.0);
|
||
assert!((n[0].0 - 0.5).abs() < 1e-6);
|
||
assert!((n[0].1 - 0.25).abs() < 1e-6);
|
||
assert!((n[2].0 - 1.0).abs() < 1e-6);
|
||
}
|
||
|
||
fn gradient(width: u32, height: u32) -> Vec<u8> {
|
||
let mut v = vec![0u8; (width * height * 4) as usize];
|
||
for (i, px) in v.chunks_exact_mut(4).enumerate() {
|
||
// A gradient, so a mis-placed crop shows up as the wrong value
|
||
// rather than as more of the same colour.
|
||
px[0] = (i % 251) as u8;
|
||
px[1] = 40;
|
||
px[2] = 90;
|
||
px[3] = 255;
|
||
}
|
||
v
|
||
}
|
||
|
||
fn stored_face(x: f32, y: f32, w: f32, h: f32) -> faces::Face {
|
||
faces::Face {
|
||
id: faces::FaceId(1),
|
||
image_id: ImageId(1),
|
||
x,
|
||
y,
|
||
w,
|
||
h,
|
||
landmarks: [(0.0, 0.0); 5],
|
||
confidence: 0.9,
|
||
crop_px: 120.0,
|
||
quality: None,
|
||
eyes: None,
|
||
landmarks_dense: Vec::new(),
|
||
model_id: "w600k_mbf".into(),
|
||
person: None,
|
||
probability: 0.0,
|
||
confirmed: false,
|
||
}
|
||
}
|
||
|
||
#[test]
|
||
fn a_face_crop_is_square_and_the_size_asked_for() {
|
||
let px = gradient(1024, 683);
|
||
let c = crop_face(&px, 1024, 683, &stored_face(0.3, 0.2, 0.1, 0.15), 96).unwrap();
|
||
assert_eq!((c.width, c.height), (96, 96));
|
||
assert_eq!(c.rgba.len(), 96 * 96 * 4);
|
||
}
|
||
|
||
#[test]
|
||
fn a_degenerate_box_yields_no_crop_rather_than_a_panic() {
|
||
let px = gradient(64, 64);
|
||
assert!(crop_face(&px, 64, 64, &stored_face(0.5, 0.5, 0.0, 0.0), 96).is_none());
|
||
assert!(crop_face(&px, 0, 0, &stored_face(0.1, 0.1, 0.2, 0.2), 96).is_none());
|
||
assert!(crop_face(&px, 64, 64, &stored_face(0.1, 0.1, 0.2, 0.2), 0).is_none());
|
||
}
|
||
|
||
/// A face at the very edge of the frame is drawn short, not smeared: the
|
||
/// out-of-frame margin stays transparent.
|
||
#[test]
|
||
fn a_face_at_the_edge_keeps_a_transparent_margin() {
|
||
let px = gradient(200, 200);
|
||
let c = crop_face(&px, 200, 200, &stored_face(0.0, 0.0, 0.1, 0.1), 32).unwrap();
|
||
assert_eq!(c.rgba[3], 0, "outside the frame should be transparent");
|
||
let centre = ((16 * 32 + 16) * 4 + 3) as usize;
|
||
assert_eq!(c.rgba[centre], 255, "the face itself should be opaque");
|
||
}
|
||
|
||
/// The normalised box means a proxy regenerated at another resolution still
|
||
/// crops the same part of the picture — what the catalog's normalisation is
|
||
/// for.
|
||
#[test]
|
||
fn the_same_face_crops_the_same_region_at_two_proxy_sizes() {
|
||
let face = stored_face(0.25, 0.25, 0.2, 0.2);
|
||
let small = crop_face(&gradient(400, 400), 400, 400, &face, 16).unwrap();
|
||
let large = crop_face(&gradient(800, 800), 800, 800, &face, 16).unwrap();
|
||
assert!(small.rgba.chunks_exact(4).all(|p| p[3] == 255));
|
||
assert!(large.rgba.chunks_exact(4).all(|p| p[3] == 255));
|
||
}
|
||
|
||
#[test]
|
||
fn the_crop_keeps_margin_around_the_detection_box() {
|
||
// A 0.1-wide face in a 1000px frame is 100px; with the margin the crop
|
||
// spans 100 * 1.35 = 135px of source.
|
||
let face = stored_face(0.4, 0.4, 0.1, 0.1);
|
||
let c = crop_face(&gradient(1000, 1000), 1000, 1000, &face, 135).unwrap();
|
||
assert_eq!(c.width, 135);
|
||
// Fully inside the frame, so nothing is transparent.
|
||
assert!(c.rgba.chunks_exact(4).all(|p| p[3] == 255));
|
||
}
|
||
|
||
#[test]
|
||
fn rgba_becomes_packed_rgb_dropping_alpha() {
|
||
let rgba = [255u8, 128, 0, 255, 0, 0, 0, 128];
|
||
let rgb = rgba_to_rgb_f32(&rgba);
|
||
assert_eq!(rgb.len(), 6);
|
||
assert!((rgb[0] - 1.0).abs() < 1e-6);
|
||
assert!((rgb[1] - 128.0 / 255.0).abs() < 1e-6);
|
||
assert!((rgb[2] - 0.0).abs() < 1e-6);
|
||
assert!(rgb[3..6].iter().all(|&v| v == 0.0));
|
||
}
|
||
|
||
// ── setting a group aside has to survive regrouping ───────────────────
|
||
|
||
const TEST_MODEL: &str = "w600k_mbf";
|
||
|
||
/// A catalog holding `n` images and nothing else.
|
||
fn catalog_with(n: usize) -> Catalog {
|
||
let c = Catalog::in_memory().unwrap();
|
||
let conn = c.connection();
|
||
conn.execute(
|
||
"INSERT OR IGNORE INTO roots(id, kind, label) VALUES (1, 'local', 'lib')",
|
||
[],
|
||
)
|
||
.unwrap();
|
||
for i in 0..n {
|
||
conn.execute(
|
||
&format!(
|
||
"INSERT INTO images(id, root_id, source_ref, added_at)
|
||
VALUES ({}, 1, 'IMG_{i}.CR3', 0)",
|
||
i + 1
|
||
),
|
||
[],
|
||
)
|
||
.unwrap();
|
||
}
|
||
c
|
||
}
|
||
|
||
/// A unit embedding pointing at `identity`, `cosine` of the way there.
|
||
fn embedding(identity: usize, cosine: f32) -> Vec<u8> {
|
||
let mut v = Box::new([0.0_f32; dr_face::EMBEDDING_DIM]);
|
||
v[identity * 2] = cosine;
|
||
v[identity * 2 + 1] = (1.0 - cosine * cosine).max(0.0).sqrt();
|
||
dr_face::Embedding {
|
||
model: ModelId::new(TEST_MODEL.to_string()),
|
||
v,
|
||
}
|
||
.to_f16_bytes()
|
||
}
|
||
|
||
fn put_face(catalog: &Catalog, image: u64, identity: usize, cosine: f32) {
|
||
let f = DetectedFace {
|
||
x: 0.1,
|
||
y: 0.1,
|
||
w: 0.2,
|
||
h: 0.2,
|
||
landmarks: [(0.0, 0.0); 5],
|
||
confidence: 0.9,
|
||
embedding: embedding(identity, cosine),
|
||
crop_px: 150.0,
|
||
quality: None,
|
||
eyes: None,
|
||
landmarks_dense: Vec::new(),
|
||
model_id: TEST_MODEL.to_string(),
|
||
crop: Vec::new(),
|
||
};
|
||
faces::record_detections(
|
||
catalog.connection(),
|
||
ImageId(image),
|
||
TEST_MODEL,
|
||
1024,
|
||
std::slice::from_ref(&f),
|
||
)
|
||
.unwrap();
|
||
}
|
||
|
||
/// The bug this pins: "not interested" only ever covered *suggested* faces,
|
||
/// and reclustering anchored confirmations alone. So the next Regroup cut
|
||
/// the ignored group's faces loose, built fresh unnamed groups out of them,
|
||
/// and every stranger the user had dismissed came straight back.
|
||
#[test]
|
||
fn a_group_set_aside_does_not_come_back_on_the_next_regroup() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let people = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(people.len(), 1, "the two faces should have grouped");
|
||
let stranger = people[0].id;
|
||
assert_eq!(people[0].suggested_faces, 2);
|
||
|
||
faces::set_ignored(catalog.connection(), stranger, true).unwrap();
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(
|
||
after.len(),
|
||
1,
|
||
"regrouping resurrected the group that was set aside: {after:?}"
|
||
);
|
||
assert_eq!(after[0].id, stranger);
|
||
assert!(after[0].ignored, "the group stopped being set aside");
|
||
assert_eq!(
|
||
after[0].suggested_faces, 2,
|
||
"the faces left the group they were set aside in"
|
||
);
|
||
}
|
||
|
||
/// And it holds as the library grows: a stranger photographed again joins
|
||
/// the group that was set aside rather than arriving as somebody new.
|
||
#[test]
|
||
fn a_new_face_joins_the_group_it_matches_even_when_that_group_is_set_aside() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let stranger = faces::people(catalog.connection()).unwrap()[0].id;
|
||
faces::set_ignored(catalog.connection(), stranger, true).unwrap();
|
||
|
||
// The same person turns up in a third photograph.
|
||
put_face(&catalog, 3, 0, 0.98);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(after.len(), 1, "a new face made a second group: {after:?}");
|
||
assert!(after[0].ignored);
|
||
assert_eq!(after[0].suggested_faces, 3);
|
||
}
|
||
|
||
/// The other half of the promise: bringing them back really does.
|
||
#[test]
|
||
fn bringing_a_group_back_makes_it_ordinary_again() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
|
||
let id = faces::people(catalog.connection()).unwrap()[0].id;
|
||
faces::set_ignored(catalog.connection(), id, true).unwrap();
|
||
faces::set_ignored(catalog.connection(), id, false).unwrap();
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(after.len(), 1);
|
||
assert!(!after[0].ignored);
|
||
}
|
||
|
||
/// Naming a group does not confirm its faces, so before this they were
|
||
/// still only suggestions — and the next Regroup cut them loose, built a
|
||
/// new person out of them, and left the named one empty. Do that a few
|
||
/// times and the rail fills with same-named people holding nothing while
|
||
/// the faces sit under whichever one was made last.
|
||
#[test]
|
||
fn a_named_group_keeps_its_faces_through_the_next_regroup() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let people = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(people.len(), 1);
|
||
let her = people[0].id;
|
||
assert_eq!(people[0].suggested_faces, 2);
|
||
|
||
// Named, and nothing else — no confirmations, which is what a user who
|
||
// types a name and moves on has done.
|
||
faces::rename_person(catalog.connection(), her, "Catherine").unwrap();
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(
|
||
after.len(),
|
||
1,
|
||
"regrouping left a second person behind: {after:?}"
|
||
);
|
||
assert_eq!(after[0].id, her);
|
||
assert_eq!(after[0].name, "Catherine");
|
||
assert_eq!(
|
||
after[0].suggested_faces, 2,
|
||
"the named group lost the faces it was named for"
|
||
);
|
||
}
|
||
|
||
/// And a face found later joins the person it matches rather than arriving
|
||
/// as somebody new — the same benefit anchoring gives an ignored group.
|
||
#[test]
|
||
fn a_new_face_joins_a_named_person_rather_than_starting_a_rival() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let her = faces::people(catalog.connection()).unwrap()[0].id;
|
||
faces::rename_person(catalog.connection(), her, "Catherine").unwrap();
|
||
|
||
put_face(&catalog, 3, 0, 0.98);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(after.len(), 1, "a second Catherine appeared: {after:?}");
|
||
assert_eq!(after[0].suggested_faces, 3);
|
||
}
|
||
|
||
/// A person over the reference cap is stood for by a chosen few, and the
|
||
/// rest of their faces stay where they are: not compared, not released,
|
||
/// and still theirs when the next face arrives.
|
||
#[test]
|
||
fn a_well_covered_person_is_stood_for_by_their_references() {
|
||
let many = dr_face::references::MAX_REFERENCES + 50;
|
||
let catalog = catalog_with(many + 1);
|
||
for i in 0..many {
|
||
put_face(&catalog, i as u64 + 1, 0, 1.0 - (i as f32) * 1e-3);
|
||
}
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let him = faces::people(catalog.connection()).unwrap()[0].id;
|
||
faces::rename_person(catalog.connection(), him, "Ian").unwrap();
|
||
|
||
// Every one of his faces lies in one plane, so two references span
|
||
// them all and the selector stops there rather than filling the cap.
|
||
let pop = Population::read(&catalog, TEST_MODEL).unwrap();
|
||
let standing = pop
|
||
.candidates
|
||
.iter()
|
||
.filter(|c| c.confirmed_person.is_some())
|
||
.count();
|
||
assert!(
|
||
(2..=dr_face::references::MAX_REFERENCES).contains(&standing),
|
||
"{standing} of {many} faces entered the scan"
|
||
);
|
||
assert_eq!(pop.anchors.len(), many, "the rest lost their anchor");
|
||
|
||
put_face(&catalog, many as u64 + 1, 0, 0.98);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(after.len(), 1, "a second Ian appeared: {after:?}");
|
||
assert_eq!(
|
||
after[0].suggested_faces,
|
||
(many + 1) as u64,
|
||
"faces were released"
|
||
);
|
||
}
|
||
|
||
/// `dr-types` sits below the face engine and cannot name its constant, so
|
||
/// it restates the number. This is the thing that stops the two drifting:
|
||
/// a settings page marking 0.80 as "default" while the engine had moved to
|
||
/// 0.75 would put the reset dot on a value nothing else agreed with.
|
||
#[test]
|
||
fn the_settings_default_is_the_engines_own_tuned_value() {
|
||
assert_eq!(
|
||
FaceSettings::default().merge_probability,
|
||
dr_face::DEFAULT_MERGE_PROBABILITY
|
||
);
|
||
}
|
||
|
||
/// The smallest-group setting has to change what is on the rail, not just
|
||
/// what the *next* pass would build. Before the release step in
|
||
/// [`recluster`], raising it left the previous pass's small groups sitting
|
||
/// there full of suggestions — not empty, so not pruned — and the control
|
||
/// looked broken.
|
||
#[test]
|
||
fn raising_the_smallest_group_takes_the_small_groups_off_the_rail() {
|
||
let catalog = catalog_with(5);
|
||
// One pair, and one trio: two groups at the default of two.
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
put_face(&catalog, 3, 1, 1.0);
|
||
put_face(&catalog, 4, 1, 0.99);
|
||
put_face(&catalog, 5, 1, 0.98);
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
assert_eq!(
|
||
faces::people(catalog.connection()).unwrap().len(),
|
||
2,
|
||
"the two groups should have formed at the default"
|
||
);
|
||
|
||
recluster(
|
||
&catalog,
|
||
TEST_MODEL,
|
||
&FaceSettings {
|
||
min_group_size: 3,
|
||
..FaceSettings::default()
|
||
},
|
||
)
|
||
.unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(
|
||
after.len(),
|
||
1,
|
||
"the pair survived a smallest-group of three: {after:?}"
|
||
);
|
||
assert_eq!(after[0].suggested_faces, 3, "the trio lost members");
|
||
}
|
||
|
||
/// And the setting does not overrule the user. A group they named is theirs
|
||
/// (FR-CULL-12), however few faces it holds.
|
||
#[test]
|
||
fn a_named_group_survives_a_smallest_group_it_is_under() {
|
||
let catalog = catalog_with(3);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 0, 0.99);
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
let her = faces::people(catalog.connection()).unwrap()[0].id;
|
||
faces::rename_person(catalog.connection(), her, "Catherine").unwrap();
|
||
|
||
recluster(
|
||
&catalog,
|
||
TEST_MODEL,
|
||
&FaceSettings {
|
||
min_group_size: 6,
|
||
..FaceSettings::default()
|
||
},
|
||
)
|
||
.unwrap();
|
||
|
||
let after = faces::people(catalog.connection()).unwrap();
|
||
assert_eq!(after.len(), 1, "Catherine was dropped: {after:?}");
|
||
assert_eq!(after[0].name, "Catherine");
|
||
assert_eq!(after[0].suggested_faces, 2);
|
||
}
|
||
|
||
/// Down at one, every stray becomes a group of its own — which is what a
|
||
/// user hunting for a face the grouping missed has asked for.
|
||
#[test]
|
||
fn a_smallest_group_of_one_shows_the_strays() {
|
||
let catalog = catalog_with(2);
|
||
put_face(&catalog, 1, 0, 1.0);
|
||
put_face(&catalog, 2, 1, 1.0);
|
||
|
||
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
|
||
assert!(
|
||
faces::people(catalog.connection()).unwrap().is_empty(),
|
||
"two unrelated faces made a person at the default"
|
||
);
|
||
|
||
recluster(
|
||
&catalog,
|
||
TEST_MODEL,
|
||
&FaceSettings {
|
||
min_group_size: 1,
|
||
..FaceSettings::default()
|
||
},
|
||
)
|
||
.unwrap();
|
||
assert_eq!(faces::people(catalog.connection()).unwrap().len(), 2);
|
||
}
|
||
}
|