Name segmented people from the faces already recognised in them

The segmenter knows it found a person; the face index knows which person.
Joining them turns "person" in the mask list into "Anna", which is the
difference between a vocabulary of eighty COCO classes and one that
includes the user's family. Selecting a subject in a group photograph
stops being a guessing game between three identical rows.

Containment, not IoU. A face is a small part of the person it belongs to,
so a correct pairing has an IoU near zero and anything IoU-based would
reject every true match.

Confirmed names only. A suggestion is the system's guess, and printing a
guessed name onto a mask region would launder it into a fact.

Writing the tests corrected the design once: a tight head-and-shoulders
portrait, where the face fills most of the person box, is the case where
naming is most certain, not least. An earlier guard rejected exactly that
and has been removed, with the reasoning left as a test because it is
easy to get backwards a second time.

The names hang on the develop session, set when the image opens because
that is the one moment the catalog and the image id are both in reach.
Every segmentation run afterwards picks them up for free, and a library
with no face indexing behaves exactly as it did before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-26 23:10:46 +02:00
co-authored by Claude Opus 5
parent d562ceaaf4
commit b55812812a
7 changed files with 590 additions and 22 deletions
+158
View File
@@ -240,6 +240,108 @@ fn decode_proxy(
dr_thumbs::codec::decode_rgba(&thumb.bytes).ok()
}
/// A named face box normalised to the image's long edge: `(x, y, w, h, name)`.
///
/// Named because it crosses three layers — catalog, develop session,
/// segmentation job — and "the fourth float" is not something anyone should
/// have to count out at each one.
pub type NormalisedNamedBox = (f32, f32, f32, f32, String);
/// The confirmed, named faces in one image, **normalised to the long edge**.
///
/// The form a develop session carries, because the proxy a segmentation run
/// settles on is not known when the image opens — so the scaling happens at
/// the far end, in [`crate::develop::SegmentationJob`].
///
/// **Confirmed names only.** A suggestion is the system's guess, and printing
/// a guessed name onto a mask region would launder it into a fact — the exact
/// conflation the confirmed/suggested split exists to prevent (FR-CULL-10). An
/// unrecognised or merely-suggested person leaves the region as "person",
/// which is honest.
pub fn named_boxes_normalised(
catalog: &Catalog,
image: ImageId,
) -> Result<Vec<NormalisedNamedBox>, dr_catalog::CatalogError> {
let conn = catalog.connection();
let mut names: std::collections::HashMap<PersonId, String> = std::collections::HashMap::new();
for p in faces::people(conn)? {
if !p.name.is_empty() {
names.insert(p.id, p.name);
}
}
if names.is_empty() {
return Ok(Vec::new());
}
Ok(faces::for_image(conn, image)?
.into_iter()
.filter(|f| f.confirmed)
.filter_map(|f| {
let name = f.person.and_then(|p| names.get(&p))?.clone();
Some((f.x, f.y, f.w, f.h, name))
})
.collect())
}
/// A named face box for one image, in proxy pixels.
///
/// Owned rather than borrowed because it crosses from a catalog read into a
/// segmentation pass that outlives the query.
#[derive(Debug, Clone, PartialEq)]
pub struct NamedBox {
pub bbox: (f32, f32, f32, f32),
pub name: String,
}
/// The named faces in one image, ready to label its segmented regions.
///
/// **Confirmed names only.** A suggestion is the system's guess, and printing
/// a guessed name on a mask region would launder it into a fact — the exact
/// conflation the confirmed/suggested split exists to prevent (FR-CULL-10).
/// An unrecognised or merely-suggested person leaves the region as "person",
/// which is honest.
///
/// Boxes come back in the proxy pixel space the caller names, because that is
/// what `Segmentation` works in — the catalog stores them normalised to the
/// long edge precisely so this conversion is possible at any resolution.
pub fn named_boxes_for_image(
catalog: &Catalog,
image: ImageId,
proxy_w: usize,
proxy_h: usize,
) -> Result<Vec<NamedBox>, dr_catalog::CatalogError> {
let conn = catalog.connection();
let long_edge = proxy_w.max(proxy_h) as f32;
if long_edge <= 0.0 {
return Ok(Vec::new());
}
let mut names: std::collections::HashMap<PersonId, String> = std::collections::HashMap::new();
for p in faces::people(conn)? {
if !p.name.is_empty() {
names.insert(p.id, p.name);
}
}
Ok(faces::for_image(conn, image)?
.into_iter()
.filter(|f| f.confirmed)
.filter_map(|f| {
let name = f.person.and_then(|p| names.get(&p))?.clone();
Some(NamedBox {
bbox: (
f.x * long_edge,
f.y * long_edge,
(f.x + f.w) * long_edge,
(f.y + f.h) * long_edge,
),
name,
})
})
.collect())
}
// ── the user's decisions ──────────────────────────────────────────────────
/// Name a group, or rename a person.
@@ -596,6 +698,62 @@ mod tests {
assert_eq!(load_people(&c, "w600k_mbf").unwrap().people.len(), 1);
}
/// The segmentation link: a confirmed face inside a `person` region should
/// put that person's name on it.
#[test]
fn named_boxes_come_back_in_proxy_pixels() {
let c = catalog();
let img = image(&c, 1);
let ids = faces::record_detections(c.connection(), img, "w600k_mbf", 1024, &[face(1)])
.unwrap();
let anna = faces::create_person(c.connection(), "Anna").unwrap();
faces::confirm(c.connection(), ids[0], anna).unwrap();
// The fixture's face is x=0.1 y=0.1 w=0.2 h=0.3, normalised to the
// long edge — so at 1024×683 it spans 102.4..307.2 across.
let boxes = named_boxes_for_image(&c, img, 1024, 683).unwrap();
assert_eq!(boxes.len(), 1);
assert_eq!(boxes[0].name, "Anna");
assert!((boxes[0].bbox.0 - 102.4).abs() < 0.1, "{:?}", boxes[0].bbox);
assert!((boxes[0].bbox.2 - 307.2).abs() < 0.1, "{:?}", boxes[0].bbox);
// And the same face at another proxy size lands proportionally — the
// reason the catalog stores these normalised.
let bigger = named_boxes_for_image(&c, img, 2048, 1366).unwrap();
assert!((bigger[0].bbox.0 - 204.8).abs() < 0.1);
}
/// A suggestion is the system's guess. Printing a guessed name onto a mask
/// region would launder it into a fact.
#[test]
fn a_suggested_person_does_not_name_a_region() {
let c = catalog();
let img = image(&c, 1);
let ids = faces::record_detections(c.connection(), img, "w600k_mbf", 1024, &[face(1)])
.unwrap();
let anna = faces::create_person(c.connection(), "Anna").unwrap();
faces::suggest(c.connection(), ids[0], anna, 0.95).unwrap();
assert!(named_boxes_for_image(&c, img, 1024, 683).unwrap().is_empty());
// Confirming it makes the name appear.
faces::confirm(c.connection(), ids[0], anna).unwrap();
assert_eq!(named_boxes_for_image(&c, img, 1024, 683).unwrap().len(), 1);
}
/// An unnamed cluster has nothing to say about a region.
#[test]
fn an_unnamed_group_does_not_name_a_region() {
let c = catalog();
let img = image(&c, 1);
let ids = faces::record_detections(c.connection(), img, "w600k_mbf", 1024, &[face(1)])
.unwrap();
let nameless = faces::create_person(c.connection(), "").unwrap();
faces::confirm(c.connection(), ids[0], nameless).unwrap();
assert!(named_boxes_for_image(&c, img, 1024, 683).unwrap().is_empty());
}
#[test]
fn deleting_everything_empties_the_screen() {
let c = catalog();