Name segmented people from the faces already recognised in them
The segmenter knows it found a person; the face index knows which person. Joining them turns "person" in the mask list into "Anna", which is the difference between a vocabulary of eighty COCO classes and one that includes the user's family. Selecting a subject in a group photograph stops being a guessing game between three identical rows. Containment, not IoU. A face is a small part of the person it belongs to, so a correct pairing has an IoU near zero and anything IoU-based would reject every true match. Confirmed names only. A suggestion is the system's guess, and printing a guessed name onto a mask region would launder it into a fact. Writing the tests corrected the design once: a tight head-and-shoulders portrait, where the face fills most of the person box, is the case where naming is most certain, not least. An earlier guard rejected exactly that and has been removed, with the reasoning left as a test because it is easy to get backwards a second time. The names hang on the develop session, set when the image opens because that is the one moment the catalog and the image id are both in reach. Every segmentation run afterwards picks them up for free, and a library with no face indexing behaves exactly as it did before. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+57
-1
@@ -116,6 +116,14 @@ pub struct SegmentationJob {
|
||||
source: Arc<DemosaicedImage>,
|
||||
session: SessionId,
|
||||
abandon: Abandon,
|
||||
/// Confirmed faces in this photograph, **normalised to the long edge**.
|
||||
///
|
||||
/// Carried rather than looked up, because the job runs on a thread with no
|
||||
/// catalog in reach — the same reason it carries the pixels. Normalised
|
||||
/// rather than in pixels because the proxy size is only settled inside
|
||||
/// `run`, and these have to survive being scaled to whatever it turns out
|
||||
/// to be.
|
||||
names: Vec<crate::identity::NormalisedNamedBox>,
|
||||
}
|
||||
|
||||
impl SegmentationJob {
|
||||
@@ -158,7 +166,32 @@ impl SegmentationJob {
|
||||
return Ok(None);
|
||||
}
|
||||
|
||||
let seg = segmentation::compute(&self.ctx, &rgb, rw, rh, options)?;
|
||||
let mut seg = segmentation::compute(&self.ctx, &rgb, rw, rh, options)?;
|
||||
|
||||
// Put names on the people the segmenter found. Scaled here rather than
|
||||
// at the call site because this is where the proxy size is finally
|
||||
// known.
|
||||
if !self.names.is_empty() {
|
||||
let long_edge = rw.max(rh) as f32;
|
||||
let boxes: Vec<dr_face::NamedFace<'_>> = self
|
||||
.names
|
||||
.iter()
|
||||
.map(|(x, y, w, h, name)| dr_face::NamedFace {
|
||||
bbox: (
|
||||
x * long_edge,
|
||||
y * long_edge,
|
||||
(x + w) * long_edge,
|
||||
(y + h) * long_edge,
|
||||
),
|
||||
name,
|
||||
})
|
||||
.collect();
|
||||
let named = seg.apply_names(&boxes);
|
||||
if named > 0 {
|
||||
log::info!("named {named} segmented region(s) from known faces");
|
||||
}
|
||||
}
|
||||
|
||||
log::info!(
|
||||
"segmented {rw}×{rh}: {} subject(s), proxy {:.0} ms, total {:.0} ms",
|
||||
seg.instances().len(),
|
||||
@@ -478,6 +511,13 @@ fn bilinear_sample(mask: &[f32], w: usize, h: usize, x: f32, y: f32) -> f32 {
|
||||
pub struct DevelopSession {
|
||||
/// This session's name, for work that outlives the frame it started on.
|
||||
id: SessionId,
|
||||
/// TRACES: FR-CULL-10
|
||||
/// Confirmed faces in this photograph, normalised to the long edge.
|
||||
///
|
||||
/// Empty until [`DevelopSession::set_face_names`] is called, and empty for
|
||||
/// ever on a library with no face indexing — in which case segmentation
|
||||
/// behaves exactly as it did before, which is the point.
|
||||
face_names: Vec<crate::identity::NormalisedNamedBox>,
|
||||
/// Kept so the session can build GPU resources after construction.
|
||||
///
|
||||
/// The distance fields behind a subject mask are made when a layer is
|
||||
@@ -606,6 +646,7 @@ impl DevelopSession {
|
||||
let history = History::new(&graph);
|
||||
Self {
|
||||
id: SessionId::next(),
|
||||
face_names: Vec::new(),
|
||||
ctx: ctx.clone(),
|
||||
graph,
|
||||
history,
|
||||
@@ -1593,9 +1634,24 @@ impl DevelopSession {
|
||||
source: self.demosaiced.clone(),
|
||||
session: self.id,
|
||||
abandon: Abandon::default(),
|
||||
names: self.face_names.clone(),
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-CULL-10 | FR-DEV-3
|
||||
/// The confirmed faces in this photograph, for naming segmented regions.
|
||||
///
|
||||
/// Set once when the image opens, because that is the only moment the
|
||||
/// catalog and the image id are both in reach — the develop session
|
||||
/// deliberately knows nothing about either, and every segmentation run
|
||||
/// after this point picks the names up for free.
|
||||
///
|
||||
/// Boxes are normalised to the long edge, as the catalog stores them, so
|
||||
/// they survive whatever proxy size a run settles on.
|
||||
pub fn set_face_names(&mut self, names: Vec<crate::identity::NormalisedNamedBox>) {
|
||||
self.face_names = names;
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3
|
||||
/// Take on a segmentation found elsewhere.
|
||||
///
|
||||
|
||||
Reference in New Issue
Block a user