Turn face boxes back into sensor space before matching regions
Faces are found on the thumbnail, which is cached the right way up -- the grid would lie on its side otherwise. Segmentation runs on a proxy rendered through a neutral edit graph, which carries no orientation and is therefore in sensor order. For anything shot in portrait the two differ by a quarter turn, so a face and the person containing it were being compared in spaces 90 degrees apart: no match, or worse, a match against somebody else's region. The transform goes on the face rather than on the proxy. Instance masks are defined in the proxy's space and sampled long afterwards, so turning that space would be a far larger change than naming a region warrants. Also two things the first screenshot of the running app showed that no test would have: 110 of 23,528 displayed as "0%", which reads as the feature having done nothing. One decimal below ten percent, and a floor so real progress never shows as none. The rail picked some near-black covers, because the largest face in a group is often the nearest one in a badly lit frame and a black square beside a name identifies nobody. It now cuts the best few and takes the first legible one, falling back to the largest when a person's every photograph is dark -- which happens, and showing it beats showing nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+33
-6
@@ -116,7 +116,8 @@ pub struct SegmentationJob {
|
||||
source: Arc<DemosaicedImage>,
|
||||
session: SessionId,
|
||||
abandon: Abandon,
|
||||
/// Confirmed faces in this photograph, **normalised to the long edge**.
|
||||
/// Confirmed faces in this photograph, **normalised to the long edge** of
|
||||
/// the *upright* image.
|
||||
///
|
||||
/// Carried rather than looked up, because the job runs on a thread with no
|
||||
/// catalog in reach — the same reason it carries the pixels. Normalised
|
||||
@@ -124,6 +125,8 @@ pub struct SegmentationJob {
|
||||
/// `run`, and these have to survive being scaled to whatever it turns out
|
||||
/// to be.
|
||||
names: Vec<crate::identity::NormalisedNamedBox>,
|
||||
/// Undone before matching, since the proxy is in sensor order.
|
||||
orientation: dr_types::Orientation,
|
||||
}
|
||||
|
||||
impl SegmentationJob {
|
||||
@@ -172,16 +175,31 @@ impl SegmentationJob {
|
||||
// at the call site because this is where the proxy size is finally
|
||||
// known.
|
||||
if !self.names.is_empty() {
|
||||
// The proxy is in sensor order; the faces are upright. The long
|
||||
// edge is the same number either way — `max` survives the swap —
|
||||
// but the axes do not, so the boxes are turned back before they
|
||||
// are compared with anything.
|
||||
let long_edge = rw.max(rh) as f32;
|
||||
let displayed = if self.orientation.swaps_axes() {
|
||||
(rh as f32, rw as f32)
|
||||
} else {
|
||||
(rw as f32, rh as f32)
|
||||
};
|
||||
let boxes: Vec<dr_face::NamedFace<'_>> = self
|
||||
.names
|
||||
.iter()
|
||||
.map(|(x, y, w, h, name)| dr_face::NamedFace {
|
||||
bbox: (
|
||||
x * long_edge,
|
||||
y * long_edge,
|
||||
(x + w) * long_edge,
|
||||
(y + h) * long_edge,
|
||||
bbox: dr_face::naming::to_sensor_space(
|
||||
(
|
||||
x * long_edge,
|
||||
y * long_edge,
|
||||
(x + w) * long_edge,
|
||||
(y + h) * long_edge,
|
||||
),
|
||||
displayed,
|
||||
self.orientation.quarter_turns,
|
||||
self.orientation.flip_h,
|
||||
self.orientation.flip_v,
|
||||
),
|
||||
name,
|
||||
})
|
||||
@@ -518,6 +536,13 @@ pub struct DevelopSession {
|
||||
/// ever on a library with no face indexing — in which case segmentation
|
||||
/// behaves exactly as it did before, which is the point.
|
||||
face_names: Vec<crate::identity::NormalisedNamedBox>,
|
||||
/// How this photograph is stored relative to how it is shown.
|
||||
///
|
||||
/// Kept because the segmentation proxy is rendered through a *neutral*
|
||||
/// graph and is therefore in sensor order, while faces were found on the
|
||||
/// upright thumbnail. On anything shot in portrait the two differ by a
|
||||
/// quarter turn, and matching them without undoing it finds nothing.
|
||||
orientation: dr_types::Orientation,
|
||||
/// Kept so the session can build GPU resources after construction.
|
||||
///
|
||||
/// The distance fields behind a subject mask are made when a layer is
|
||||
@@ -647,6 +672,7 @@ impl DevelopSession {
|
||||
Self {
|
||||
id: SessionId::next(),
|
||||
face_names: Vec::new(),
|
||||
orientation,
|
||||
ctx: ctx.clone(),
|
||||
graph,
|
||||
history,
|
||||
@@ -1635,6 +1661,7 @@ impl DevelopSession {
|
||||
session: self.id,
|
||||
abandon: Abandon::default(),
|
||||
names: self.face_names.clone(),
|
||||
orientation: self.orientation,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user