Measure which mask layers a crop takes out of the frame

Mask geometry is stored in source coordinates, so re-cropping tighter
never destroys a layer. It makes it invisible: the layer stays in the
panel and the sidecar, its adjustment lands on pixels nobody will see,
and nothing says so. The spec had no clause for this; FR-DEV-17 now
states it, under the ID issue #10 reserved.

dr_pipeline::orphan samples each layer's mask on a 64x64 lattice over
the source, with the gradient, radial, brush and model-raster geometry
the mask shader uses, folds the parts by their joins and inversions, and
maps the samples through the framing to see how much of the coverage
the crop keeps. `hidden_by_crop` reports the layers whose share fell
below a tenth, and only those the change newly hid, so an already
stranded layer is not announced again on every later adjustment.

Ranges follow the picture and region selections need a label map this
crate does not hold, so a layer that adds either is never reported: a
false alarm on the common path would teach the notice to be dismissed
unread.
This commit is contained in:
2026-09-24 21:52:02 -04:00
parent 733a033274
commit 9772785f81
4 changed files with 701 additions and 51 deletions
+1
View File
@@ -44,6 +44,7 @@ pub mod mask;
pub mod neutral;
pub mod operation;
pub mod ops;
pub mod orphan;
pub mod preset;
pub mod sidecar;
pub mod spot;
+637
View File
@@ -0,0 +1,637 @@
//! TRACES: FR-DEV-17
//! Whether a crop leaves a mask layer's work outside the frame.
//!
//! # Why this is a question worth asking
//!
//! Mask geometry is stored in normalised *source* coordinates (see
//! [`MaskSource::Linear`] and [`Stroke::points`]), so a tighter crop never
//! destroys a layer. It makes it invisible — which is worse, because nothing
//! announces it. The layer is still in the panel, still in the sidecar, still
//! costing a rasterisation, and its adjustment lands on pixels nobody will
//! ever see. The crop that did that is exactly the kind of edit made early
//! and quickly, and the loss is found, if at all, much later.
//!
//! This module answers one question on the CPU, cheaply enough to ask once
//! per committed crop: *which layers did this change of crop take out of the
//! picture?* The interface turns the answer into a notice. It never refuses
//! the crop — the photographer may well mean it.
//!
//! # How it is measured
//!
//! Each layer's mask is sampled on a [`GRID`]×[`GRID`] lattice over the
//! source, with the same geometry the mask shader uses (`mask.wgsl`), folded
//! part by part with the layer's joins and inversions. The sample points are
//! then mapped through the framing — crop, straighten, turns and flips — into
//! the frame, and the layer's *share inside* is the coverage that lands in
//! the crop over the coverage there is. A layer is hidden by a crop when that
//! share falls below [`HIDDEN_SHARE`] and was not already below it.
//!
//! The lattice is coarse on purpose. The question is "is this layer mostly
//! gone", not "which pixels are": the edge treatment — feather, morphology —
//! moves a boundary by a few hundredths of the frame and cannot turn a layer
//! that is mostly inside into one that is mostly outside, so it is left out.
//!
//! # What cannot be orphaned
//!
//! A range ([`MaskSource::Luminance`], [`MaskSource::Colour`]) selects by a
//! property of the picture, so wherever the crop falls it selects whatever
//! part of the picture remains — there is nothing for a crop to strand. A
//! region selection needs the segmentation's label map, which this crate does
//! not hold, and a model's selection with no raster to hand has nothing to
//! measure. A layer that *adds* any of these is therefore never reported: a
//! false alarm on the common path would teach the photographer to dismiss the
//! notice unread, which is the failure it exists to prevent. Subtracting one
//! is ignored, which can only make the layer look larger — the safe side.
use std::borrow::Cow;
use crate::framing::{CropRect, Framing};
use crate::mask::{Join, MaskLayer, MaskPart, MaskSource, MaskStack, Stroke};
/// Samples per axis over the source.
///
/// 4096 points: enough that a brush dab a twentieth of the frame across is
/// several samples wide, and few enough that the whole stack is measured in
/// well under a frame when a crop is let go.
pub const GRID: usize = 64;
/// The share of a layer's coverage below which it counts as cropped away.
///
/// "Mostly outside" rather than "entirely": a gradient reduced to a sliver
/// along one edge, or a subject of which one elbow survives, has lost the
/// work as surely as one that is wholly gone. Low enough that an ordinary
/// recomposition which trims part of a subject is not reported.
pub const HIDDEN_SHARE: f32 = 0.1;
/// A model's selection, as bytes over the source at some proxy size.
///
/// Supplied by the caller for a part whose own [`MaskPart::coverage`] is not
/// set — the live session holds its model output outside the graph and folds
/// it into the parts only when saving.
pub struct Raster<'a> {
pub values: Cow<'a, [u8]>,
pub width: usize,
pub height: usize,
}
/// One layer's coverage, sampled over the source.
#[derive(Debug, Clone, PartialEq)]
pub struct Footprint {
/// Row-major over the lattice, `0.0..=1.0`, one per cell centre.
weights: Vec<f32>,
}
impl Footprint {
/// The layer's coverage on the lattice, or `None` when it has none to
/// strand — see the module header for which layers those are.
///
/// `model` is asked for the raster behind a subject or category part that
/// carries none of its own.
pub fn of<'r>(
layer: &MaskLayer,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Option<Self> {
let mut acc: Option<Vec<f32>> = None;
for (i, part) in layer.shown_parts().enumerate() {
let adds = i == 0 || part.join == Join::Union;
let Some(values) = part_values(part, source, model) else {
if adds {
// It follows the picture, or cannot be measured: either
// way it is not a shape a crop can leave behind.
return None;
}
continue;
};
acc = Some(match acc {
None => values,
Some(mut a) => {
for (d, s) in a.iter_mut().zip(values) {
*d = part.join.apply(*d, s);
}
a
}
});
}
let mut weights = acc?;
if layer.invert {
weights.iter_mut().for_each(|w| *w = 1.0 - *w);
}
Some(Self { weights })
}
/// Of this layer's coverage, the share `inside` marks as in frame.
/// `None` when there is no coverage at all.
fn share(&self, inside: &[bool]) -> Option<f32> {
let (mut total, mut kept) = (0.0f32, 0.0f32);
for (w, &i) in self.weights.iter().zip(inside) {
total += w;
if i {
kept += w;
}
}
(total > 1e-3).then(|| kept / total)
}
/// Of this layer's coverage, the share `framing`'s crop keeps.
pub fn share_inside(&self, framing: &Framing, source: (u32, u32)) -> Option<f32> {
self.share(&in_frame(framing, source))
}
}
/// TRACES: FR-DEV-17
/// The layers that changing the framing from `before` to `after` took out of
/// the picture, in stack order.
///
/// Only those it *newly* hid: a layer already cropped away by `before` is not
/// reported again, or every later adjustment of the crop would repeat a notice
/// the photographer has already answered.
///
/// The view (zoom and pan) of either framing is ignored. It is a way of
/// looking, not the frame.
pub fn hidden_by_crop<'m, 'r>(
masks: &'m MaskStack,
before: &Framing,
after: &Framing,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Vec<&'m MaskLayer> {
if masks.is_empty() || framed_alike(before, after) {
return Vec::new();
}
let was = in_frame(before, source);
let now = in_frame(after, source);
masks
.layers()
.iter()
.filter(|layer| {
let Some(print) = Footprint::of(layer, source, model) else {
return false;
};
let (Some(was), Some(now)) = (print.share(&was), print.share(&now)) else {
return false;
};
was >= HIDDEN_SHARE && now < HIDDEN_SHARE
})
.collect()
}
/// Whether the two frame the same part of the source, zoom aside.
fn framed_alike(a: &Framing, b: &Framing) -> bool {
let mut a = *a;
let mut b = *b;
a.set_view(CropRect::default());
b.set_view(CropRect::default());
a == b
}
/// Which lattice cells the framing's crop keeps.
fn in_frame(framing: &Framing, source: (u32, u32)) -> Vec<bool> {
let mut f = *framing;
f.set_view(CropRect::default());
lattice()
.map(|uv| {
let (x, y) = f.output_at(uv, source.0, source.1);
(0.0..=1.0).contains(&x) && (0.0..=1.0).contains(&y)
})
.collect()
}
/// Cell centres, row-major, in normalised source coordinates.
fn lattice() -> impl Iterator<Item = (f32, f32)> {
let step = 1.0 / GRID as f32;
(0..GRID).flat_map(move |y| {
(0..GRID).map(move |x| ((x as f32 + 0.5) * step, (y as f32 + 0.5) * step))
})
}
/// One part's coverage on the lattice, its own inversion applied, or `None`
/// where it cannot be measured as a shape.
fn part_values<'r>(
part: &MaskPart,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Option<Vec<f32>> {
let aspect = source.0.max(1) as f32 / source.1.max(1) as f32;
let mut values: Vec<f32> = match &part.source {
MaskSource::Linear {
centre,
angle,
width,
} => {
let axis = (angle.cos(), angle.sin());
lattice()
.map(|(u, v)| {
let d = (u - centre.0) * aspect * axis.0 + (v - centre.1) * axis.1;
if *width <= 0.0 {
if d >= 0.0 {
1.0
} else {
0.0
}
} else {
smoothstep(-width * 0.5, width * 0.5, d)
}
})
.collect()
}
MaskSource::Radial {
centre,
radii,
angle,
feather,
} => {
let (sa, ca) = (-angle).sin_cos();
let radii = (radii.0.max(1e-6), radii.1.max(1e-6));
let edge = feather.clamp(0.0, 1.0);
lattice()
.map(|(u, v)| {
let d = ((u - centre.0) * aspect, v - centre.1);
let local = (d.0 * ca - d.1 * sa, d.0 * sa + d.1 * ca);
let r = (local.0 / radii.0).hypot(local.1 / radii.1);
if edge <= 0.0 {
if r <= 1.0 {
1.0
} else {
0.0
}
} else {
1.0 - smoothstep(1.0 - edge, 1.0, r)
}
})
.collect()
}
MaskSource::Brush { strokes } => brush_values(strokes, source),
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
if let Some(coverage) = &part.coverage {
// The lattice is a regular grid over the source, which is
// exactly the resample `decode_at` does.
coverage
.decode_at(GRID, GRID)
.into_iter()
.map(|b| f32::from(b) / 255.0)
.collect()
} else {
let raster = model(part)?;
if raster.width == 0
|| raster.height == 0
|| raster.values.len() < raster.width * raster.height
{
return None;
}
lattice()
.map(|(u, v)| {
let x = ((u * raster.width as f32) as usize).min(raster.width - 1);
let y = ((v * raster.height as f32) as usize).min(raster.height - 1);
f32::from(raster.values[y * raster.width + x]) / 255.0
})
.collect()
}
}
MaskSource::Regions { .. } | MaskSource::Luminance { .. } | MaskSource::Colour { .. } => {
return None
}
};
if part.invert {
values.iter_mut().for_each(|w| *w = 1.0 - *w);
}
Some(values)
}
/// Strokes composited in order, the way `fs_brush` and its blend states do.
fn brush_values(strokes: &[Stroke], source: (u32, u32)) -> Vec<f32> {
// Into units of the shorter edge, as `to_square` does, so a dab is round.
let short = source.0.min(source.1).max(1) as f32;
let scale = (source.0.max(1) as f32 / short, source.1.max(1) as f32 / short);
let square = |p: (f32, f32)| (p.0 * scale.0, p.1 * scale.1);
let cells: Vec<(f32, f32)> = lattice().map(square).collect();
let mut out = vec![0.0f32; cells.len()];
for stroke in strokes.iter().filter(|s| !s.is_empty()) {
let points: Vec<(f32, f32)> = stroke.points.iter().copied().map(square).collect();
let r = stroke.radius;
let inner = r * stroke.hardness.clamp(0.0, 1.0);
// Only cells inside the stroke's bounding box can be reached.
let (lo, hi) = points.iter().fold(
((f32::MAX, f32::MAX), (f32::MIN, f32::MIN)),
|(lo, hi), p| ((lo.0.min(p.0), lo.1.min(p.1)), (hi.0.max(p.0), hi.1.max(p.1))),
);
for (cell, dst) in cells.iter().zip(out.iter_mut()) {
if cell.0 < lo.0 - r || cell.0 > hi.0 + r || cell.1 < lo.1 - r || cell.1 > hi.1 + r {
continue;
}
let d = if points.len() == 1 {
dist(*cell, points[0])
} else {
points
.windows(2)
.map(|s| segment_distance(*cell, s[0], s[1]))
.fold(f32::MAX, f32::min)
};
let c = ((1.0 - smoothstep(inner, r, d)) * stroke.flow).clamp(0.0, 1.0);
*dst = if stroke.erase {
*dst * (1.0 - c)
} else {
*dst + c - *dst * c
};
}
}
out
}
fn dist(a: (f32, f32), b: (f32, f32)) -> f32 {
(a.0 - b.0).hypot(a.1 - b.1)
}
fn segment_distance(q: (f32, f32), a: (f32, f32), b: (f32, f32)) -> f32 {
let ab = (b.0 - a.0, b.1 - a.1);
let len2 = ab.0 * ab.0 + ab.1 * ab.1;
if len2 <= 1e-12 {
return dist(q, a);
}
let t = (((q.0 - a.0) * ab.0 + (q.1 - a.1) * ab.1) / len2).clamp(0.0, 1.0);
dist(q, (a.0 + ab.0 * t, a.1 + ab.1 * t))
}
/// WGSL's `smoothstep`, including its behaviour when the edges meet.
fn smoothstep(e0: f32, e1: f32, x: f32) -> f32 {
if e1 <= e0 {
return if x < e0 { 0.0 } else { 1.0 };
}
let t = ((x - e0) / (e1 - e0)).clamp(0.0, 1.0);
t * t * (3.0 - 2.0 * t)
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use super::*;
use crate::coverage::Coverage;
const SOURCE: (u32, u32) = (3000, 2000);
fn no_model(_: &MaskPart) -> Option<Raster<'static>> {
None
}
fn cropped(x: f32, y: f32, w: f32, h: f32) -> Framing {
let mut f = Framing::new();
f.set_crop(CropRect {
x,
y,
width: w,
height: h,
});
f
}
/// A small circle near the source's top-left corner.
fn top_left_circle() -> MaskLayer {
MaskLayer::new(
"m1",
MaskSource::Radial {
centre: (0.15, 0.15),
radii: (0.08, 0.08),
angle: 0.0,
feather: 0.2,
},
)
}
fn stack(layers: Vec<MaskLayer>) -> MaskStack {
let mut s = MaskStack::new();
for l in layers {
assert!(s.push(l));
}
s
}
fn hidden(masks: &MaskStack, before: &Framing, after: &Framing) -> Vec<String> {
hidden_by_crop(masks, before, after, SOURCE, &no_model)
.into_iter()
.map(|l| l.id.clone())
.collect()
}
#[test]
fn a_crop_away_from_a_shape_hides_it() {
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.5, 0.5, 0.5, 0.5);
assert_eq!(hidden(&masks, &Framing::new(), &after), vec!["m1"]);
}
#[test]
fn a_crop_that_keeps_the_shape_is_silent() {
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.0, 0.0, 0.6, 0.6);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_crop_that_trims_part_of_a_shape_is_silent() {
// Half the circle survives, which is a recomposition, not a loss.
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.15, 0.0, 0.85, 1.0);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_layer_already_cropped_away_is_not_reported_again() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
let after = cropped(0.6, 0.6, 0.4, 0.4);
assert!(hidden(&masks, &before, &after).is_empty());
}
#[test]
fn uncropping_brings_a_layer_back_and_says_nothing() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
assert!(hidden(&masks, &before, &Framing::new()).is_empty());
}
#[test]
fn an_unchanged_frame_is_silent_whatever_the_zoom() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
let mut after = before;
after.set_view(CropRect {
x: 0.5,
y: 0.5,
width: 0.5,
height: 0.5,
});
assert!(hidden(&masks, &Framing::new(), &after).len() == 1);
assert!(hidden(&masks, &before, &after).is_empty());
}
#[test]
fn a_range_follows_the_picture_and_is_never_stranded() {
let masks = stack(vec![
MaskLayer::new("lum", MaskSource::highlights()),
MaskLayer::new("skin", MaskSource::skin_tones()),
]);
let after = cropped(0.9, 0.9, 0.1, 0.1);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_linear_gradient_over_the_bottom_is_hidden_by_keeping_the_top() {
let masks = stack(vec![MaskLayer::new(
"grad",
MaskSource::Linear {
centre: (0.5, 0.8),
angle: std::f32::consts::FRAC_PI_2,
width: 0.05,
},
)]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 1.0, 0.5)),
vec!["grad"]
);
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.5, 1.0, 0.5)).is_empty());
}
#[test]
fn a_painted_stroke_is_measured_where_it_was_painted() {
let mut layer = MaskLayer::new("paint", MaskSource::brush());
layer.begin_stroke(0, false, 0.05, 0.8, 1.0);
for i in 0..10 {
layer.extend_stroke(0, 0.8 + i as f32 * 0.01, 0.8);
}
layer.end_stroke(0);
let masks = stack(vec![layer]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.5, 0.5)),
vec!["paint"]
);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn an_unpainted_brush_has_nothing_to_lose() {
let masks = stack(vec![MaskLayer::new("empty", MaskSource::brush())]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.3, 0.3)).is_empty());
}
#[test]
fn an_inverted_layer_is_measured_as_what_it_selects() {
// Everything *but* a corner circle: a crop into the other corner
// keeps most of it.
let mut layer = top_left_circle();
layer.invert = true;
let masks = stack(vec![layer]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn a_subject_is_measured_from_its_stored_coverage_or_the_model() {
// A subject occupying the right-hand quarter of a 40x20 proxy.
let (w, h) = (40usize, 20usize);
let bytes: Vec<u8> = (0..w * h)
.map(|i| if i % w >= 30 { 255 } else { 0 })
.collect();
let subject = MaskSource::Subject {
signature: 1,
index: 0,
class: "dog".into(),
score: 0.9,
};
let left = cropped(0.0, 0.0, 0.5, 1.0);
let mut stored = MaskLayer::new("stored", subject.clone());
stored.base_mut().coverage = Coverage::encode(&bytes, w, h, 2).map(Arc::new);
let live = MaskLayer::new("live", subject);
let masks = stack(vec![stored, live]);
// No model to hand: only the layer carrying its raster is measured.
assert_eq!(hidden(&masks, &Framing::new(), &left), vec!["stored"]);
let model = |_: &MaskPart| {
Some(Raster {
values: Cow::Borrowed(bytes.as_slice()),
width: w,
height: h,
})
};
let ids: Vec<_> = hidden_by_crop(&masks, &Framing::new(), &left, SOURCE, &model)
.into_iter()
.map(|l| l.id.as_str())
.collect();
assert_eq!(ids, vec!["stored", "live"]);
}
#[test]
fn a_crop_is_read_in_the_turned_frame() {
// Turned a quarter clockwise, the source's top-left lands at the
// frame's top-right, so keeping the right half of the frame keeps it.
let masks = stack(vec![top_left_circle()]);
let mut before = Framing::new();
before.rotate_quarters(1);
let mut right = before;
right.set_crop(CropRect {
x: 0.5,
y: 0.0,
width: 0.5,
height: 1.0,
});
let mut left = before;
left.set_crop(CropRect {
x: 0.0,
y: 0.0,
width: 0.5,
height: 1.0,
});
assert!(hidden(&masks, &before, &right).is_empty());
assert_eq!(hidden(&masks, &before, &left), vec!["m1"]);
}
#[test]
fn a_subtracted_range_does_not_hide_the_shape_it_cuts() {
let mut layer = top_left_circle();
assert!(layer.push_part(MaskPart::new(
"p2",
Join::Subtract,
MaskSource::highlights()
)));
let masks = stack(vec![layer.clone()]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)),
vec!["m1"]
);
// Added instead, the range reaches everywhere and nothing is lost.
let mut layer = top_left_circle();
assert!(layer.push_part(MaskPart::new("p2", Join::Union, MaskSource::highlights())));
let masks = stack(vec![layer]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn an_intersection_keeps_only_what_both_parts_cover() {
let circle = |id: &str, join, centre, r| {
MaskPart::new(
id,
join,
MaskSource::Radial {
centre,
radii: (r, r),
angle: 0.0,
feather: 0.2,
},
)
};
// Two circles in opposite corners: keeping the bottom-right quarter
// keeps one of them, and nothing is lost.
let mut layer = top_left_circle();
assert!(layer.push_part(circle("p2", Join::Union, (0.85, 0.85), 0.08)));
let bottom_right = cropped(0.5, 0.5, 0.5, 0.5);
let masks = stack(vec![layer.clone()]);
assert!(hidden(&masks, &Framing::new(), &bottom_right).is_empty());
// Intersected with a disc around the top-left, only that corner's
// circle survives, and the same crop takes it out of the frame.
assert!(layer.push_part(circle("p3", Join::Intersect, (0.15, 0.15), 0.3)));
let masks = stack(vec![layer]);
assert_eq!(hidden(&masks, &Framing::new(), &bottom_right), vec!["m1"]);
}
}