Put the region map behind the sliders that were already there
A mask layer holds a real develop chain, so the develop panel can edit one with no new controls: select a layer and the same sliders read and write its chain instead of the graph's. An operation declared in `ops/` tomorrow becomes locally adjustable by existing, which is the payoff for making a layer a chain rather than a handful of special-cased parameters. `segmentation.rs` joins the two arms into the one thing the view needs. The model reads the image through a neutral graph rather than the edited one, so a segmentation survives an exposure change instead of being invalidated by every slider. Arm B failing is not fatal: a missing or unreadable model leaves a working watershed map, because refusing to segment at all would trade a working feature for a strict one. The overlay colours groups by a golden-angle walk over hue. Deterministic rather than random, so a region keeps its colour across a level change and the eye can track it; boundaries drawn black over the fill, because two adjacent groups landing on near hues read as one region and telling them apart is the whole reason to look at it. Clicking the photograph creates the layer if none is selected — that is how a local adjustment begins, and making the user press "add layer" first would be a step with no decision in it. Shift-click extends, and clicking a region already selected removes it, so one gesture both adds and corrects. `segment-readback` is a new dr-gpu feature and not a loosening of `readback`. The region-graph transfer is once per image on a worker; the one AC-8 forbids is per frame in the render loop. Sharing a switch would have forced a build wanting local masking to unlock the other. F3 still stands and the feature name says so.
This commit is contained in:
+444
-6
@@ -10,7 +10,12 @@
|
||||
//! new operation appears in the panel with no change here (FR-DEV-3c).
|
||||
|
||||
use dr_decode::RawImage;
|
||||
use dr_gpu::{AdjustPass, DemosaicedImage, Demosaicer, GpuContext, Histogram, HistogramPass};
|
||||
use dr_gpu::{
|
||||
AdjustPass, DemosaicedImage, Demosaicer, GpuContext, Histogram, HistogramPass, MaskPass,
|
||||
};
|
||||
use dr_pipeline::mask::{MaskLayer, MaskSource};
|
||||
|
||||
use crate::segmentation::{self, Segmentation};
|
||||
use dr_pipeline::ops::curve;
|
||||
use dr_pipeline::{
|
||||
CropRect, Edit, EditGraph, History, OpCapability, OpId, ParamId, ParamKind, Presentation,
|
||||
@@ -40,6 +45,24 @@ pub struct DevelopSession {
|
||||
/// old driver, a device without the storage-buffer atomics it needs — the
|
||||
/// photographer loses the histogram and keeps the photograph.
|
||||
histogram: Option<HistogramPass>,
|
||||
|
||||
/// TRACES: FR-DEV-3
|
||||
/// The region map local masks select from, once it has been computed.
|
||||
///
|
||||
/// `None` until the photographer asks for it. Segmentation costs about
|
||||
/// half a second and most edits never need one, so running it on open
|
||||
/// would tax every photograph for a feature used on some of them.
|
||||
segmentation: Option<Segmentation>,
|
||||
/// Rasterises the mask layers. Built lazily for the same reason.
|
||||
masks: Option<MaskPass>,
|
||||
/// Which layer the develop panel is editing, if any.
|
||||
///
|
||||
/// This is what lets one panel serve both scopes: with a layer selected,
|
||||
/// the sliders read and write *its* chain, and the photographer is
|
||||
/// adjusting a region rather than the frame.
|
||||
active_mask: Option<String>,
|
||||
/// Whether to draw the false-coloured region overlay.
|
||||
show_overlay: bool,
|
||||
}
|
||||
|
||||
impl DevelopSession {
|
||||
@@ -97,6 +120,10 @@ impl DevelopSession {
|
||||
histogram: HistogramPass::new(ctx)
|
||||
.inspect_err(|e| log::warn!("no histogram on this device: {e}"))
|
||||
.ok(),
|
||||
segmentation: None,
|
||||
masks: None,
|
||||
active_mask: None,
|
||||
show_overlay: false,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -105,7 +132,24 @@ impl DevelopSession {
|
||||
/// Built entirely from the capability list. The `kind` string chooses the
|
||||
/// widget; nothing switches on a parameter's identity.
|
||||
pub fn rows(&self) -> Vec<ParamRow> {
|
||||
rows_from(&self.graph.capabilities())
|
||||
match self.active_layer() {
|
||||
// A selected layer takes over the panel. The rows are built from
|
||||
// the layer's own capability list, so every control the global
|
||||
// chain offers is offered here too — including ones added later,
|
||||
// which need no work to become local.
|
||||
Some(layer) => rows_from(&layer.capabilities()),
|
||||
None => rows_from(&self.graph.capabilities()),
|
||||
}
|
||||
}
|
||||
|
||||
fn active_layer(&self) -> Option<&MaskLayer> {
|
||||
let id = self.active_mask.as_ref()?;
|
||||
self.graph.masks().get(id)
|
||||
}
|
||||
|
||||
fn active_layer_mut(&mut self) -> Option<&mut MaskLayer> {
|
||||
let id = self.active_mask.clone()?;
|
||||
self.graph.masks_mut().get_mut(&id)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -463,10 +507,24 @@ impl DevelopSession {
|
||||
/// here is curve-shaped; it walks whatever parameters the operation
|
||||
/// declares.
|
||||
pub fn reset_op(&mut self, op_index: i32) {
|
||||
let caps = self.graph.capabilities();
|
||||
let caps = match self.active_layer() {
|
||||
Some(layer) => layer.capabilities(),
|
||||
None => self.graph.capabilities(),
|
||||
};
|
||||
let Some(cap) = usize::try_from(op_index).ok().and_then(|i| caps.get(i)) else {
|
||||
return;
|
||||
};
|
||||
if self.active_mask.is_some() {
|
||||
let params: Vec<_> = cap.params.iter().map(|p| (p.id, p.default)).collect();
|
||||
let id = cap.id.0;
|
||||
if let Some(layer) = self.active_layer_mut() {
|
||||
for (param, default) in params {
|
||||
layer.set_param(id, param, default);
|
||||
}
|
||||
}
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
return;
|
||||
}
|
||||
for p in &cap.params {
|
||||
self.graph.set_param(cap.id, p.id, p.default);
|
||||
}
|
||||
@@ -493,6 +551,17 @@ impl DevelopSession {
|
||||
log::warn!("control at ({op_index}, {param_index}) has no parameter");
|
||||
return;
|
||||
};
|
||||
if self.active_mask.is_some() {
|
||||
if let Some(layer) = self.active_layer_mut() {
|
||||
layer.set_param(op.0, param, value);
|
||||
}
|
||||
// Coalesced the same way a global drag is: a slider dragged across
|
||||
// a masked layer is still one gesture and must undo as one.
|
||||
let edit = Edit::for_param(&self.graph, op, param);
|
||||
self.history.record(&self.graph, edit);
|
||||
return;
|
||||
}
|
||||
|
||||
self.graph.set_param(op, param, value);
|
||||
let edit = Edit::for_param(&self.graph, op, param);
|
||||
self.history.record(&self.graph, edit);
|
||||
@@ -520,10 +589,372 @@ impl DevelopSession {
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
}
|
||||
|
||||
/// Rasterise the current mask stack, if there is one.
|
||||
///
|
||||
/// Returns `None` for a stack with no active layers, which is the common
|
||||
/// case and the one that must cost nothing: the adjust pass then binds its
|
||||
/// own placeholder and the generated shader has no layer block to read it
|
||||
/// with.
|
||||
/// Returns whether the array is now valid for the current stack.
|
||||
///
|
||||
/// Split from reading the array back because the render below needs
|
||||
/// `self.adjust` mutably while holding `self.masks` immutably. Those are
|
||||
/// disjoint fields and the borrow checker will allow it — but only when
|
||||
/// each is reached directly rather than through a method taking `self`.
|
||||
fn rasterise_masks(&mut self, w: u32, h: u32) -> bool {
|
||||
if self.graph.masks().is_neutral() {
|
||||
return false;
|
||||
}
|
||||
let labels = self.segmentation.as_ref().and_then(|s| s.labels());
|
||||
let Some(pass) = self.masks.as_mut() else {
|
||||
return false;
|
||||
};
|
||||
pass.render(self.graph.masks(), labels, w, h)
|
||||
.inspect_err(|e| log::warn!("mask rasterisation failed: {e}"))
|
||||
.is_ok()
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// Segmentation (S15, docs/segmentation.md)
|
||||
// ----------------------------------------------------------------------
|
||||
|
||||
/// TRACES: FR-DEV-3
|
||||
/// Compute the region map this image's local masks select from.
|
||||
///
|
||||
/// **Blocking, and roughly half a second.** The caller is responsible for
|
||||
/// running it off the UI thread — see the worker in `lib.rs`. It is
|
||||
/// exposed as a plain blocking call rather than something async because
|
||||
/// what it needs is a GPU context and a CPU core, not a runtime.
|
||||
pub fn segment(&mut self, ctx: &GpuContext, options: &segmentation::Options) -> Result<(), String> {
|
||||
// The model reads the photograph as captured, not as edited: the
|
||||
// segmentation must survive an exposure change, or every slider would
|
||||
// invalidate the masks that depend on it (docs/segmentation.md §3).
|
||||
let (rgb, rw, rh) = self.neutral_proxy(ctx, options.segment.max_edge)?;
|
||||
let seg = segmentation::compute(ctx, &self.demosaiced, &rgb, rw, rh, options)?;
|
||||
|
||||
if self.masks.is_none() {
|
||||
self.masks = MaskPass::new(ctx)
|
||||
.inspect_err(|e| log::warn!("no mask rasteriser on this device: {e}"))
|
||||
.ok();
|
||||
}
|
||||
self.segmentation = Some(seg);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Render the *unedited* image to a CPU buffer at proxy size.
|
||||
///
|
||||
/// Goes through a throwaway [`AdjustPass`] with a neutral graph rather
|
||||
/// than the session's own. Reusing `self.adjust` would overwrite the frame
|
||||
/// the histogram reads and leave the view showing an unedited image until
|
||||
/// the next redraw — a visible flicker for the sake of not allocating.
|
||||
///
|
||||
/// This is `export_pixels`, which is ungated: an export is not the display
|
||||
/// round-trip AC-8 forbids, and neither is this.
|
||||
fn neutral_proxy(
|
||||
&self,
|
||||
ctx: &GpuContext,
|
||||
max_edge: u32,
|
||||
) -> Result<(Vec<f32>, usize, usize), String> {
|
||||
let (sw, sh) = self.demosaiced.size();
|
||||
let scale = (max_edge as f32 / sw.max(sh) as f32).min(1.0);
|
||||
let (w, h) = (
|
||||
((sw as f32 * scale) as u32).max(1),
|
||||
((sh as f32 * scale) as u32).max(1),
|
||||
);
|
||||
|
||||
let neutral = EditGraph::default_chain();
|
||||
let mut pass = AdjustPass::new(ctx);
|
||||
pass.render(&self.demosaiced, &neutral.compose(), w, h)
|
||||
.map_err(|e| format!("could not render the segmentation proxy: {e}"))?;
|
||||
let (rgba, pw, ph) = pass
|
||||
.export_pixels()
|
||||
.map_err(|e| format!("could not read the segmentation proxy: {e}"))?;
|
||||
|
||||
// Straight to float RGB, dropping alpha. The values stay display-
|
||||
// encoded because that is what the model was trained on — one of the
|
||||
// few places in this codebase where not linearising is correct.
|
||||
let rgb = rgba
|
||||
.chunks_exact(4)
|
||||
.flat_map(|p| [p[0] as f32 / 255.0, p[1] as f32 / 255.0, p[2] as f32 / 255.0])
|
||||
.collect();
|
||||
Ok((rgb, pw as usize, ph as usize))
|
||||
}
|
||||
|
||||
pub fn has_segmentation(&self) -> bool {
|
||||
self.segmentation.is_some()
|
||||
}
|
||||
|
||||
pub fn segmentation_level(&self) -> u32 {
|
||||
self.segmentation.as_ref().map_or(0, |s| s.level())
|
||||
}
|
||||
|
||||
pub fn segmentation_region_count(&self) -> usize {
|
||||
self.segmentation.as_ref().map_or(0, |s| s.region_count())
|
||||
}
|
||||
|
||||
/// Move the granularity ladder — the scroll wheel over the canvas.
|
||||
pub fn set_segmentation_level(&mut self, level: u32) {
|
||||
if let Some(seg) = self.segmentation.as_mut() {
|
||||
seg.set_level(level);
|
||||
}
|
||||
}
|
||||
|
||||
/// The subjects the model recognised, as `(label, confidence)`.
|
||||
///
|
||||
/// Confidence is shown rather than hidden because the detector is offered
|
||||
/// as a shortcut, not as an authority: a 0.42 "dog" is worth listing and
|
||||
/// worth flagging, and a list that presented it identically to a 0.95 one
|
||||
/// would make the tool look wrong when the guess was merely weak.
|
||||
pub fn detected_subjects(&self) -> Vec<(String, f32)> {
|
||||
self.segmentation
|
||||
.as_ref()
|
||||
.map(|s| {
|
||||
s.instances()
|
||||
.iter()
|
||||
.map(|i| (i.class_name.to_string(), i.score))
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// The region overlay
|
||||
// ----------------------------------------------------------------------
|
||||
|
||||
pub fn overlay_enabled(&self) -> bool {
|
||||
self.show_overlay
|
||||
}
|
||||
|
||||
pub fn set_overlay(&mut self, on: bool) {
|
||||
self.show_overlay = on;
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3
|
||||
/// A false-coloured picture of the current grouping, for the canvas.
|
||||
///
|
||||
/// Returned as a CPU image rather than a texture, and deliberately: it is
|
||||
/// regenerated only when the level changes, it is proxy-sized rather than
|
||||
/// viewport-sized, and Slint scales and composites it for free. Putting it
|
||||
/// on the GPU would buy nothing and add a second texture to keep in step
|
||||
/// with the view.
|
||||
///
|
||||
/// `None` when the overlay is off or nothing has been segmented, so the
|
||||
/// caller can bind this straight to an image source.
|
||||
pub fn overlay_image(&self) -> Option<slint::Image> {
|
||||
if !self.show_overlay {
|
||||
return None;
|
||||
}
|
||||
let (rgba, w, h) = self.segmentation.as_ref()?.overlay_rgba();
|
||||
let buffer = slint::SharedPixelBuffer::<slint::Rgba8Pixel>::clone_from_slice(&rgba, w, h);
|
||||
Some(slint::Image::from_rgba8(buffer))
|
||||
}
|
||||
|
||||
// ----------------------------------------------------------------------
|
||||
// Mask layers
|
||||
// ----------------------------------------------------------------------
|
||||
|
||||
/// The layers, as `(id, name, enabled, is_active_selection)`.
|
||||
pub fn mask_layers(&self) -> Vec<(String, String, bool, bool)> {
|
||||
self.graph
|
||||
.masks()
|
||||
.layers()
|
||||
.iter()
|
||||
.map(|l| {
|
||||
(
|
||||
l.id.clone(),
|
||||
l.display_name().to_string(),
|
||||
l.enabled,
|
||||
Some(&l.id) == self.active_mask.as_ref(),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
pub fn active_mask(&self) -> Option<&str> {
|
||||
self.active_mask.as_deref()
|
||||
}
|
||||
|
||||
/// Select a layer for editing, or `None` to return the panel to the
|
||||
/// global chain.
|
||||
pub fn set_active_mask(&mut self, id: Option<&str>) {
|
||||
self.active_mask = id
|
||||
.filter(|id| self.graph.masks().get(id).is_some())
|
||||
.map(|id| id.to_string());
|
||||
}
|
||||
|
||||
/// Select the region under a normalised image point.
|
||||
///
|
||||
/// `add` extends the selected layer instead of replacing its selection,
|
||||
/// which is the shift-click every selection tool has. With no layer
|
||||
/// selected a new one is created, because clicking the photograph is how a
|
||||
/// local adjustment begins and requiring "add layer" first would be a step
|
||||
/// with no decision in it.
|
||||
///
|
||||
/// Returns the layer that now holds the selection.
|
||||
pub fn select_region_at(&mut self, x: f32, y: f32, add: bool) -> Option<String> {
|
||||
let seg = self.segmentation.as_ref()?;
|
||||
let picked = seg.regions_at(x, y);
|
||||
if picked.is_empty() {
|
||||
return None;
|
||||
}
|
||||
let (signature, level) = (seg.signature(), seg.level());
|
||||
|
||||
let id = match self.active_mask.clone() {
|
||||
Some(id) => id,
|
||||
None => {
|
||||
let id = self.graph.masks().next_id();
|
||||
let layer = MaskLayer::new(
|
||||
id.clone(),
|
||||
MaskSource::Regions {
|
||||
signature,
|
||||
level,
|
||||
ids: Vec::new(),
|
||||
},
|
||||
);
|
||||
if !self.graph.masks_mut().push(layer) {
|
||||
return None;
|
||||
}
|
||||
self.active_mask = Some(id.clone());
|
||||
id
|
||||
}
|
||||
};
|
||||
|
||||
let layer = self.graph.masks_mut().get_mut(&id)?;
|
||||
let mut ids = match (&layer.source, add) {
|
||||
(MaskSource::Regions { ids, .. }, true) => ids.clone(),
|
||||
_ => Vec::new(),
|
||||
};
|
||||
|
||||
// Clicking a region already in the selection removes it, so one
|
||||
// gesture both adds and corrects — the alternative is a modifier for
|
||||
// subtract that nobody remembers.
|
||||
if add && picked.iter().all(|r| ids.contains(r)) {
|
||||
ids.retain(|r| !picked.contains(r));
|
||||
} else {
|
||||
ids.extend(picked);
|
||||
}
|
||||
ids.sort_unstable();
|
||||
ids.dedup();
|
||||
|
||||
layer.source = MaskSource::Regions {
|
||||
signature,
|
||||
level,
|
||||
ids,
|
||||
};
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// Add a layer selecting one detected subject.
|
||||
pub fn add_subject_mask(&mut self, index: usize) -> Option<String> {
|
||||
let seg = self.segmentation.as_ref()?;
|
||||
let instance = seg.instances().get(index)?;
|
||||
let (signature, level) = (seg.signature(), seg.level());
|
||||
let (name, ids) = (instance.class_name.to_string(), instance.regions.clone());
|
||||
|
||||
let id = self.graph.masks().next_id();
|
||||
let mut layer = MaskLayer::new(
|
||||
id.clone(),
|
||||
MaskSource::Regions {
|
||||
signature,
|
||||
level,
|
||||
ids,
|
||||
},
|
||||
);
|
||||
layer.name = name;
|
||||
if !self.graph.masks_mut().push(layer) {
|
||||
return None;
|
||||
}
|
||||
self.active_mask = Some(id.clone());
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
Some(id)
|
||||
}
|
||||
|
||||
/// Add a gradient layer, which needs no segmentation.
|
||||
pub fn add_gradient_mask(&mut self, radial: bool) -> Option<String> {
|
||||
let id = self.graph.masks().next_id();
|
||||
let source = if radial {
|
||||
MaskSource::Radial {
|
||||
centre: (0.5, 0.5),
|
||||
radii: (0.35, 0.35),
|
||||
angle: 0.0,
|
||||
feather: 0.5,
|
||||
}
|
||||
} else {
|
||||
MaskSource::Linear {
|
||||
centre: (0.5, 0.5),
|
||||
angle: std::f32::consts::FRAC_PI_2,
|
||||
width: 0.3,
|
||||
}
|
||||
};
|
||||
if !self.graph.masks_mut().push(MaskLayer::new(id.clone(), source)) {
|
||||
return None;
|
||||
}
|
||||
self.active_mask = Some(id.clone());
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
Some(id)
|
||||
}
|
||||
|
||||
pub fn remove_mask(&mut self, id: &str) {
|
||||
if self.graph.masks_mut().remove(id).is_some() {
|
||||
if self.active_mask.as_deref() == Some(id) {
|
||||
self.active_mask = None;
|
||||
}
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set_mask_enabled(&mut self, id: &str, enabled: bool) {
|
||||
if let Some(layer) = self.graph.masks_mut().get_mut(id) {
|
||||
layer.enabled = enabled;
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set_mask_invert(&mut self, id: &str, invert: bool) {
|
||||
if let Some(layer) = self.graph.masks_mut().get_mut(id) {
|
||||
layer.invert = invert;
|
||||
self.history.record(&self.graph, Edit::Discrete);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn set_mask_opacity(&mut self, id: &str, opacity: f32) {
|
||||
if let Some(layer) = self.graph.masks_mut().get_mut(id) {
|
||||
layer.opacity = opacity.clamp(0.0, 1.0);
|
||||
// `Op` rather than `Discrete`: opacity is dragged, and a drag is
|
||||
// one decision however many values it passes through. `Discrete`
|
||||
// would put every intermediate position on the undo stack.
|
||||
self.history.record(&self.graph, Edit::Op(OpId("mask-opacity")));
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a layer's region ids belong to a segmentation other than the
|
||||
/// one currently loaded — a mask restored from a sidecar written under
|
||||
/// different tuning.
|
||||
pub fn mask_is_stale(&self, id: &str) -> bool {
|
||||
let Some(layer) = self.graph.masks().get(id) else {
|
||||
return false;
|
||||
};
|
||||
match self.segmentation.as_ref() {
|
||||
Some(seg) => layer.is_stale(seg.signature()),
|
||||
// Nothing loaded to compare against. Not stale, just unrenderable
|
||||
// — the distinction matters because "stale" invites the user to
|
||||
// recompute the selection and this only needs the segmentation
|
||||
// running.
|
||||
None => false,
|
||||
}
|
||||
}
|
||||
|
||||
fn lookup(&self, op_index: i32, param_index: i32) -> Option<(OpId, ParamId)> {
|
||||
// Rows are emitted in capability order, so the flat index is the sum
|
||||
// of preceding parameter counts.
|
||||
let caps = self.graph.capabilities();
|
||||
// of preceding parameter counts. Taken from whichever scope `rows`
|
||||
// last described — the indices the interface is holding are positions
|
||||
// in *that* list, and reading the global chain while a layer is
|
||||
// selected would map a slider onto a different operation.
|
||||
let caps = match self.active_layer() {
|
||||
Some(layer) => layer.capabilities(),
|
||||
None => self.graph.capabilities(),
|
||||
};
|
||||
let op = caps.get(usize::try_from(op_index).ok()?)?;
|
||||
let param = op.params.get(usize::try_from(param_index).ok()?)?;
|
||||
Some((op.id, param.id))
|
||||
@@ -560,9 +991,16 @@ impl DevelopSession {
|
||||
let (w, h) = fit(fw, fh, width.max(1), height.max(1));
|
||||
|
||||
let shader = self.graph.compose();
|
||||
|
||||
// Rasterise the masks first: the shader addresses array slices by
|
||||
// index, so the array has to describe *this* stack before it is bound.
|
||||
let masks = self
|
||||
.rasterise_masks(w, h)
|
||||
.then(|| self.masks.as_ref().and_then(|p| p.array()))
|
||||
.flatten();
|
||||
let texture = self
|
||||
.adjust
|
||||
.render(&self.demosaiced, &shader, w, h)
|
||||
.render_masked(&self.demosaiced, &shader, w, h, masks)
|
||||
.map_err(|e| e.to_string())?;
|
||||
|
||||
// The import is fallible on format and usage only, and both are fixed
|
||||
|
||||
Reference in New Issue
Block a user