dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Let one mask be built from more than one selection, and paint into it

Browse Source 
A mask the model draws arrives approximately right — stopping inside a
shoulder, leaking into the hair — and FR-DEV-3's edge controls move the
*whole* boundary, so no value of feather or dilation fixes two errors that
go opposite ways. What fixes them is a second selection joined to the first,
and a layer that held exactly one source had nowhere to put one. The brush
the core has had all along was reachable from no control in the application.

A layer is now an ordered list of parts. Each names a source and how it
joins the mask before it — added to it, or taken out of it — and carries its
own edge treatment, because a model's soft coverage and a stroke painted
where it stopped short do not want the same feather. Invert and opacity stay
on the layer, where the composed shader already reads them.

The sidecar grows `[part]` blocks and nothing else. A layer of one part
writes exactly the bytes it always did; a mask block with no part blocks
after it reads back as one part; and a stroke, a join or a source this build
cannot read costs that part rather than the layer. So every sidecar in every
library still parses to the edit it always was.

On the device the parts fold into the layer's one slice, so eight layers
still cost eight channels: union is a `max` blend and subtraction is the
erase blend the brush already used. A part is drawn into a scratch texture
before it is joined, and that is not incidental — an erase stroke means a
hole in *that part*, not a hole in the mask, and drawn straight onto the
accumulator it would punch through the subject underneath. A layer of one
part skips all of it and takes the path it always took.

In the interface: a part list under the selected layer with a chip saying
which way each joins, Add and Subtract beside it, a Select/Paint/Erase strip
with the brush's size, hardness and flow, and a drag on the photograph that
paints. Pressing Paint on a mask that cannot hold a stroke joins a part that
can, rather than explaining that a subject is not a brush. A whole stroke is
one step in the history.

The edge controls now shape the part that is selected rather than the layer,
which is the one behaviour change to an existing control: with a correction
selected, the feather slider softens the correction and leaves the model's
mask alone.
This commit is contained in:
dtourolle
2026-09-07 20:00:40 +02:00
parent 9ede23073d
commit df741a8a49
20 changed files with 2998 additions and 654 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-gpu/src/mask.rs
+397 -102
View File
@@ -41,7 +41,7 @@
//! `region_count`. The compaction is CPU-side and once per image, which is the
//! same place and cadence the region adjacency graph is already built at.
use dr_pipeline::mask::{MaskSource, MaskStack, Stroke, MAX_LAYERS};
use dr_pipeline::mask::{Join, MaskSource, MaskStack, Stroke, MAX_LAYERS};
use wgpu::util::DeviceExt;
@@ -95,6 +95,12 @@ struct MaskParams {
/// The same packing the generated adjust shader uses, so the two agree by
/// construction rather than by inspection.
as_shot_wb: [f32; 4],
/// Whether this part is turned over before it joins the mask. Read by the
/// combine pass and by nothing else — see `fs_combine`.
invert: u32,
/// A uniform buffer is a multiple of sixteen bytes, and the flag above
/// takes four of them.
_pad: [u32; 3],
}
/// One stroke, as `mask.wgsl`'s `StrokeHeader` expects it.
@@ -383,6 +389,19 @@ pub struct MaskPass {
/// `dst(1 - a)` to erase.
brush_add: wgpu::RenderPipeline,
brush_erase: wgpu::RenderPipeline,
/// Reads a part back out of [`Self::scratch`] and blends it into the
/// layer's slice. The set operation is the blend state, so these two are
/// one shader as well.
combine_layout: wgpu::BindGroupLayout,
combine_union: wgpu::RenderPipeline,
combine_subtract: wgpu::RenderPipeline,
/// Where a part is drawn before it is joined.
///
/// One texture for the whole stack rather than one per layer, because
/// layers rasterise in sequence and a part is read back immediately after
/// it is drawn. Allocated the first time a layer has more than one part,
/// so a library of unedited masks never pays for it.
scratch: Option<Scratch>,
array: Option<MaskArray>,
/// How many times the array texture has been (re)allocated.
///
@@ -551,6 +570,88 @@ impl MaskPass {
"mask-brush-add",
blend_state(wgpu::BlendFactor::One, wgpu::BlendFactor::OneMinusSrc),
);
// The pipelines that join one part to the mask so far. The blend
// state is the set operation and the shader is the same three
// vertices either way — which is why adding a way to combine masks
// cost no shader arithmetic at all.
let combine_layout =
ctx.device
.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
label: Some("mask-combine-bgl"),
entries: &[
uniform_entry(0),
wgpu::BindGroupLayoutEntry {
binding: 7,
visibility: wgpu::ShaderStages::FRAGMENT,
ty: wgpu::BindingType::Texture {
// Loaded texel by texel at matching size, so
// there is nothing to filter and no sampler.
sample_type: wgpu::TextureSampleType::Float { filterable: false },
view_dimension: wgpu::TextureViewDimension::D2,
multisampled: false,
},
count: None,
},
],
});
let combine_pipeline_layout =
ctx.device
.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
label: Some("mask-combine-layout"),
bind_group_layouts: &[Some(&combine_layout)],
immediate_size: 0,
});
let combine = |label, blend| {
ctx.device
.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
label: Some(label),
layout: Some(&combine_pipeline_layout),
vertex: wgpu::VertexState {
module: &module,
entry_point: Some("vs"),
compilation_options: Default::default(),
buffers: &[],
},
fragment: Some(wgpu::FragmentState {
module: &module,
entry_point: Some("fs_combine"),
compilation_options: Default::default(),
targets: &[Some(wgpu::ColorTargetState {
format: MaskArray::FORMAT,
blend: Some(blend),
write_mask: wgpu::ColorWrites::ALL,
})],
}),
primitive: wgpu::PrimitiveState::default(),
depth_stencil: None,
multisample: wgpu::MultisampleState::default(),
multiview_mask: None,
cache: None,
})
};
// `max`, not source-over: a union must not build up where two parts
// overlap. Two selections that both half-cover a pixel select it half
// — adding them would make the overlap of two soft edges harder than
// either, which is a seam exactly where a photographer joined two
// things to avoid one.
let combine_union = combine(
"mask-combine-union",
wgpu::BlendState {
color: MAX_BLEND,
alpha: MAX_BLEND,
},
);
// `dst * (1 - src)`, which is the erase blend one level up: what the
// mask had, minus what this part covers, in proportion to how much of
// it the part covers.
let combine_subtract = combine(
"mask-combine-subtract",
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::OneMinusSrc),
);
// The same, with the deposit thrown away: coverage is only ever taken
// off what earlier strokes on this layer put down. There is no negative
// coverage to accumulate, so erasing an unpainted layer is a no-op
@@ -577,6 +678,10 @@ impl MaskPass {
brush_layout,
brush_add,
brush_erase,
combine_layout,
combine_union,
combine_subtract,
scratch: None,
array: None,
allocations: 0,
placeholder,
@@ -617,77 +722,130 @@ impl MaskPass {
});
for (slot, layer) in stack.active().enumerate().take(MAX_LAYERS) {
let field = match (&layer.source, labels) {
(MaskSource::Regions { .. }, None) => {
log::warn!(
"mask layer {} is a region mask with no segmentation loaded; skipping",
layer.id
);
continue;
}
(MaskSource::Regions { .. }, Some(f)) => f,
(_, _) => &self.placeholder,
};
// **The path a mask with one part takes is the path every mask
// took before parts existed**: drawn straight into the layer's
// slice, cleared by the draw itself. Nothing about an unedited
// library's rendering changes, and the scratch texture is never
// allocated for it.
//
// An inverted base is the exception, because turning a part over
// is done where it is read back rather than where it is drawn —
// a brush deposits dabs and cannot know what the rest of the
// frame is. See `fs_combine`.
let direct = layer.parts().len() == 1 && !layer.base().invert;
if !direct {
self.ensure_scratch(width, height)?;
}
// A subject layer whose instance is missing is skipped for the
// same reason a region layer without a segmentation is: an absent
// mask that defaults to "everything" would apply the adjustment to
// the whole photograph, which is a much louder failure than none.
// Indexed by *slot*, not by the instance the layer names: the
// fields are built per layer, in this same order, because two
// layers over one subject can carry different morphology.
let subject = match &layer.source {
// Category alongside Subject: both are model coverage turned
// into a distance field, both are built per layer in this same
// order, and leaving a category out of here is precisely the
// failure the comment above warns about — it binds the 1x1
// placeholder, so the mask covers everything and the
// adjustment silently goes global.
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
match subjects.filter(|s| slot < s.len()) {
Some(s) => (s, slot),
None => {
log::warn!("mask layer {} has no distance field; skipping", layer.id);
continue;
for (index, part) in layer.parts().iter().enumerate() {
let base = index == 0;
let field = match (&part.source, labels) {
(MaskSource::Regions { .. }, None) => {
log::warn!(
"mask layer {} is a region mask with no segmentation loaded; skipping",
layer.id
);
if base {
break;
}
continue;
}
(MaskSource::Regions { .. }, Some(f)) => f,
(_, _) => &self.placeholder,
};
// A subject part whose instance is missing is skipped for the
// same reason a region part without a segmentation is: an
// absent mask that defaults to "everything" would apply the
// adjustment to the whole photograph, which is a much louder
// failure than none.
//
// Indexed by *slot*, not by the instance the part names: the
// fields are built per layer, in this same order, because two
// layers over one subject can carry different morphology.
// Which is also why only a base part can have one — a model
// part joined to a mask has no field built for it yet, and it
// is skipped rather than drawn against a placeholder that
// would cover the frame.
let subject = match &part.source {
// Category alongside Subject: both are model coverage
// turned into a distance field, both are built per layer
// in this same order, and leaving a category out of here
// is precisely the failure the comment above warns about —
// it binds the 1x1 placeholder, so the mask covers
// everything and the adjustment silently goes global.
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
match subjects.filter(|s| base && slot < s.len()) {
Some(s) => (s, slot),
None => {
log::warn!(
"part {} of mask layer {} has no distance field; skipping",
part.id,
layer.id
);
if base {
break;
}
continue;
}
}
}
}
_ => (&self.empty_subject, 0),
};
_ => (&self.empty_subject, 0),
};
// TRACES: FR-DEV-10
// A range layer with no photograph bound is skipped rather than
// drawn against the placeholder, on exactly the rule the two
// cases above follow: an absent mask that defaults to "everything"
// takes a local adjustment global, which is a far quieter failure
// than a layer that visibly did not render.
let image = match (&layer.source, source) {
(s, None) if s.is_range() => {
log::warn!(
"mask layer {} selects a range with no image loaded; skipping",
layer.id
);
continue;
}
(_, image) => image,
};
// TRACES: FR-DEV-10
// A range part with no photograph bound is skipped rather than
// drawn against the placeholder, on exactly the rule the two
// cases above follow: an absent mask that defaults to
// "everything" takes a local adjustment global, which is a far
// quieter failure than a part that visibly did not render.
let image = match (&part.source, source) {
(s, None) if s.is_range() => {
log::warn!(
"mask layer {} selects a range with no image loaded; skipping",
layer.id
);
if base {
break;
}
continue;
}
(_, image) => image,
};
let params = self.params(layer, field, image, width, height);
match &layer.source {
MaskSource::Brush { strokes } => {
self.draw_brush(&mut encoder, slot as u32, &params, strokes, width, height)
let params = self.params(part, field, image, width, height);
let target = if direct {
self.slice_view(slot as u32)
} else {
self.scratch_view()
};
match &part.source {
MaskSource::Brush { strokes } => {
self.draw_brush(&mut encoder, &target, &params, strokes, width, height)
}
_ => {
let selected = self.selection_buffer(part, field);
self.draw(
&mut encoder,
&target,
&params,
field,
&selected,
subject,
image,
);
}
}
_ => {
let selected = self.selection_buffer(layer, field);
self.draw(
&mut encoder,
slot as u32,
&params,
field,
&selected,
subject,
image,
);
if !direct {
// The first part joins a cleared slice, so it lands
// exactly as it was drawn whichever way it says it joins —
// there is nothing yet for a subtraction to take away
// from, and a mask that began by subtracting from nothing
// would render as empty however it was painted afterwards.
let join = if base { Join::Union } else { part.join };
self.combine(&mut encoder, slot as u32, join, base, &params);
}
}
}
@@ -708,7 +866,7 @@ impl MaskPass {
fn params(
&self,
layer: &dr_pipeline::mask::MaskLayer,
part: &dr_pipeline::mask::MaskPart,
field: &LabelField,
source: Option<&DemosaicedImage>,
width: u32,
@@ -758,9 +916,11 @@ impl MaskPass {
[m[6], m[7], m[8], 0.0],
],
as_shot_wb: [wb[0], wb[1], wb[2], if non_linear { 1.0 } else { 0.0 }],
invert: u32::from(part.invert),
_pad: [0; 3],
};
match &layer.source {
match &part.source {
// `softness` carries the layer's feather. The model's coverage is
// already a soft sigmoid, so zero means "use the edge the model
// drew" rather than "hard edge" — the one place in this shader
@@ -780,12 +940,12 @@ impl MaskPass {
MaskParams {
mode: MODE_SUBJECT,
// `softness` is the feather half-width in pixels.
softness: (layer.feather * short).max(0.0),
softness: (part.feather * short).max(0.0),
// `angle` carries the morphology offset — reused rather
// than padded, since a subject layer has no ellipse to
// rotate.
angle: morph_offset(layer) * short,
falloff: falloff_code(layer.falloff),
angle: morph_offset(part) * short,
falloff: falloff_code(part.falloff),
..base
}
}
@@ -871,7 +1031,7 @@ impl MaskPass {
fn draw_brush(
&self,
encoder: &mut wgpu::CommandEncoder,
slot: u32,
target: &wgpu::TextureView,
params: &MaskParams,
strokes: &[Stroke],
width: u32,
@@ -933,19 +1093,10 @@ impl MaskPass {
})
});
let array = self.array.as_ref().expect("array ensured by caller");
let view = array.texture.create_view(&wgpu::TextureViewDescriptor {
label: Some("mask-slice"),
dimension: Some(wgpu::TextureViewDimension::D2),
base_array_layer: slot,
array_layer_count: Some(1),
..Default::default()
});
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("mask-brush-pass"),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view: &view,
view: target,
depth_slice: None,
resolve_target: None,
ops: wgpu::Operations {
@@ -991,11 +1142,11 @@ impl MaskPass {
/// One byte-flag per region, or a single zero for a non-region layer.
fn selection_buffer(
&self,
layer: &dr_pipeline::mask::MaskLayer,
part: &dr_pipeline::mask::MaskPart,
field: &LabelField,
) -> wgpu::Buffer {
let mut flags = vec![0u32; field.region_count.max(1) as usize];
if let MaskSource::Regions { ids, .. } = &layer.source {
if let MaskSource::Regions { ids, .. } = &part.source {
for &id in ids {
if let Some(slot) = flags.get_mut(id as usize) {
*slot = 1;
@@ -1016,7 +1167,7 @@ impl MaskPass {
fn draw(
&self,
encoder: &mut wgpu::CommandEncoder,
slot: u32,
target: &wgpu::TextureView,
params: &MaskParams,
field: &LabelField,
selected: &wgpu::Buffer,
@@ -1072,19 +1223,10 @@ impl MaskPass {
// The array slice is selected by the attachment rather than by a
// uniform the shader reads — one fewer value that can disagree with
// where the pass actually writes.
let array = self.array.as_ref().expect("array ensured by caller");
let view = array.texture.create_view(&wgpu::TextureViewDescriptor {
label: Some("mask-slice"),
dimension: Some(wgpu::TextureViewDimension::D2),
base_array_layer: slot,
array_layer_count: Some(1),
..Default::default()
});
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("mask-pass"),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view: &view,
view: target,
depth_slice: None,
resolve_target: None,
ops: wgpu::Operations {
@@ -1105,6 +1247,140 @@ impl MaskPass {
pass.draw(0..3, 0..1);
}
/// A view of one layer's slice of the array.
fn slice_view(&self, slot: u32) -> wgpu::TextureView {
// The array slice is selected by the attachment rather than by a
// uniform the shader reads — one fewer value that can disagree with
// where the pass actually writes.
let array = self.array.as_ref().expect("array ensured by caller");
array.texture.create_view(&wgpu::TextureViewDescriptor {
label: Some("mask-slice"),
dimension: Some(wgpu::TextureViewDimension::D2),
base_array_layer: slot,
array_layer_count: Some(1),
..Default::default()
})
}
fn scratch_view(&self) -> wgpu::TextureView {
self.scratch
.as_ref()
.expect("scratch ensured by caller")
.texture
.create_view(&wgpu::TextureViewDescriptor {
label: Some("mask-part"),
..Default::default()
})
}
/// Blend the part sitting in [`Self::scratch`] into a layer's slice.
///
/// `first` clears the slice instead of loading it, which is both cheaper
/// on a tiler and the only thing that makes the fold start from nothing
/// covered rather than from whatever the last rasterisation left.
fn combine(
&self,
encoder: &mut wgpu::CommandEncoder,
slot: u32,
join: Join,
first: bool,
params: &MaskParams,
) {
let params_buf = self
.ctx
.device
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
label: Some("mask-combine-params"),
contents: bytemuck::bytes_of(params),
usage: wgpu::BufferUsages::UNIFORM,
});
let bind_group = self
.ctx
.device
.create_bind_group(&wgpu::BindGroupDescriptor {
label: Some("mask-combine-bind"),
layout: &self.combine_layout,
entries: &[
wgpu::BindGroupEntry {
binding: 0,
resource: params_buf.as_entire_binding(),
},
wgpu::BindGroupEntry {
binding: 7,
resource: wgpu::BindingResource::TextureView(&self.scratch_view()),
},
],
});
let target = self.slice_view(slot);
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
label: Some("mask-combine-pass"),
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
view: &target,
depth_slice: None,
resolve_target: None,
ops: wgpu::Operations {
load: if first {
wgpu::LoadOp::Clear(wgpu::Color::BLACK)
} else {
wgpu::LoadOp::Load
},
store: wgpu::StoreOp::Store,
},
})],
depth_stencil_attachment: None,
timestamp_writes: None,
occlusion_query_set: None,
multiview_mask: None,
});
pass.set_pipeline(match join {
Join::Union => &self.combine_union,
Join::Subtract => &self.combine_subtract,
});
pass.set_bind_group(0, &bind_group, &[]);
pass.draw(0..3, 0..1);
}
/// The texture a part is drawn in before it is joined.
///
/// Allocated on the first mask that has more than one part and kept at the
/// rasterisation size, which is the same size the array is: a part and the
/// slice it joins are compared texel for texel, so there is nothing to
/// scale and nothing to sample between.
fn ensure_scratch(&mut self, width: u32, height: u32) -> Result<(), GpuError> {
if self
.scratch
.as_ref()
.is_some_and(|s| s.width == width && s.height == height)
{
return Ok(());
}
let texture = self.ctx.device.create_texture(&wgpu::TextureDescriptor {
label: Some("mask-scratch"),
size: wgpu::Extent3d {
width,
height,
depth_or_array_layers: 1,
},
mip_level_count: 1,
sample_count: 1,
dimension: wgpu::TextureDimension::D2,
format: MaskArray::FORMAT,
usage: wgpu::TextureUsages::RENDER_ATTACHMENT | wgpu::TextureUsages::TEXTURE_BINDING,
view_formats: &[],
});
self.scratch = Some(Scratch {
texture,
width,
height,
});
Ok(())
}
fn ensure_array(&mut self, width: u32, height: u32, layers: u32) -> Result<(), GpuError> {
if self
.array
@@ -1147,6 +1423,25 @@ impl MaskPass {
}
}
/// The texture one part is drawn into on its way into a layer's slice.
struct Scratch {
texture: wgpu::Texture,
width: u32,
height: u32,
}
/// `max(dst, src)` — the union of two parts.
///
/// Not source-over, which would build up: two parts that each half-cover a
/// pixel select it half, and adding them would make the overlap of two soft
/// edges harder than either of them, drawing a seam exactly where a
/// photographer joined two selections to avoid one.
const MAX_BLEND: wgpu::BlendComponent = wgpu::BlendComponent {
src_factor: wgpu::BlendFactor::One,
dst_factor: wgpu::BlendFactor::One,
operation: wgpu::BlendOperation::Max,
};
/// TRACES: FR-DEV-10
/// A single black texel, bound at the image slot for a mask that is not a
/// range.
@@ -1192,11 +1487,11 @@ fn field_short_edge(width: u32, height: u32) -> f32 {
/// Zero for closing and opening: those are folded into the field itself when
/// it is built, because their second half acts on a shape the original field
/// does not describe.
fn morph_offset(layer: &dr_pipeline::mask::MaskLayer) -> f32 {
fn morph_offset(part: &dr_pipeline::mask::MaskPart) -> f32 {
use dr_pipeline::mask::Morphology;
match layer.morphology {
Morphology::Dilate => layer.morph_radius,
Morphology::Erode => -layer.morph_radius,
match part.morphology {
Morphology::Dilate => part.morph_radius,
Morphology::Erode => -part.morph_radius,
Morphology::None | Morphology::Close | Morphology::Open => 0.0,
}
}
@@ -1362,11 +1657,11 @@ mod tests {
fn painted(gestures: &[Gesture]) -> MaskLayer {
let mut layer = lit(MaskSource::brush());
for (erase, radius, path) in gestures {
layer.begin_stroke(*erase, *radius, 0.5, 1.0);
layer.begin_stroke(0, *erase, *radius, 0.5, 1.0);
for &(x, y) in path {
layer.extend_stroke(x, y);
layer.extend_stroke(0, x, y);
}
layer.end_stroke();
layer.end_stroke(0);
}
layer
}