Paint a mask without ever rasterising one on the CPU
The last line of FR-DEV-3, and the mask ARCH §5.4 was written for. darktable rasterises drawn masks on the CPU and users call the result unworkable; the architecture's answer is that a stroke arrives as *parameters* and the device draws it. This is that, from the model through the sidecar to the pixels — but not the finger: the canvas is somebody else's change, and this leaves it a seam rather than reaching into it. **A stroke is a swept disc along a polyline**, plus erase, radius, hardness and flow. `MaskSource::Brush` holds an ordered list of them, and the order is the mask: an erase after an add takes it away and the same pair reversed does not. Nothing about it is pixels, which is what makes a mask that costs a line of text, diffs by the gesture, and survives a crop, a straighten and an export at any size — the properties a stored raster has none of, and the same argument the region ids were chosen for. Two things keep the point count honest. While the finger is down, a position closer to the last than an eighth of the radius is dropped: a touch screen reports 120 a second, so a finger held still for five seconds is six hundred points in the same place, and simplification would only remove them once the gesture had ended — after every frame in between had drawn all of them. When it ends, Douglas–Peucker at an eighth of the radius removes what a disc that wide cannot express: a swept circle moved by r/8 moves its own edge by r/8, which is inside the soft part of any brush. Coordinates snap to a ten-thousandth of the frame on the way in *and* are written at that precision, so a round trip is exact rather than nearly exact — a file that drifts in the sixth decimal every save is a per-field merge conflict a day, over nothing. **Cost is why the strokes are not drawn by the full-screen triangle the other masks use.** A swept disc is the minimum distance to any of its segments, so a stroke over the whole frame costs `pixels × segments` and both terms grow together — the quadratic that is darktable's problem moved onto the GPU rather than solved. Each stroke is instead drawn over its own bounding box, grown by the radius, so the rasteriser never invokes the shader for a pixel the stroke cannot reach: `area(box) × segments`, which for a dab or a swipe is a small fraction of the frame. A gesture past 256 points continues as a second stroke for the same reason, since a shorter stroke has a smaller box. Add and erase are `dst + a(1 - dst)` and `dst(1 - a)`, which are exactly a source-over and a one-minus-source blend — so they are blend state, not arithmetic, and no pass ever reads the slice it is writing. That is what permits one draw per stroke at all. Within a stroke the coverage is the *minimum* distance over its segments rather than a sum: a path that crosses itself must not build up where it did, or every circle and every scribble would be blotchy wherever consecutive dabs overlap, which is everywhere. Not a distance field, deliberately. `dr-segment`'s transform documents the two conditions that make CPU work right there — once per mask edit, over input already CPU-side — and a stroke fails both: it changes while the finger moves, and its input is a handful of coordinates that never needed to be pixels. It also needs no transform, because the distance to a swept disc is closed form. A stroke is the one mask whose distance field is known without computing one. An unpainted brush layer is inactive rather than empty, which is not an optimisation: `invert` turns empty into everything, so a layer created with invert already set would apply its adjustment to the whole photograph before a single stroke was made. That is the loud, confident kind of wrong this codebase refuses everywhere else a mask can go missing, and there is a rendered test for it. The tests read pixels back off a device rather than checking that the two halves agree with each other. What they pin down is what is silent when wrong: the y flip between mask space and clip space, which a centred stroke would not notice; a bounding box not grown by the radius, which makes a tap draw nothing at all; an aspect ratio ignored, which makes a dab an ellipse on any frame that is not square; a stroke doubling back and building up; and an erase that lost its place in the order and put back paint the user had taken off. Not done here: the interaction. The canvas needs to begin, extend and end a stroke on the active layer, and `DevelopSession::rasterise_masks` still returns early without a segmentation — it takes the proxy size from one, and a brush needs no model to have run over the photograph first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
+434
-5
@@ -9,8 +9,17 @@
|
||||
//! Rasterising is **not** on the slider path. Dragging exposure on a masked
|
||||
//! layer changes uniforms only; the mask array is reused untouched. This pass
|
||||
//! runs when a mask's *shape* changes — a different selection, a moved
|
||||
//! gradient, a resized output — which is what keeps a local adjustment as
|
||||
//! responsive as a global one.
|
||||
//! gradient, a new stroke, a resized output — which is what keeps a local
|
||||
//! adjustment as responsive as a global one.
|
||||
//!
|
||||
//! # The two shapes of pass
|
||||
//!
|
||||
//! A parametric mask is a function of the whole frame, so it is one full-screen
|
||||
//! triangle. A brush is not: a stroke reaches a bounded part of the picture,
|
||||
//! and drawing it over the whole frame would cost `pixels × segments` for a
|
||||
//! mark the size of a thumb. So strokes are drawn over their own bounding
|
||||
//! boxes, one draw each, compositing onto the slice with blend state — see the
|
||||
//! second half of `mask.wgsl`.
|
||||
//!
|
||||
//! # The label field
|
||||
//!
|
||||
@@ -21,7 +30,7 @@
|
||||
//! `region_count`. The compaction is CPU-side and once per image, which is the
|
||||
//! same place and cadence the region adjacency graph is already built at.
|
||||
|
||||
use dr_pipeline::mask::{MaskSource, MaskStack, MAX_LAYERS};
|
||||
use dr_pipeline::mask::{MaskSource, MaskStack, Stroke, MAX_LAYERS};
|
||||
use wgpu::util::DeviceExt;
|
||||
|
||||
use crate::{GpuContext, GpuError};
|
||||
@@ -31,6 +40,12 @@ const MODE_REGIONS: u32 = 0;
|
||||
const MODE_LINEAR: u32 = 1;
|
||||
const MODE_RADIAL: u32 = 2;
|
||||
const MODE_SUBJECT: u32 = 3;
|
||||
/// Brush layers go through their own entry points rather than the `switch`, so
|
||||
/// this is only ever read by a person looking at a captured frame.
|
||||
const MODE_BRUSH: u32 = 4;
|
||||
|
||||
/// Six vertices — two triangles — per stroke. See `vs_brush`.
|
||||
const VERTICES_PER_STROKE: u32 = 6;
|
||||
|
||||
#[repr(C)]
|
||||
#[derive(Copy, Clone, bytemuck::Pod, bytemuck::Zeroable)]
|
||||
@@ -54,6 +69,105 @@ struct MaskParams {
|
||||
_pad1: [f32; 2],
|
||||
}
|
||||
|
||||
/// One stroke, as `mask.wgsl`'s `StrokeHeader` expects it.
|
||||
///
|
||||
/// The bounding box is computed here rather than in the shader because the
|
||||
/// vertex stage needs it before there is anything to compute it from — that is
|
||||
/// the whole trick: the box is what stops the fragment shader running over
|
||||
/// pixels the stroke cannot reach. Finding it is a pass over a few hundred
|
||||
/// coordinates, which is not rasterising a mask on the CPU by any reading of
|
||||
/// ARCH §5.4: no pixel is produced, and the output is four floats.
|
||||
#[repr(C)]
|
||||
#[derive(Copy, Clone, bytemuck::Pod, bytemuck::Zeroable)]
|
||||
struct StrokeHeader {
|
||||
lo: [f32; 2],
|
||||
hi: [f32; 2],
|
||||
radius: f32,
|
||||
hardness: f32,
|
||||
flow: f32,
|
||||
first: u32,
|
||||
count: u32,
|
||||
_pad: u32,
|
||||
}
|
||||
|
||||
/// The strokes of one layer, packed for the shader.
|
||||
///
|
||||
/// Empty when the layer has nothing to draw, which is not the same as an error:
|
||||
/// a brush layer with no strokes is a mask covering nothing, and a mask
|
||||
/// covering nothing is what an unpainted layer should be.
|
||||
struct StrokeBatch {
|
||||
headers: Vec<StrokeHeader>,
|
||||
points: Vec<[f32; 2]>,
|
||||
/// Whether each header erases, in step with `headers`. Not in the header
|
||||
/// itself because it selects a *pipeline* rather than a value the shader
|
||||
/// reads: add and erase are two blend states over one fragment shader.
|
||||
erases: Vec<bool>,
|
||||
}
|
||||
|
||||
impl StrokeBatch {
|
||||
/// Pack `strokes` for a mask of `width`×`height`.
|
||||
fn pack(strokes: &[Stroke], width: u32, height: u32) -> Self {
|
||||
let short = field_short_edge(width, height);
|
||||
// Back out of shorter-edge units into normalised ones, per axis. The
|
||||
// radius is a fraction of the shorter edge, so on a landscape frame it
|
||||
// is a smaller fraction of the width than of the height, and growing
|
||||
// the box by the same amount in both would clip the ends of a stroke
|
||||
// along the long axis.
|
||||
let margin = |extent: u32| short / extent.max(1) as f32;
|
||||
let (mx, my) = (margin(width), margin(height));
|
||||
let texel = (1.0 / width.max(1) as f32).max(1.0 / height.max(1) as f32);
|
||||
|
||||
let mut out = Self {
|
||||
headers: Vec::with_capacity(strokes.len()),
|
||||
points: Vec::new(),
|
||||
erases: Vec::with_capacity(strokes.len()),
|
||||
};
|
||||
|
||||
for stroke in strokes {
|
||||
if stroke.points.is_empty() {
|
||||
continue;
|
||||
}
|
||||
|
||||
let mut lo = [f32::MAX, f32::MAX];
|
||||
let mut hi = [f32::MIN, f32::MIN];
|
||||
for &(x, y) in &stroke.points {
|
||||
lo = [lo[0].min(x), lo[1].min(y)];
|
||||
hi = [hi[0].max(x), hi[1].max(y)];
|
||||
}
|
||||
|
||||
// Grown by the radius, or a stroke would be drawn only where its
|
||||
// centre line ran — and a tap, whose box has no area at all, would
|
||||
// draw nothing whatever.
|
||||
let grow = [stroke.radius * mx + texel, stroke.radius * my + texel];
|
||||
out.headers.push(StrokeHeader {
|
||||
lo: [
|
||||
(lo[0] - grow[0]).clamp(0.0, 1.0),
|
||||
(lo[1] - grow[1]).clamp(0.0, 1.0),
|
||||
],
|
||||
hi: [
|
||||
(hi[0] + grow[0]).clamp(0.0, 1.0),
|
||||
(hi[1] + grow[1]).clamp(0.0, 1.0),
|
||||
],
|
||||
radius: stroke.radius,
|
||||
hardness: stroke.hardness,
|
||||
flow: stroke.flow,
|
||||
first: out.points.len() as u32,
|
||||
count: stroke.points.len() as u32,
|
||||
_pad: 0,
|
||||
});
|
||||
out.erases.push(stroke.erase);
|
||||
out.points
|
||||
.extend(stroke.points.iter().map(|&(x, y)| [x, y]));
|
||||
}
|
||||
|
||||
out
|
||||
}
|
||||
|
||||
fn is_empty(&self) -> bool {
|
||||
self.headers.is_empty()
|
||||
}
|
||||
}
|
||||
|
||||
/// The segmentation a region mask indexes into, resident on the GPU.
|
||||
///
|
||||
/// Uploaded once per image. Holds the compacted label field and nothing else —
|
||||
@@ -229,6 +343,18 @@ pub struct MaskPass {
|
||||
ctx: GpuContext,
|
||||
layout: wgpu::BindGroupLayout,
|
||||
pipeline: wgpu::RenderPipeline,
|
||||
/// The brush's own bindings: the parameters, plus the stroke buffers.
|
||||
///
|
||||
/// A second layout rather than two more entries on the first, because a
|
||||
/// brush reads neither the label field nor a distance field and the
|
||||
/// parametric masks read no strokes. Sharing one layout would mean binding
|
||||
/// a placeholder in every draw for something that pass provably cannot
|
||||
/// touch.
|
||||
brush_layout: wgpu::BindGroupLayout,
|
||||
/// One fragment shader, two blend states: `dst + a(1 - dst)` to paint and
|
||||
/// `dst(1 - a)` to erase.
|
||||
brush_add: wgpu::RenderPipeline,
|
||||
brush_erase: wgpu::RenderPipeline,
|
||||
array: Option<MaskArray>,
|
||||
/// How many times the array texture has been (re)allocated.
|
||||
///
|
||||
@@ -314,6 +440,76 @@ impl MaskPass {
|
||||
cache: None,
|
||||
});
|
||||
|
||||
let brush_layout = ctx
|
||||
.device
|
||||
.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
|
||||
label: Some("mask-brush-bgl"),
|
||||
entries: &[
|
||||
uniform_entry(0),
|
||||
// Visible to the vertex stage too: the stroke headers are
|
||||
// where the bounding box comes from, and the box is what
|
||||
// the vertex shader draws.
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
visibility: wgpu::ShaderStages::VERTEX_FRAGMENT,
|
||||
..storage_entry(4)
|
||||
},
|
||||
storage_entry(5),
|
||||
],
|
||||
});
|
||||
|
||||
let brush_pipeline_layout =
|
||||
ctx.device
|
||||
.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
|
||||
label: Some("mask-brush-layout"),
|
||||
bind_group_layouts: &[Some(&brush_layout)],
|
||||
immediate_size: 0,
|
||||
});
|
||||
|
||||
let brush = |label, blend| {
|
||||
ctx.device
|
||||
.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
|
||||
label: Some(label),
|
||||
layout: Some(&brush_pipeline_layout),
|
||||
vertex: wgpu::VertexState {
|
||||
module: &module,
|
||||
entry_point: Some("vs_brush"),
|
||||
compilation_options: Default::default(),
|
||||
buffers: &[],
|
||||
},
|
||||
fragment: Some(wgpu::FragmentState {
|
||||
module: &module,
|
||||
entry_point: Some("fs_brush"),
|
||||
compilation_options: Default::default(),
|
||||
targets: &[Some(wgpu::ColorTargetState {
|
||||
format: MaskArray::FORMAT,
|
||||
blend: Some(blend),
|
||||
write_mask: wgpu::ColorWrites::ALL,
|
||||
})],
|
||||
}),
|
||||
primitive: wgpu::PrimitiveState::default(),
|
||||
depth_stencil: None,
|
||||
multisample: wgpu::MultisampleState::default(),
|
||||
multiview_mask: None,
|
||||
cache: None,
|
||||
})
|
||||
};
|
||||
|
||||
// Source-over: what the stroke deposits, plus what it did not cover of
|
||||
// whatever was already there. Two strokes at half flow reach three
|
||||
// quarters rather than one, which is what "build up" means.
|
||||
let brush_add = brush(
|
||||
"mask-brush-add",
|
||||
blend_state(wgpu::BlendFactor::One, wgpu::BlendFactor::OneMinusSrc),
|
||||
);
|
||||
// The same, with the deposit thrown away: coverage is only ever taken
|
||||
// off what earlier strokes on this layer put down. There is no negative
|
||||
// coverage to accumulate, so erasing an unpainted layer is a no-op
|
||||
// rather than a mask that comes back inverted.
|
||||
let brush_erase = brush(
|
||||
"mask-brush-erase",
|
||||
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::OneMinusSrc),
|
||||
);
|
||||
|
||||
if let Some(err) = pollster::block_on(scope.pop()) {
|
||||
return Err(GpuError::ShaderCompilation(err.to_string()));
|
||||
}
|
||||
@@ -327,6 +523,9 @@ impl MaskPass {
|
||||
ctx: ctx.clone(),
|
||||
layout,
|
||||
pipeline,
|
||||
brush_layout,
|
||||
brush_add,
|
||||
brush_erase,
|
||||
array: None,
|
||||
allocations: 0,
|
||||
placeholder,
|
||||
@@ -394,8 +593,15 @@ impl MaskPass {
|
||||
};
|
||||
|
||||
let params = self.params(layer, field, width, height);
|
||||
let selected = self.selection_buffer(layer, field);
|
||||
self.draw(&mut encoder, slot as u32, ¶ms, field, &selected, subject);
|
||||
match &layer.source {
|
||||
MaskSource::Brush { strokes } => {
|
||||
self.draw_brush(&mut encoder, slot as u32, ¶ms, strokes, width, height)
|
||||
}
|
||||
_ => {
|
||||
let selected = self.selection_buffer(layer, field);
|
||||
self.draw(&mut encoder, slot as u32, ¶ms, field, &selected, subject);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
self.ctx.queue.submit([encoder.finish()]);
|
||||
@@ -488,6 +694,145 @@ impl MaskPass {
|
||||
angle: *angle,
|
||||
..base
|
||||
},
|
||||
// A brush carries everything else per stroke, so the only fields it
|
||||
// reads here are the output dimensions — which it needs for the
|
||||
// aspect ratio, not for a coordinate.
|
||||
MaskSource::Brush { .. } => MaskParams {
|
||||
mode: MODE_BRUSH,
|
||||
..base
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/// Paint one brush layer's slice.
|
||||
///
|
||||
/// The slice is cleared and then the strokes are blended onto it in the
|
||||
/// order they were painted, which is why this is a pass of its own rather
|
||||
/// than a variation on [`Self::draw`]: the accumulating mask *is* the
|
||||
/// attachment, so an erase can take away what an add put down without
|
||||
/// either of them reading the texture.
|
||||
///
|
||||
/// Consecutive strokes that composite the same way go out as one draw,
|
||||
/// since the only thing that changes between them is the pipeline. A layer
|
||||
/// painted and never erased is therefore one draw call however many strokes
|
||||
/// it holds.
|
||||
fn draw_brush(
|
||||
&self,
|
||||
encoder: &mut wgpu::CommandEncoder,
|
||||
slot: u32,
|
||||
params: &MaskParams,
|
||||
strokes: &[Stroke],
|
||||
width: u32,
|
||||
height: u32,
|
||||
) {
|
||||
let batch = StrokeBatch::pack(strokes, width, height);
|
||||
|
||||
// Still worth beginning the pass: the slice has to be cleared, or an
|
||||
// unpainted layer would show whatever the last edit left in it.
|
||||
let bind_group = (!batch.is_empty()).then(|| {
|
||||
let params_buf = self
|
||||
.ctx
|
||||
.device
|
||||
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
||||
label: Some("mask-brush-params"),
|
||||
contents: bytemuck::bytes_of(params),
|
||||
usage: wgpu::BufferUsages::UNIFORM,
|
||||
});
|
||||
// Rebuilt per rasterisation rather than kept and patched. This runs
|
||||
// when a mask's shape changes, not per frame, and a few kilobytes
|
||||
// of stroke geometry is cheaper to upload than a residency scheme
|
||||
// is to get wrong.
|
||||
let headers = self
|
||||
.ctx
|
||||
.device
|
||||
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
||||
label: Some("mask-strokes"),
|
||||
contents: bytemuck::cast_slice(&batch.headers),
|
||||
usage: wgpu::BufferUsages::STORAGE,
|
||||
});
|
||||
let points = self
|
||||
.ctx
|
||||
.device
|
||||
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
||||
label: Some("mask-stroke-points"),
|
||||
contents: bytemuck::cast_slice(&batch.points),
|
||||
usage: wgpu::BufferUsages::STORAGE,
|
||||
});
|
||||
|
||||
self.ctx
|
||||
.device
|
||||
.create_bind_group(&wgpu::BindGroupDescriptor {
|
||||
label: Some("mask-brush-bind"),
|
||||
layout: &self.brush_layout,
|
||||
entries: &[
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 0,
|
||||
resource: params_buf.as_entire_binding(),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 4,
|
||||
resource: headers.as_entire_binding(),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 5,
|
||||
resource: points.as_entire_binding(),
|
||||
},
|
||||
],
|
||||
})
|
||||
});
|
||||
|
||||
let array = self.array.as_ref().expect("array ensured by caller");
|
||||
let view = array.texture.create_view(&wgpu::TextureViewDescriptor {
|
||||
label: Some("mask-slice"),
|
||||
dimension: Some(wgpu::TextureViewDimension::D2),
|
||||
base_array_layer: slot,
|
||||
array_layer_count: Some(1),
|
||||
..Default::default()
|
||||
});
|
||||
|
||||
let mut pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
|
||||
label: Some("mask-brush-pass"),
|
||||
color_attachments: &[Some(wgpu::RenderPassColorAttachment {
|
||||
view: &view,
|
||||
depth_slice: None,
|
||||
resolve_target: None,
|
||||
ops: wgpu::Operations {
|
||||
// Nothing at all until a stroke covers it, which is what
|
||||
// makes an unpainted brush layer mask nothing rather than
|
||||
// everything.
|
||||
load: wgpu::LoadOp::Clear(wgpu::Color::BLACK),
|
||||
store: wgpu::StoreOp::Store,
|
||||
},
|
||||
})],
|
||||
depth_stencil_attachment: None,
|
||||
timestamp_writes: None,
|
||||
occlusion_query_set: None,
|
||||
multiview_mask: None,
|
||||
});
|
||||
|
||||
let Some(bind_group) = bind_group else {
|
||||
return;
|
||||
};
|
||||
pass.set_bind_group(0, &bind_group, &[]);
|
||||
|
||||
let mut run = 0;
|
||||
while run < batch.erases.len() {
|
||||
let erases = batch.erases[run];
|
||||
let mut end = run + 1;
|
||||
while end < batch.erases.len() && batch.erases[end] == erases {
|
||||
end += 1;
|
||||
}
|
||||
|
||||
pass.set_pipeline(if erases {
|
||||
&self.brush_erase
|
||||
} else {
|
||||
&self.brush_add
|
||||
});
|
||||
pass.draw(
|
||||
run as u32 * VERTICES_PER_STROKE..end as u32 * VERTICES_PER_STROKE,
|
||||
0..1,
|
||||
);
|
||||
run = end;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -675,6 +1020,24 @@ fn falloff_code(falloff: dr_pipeline::mask::Falloff) -> u32 {
|
||||
}
|
||||
}
|
||||
|
||||
/// `src * src_factor + dst * dst_factor`, on both components.
|
||||
///
|
||||
/// The mask is a single channel, so the alpha component is never written — but
|
||||
/// a target still has to declare one, and declaring something different there
|
||||
/// would be a difference nothing could observe and everything could be confused
|
||||
/// by.
|
||||
fn blend_state(src: wgpu::BlendFactor, dst: wgpu::BlendFactor) -> wgpu::BlendState {
|
||||
let component = wgpu::BlendComponent {
|
||||
src_factor: src,
|
||||
dst_factor: dst,
|
||||
operation: wgpu::BlendOperation::Add,
|
||||
};
|
||||
wgpu::BlendState {
|
||||
color: component,
|
||||
alpha: component,
|
||||
}
|
||||
}
|
||||
|
||||
fn uniform_entry(binding: u32) -> wgpu::BindGroupLayoutEntry {
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
binding,
|
||||
@@ -796,6 +1159,72 @@ mod tests {
|
||||
assert_eq!(array.layers(), 2, "one slice per active layer");
|
||||
}
|
||||
|
||||
/// One gesture: whether it erases, its radius, and its path.
|
||||
type Gesture = (bool, f32, Vec<(f32, f32)>);
|
||||
|
||||
fn painted(gestures: &[Gesture]) -> MaskLayer {
|
||||
let mut layer = lit(MaskSource::brush());
|
||||
for (erase, radius, path) in gestures {
|
||||
layer.begin_stroke(*erase, *radius, 0.5, 1.0);
|
||||
for &(x, y) in path {
|
||||
layer.extend_stroke(x, y);
|
||||
}
|
||||
layer.end_stroke();
|
||||
}
|
||||
layer
|
||||
}
|
||||
|
||||
/// A brush is the one mask that needs nothing uploaded first — no
|
||||
/// segmentation, no distance field, no label. Requiring one would mean a
|
||||
/// photograph could not be painted on until a model had run over it.
|
||||
#[test]
|
||||
fn a_brush_needs_no_segmentation() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(painted(&[(false, 0.1, vec![(0.2, 0.2), (0.8, 0.8)])]));
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
let array = pass.render(&stack, None, None, 32, 32).expect("render");
|
||||
assert_eq!(array.layers(), 1);
|
||||
}
|
||||
|
||||
/// The box a stroke is drawn over has to be grown by its radius. Packed
|
||||
/// from the points alone, a tap's box has no area at all and the stroke
|
||||
/// would be silently missing from the mask.
|
||||
#[test]
|
||||
fn a_taps_box_has_room_for_its_dab() {
|
||||
let layer = painted(&[(false, 0.25, vec![(0.5, 0.5)])]);
|
||||
let batch = StrokeBatch::pack(layer.strokes(), 64, 32);
|
||||
assert_eq!(batch.headers.len(), 1);
|
||||
|
||||
let h = &batch.headers[0];
|
||||
assert!(h.hi[0] - h.lo[0] > 0.2, "wide enough for the dab: {h:?}", h = (h.lo, h.hi));
|
||||
assert!(
|
||||
h.hi[1] - h.lo[1] > h.hi[0] - h.lo[0],
|
||||
"and taller than it is wide in normalised units, since the radius \
|
||||
is a fraction of the shorter edge"
|
||||
);
|
||||
}
|
||||
|
||||
/// The pipeline is chosen per stroke, so the packed order has to be the
|
||||
/// painted order — an erase that ended up before its add would put paint
|
||||
/// back that the user removed.
|
||||
#[test]
|
||||
fn packing_keeps_the_painted_order() {
|
||||
let layer = painted(&[
|
||||
(false, 0.1, vec![(0.2, 0.5), (0.4, 0.5)]),
|
||||
(true, 0.1, vec![(0.3, 0.5)]),
|
||||
(false, 0.1, vec![(0.8, 0.5)]),
|
||||
]);
|
||||
let batch = StrokeBatch::pack(layer.strokes(), 32, 32);
|
||||
assert_eq!(batch.erases, [false, true, false]);
|
||||
assert_eq!(batch.headers[0].first, 0);
|
||||
assert_eq!(batch.headers[1].first, batch.headers[0].count);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_stack_still_yields_a_bindable_array() {
|
||||
let Some(ctx) = ctx() else {
|
||||
|
||||
@@ -28,7 +28,11 @@ struct MaskParams {
|
||||
label_width: u32,
|
||||
label_height: u32,
|
||||
|
||||
// 0 = regions, 1 = linear, 2 = radial, 3 = subject.
|
||||
// 0 = regions, 1 = linear, 2 = radial, 3 = subject, 4 = brush.
|
||||
//
|
||||
// A brush does not read this — it has its own entry points, because it is
|
||||
// the one mask that is not a function of the whole frame — but it is set
|
||||
// anyway so a captured frame says which kind of mask a pass was drawing.
|
||||
mode: u32,
|
||||
// How many regions the label field holds, so an out-of-range label is
|
||||
// caught rather than read past the end of `selected`.
|
||||
@@ -220,3 +224,150 @@ fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
|
||||
|
||||
return vec4<f32>(clamp(m, 0.0, 1.0), 0.0, 0.0, 1.0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Brush strokes (ARCH §5.4)
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// The mask the architecture was written for. What arrives is a list of
|
||||
// positions, a radius, a hardness and a flow; what leaves is pixels. Nothing
|
||||
// between the two ever exists in CPU memory, which is the whole difference from
|
||||
// darktable, where the same strokes are rasterised on the CPU and the lag makes
|
||||
// painting unusable.
|
||||
//
|
||||
// # Why the strokes are not drawn by the full-screen triangle above
|
||||
//
|
||||
// Cost. A swept disc is the minimum distance to any segment of its polyline, so
|
||||
// evaluating one stroke costs a distance per segment *per pixel*. Over the
|
||||
// whole frame that is `pixels × segments`, and a stroke that wandered across
|
||||
// the photograph has both terms large at once.
|
||||
//
|
||||
// So each stroke is drawn over its own bounding box instead, expanded by the
|
||||
// radius. The rasteriser then never invokes the fragment shader for a pixel the
|
||||
// stroke cannot reach, and the cost becomes `area(box) × segments` — for the
|
||||
// ordinary case, a dab or a swipe, a small fraction of the frame. The model
|
||||
// splits a long gesture into strokes of bounded length for the same reason:
|
||||
// both terms of that product grow with how far one stroke travelled.
|
||||
//
|
||||
// # Why the strokes composite with fixed-function blending
|
||||
//
|
||||
// Add is `dst + a(1 - dst)` and erase is `dst(1 - a)`, which are exactly a
|
||||
// source-over and a one-minus-source blend. Expressing them as blend state
|
||||
// rather than as arithmetic in the shader is what allows one draw per stroke:
|
||||
// the accumulating mask is the attachment, and no pass ever has to read the
|
||||
// slice it is writing.
|
||||
|
||||
struct StrokeHeader {
|
||||
// Bounding box in normalised coordinates, already grown by the radius and
|
||||
// a texel — the vertex shader trusts it and draws nothing outside it.
|
||||
lo: vec2<f32>,
|
||||
hi: vec2<f32>,
|
||||
// Radius in units of the frame's shorter edge, so a dab is round on a frame
|
||||
// that is not square.
|
||||
radius: f32,
|
||||
// Fraction of the radius that is fully covered.
|
||||
hardness: f32,
|
||||
// Coverage deposited where the stroke is solid.
|
||||
flow: f32,
|
||||
// Window into `stroke_points`.
|
||||
first: u32,
|
||||
count: u32,
|
||||
_pad: u32,
|
||||
}
|
||||
|
||||
@group(0) @binding(4) var<storage, read> strokes: array<StrokeHeader>;
|
||||
@group(0) @binding(5) var<storage, read> stroke_points: array<vec2<f32>>;
|
||||
|
||||
struct BrushVertex {
|
||||
@builtin(position) pos: vec4<f32>,
|
||||
// Flat: a stroke index interpolated across its own quad would name a
|
||||
// different stroke in the middle of it.
|
||||
@location(0) @interpolate(flat) stroke: u32,
|
||||
}
|
||||
|
||||
// Six vertices per stroke, non-instanced.
|
||||
//
|
||||
// Deliberately not one instance per stroke: `@builtin(instance_index)` with a
|
||||
// non-zero first instance needs base-instance support, which the GL backend
|
||||
// this has to run on under Android cannot promise. Dividing the vertex index
|
||||
// costs one integer operation and works everywhere.
|
||||
@vertex
|
||||
fn vs_brush(@builtin(vertex_index) v: u32) -> BrushVertex {
|
||||
var quad = array<vec2<f32>, 6>(
|
||||
vec2<f32>(0.0, 0.0), vec2<f32>(1.0, 0.0), vec2<f32>(0.0, 1.0),
|
||||
vec2<f32>(0.0, 1.0), vec2<f32>(1.0, 0.0), vec2<f32>(1.0, 1.0),
|
||||
);
|
||||
|
||||
let i = v / 6u;
|
||||
let s = strokes[i];
|
||||
let uv = mix(s.lo, s.hi, quad[v % 6u]);
|
||||
|
||||
var out: BrushVertex;
|
||||
// y is flipped because normalised mask coordinates run downwards, the way
|
||||
// the fragment shader above reads them, and clip space runs upwards. A
|
||||
// stroke drawn without this lands mirrored about the horizon, which is
|
||||
// plausible enough on a symmetric test image to survive a careless check.
|
||||
out.pos = vec4<f32>(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0, 0.0, 1.0);
|
||||
out.stroke = i;
|
||||
return out;
|
||||
}
|
||||
|
||||
// Into units of the frame's shorter edge.
|
||||
//
|
||||
// Without this the brush would be a circle in normalised coordinates, which on
|
||||
// a 3:2 frame is an ellipse half again as wide as it is tall. A brush whose dab
|
||||
// is not round is not a brush.
|
||||
fn to_square(uv: vec2<f32>) -> vec2<f32> {
|
||||
let dims = vec2<f32>(f32(p.width), f32(p.height));
|
||||
return uv * dims / min(dims.x, dims.y);
|
||||
}
|
||||
|
||||
fn segment_distance(q: vec2<f32>, a: vec2<f32>, b: vec2<f32>) -> f32 {
|
||||
let ab = b - a;
|
||||
let len2 = dot(ab, ab);
|
||||
// A finger that stopped and went back leaves a zero-length segment, and
|
||||
// dividing by its length is a NaN — which propagates through the min()
|
||||
// below and takes the whole stroke with it.
|
||||
if (len2 <= 1e-12) {
|
||||
return length(q - a);
|
||||
}
|
||||
let t = clamp(dot(q - a, ab) / len2, 0.0, 1.0);
|
||||
return length(q - (a + ab * t));
|
||||
}
|
||||
|
||||
@fragment
|
||||
fn fs_brush(in: BrushVertex) -> @location(0) vec4<f32> {
|
||||
let s = strokes[in.stroke];
|
||||
let q = to_square(vec2<f32>(in.pos.x / f32(p.width), in.pos.y / f32(p.height)));
|
||||
|
||||
// The *minimum* over the segments, which is the maximum of their coverage.
|
||||
// Accumulating the segments instead would make a stroke that crosses itself
|
||||
// — every circle, every scribble — build up a bright patch where it did,
|
||||
// and a soft brush would go blotchy along any curve tight enough for
|
||||
// consecutive dabs to overlap, which is all of them.
|
||||
var d = 1e30;
|
||||
if (s.count == 1u) {
|
||||
// A tap. One point is a legitimate stroke, and it paints one dab.
|
||||
d = length(q - to_square(stroke_points[s.first]));
|
||||
} else {
|
||||
for (var k = 0u; k + 1u < s.count; k = k + 1u) {
|
||||
d = min(
|
||||
d,
|
||||
segment_distance(
|
||||
q,
|
||||
to_square(stroke_points[s.first + k]),
|
||||
to_square(stroke_points[s.first + k + 1u]),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Even at full hardness the edge keeps a one-pixel ramp. A true step would
|
||||
// alias into a staircase, and the mask is sampled bilinearly at whatever
|
||||
// zoom the user is inspecting it at — which is where an edge is judged.
|
||||
let texel = 1.0 / f32(min(p.width, p.height));
|
||||
let inner = min(s.radius * clamp(s.hardness, 0.0, 1.0), max(s.radius - texel, 0.0));
|
||||
let coverage = 1.0 - smoothstep(inner, s.radius, d);
|
||||
|
||||
return vec4<f32>(clamp(coverage * s.flow, 0.0, 1.0), 0.0, 0.0, 1.0);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user