Paint a mask without ever rasterising one on the CPU

The last line of FR-DEV-3, and the mask ARCH §5.4 was written for. darktable
rasterises drawn masks on the CPU and users call the result unworkable; the
architecture's answer is that a stroke arrives as *parameters* and the device
draws it. This is that, from the model through the sidecar to the pixels — but
not the finger: the canvas is somebody else's change, and this leaves it a
seam rather than reaching into it.

**A stroke is a swept disc along a polyline**, plus erase, radius, hardness and
flow. `MaskSource::Brush` holds an ordered list of them, and the order is the
mask: an erase after an add takes it away and the same pair reversed does not.
Nothing about it is pixels, which is what makes a mask that costs a line of
text, diffs by the gesture, and survives a crop, a straighten and an export at
any size — the properties a stored raster has none of, and the same argument
the region ids were chosen for.

Two things keep the point count honest. While the finger is down, a position
closer to the last than an eighth of the radius is dropped: a touch screen
reports 120 a second, so a finger held still for five seconds is six hundred
points in the same place, and simplification would only remove them once the
gesture had ended — after every frame in between had drawn all of them. When
it ends, Douglas–Peucker at an eighth of the radius removes what a disc that
wide cannot express: a swept circle moved by r/8 moves its own edge by r/8,
which is inside the soft part of any brush. Coordinates snap to a
ten-thousandth of the frame on the way in *and* are written at that precision,
so a round trip is exact rather than nearly exact — a file that drifts in the
sixth decimal every save is a per-field merge conflict a day, over nothing.

**Cost is why the strokes are not drawn by the full-screen triangle the other
masks use.** A swept disc is the minimum distance to any of its segments, so a
stroke over the whole frame costs `pixels × segments` and both terms grow
together — the quadratic that is darktable's problem moved onto the GPU rather
than solved. Each stroke is instead drawn over its own bounding box, grown by
the radius, so the rasteriser never invokes the shader for a pixel the stroke
cannot reach: `area(box) × segments`, which for a dab or a swipe is a small
fraction of the frame. A gesture past 256 points continues as a second stroke
for the same reason, since a shorter stroke has a smaller box.

Add and erase are `dst + a(1 - dst)` and `dst(1 - a)`, which are exactly a
source-over and a one-minus-source blend — so they are blend state, not
arithmetic, and no pass ever reads the slice it is writing. That is what
permits one draw per stroke at all. Within a stroke the coverage is the
*minimum* distance over its segments rather than a sum: a path that crosses
itself must not build up where it did, or every circle and every scribble
would be blotchy wherever consecutive dabs overlap, which is everywhere.

Not a distance field, deliberately. `dr-segment`'s transform documents the two
conditions that make CPU work right there — once per mask edit, over input
already CPU-side — and a stroke fails both: it changes while the finger moves,
and its input is a handful of coordinates that never needed to be pixels. It
also needs no transform, because the distance to a swept disc is closed form.
A stroke is the one mask whose distance field is known without computing one.

An unpainted brush layer is inactive rather than empty, which is not an
optimisation: `invert` turns empty into everything, so a layer created with
invert already set would apply its adjustment to the whole photograph before a
single stroke was made. That is the loud, confident kind of wrong this codebase
refuses everywhere else a mask can go missing, and there is a rendered test for
it.

The tests read pixels back off a device rather than checking that the two
halves agree with each other. What they pin down is what is silent when wrong:
the y flip between mask space and clip space, which a centred stroke would not
notice; a bounding box not grown by the radius, which makes a tap draw nothing
at all; an aspect ratio ignored, which makes a dab an ellipse on any frame that
is not square; a stroke doubling back and building up; and an erase that lost
its place in the order and put back paint the user had taken off.

Not done here: the interaction. The canvas needs to begin, extend and end a
stroke on the active layer, and `DevelopSession::rasterise_masks` still returns
early without a segmentation — it takes the proxy size from one, and a brush
needs no model to have run over the photograph first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-22 12:37:42 +02:00
co-authored by Claude Opus 5
parent 586698db00
commit c396a22dfd
6 changed files with 1787 additions and 20 deletions
+152 -1
View File
@@ -28,7 +28,11 @@ struct MaskParams {
label_width: u32,
label_height: u32,
// 0 = regions, 1 = linear, 2 = radial, 3 = subject.
// 0 = regions, 1 = linear, 2 = radial, 3 = subject, 4 = brush.
//
// A brush does not read this — it has its own entry points, because it is
// the one mask that is not a function of the whole frame — but it is set
// anyway so a captured frame says which kind of mask a pass was drawing.
mode: u32,
// How many regions the label field holds, so an out-of-range label is
// caught rather than read past the end of `selected`.
@@ -220,3 +224,150 @@ fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
return vec4<f32>(clamp(m, 0.0, 1.0), 0.0, 0.0, 1.0);
}
// ---------------------------------------------------------------------------
// Brush strokes (ARCH §5.4)
// ---------------------------------------------------------------------------
//
// The mask the architecture was written for. What arrives is a list of
// positions, a radius, a hardness and a flow; what leaves is pixels. Nothing
// between the two ever exists in CPU memory, which is the whole difference from
// darktable, where the same strokes are rasterised on the CPU and the lag makes
// painting unusable.
//
// # Why the strokes are not drawn by the full-screen triangle above
//
// Cost. A swept disc is the minimum distance to any segment of its polyline, so
// evaluating one stroke costs a distance per segment *per pixel*. Over the
// whole frame that is `pixels × segments`, and a stroke that wandered across
// the photograph has both terms large at once.
//
// So each stroke is drawn over its own bounding box instead, expanded by the
// radius. The rasteriser then never invokes the fragment shader for a pixel the
// stroke cannot reach, and the cost becomes `area(box) × segments` — for the
// ordinary case, a dab or a swipe, a small fraction of the frame. The model
// splits a long gesture into strokes of bounded length for the same reason:
// both terms of that product grow with how far one stroke travelled.
//
// # Why the strokes composite with fixed-function blending
//
// Add is `dst + a(1 - dst)` and erase is `dst(1 - a)`, which are exactly a
// source-over and a one-minus-source blend. Expressing them as blend state
// rather than as arithmetic in the shader is what allows one draw per stroke:
// the accumulating mask is the attachment, and no pass ever has to read the
// slice it is writing.
struct StrokeHeader {
// Bounding box in normalised coordinates, already grown by the radius and
// a texel — the vertex shader trusts it and draws nothing outside it.
lo: vec2<f32>,
hi: vec2<f32>,
// Radius in units of the frame's shorter edge, so a dab is round on a frame
// that is not square.
radius: f32,
// Fraction of the radius that is fully covered.
hardness: f32,
// Coverage deposited where the stroke is solid.
flow: f32,
// Window into `stroke_points`.
first: u32,
count: u32,
_pad: u32,
}
@group(0) @binding(4) var<storage, read> strokes: array<StrokeHeader>;
@group(0) @binding(5) var<storage, read> stroke_points: array<vec2<f32>>;
struct BrushVertex {
@builtin(position) pos: vec4<f32>,
// Flat: a stroke index interpolated across its own quad would name a
// different stroke in the middle of it.
@location(0) @interpolate(flat) stroke: u32,
}
// Six vertices per stroke, non-instanced.
//
// Deliberately not one instance per stroke: `@builtin(instance_index)` with a
// non-zero first instance needs base-instance support, which the GL backend
// this has to run on under Android cannot promise. Dividing the vertex index
// costs one integer operation and works everywhere.
@vertex
fn vs_brush(@builtin(vertex_index) v: u32) -> BrushVertex {
var quad = array<vec2<f32>, 6>(
vec2<f32>(0.0, 0.0), vec2<f32>(1.0, 0.0), vec2<f32>(0.0, 1.0),
vec2<f32>(0.0, 1.0), vec2<f32>(1.0, 0.0), vec2<f32>(1.0, 1.0),
);
let i = v / 6u;
let s = strokes[i];
let uv = mix(s.lo, s.hi, quad[v % 6u]);
var out: BrushVertex;
// y is flipped because normalised mask coordinates run downwards, the way
// the fragment shader above reads them, and clip space runs upwards. A
// stroke drawn without this lands mirrored about the horizon, which is
// plausible enough on a symmetric test image to survive a careless check.
out.pos = vec4<f32>(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0, 0.0, 1.0);
out.stroke = i;
return out;
}
// Into units of the frame's shorter edge.
//
// Without this the brush would be a circle in normalised coordinates, which on
// a 3:2 frame is an ellipse half again as wide as it is tall. A brush whose dab
// is not round is not a brush.
fn to_square(uv: vec2<f32>) -> vec2<f32> {
let dims = vec2<f32>(f32(p.width), f32(p.height));
return uv * dims / min(dims.x, dims.y);
}
fn segment_distance(q: vec2<f32>, a: vec2<f32>, b: vec2<f32>) -> f32 {
let ab = b - a;
let len2 = dot(ab, ab);
// A finger that stopped and went back leaves a zero-length segment, and
// dividing by its length is a NaN — which propagates through the min()
// below and takes the whole stroke with it.
if (len2 <= 1e-12) {
return length(q - a);
}
let t = clamp(dot(q - a, ab) / len2, 0.0, 1.0);
return length(q - (a + ab * t));
}
@fragment
fn fs_brush(in: BrushVertex) -> @location(0) vec4<f32> {
let s = strokes[in.stroke];
let q = to_square(vec2<f32>(in.pos.x / f32(p.width), in.pos.y / f32(p.height)));
// The *minimum* over the segments, which is the maximum of their coverage.
// Accumulating the segments instead would make a stroke that crosses itself
// — every circle, every scribble — build up a bright patch where it did,
// and a soft brush would go blotchy along any curve tight enough for
// consecutive dabs to overlap, which is all of them.
var d = 1e30;
if (s.count == 1u) {
// A tap. One point is a legitimate stroke, and it paints one dab.
d = length(q - to_square(stroke_points[s.first]));
} else {
for (var k = 0u; k + 1u < s.count; k = k + 1u) {
d = min(
d,
segment_distance(
q,
to_square(stroke_points[s.first + k]),
to_square(stroke_points[s.first + k + 1u]),
),
);
}
}
// Even at full hardness the edge keeps a one-pixel ramp. A true step would
// alias into a staircase, and the mask is sampled bilinearly at whatever
// zoom the user is inspecting it at — which is where an edge is judged.
let texel = 1.0 / f32(min(p.width, p.height));
let inner = min(s.radius * clamp(s.hardness, 0.0, 1.0), max(s.radius - texel, 0.0));
let coverage = 1.0 - smoothstep(inner, s.radius, d);
return vec4<f32>(clamp(coverage * s.flow, 0.0, 1.0), 0.0, 0.0, 1.0);
}