Paint a mask without ever rasterising one on the CPU
The last line of FR-DEV-3, and the mask ARCH §5.4 was written for. darktable rasterises drawn masks on the CPU and users call the result unworkable; the architecture's answer is that a stroke arrives as *parameters* and the device draws it. This is that, from the model through the sidecar to the pixels — but not the finger: the canvas is somebody else's change, and this leaves it a seam rather than reaching into it. **A stroke is a swept disc along a polyline**, plus erase, radius, hardness and flow. `MaskSource::Brush` holds an ordered list of them, and the order is the mask: an erase after an add takes it away and the same pair reversed does not. Nothing about it is pixels, which is what makes a mask that costs a line of text, diffs by the gesture, and survives a crop, a straighten and an export at any size — the properties a stored raster has none of, and the same argument the region ids were chosen for. Two things keep the point count honest. While the finger is down, a position closer to the last than an eighth of the radius is dropped: a touch screen reports 120 a second, so a finger held still for five seconds is six hundred points in the same place, and simplification would only remove them once the gesture had ended — after every frame in between had drawn all of them. When it ends, Douglas–Peucker at an eighth of the radius removes what a disc that wide cannot express: a swept circle moved by r/8 moves its own edge by r/8, which is inside the soft part of any brush. Coordinates snap to a ten-thousandth of the frame on the way in *and* are written at that precision, so a round trip is exact rather than nearly exact — a file that drifts in the sixth decimal every save is a per-field merge conflict a day, over nothing. **Cost is why the strokes are not drawn by the full-screen triangle the other masks use.** A swept disc is the minimum distance to any of its segments, so a stroke over the whole frame costs `pixels × segments` and both terms grow together — the quadratic that is darktable's problem moved onto the GPU rather than solved. Each stroke is instead drawn over its own bounding box, grown by the radius, so the rasteriser never invokes the shader for a pixel the stroke cannot reach: `area(box) × segments`, which for a dab or a swipe is a small fraction of the frame. A gesture past 256 points continues as a second stroke for the same reason, since a shorter stroke has a smaller box. Add and erase are `dst + a(1 - dst)` and `dst(1 - a)`, which are exactly a source-over and a one-minus-source blend — so they are blend state, not arithmetic, and no pass ever reads the slice it is writing. That is what permits one draw per stroke at all. Within a stroke the coverage is the *minimum* distance over its segments rather than a sum: a path that crosses itself must not build up where it did, or every circle and every scribble would be blotchy wherever consecutive dabs overlap, which is everywhere. Not a distance field, deliberately. `dr-segment`'s transform documents the two conditions that make CPU work right there — once per mask edit, over input already CPU-side — and a stroke fails both: it changes while the finger moves, and its input is a handful of coordinates that never needed to be pixels. It also needs no transform, because the distance to a swept disc is closed form. A stroke is the one mask whose distance field is known without computing one. An unpainted brush layer is inactive rather than empty, which is not an optimisation: `invert` turns empty into everything, so a layer created with invert already set would apply its adjustment to the whole photograph before a single stroke was made. That is the loud, confident kind of wrong this codebase refuses everywhere else a mask can go missing, and there is a rendered test for it. The tests read pixels back off a device rather than checking that the two halves agree with each other. What they pin down is what is silent when wrong: the y flip between mask space and clip space, which a centred stroke would not notice; a bounding box not grown by the radius, which makes a tap draw nothing at all; an aspect ratio ignored, which makes a dab an ellipse on any frame that is not square; a stroke doubling back and building up; and an erase that lost its place in the order and put back paint the user had taken off. Not done here: the interaction. The canvas needs to begin, extend and end a stroke on the active layer, and `DevelopSession::rasterise_masks` still returns early without a segmentation — it takes the proxy size from one, and a brush needs no model to have run over the photograph first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -28,7 +28,11 @@ struct MaskParams {
|
||||
label_width: u32,
|
||||
label_height: u32,
|
||||
|
||||
// 0 = regions, 1 = linear, 2 = radial, 3 = subject.
|
||||
// 0 = regions, 1 = linear, 2 = radial, 3 = subject, 4 = brush.
|
||||
//
|
||||
// A brush does not read this — it has its own entry points, because it is
|
||||
// the one mask that is not a function of the whole frame — but it is set
|
||||
// anyway so a captured frame says which kind of mask a pass was drawing.
|
||||
mode: u32,
|
||||
// How many regions the label field holds, so an out-of-range label is
|
||||
// caught rather than read past the end of `selected`.
|
||||
@@ -220,3 +224,150 @@ fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
|
||||
|
||||
return vec4<f32>(clamp(m, 0.0, 1.0), 0.0, 0.0, 1.0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Brush strokes (ARCH §5.4)
|
||||
// ---------------------------------------------------------------------------
|
||||
//
|
||||
// The mask the architecture was written for. What arrives is a list of
|
||||
// positions, a radius, a hardness and a flow; what leaves is pixels. Nothing
|
||||
// between the two ever exists in CPU memory, which is the whole difference from
|
||||
// darktable, where the same strokes are rasterised on the CPU and the lag makes
|
||||
// painting unusable.
|
||||
//
|
||||
// # Why the strokes are not drawn by the full-screen triangle above
|
||||
//
|
||||
// Cost. A swept disc is the minimum distance to any segment of its polyline, so
|
||||
// evaluating one stroke costs a distance per segment *per pixel*. Over the
|
||||
// whole frame that is `pixels × segments`, and a stroke that wandered across
|
||||
// the photograph has both terms large at once.
|
||||
//
|
||||
// So each stroke is drawn over its own bounding box instead, expanded by the
|
||||
// radius. The rasteriser then never invokes the fragment shader for a pixel the
|
||||
// stroke cannot reach, and the cost becomes `area(box) × segments` — for the
|
||||
// ordinary case, a dab or a swipe, a small fraction of the frame. The model
|
||||
// splits a long gesture into strokes of bounded length for the same reason:
|
||||
// both terms of that product grow with how far one stroke travelled.
|
||||
//
|
||||
// # Why the strokes composite with fixed-function blending
|
||||
//
|
||||
// Add is `dst + a(1 - dst)` and erase is `dst(1 - a)`, which are exactly a
|
||||
// source-over and a one-minus-source blend. Expressing them as blend state
|
||||
// rather than as arithmetic in the shader is what allows one draw per stroke:
|
||||
// the accumulating mask is the attachment, and no pass ever has to read the
|
||||
// slice it is writing.
|
||||
|
||||
struct StrokeHeader {
|
||||
// Bounding box in normalised coordinates, already grown by the radius and
|
||||
// a texel — the vertex shader trusts it and draws nothing outside it.
|
||||
lo: vec2<f32>,
|
||||
hi: vec2<f32>,
|
||||
// Radius in units of the frame's shorter edge, so a dab is round on a frame
|
||||
// that is not square.
|
||||
radius: f32,
|
||||
// Fraction of the radius that is fully covered.
|
||||
hardness: f32,
|
||||
// Coverage deposited where the stroke is solid.
|
||||
flow: f32,
|
||||
// Window into `stroke_points`.
|
||||
first: u32,
|
||||
count: u32,
|
||||
_pad: u32,
|
||||
}
|
||||
|
||||
@group(0) @binding(4) var<storage, read> strokes: array<StrokeHeader>;
|
||||
@group(0) @binding(5) var<storage, read> stroke_points: array<vec2<f32>>;
|
||||
|
||||
struct BrushVertex {
|
||||
@builtin(position) pos: vec4<f32>,
|
||||
// Flat: a stroke index interpolated across its own quad would name a
|
||||
// different stroke in the middle of it.
|
||||
@location(0) @interpolate(flat) stroke: u32,
|
||||
}
|
||||
|
||||
// Six vertices per stroke, non-instanced.
|
||||
//
|
||||
// Deliberately not one instance per stroke: `@builtin(instance_index)` with a
|
||||
// non-zero first instance needs base-instance support, which the GL backend
|
||||
// this has to run on under Android cannot promise. Dividing the vertex index
|
||||
// costs one integer operation and works everywhere.
|
||||
@vertex
|
||||
fn vs_brush(@builtin(vertex_index) v: u32) -> BrushVertex {
|
||||
var quad = array<vec2<f32>, 6>(
|
||||
vec2<f32>(0.0, 0.0), vec2<f32>(1.0, 0.0), vec2<f32>(0.0, 1.0),
|
||||
vec2<f32>(0.0, 1.0), vec2<f32>(1.0, 0.0), vec2<f32>(1.0, 1.0),
|
||||
);
|
||||
|
||||
let i = v / 6u;
|
||||
let s = strokes[i];
|
||||
let uv = mix(s.lo, s.hi, quad[v % 6u]);
|
||||
|
||||
var out: BrushVertex;
|
||||
// y is flipped because normalised mask coordinates run downwards, the way
|
||||
// the fragment shader above reads them, and clip space runs upwards. A
|
||||
// stroke drawn without this lands mirrored about the horizon, which is
|
||||
// plausible enough on a symmetric test image to survive a careless check.
|
||||
out.pos = vec4<f32>(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0, 0.0, 1.0);
|
||||
out.stroke = i;
|
||||
return out;
|
||||
}
|
||||
|
||||
// Into units of the frame's shorter edge.
|
||||
//
|
||||
// Without this the brush would be a circle in normalised coordinates, which on
|
||||
// a 3:2 frame is an ellipse half again as wide as it is tall. A brush whose dab
|
||||
// is not round is not a brush.
|
||||
fn to_square(uv: vec2<f32>) -> vec2<f32> {
|
||||
let dims = vec2<f32>(f32(p.width), f32(p.height));
|
||||
return uv * dims / min(dims.x, dims.y);
|
||||
}
|
||||
|
||||
fn segment_distance(q: vec2<f32>, a: vec2<f32>, b: vec2<f32>) -> f32 {
|
||||
let ab = b - a;
|
||||
let len2 = dot(ab, ab);
|
||||
// A finger that stopped and went back leaves a zero-length segment, and
|
||||
// dividing by its length is a NaN — which propagates through the min()
|
||||
// below and takes the whole stroke with it.
|
||||
if (len2 <= 1e-12) {
|
||||
return length(q - a);
|
||||
}
|
||||
let t = clamp(dot(q - a, ab) / len2, 0.0, 1.0);
|
||||
return length(q - (a + ab * t));
|
||||
}
|
||||
|
||||
@fragment
|
||||
fn fs_brush(in: BrushVertex) -> @location(0) vec4<f32> {
|
||||
let s = strokes[in.stroke];
|
||||
let q = to_square(vec2<f32>(in.pos.x / f32(p.width), in.pos.y / f32(p.height)));
|
||||
|
||||
// The *minimum* over the segments, which is the maximum of their coverage.
|
||||
// Accumulating the segments instead would make a stroke that crosses itself
|
||||
// — every circle, every scribble — build up a bright patch where it did,
|
||||
// and a soft brush would go blotchy along any curve tight enough for
|
||||
// consecutive dabs to overlap, which is all of them.
|
||||
var d = 1e30;
|
||||
if (s.count == 1u) {
|
||||
// A tap. One point is a legitimate stroke, and it paints one dab.
|
||||
d = length(q - to_square(stroke_points[s.first]));
|
||||
} else {
|
||||
for (var k = 0u; k + 1u < s.count; k = k + 1u) {
|
||||
d = min(
|
||||
d,
|
||||
segment_distance(
|
||||
q,
|
||||
to_square(stroke_points[s.first + k]),
|
||||
to_square(stroke_points[s.first + k + 1u]),
|
||||
),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Even at full hardness the edge keeps a one-pixel ramp. A true step would
|
||||
// alias into a staircase, and the mask is sampled bilinearly at whatever
|
||||
// zoom the user is inspecting it at — which is where an edge is judged.
|
||||
let texel = 1.0 / f32(min(p.width, p.height));
|
||||
let inner = min(s.radius * clamp(s.hardness, 0.0, 1.0), max(s.radius - texel, 0.0));
|
||||
let coverage = 1.0 - smoothstep(inner, s.radius, d);
|
||||
|
||||
return vec4<f32>(clamp(coverage * s.flow, 0.0, 1.0), 0.0, 0.0, 1.0);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user