// Rasterise one local-adjustment mask into a layer of the mask array. // // ARCH §5.4: every mask becomes pixels here and never in CPU memory. One draw // per layer, each targeting its own array slice, run only when a mask's // *shape* changes — moving a slider on a masked layer re-runs the adjust // shader and not this one. // // # Why this is a render pass and not a compute one // // The natural shape for this is a compute shader writing a storage texture, // and the format is what rules that out: **R8Unorm is not a core storage // format**, so a compute path has to widen the mask to R32Float or RGBA8 — // four bytes per pixel per layer. At eight layers over a 24 MP export that is // 768 MB of masks, against 192 MB at one byte. A colour attachment takes // R8Unorm happily, so the mask stays one byte and the pass becomes a // full-screen triangle. // // The array slice is chosen by the *view* the caller attaches, so there is no // slot uniform here — one less thing that can disagree with the shader. struct MaskParams { // Output size, which is the render size rather than the segmentation's. width: u32, height: u32, // Label field size. Different from the above: the watershed runs at a // proxy resolution, and the mask is drawn at whatever the display or the // export asked for. label_width: u32, label_height: u32, // 0 = regions, 1 = linear, 2 = radial, 3 = subject, 4 = brush, // 5 = luminance range, 6 = colour range. // // A brush does not read this — it has its own entry points, because it is // the one mask that is not a function of the whole frame — but it is set // anyway so a captured frame says which kind of mask a pass was drawing. mode: u32, // How many regions the label field holds, so an out-of-range label is // caught rather than read past the end of `selected`. region_count: u32, // Softening applied to a region mask, in output pixels. feather: f32, // 0 hard, 1 linear, 2 smooth, 3 gaussian, 4 exponential. Kept in step with // `falloff_code` on the Rust side. falloff: u32, // Geometry. Meaning depends on `mode`. The centre is in normalised 0..1 // coordinates; every distance below it is in the isotropic frame units // `frame_delta` establishes. centre: vec2, // Linear: (cos, sin) of the ramp direction. Radial: semi-axes. axis: vec2, // Linear: ramp width. Radial: edge falloff as a fraction of the radius. // A range: the fade at each edge of its band, in the band's own units. softness: f32, // Radial only: rotation of the ellipse. angle: f32, // TRACES: FR-DEV-10 // How many source texels one mask texel spans, per axis. // // The mask array is rasterised at a proxy size and the photograph is not, // so one texel here covers several there. A range mask is a function of // pixel *values*, and point-sampling one source texel in four would make // its edge follow the sensor's noise wherever the picture has fine // texture — speckle that is then a mask, and therefore visible in the // adjustment. Averaging the footprint is what makes the band land on the // tone the area actually is. source_step: vec2, // Camera RGB → linear sRGB, one row each. Only a range reads these: it is // the one mask that looks at the photograph, and a hue is the body's own // primaries until this matrix has been applied — so the same stored arc // would select a different set of colours on every make of sensor. cam_to_srgb_0: vec4, cam_to_srgb_1: vec4, cam_to_srgb_2: vec4, // rgb: as-shot white balance. w: non-zero when the source arrived // gamma-encoded rather than linear. as_shot_wb: vec4, // Whether this part is turned over before it joins the mask. // // Read by `fs_combine` and by nothing else, deliberately. A brush deposits // dabs onto an empty field and has no idea what the rest of the frame is, // so a stroke shader cannot invert anything; doing it where the finished // part is read back is the one place that works for every kind of source. invert: u32, // Three scalars rather than a `vec3`: a three-component vector is // aligned to sixteen bytes in the uniform address space, so it would sit // at offset 144 and make this struct 160 bytes against the Rust side's // 144 — a mismatch wgpu reports as a binding too small for the shader, // several layers away from the padding that caused it. _pad0: u32, _pad1: u32, _pad2: u32, } @group(0) @binding(0) var p: MaskParams; // Compacted region id per pixel of the label field. Compacted rather than the // watershed's raw basin roots: the roots are sparse indices into pixel space, // so indexing a per-region array by one would need a table as large as the // image. The compaction happens once, when the segmentation is built. @group(0) @binding(1) var labels: array; // One entry per region: non-zero if the region is in this mask. Small — a few // thousand bytes — which is what makes changing a selection cheap. @group(0) @binding(2) var selected: array; // The **signed distance** from one subject's boundary, in proxy pixels: // positive inside, negative outside. A 1x1 placeholder when the layer is not a // subject — the binding is fixed, and a second pipeline differing only in what // it ignores would be worse than a wasted texel. // // A distance field rather than a finished alpha is what makes growing, // shrinking and feathering free: each is arithmetic on this, so a slider moves // a uniform instead of rebuilding a mask. @group(0) @binding(3) var subject: texture_2d; // TRACES: FR-DEV-10 // The photograph itself, as the demosaicer left it: camera RGB, unbalanced, // with no edit applied. A 1x1 placeholder for every mask that is a shape, // because the bindings are fixed and a second pipeline differing only in what // it ignores would cost more than one texel. // // **The unedited image, and that is the design rather than an accident of // pass order.** A band over the *edited* result would move as the edit was // made: raising the highlights would change which pixels counted as // highlights, so the slider would chase its own mask. Measuring what the // camera recorded means the selection stays where the photographer put it // while they work on it. @group(0) @binding(6) var image: texture_2d; // A full-screen triangle rather than a quad: three vertices instead of six, // no shared edge for the rasteriser to crack along, and no vertex buffer. @vertex fn vs(@builtin(vertex_index) i: u32) -> @builtin(position) vec4 { let x = f32(i32(i) / 2) * 4.0 - 1.0; let y = f32(i32(i) & 1) * 4.0 - 1.0; return vec4(x, y, 0.0, 1.0); } fn region_at(px: vec2) -> u32 { // Nearest-neighbour from output space into the label field. Deliberately // not bilinear: region ids are *names*, and the average of region 4 and // region 9 is not region 6. let fx = (f32(px.x) + 0.5) / f32(p.width); let fy = (f32(px.y) + 0.5) / f32(p.height); let lx = clamp(i32(fx * f32(p.label_width)), 0, i32(p.label_width) - 1); let ly = clamp(i32(fy * f32(p.label_height)), 0, i32(p.label_height) - 1); return labels[u32(ly) * p.label_width + u32(lx)]; } fn in_selection(px: vec2) -> f32 { let r = region_at(px); if (r >= p.region_count) { return 0.0; } return select(0.0, 1.0, selected[r] != 0u); } fn region_mask(px: vec2) -> f32 { let hard = in_selection(px); if (p.feather <= 0.0) { return hard; } // Box-average the binary selection over the feather radius. Cheap, and it // is the whole reason a region mask does not look cut out with scissors: // the watershed boundary is pixel-exact, which is correct and also harsher // than any edit wants at a subject's edge. let r = i32(ceil(p.feather)); var total = 0.0; var n = 0.0; for (var dy = -r; dy <= r; dy = dy + 1) { for (var dx = -r; dx <= r; dx = dx + 1) { let q = clamp( px + vec2(dx, dy), vec2(0, 0), vec2(i32(p.width) - 1, i32(p.height) - 1), ); total = total + in_selection(q); n = n + 1.0; } } return total / n; } // Offset from a gradient's centre, in the frame's own **isotropic** units: // y spans 0..1 and x spans 0..aspect, so a step of the same length means the // same distance whichever way it points. // // Without this the geometry lives in raw 0..1, where one axis is compressed // against the other by the aspect ratio — so a 45° ramp is not at 45° on // anything but a square frame, and a radial with equal radii draws an ellipse. // Both faults are invisible in the stored numbers and obvious the moment a // handle is dragged on a photograph, which is what this exists for. fn frame_delta(uv: vec2) -> vec2 { let aspect = vec2(f32(p.width) / f32(max(p.height, 1u)), 1.0); return (uv - p.centre) * aspect; } fn linear_mask(uv: vec2) -> f32 { // Signed distance along the ramp direction, from the centre. let d = dot(frame_delta(uv), p.axis); if (p.softness <= 0.0) { return select(0.0, 1.0, d >= 0.0); } return smoothstep(-p.softness * 0.5, p.softness * 0.5, d); } fn radial_mask(uv: vec2) -> f32 { let ca = cos(-p.angle); let sa = sin(-p.angle); let d = frame_delta(uv); // Into the ellipse's own frame, then normalised by its semi-axes so the // problem becomes a unit circle. let local = vec2(d.x * ca - d.y * sa, d.x * sa + d.y * ca); let r = length(local / max(p.axis, vec2(1e-6))); let edge = clamp(p.softness, 0.0, 1.0); if (edge <= 0.0) { return select(0.0, 1.0, r <= 1.0); } return 1.0 - smoothstep(1.0 - edge, 1.0, r); } // Coverage for one object, from its distance field. // // Bilinear on the *distance*, which is the reason this is a distance field at // all: distance varies smoothly across the boundary where coverage does not, // so interpolating it gives a clean sub-pixel edge even though the model's // own mask was quarter-resolution. fn subject_mask(uv: vec2) -> f32 { let dims = vec2(textureDimensions(subject)); let last = vec2(dims) - vec2(1); let t = uv * dims - vec2(0.5); let base = vec2(floor(t)); let f = fract(t); let p0 = clamp(base, vec2(0), last); let p1 = clamp(base + vec2(1), vec2(0), last); let a = textureLoad(subject, vec2(p0.x, p0.y), 0).r; let b = textureLoad(subject, vec2(p1.x, p0.y), 0).r; let c = textureLoad(subject, vec2(p0.x, p1.y), 0).r; let d = textureLoad(subject, vec2(p1.x, p1.y), 0).r; // `angle` carries the morphology offset in pixels: positive grows the // mask, negative shrinks it. Adding it before the falloff is what makes // dilation move the boundary rather than merely brighten the edge. let dist = mix(mix(a, b, f.x), mix(c, d, f.x), f.y) + p.angle; // `softness` is the feather half-width, also in pixels. if (p.softness <= 0.0) { return select(0.0, 1.0, dist >= 0.0); } let t_norm = dist / p.softness; // Every curve is 0.5 at the boundary, so changing the falloff changes how // the transition looks and never where it sits. switch p.falloff { case 0u: { return select(0.0, 1.0, dist >= 0.0); } case 1u: { return clamp(t_norm * 0.5 + 0.5, 0.0, 1.0); } case 3u: { return 1.0 / (1.0 + exp(-3.0 * t_norm)); } case 4u: { if (t_norm >= 0.0) { return 1.0 - 0.5 * exp(-3.0 * t_norm); } return 0.5 * exp(3.0 * t_norm); } default: { let x = clamp(t_norm * 0.5 + 0.5, 0.0, 1.0); return x * x * (3.0 - 2.0 * x); } } } // --------------------------------------------------------------------------- // Range masks (FR-DEV-10) // --------------------------------------------------------------------------- // // The masks that select by what a pixel *is* rather than by where it sits. // Nothing below reads `frame_delta`, and that absence is the point: a range is // not a function of position, so it cannot be stretched by an aspect ratio, // cannot drift under a crop, and comes out the same at a proxy size and at an // export because the only thing it depends on is the photograph's own values. // // The band arrives entirely in the fields the gradients use — `axis` is the // pair of bounds, `centre` is a colour range's arc, `softness` is the fade — // so a range costs nothing in the uniform beyond the image transform above. // Display-encoded sRGB back to linear. // // A JPEG is uploaded with its bytes untouched, so its values are gamma-encoded // where the demosaicer's are linear. The same undoing the generated adjust // shader does, at the same point and for the same reason: a band over // brightness is meaningless if two sources disagree about what a value means. fn decode_srgb(c: vec3) -> vec3 { let lo = c / 12.92; let hi = pow((max(c, vec3(0.04045)) + 0.055) / 1.055, vec3(2.4)); return select(hi, lo, c <= vec3(0.04045)); } // One source texel, as linear sRGB. // // This is the prologue of the generated adjust shader, repeated: decode, // balance, pull a clipped pixel back to neutral, then the camera matrix. It is // repeated rather than shared because the composer emits WGSL for the *edit* // and this pass is not one — but it must agree with it, since a range mask // exists to select the values the layer's own adjustments will then see. // // The highlight desaturation is the part that looks skippable and is not. A // fully clipped photosite arrives as (1,1,1), carrying no colour at all; the // as-shot multipliers are far from neutral, so balancing it and passing it // through the matrix produces a strong magenta. A colour range would then // select every blown sky as if the photographer had asked for magenta. fn source_texel(px: vec2) -> vec3 { var c = textureLoad(image, px, 0).rgb; if (p.as_shot_wb.w > 0.5) { c = decode_srgb(c); } let clipped = smoothstep(0.985, 1.0, max(c.r, max(c.g, c.b))); c = c * p.as_shot_wb.rgb; if (clipped > 0.0) { c = mix(c, vec3(max(c.r, max(c.g, c.b))), clipped); } return vec3( dot(p.cam_to_srgb_0.rgb, c), dot(p.cam_to_srgb_1.rgb, c), dot(p.cam_to_srgb_2.rgb, c), ); } // The most taps one mask texel averages, per axis. // // A cap rather than the true footprint. At a 1600 px proxy over a 24 MP frame // the ratio is under four, so this is the whole footprint for every ordinary // photograph; past it the taps stride across the footprint instead of // covering it, which is a sample of the area rather than its mean. That is the // right way to run out of budget here — the estimate gets noisier, it does not // start measuring somewhere else. const MAX_SOURCE_TAPS: i32 = 4; // The photograph's value under one mask texel, in linear sRGB. fn image_value(px: vec2) -> vec3 { let dims = vec2(textureDimensions(image)); let last = dims - vec2(1); // The footprint's top-left corner in source texels. Not a centre plus a // radius: the mask texel is a *box* over the source, and sampling // symmetrically about its centre would weight the middle of every // footprint twice at odd tap counts. let origin = vec2(px) * p.source_step; let taps = clamp(vec2(ceil(p.source_step)), vec2(1), vec2(MAX_SOURCE_TAPS)); // `stride`, not `step`: WGSL has a builtin of that name, and a local that // shadows one is legal and unreadable in the same breath. let stride = p.source_step / vec2(taps); var total = vec3(0.0); for (var y = 0; y < taps.y; y = y + 1) { for (var x = 0; x < taps.x; x = x + 1) { let at = origin + (vec2(f32(x), f32(y)) + vec2(0.5)) * stride; total = total + source_texel(clamp(vec2(at), vec2(0), last)); } } return total / f32(taps.x * taps.y); } // A soft band: one inside, nothing outside, a smooth ramp across each edge. // // The `min` rather than a product of the two ramps. A band narrower than twice // its softness has no plateau, and multiplying the rising and falling ramps // would then peak well below one — so "select the highlights" would come out // at sixty per cent and the photographer would compensate with opacity, // against a mask that was quietly weaker than it said. `min` keeps the // plateau where there is one and degrades to a single peak where there is not. fn band(v: f32, lo: f32, hi: f32, soft: f32) -> f32 { if (soft <= 0.0) { return select(0.0, 1.0, v >= lo && v <= hi); } return min(smoothstep(lo - soft, lo, v), 1.0 - smoothstep(hi, hi + soft, v)); } fn luminance_mask(px: vec2) -> f32 { let y = dot(image_value(px), vec3(0.2126, 0.7152, 0.0722)); // Onto the perceptual position `tone_position` in `ops/_helpers.yaml` // establishes, which is where the stored bounds are measured. Linear light // puts middle grey at 0.18, so a band stated in it would spend four fifths // of its travel inside the shadows. let t = clamp(pow(max(y, 0.0), 1.0 / 3.0), 0.0, 1.0); return band(t, p.axis.x, p.axis.y, p.softness); } // Hue in turns, 0 at red and increasing through yellow. // // The plain six-sector definition. Zero for a neutral, which is a value the // caller must not act on — the chroma bound below is what keeps a colour range // away from the greys where this number is rounding noise. fn hue_of(c: vec3) -> f32 { let hi = max(c.r, max(c.g, c.b)); let lo = min(c.r, min(c.g, c.b)); let d = hi - lo; if (d <= 0.0) { return 0.0; } var h = 0.0; if (hi == c.r) { h = (c.g - c.b) / d; } else if (hi == c.g) { h = (c.b - c.r) / d + 2.0; } else { h = (c.r - c.g) / d + 4.0; } return fract(h / 6.0); } fn colour_mask(px: vec2) -> f32 { let c = max(image_value(px), vec3(0.0)); let hi = max(c.r, max(c.g, c.b)); let lo = min(c.r, min(c.g, c.b)); // The max-minus-min chroma `colour_saturation` uses, so the number the // band is stated in is the one the rest of the pipeline means by // 'colourfulness'. var chroma = 0.0; if (hi > 0.0) { chroma = (hi - lo) / hi; } // Distance round the circle, so an arc centred near red reaches both ways // past zero. Written as a wrap rather than as two comparisons because red // is exactly where skin sits, and an arc that stopped at the seam would // select half of it. let d = abs(fract(hue_of(c) - p.centre.x + 0.5) - 0.5); var arc = 0.0; if (p.softness <= 0.0) { arc = select(0.0, 1.0, d <= p.centre.y); } else { arc = 1.0 - smoothstep(p.centre.y, p.centre.y + p.softness, d); } // Both, not either: an arc alone selects a haze of noise everywhere the // picture is nearly grey, because a hue rounded out of three almost-equal // channels is still a hue. return min(arc, band(chroma, p.axis.x, p.axis.y, p.softness)); } @fragment fn fs(@builtin(position) pos: vec4) -> @location(0) vec4 { let px = vec2(i32(pos.x), i32(pos.y)); // Normalised, so a gradient's geometry survives a crop or an export at // another size — the mask is defined on the frame, not on a pixel count. let uv = vec2(pos.x / f32(p.width), pos.y / f32(p.height)); var m = 0.0; switch p.mode { case 0u: { m = region_mask(px); } case 1u: { m = linear_mask(uv); } case 2u: { m = radial_mask(uv); } case 3u: { m = subject_mask(uv); } case 5u: { m = luminance_mask(px); } case 6u: { m = colour_mask(px); } default: { m = 0.0; } } return vec4(clamp(m, 0.0, 1.0), 0.0, 0.0, 1.0); } // --------------------------------------------------------------------------- // Brush strokes (ARCH §5.4) // --------------------------------------------------------------------------- // // The mask the architecture was written for. What arrives is a list of // positions, a radius, a hardness and a flow; what leaves is pixels. Nothing // between the two ever exists in CPU memory, which is the whole difference from // darktable, where the same strokes are rasterised on the CPU and the lag makes // painting unusable. // // # Why the strokes are not drawn by the full-screen triangle above // // Cost. A swept disc is the minimum distance to any segment of its polyline, so // evaluating one stroke costs a distance per segment *per pixel*. Over the // whole frame that is `pixels × segments`, and a stroke that wandered across // the photograph has both terms large at once. // // So each stroke is drawn over its own bounding box instead, expanded by the // radius. The rasteriser then never invokes the fragment shader for a pixel the // stroke cannot reach, and the cost becomes `area(box) × segments` — for the // ordinary case, a dab or a swipe, a small fraction of the frame. The model // splits a long gesture into strokes of bounded length for the same reason: // both terms of that product grow with how far one stroke travelled. // // # Why the strokes composite with fixed-function blending // // Add is `dst + a(1 - dst)` and erase is `dst(1 - a)`, which are exactly a // source-over and a one-minus-source blend. Expressing them as blend state // rather than as arithmetic in the shader is what allows one draw per stroke: // the accumulating mask is the attachment, and no pass ever has to read the // slice it is writing. struct StrokeHeader { // Bounding box in normalised coordinates, already grown by the radius and // a texel — the vertex shader trusts it and draws nothing outside it. lo: vec2, hi: vec2, // Radius in units of the frame's shorter edge, so a dab is round on a frame // that is not square. radius: f32, // Fraction of the radius that is fully covered. hardness: f32, // Coverage deposited where the stroke is solid. flow: f32, // Window into `stroke_points`. first: u32, count: u32, _pad: u32, } @group(0) @binding(4) var strokes: array; @group(0) @binding(5) var stroke_points: array>; struct BrushVertex { @builtin(position) pos: vec4, // Flat: a stroke index interpolated across its own quad would name a // different stroke in the middle of it. @location(0) @interpolate(flat) stroke: u32, } // Six vertices per stroke, non-instanced. // // Deliberately not one instance per stroke: `@builtin(instance_index)` with a // non-zero first instance needs base-instance support, which the GL backend // this has to run on under Android cannot promise. Dividing the vertex index // costs one integer operation and works everywhere. @vertex fn vs_brush(@builtin(vertex_index) v: u32) -> BrushVertex { var quad = array, 6>( vec2(0.0, 0.0), vec2(1.0, 0.0), vec2(0.0, 1.0), vec2(0.0, 1.0), vec2(1.0, 0.0), vec2(1.0, 1.0), ); let i = v / 6u; let s = strokes[i]; let uv = mix(s.lo, s.hi, quad[v % 6u]); var out: BrushVertex; // y is flipped because normalised mask coordinates run downwards, the way // the fragment shader above reads them, and clip space runs upwards. A // stroke drawn without this lands mirrored about the horizon, which is // plausible enough on a symmetric test image to survive a careless check. out.pos = vec4(uv.x * 2.0 - 1.0, 1.0 - uv.y * 2.0, 0.0, 1.0); out.stroke = i; return out; } // Into units of the frame's shorter edge. // // Without this the brush would be a circle in normalised coordinates, which on // a 3:2 frame is an ellipse half again as wide as it is tall. A brush whose dab // is not round is not a brush. fn to_square(uv: vec2) -> vec2 { let dims = vec2(f32(p.width), f32(p.height)); return uv * dims / min(dims.x, dims.y); } fn segment_distance(q: vec2, a: vec2, b: vec2) -> f32 { let ab = b - a; let len2 = dot(ab, ab); // A finger that stopped and went back leaves a zero-length segment, and // dividing by its length is a NaN — which propagates through the min() // below and takes the whole stroke with it. if (len2 <= 1e-12) { return length(q - a); } let t = clamp(dot(q - a, ab) / len2, 0.0, 1.0); return length(q - (a + ab * t)); } @fragment fn fs_brush(in: BrushVertex) -> @location(0) vec4 { let s = strokes[in.stroke]; let q = to_square(vec2(in.pos.x / f32(p.width), in.pos.y / f32(p.height))); // The *minimum* over the segments, which is the maximum of their coverage. // Accumulating the segments instead would make a stroke that crosses itself // — every circle, every scribble — build up a bright patch where it did, // and a soft brush would go blotchy along any curve tight enough for // consecutive dabs to overlap, which is all of them. var d = 1e30; if (s.count == 1u) { // A tap. One point is a legitimate stroke, and it paints one dab. d = length(q - to_square(stroke_points[s.first])); } else { for (var k = 0u; k + 1u < s.count; k = k + 1u) { d = min( d, segment_distance( q, to_square(stroke_points[s.first + k]), to_square(stroke_points[s.first + k + 1u]), ), ); } } // Even at full hardness the edge keeps a one-pixel ramp. A true step would // alias into a staircase, and the mask is sampled bilinearly at whatever // zoom the user is inspecting it at — which is where an edge is judged. let texel = 1.0 / f32(min(p.width, p.height)); let inner = min(s.radius * clamp(s.hardness, 0.0, 1.0), max(s.radius - texel, 0.0)); let coverage = 1.0 - smoothstep(inner, s.radius, d); return vec4(clamp(coverage * s.flow, 0.0, 1.0), 0.0, 0.0, 1.0); } // --------------------------------------------------------------------------- // Joining one part to the mask so far // --------------------------------------------------------------------------- // // A layer's mask is a fold over its parts, and the set operation is the *blend // state* rather than arithmetic here: union is `max(dst, src)`, subtraction is // `dst * (1 - src)`. Both are fixed-function, so joining a part costs one // full-screen draw and no second texture beyond the one being read. // // # Why a part is drawn aside first, rather than straight onto the mask // // Because an erase stroke inside a part means "a hole in *this* part", not "a // hole in the mask". Painted straight onto the accumulator it would take away // whatever the parts before it had put there — so a correction that tidied its // own edge would punch through the subject underneath, and the failure would // look like the model's mask had holes in it. @group(0) @binding(7) var part_mask: texture_2d; @fragment fn fs_combine(@builtin(position) pos: vec4) -> @location(0) vec4 { let v = textureLoad(part_mask, vec2(i32(pos.x), i32(pos.y)), 0).r; let m = select(v, 1.0 - v, p.invert != 0u); return vec4(clamp(m, 0.0, 1.0), 0.0, 0.0, 1.0); }