Mask the subject the model found, not the regions underneath it

The watershed hierarchy does not survive a photograph, so local masking
stops depending on it. A layer can now be one recognised object, and the
object's own coverage is the mask.

`Options::watershed` defaults off. It costs ~80 ms plus a full-resolution
readback to produce a ladder that collapses, and paying that on every
photograph buys a control that misleads. Kept switchable rather than
deleted: the passes and the hierarchy are correct in themselves and it is
the merge criterion that fails, which is a change to one function.

Masks now rasterise in **source** space at proxy resolution and are sampled
by the composed shader after the framing map. That fixes a real bug: they
were rasterised in output space, so zooming slid the photograph underneath
a mask that stayed pinned to the viewport, and cropping moved every
adjustment to a different part of the picture. Doing it this way also
leaves the framing map in exactly one place — a second copy in the mask
shader would have been a second thing to keep in step, failing only when
straightened.

A subject is stored as identity, not pixels: the mask is megabytes and is
reproducible by running the same model over the same image, so the sidecar
carries the index, the class and the score, and the session carries the
pixels. The class is there to be checked — if instance 3 comes back a "car"
where it was a "dog", something changed and the layer is stale rather than
silently masking the wrong thing.

The overlay now draws instances and is transparent everywhere else. The
region version covered every pixel and so hid the photograph it was drawn
over; the question it exists to answer is whether an outline follows the
subject, which you can only answer by seeing both.

`examples/local.rs` is the worked example: subject in colour with the rest
monochrome, and the subject lifted out of its background. Run on a 5472x3648
CR2 it finds two people and two cars, and the colour-pop keeps her hat and
hair while the wall and grass behind go grey.
This commit is contained in:
2026-08-22 08:39:17 +02:00
parent b1433ad4a9
commit ee10097435
13 changed files with 963 additions and 101 deletions
+39 -1
View File
@@ -28,7 +28,7 @@ struct MaskParams {
label_width: u32,
label_height: u32,
// 0 = regions, 1 = linear, 2 = radial.
// 0 = regions, 1 = linear, 2 = radial, 3 = subject.
mode: u32,
// How many regions the label field holds, so an out-of-range label is
// caught rather than read past the end of `selected`.
@@ -57,6 +57,10 @@ struct MaskParams {
// One entry per region: non-zero if the region is in this mask. Small — a few
// thousand bytes — which is what makes changing a selection cheap.
@group(0) @binding(2) var<storage, read> selected: array<u32>;
// One recognised object's coverage, for a subject mask. A 1x1 placeholder
// when the layer is not one — the binding is fixed, and a second pipeline
// differing only in what it ignores would be worse than a wasted texel.
@group(0) @binding(3) var subject: texture_2d<f32>;
// A full-screen triangle rather than a quad: three vertices instead of six,
// no shared edge for the rasteriser to crack along, and no vertex buffer.
@@ -138,6 +142,39 @@ fn radial_mask(uv: vec2<f32>) -> f32 {
return 1.0 - smoothstep(1.0 - edge, 1.0, r);
}
// The model's coverage for one object, resampled to the mask's own grid.
//
// Bilinear, unlike the region lookup above: this is a *quantity*, not a name,
// so the value between two samples is meaningful. The model's own mask is a
// quarter-resolution sigmoid, and interpolating it is what stops the outline
// stair-stepping in blocks of four.
fn subject_mask(uv: vec2<f32>) -> f32 {
let dims = vec2<f32>(textureDimensions(subject));
let last = vec2<i32>(dims) - vec2<i32>(1);
let t = uv * dims - vec2<f32>(0.5);
let base = vec2<i32>(floor(t));
let f = fract(t);
let p0 = clamp(base, vec2<i32>(0), last);
let p1 = clamp(base + vec2<i32>(1), vec2<i32>(0), last);
let a = textureLoad(subject, vec2<i32>(p0.x, p0.y), 0).r;
let b = textureLoad(subject, vec2<i32>(p1.x, p0.y), 0).r;
let c = textureLoad(subject, vec2<i32>(p0.x, p1.y), 0).r;
let d = textureLoad(subject, vec2<i32>(p1.x, p1.y), 0).r;
let cov = mix(mix(a, b, f.x), mix(c, d, f.x), f.y);
// `softness` carries the layer's feather here. Zero gives the model's own
// soft edge untouched, which is a perfectly good mask edge and a better
// default than imposing a ramp on top of one that already exists.
if (p.softness <= 0.0) {
return cov;
}
return smoothstep(0.5 - p.softness, 0.5 + p.softness, cov);
}
@fragment
fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
let px = vec2<i32>(i32(pos.x), i32(pos.y));
@@ -151,6 +188,7 @@ fn fs(@builtin(position) pos: vec4<f32>) -> @location(0) vec4<f32> {
case 0u: { m = region_mask(px); }
case 1u: { m = linear_mask(uv); }
case 2u: { m = radial_mask(uv); }
case 3u: { m = subject_mask(uv); }
default: { m = 0.0; }
}