Let a mask start from a tone or a colour, not only a shape
Every local adjustment began from a shape: painted, drawn with a handle, or found by a model. So the only way to hold back a sky was to draw a line near where it ended, and the only way to warm skin was to paint round it — both of which put the edit's edge where the photographer put a gesture rather than where the picture changes. A gradient across a treeline halos, and an adjustment traced round a face stops on the outline of a hand. MaskSource grows two variants that select by what a pixel *is*. Luminance carries two bounds on the perceptual tone scale plus a softness; Colour carries an arc of hue, a range of chroma, and one softness for every edge of both. Five floats and three, so they diff, sync and merge per field under FR-NC-9 exactly as a gradient's geometry does — the property a stored raster has none of, and the reason the model's coverage had to sit beside its source rather than inside it. The pixels are the shader's business and nowhere else's. `mask.wgsl` takes the demosaiced source as a sixth binding and two new modes read it: decode, balance, pull a clipped photosite back to neutral, apply the camera matrix, then weigh the band. Nothing crosses to the CPU but the numbers and the matrix, and each mask texel averages its own footprint in the source, so a band lands on the tone an area is rather than on whichever texel a proxy grid happened to land on. The photograph it measures is the one the camera recorded, before this edit. A band over the edited result would slide out from under the edit as the edit was made — raising the highlights would change which pixels counted as highlights, and the slider would chase its own mask. Feather, falloff and morphology stay off a range layer, which is what `shapeable` already meant. All three are functions of the signed distance from a boundary, and a range has no boundary to be at a distance from; its edge is the softness of its own band, in the band's units. Offering them would be four controls that move and change nothing.
This commit is contained in:
+193
-12
@@ -21,6 +21,17 @@
|
||||
//! boxes, one draw each, compositing onto the slice with blend state — see the
|
||||
//! second half of `mask.wgsl`.
|
||||
//!
|
||||
//! # The photograph, bound as an input
|
||||
//!
|
||||
//! A range mask (FR-DEV-10) selects by what a pixel *is*, so this pass reads
|
||||
//! the demosaiced source as well as writing masks. It is bound for every draw
|
||||
//! and looked at by two modes; everything else gets a 1x1 placeholder, for the
|
||||
//! reason the label field below does — the bindings are fixed, and a second
|
||||
//! pipeline differing only in what it ignores costs more than a texel.
|
||||
//!
|
||||
//! Nothing is read back and nothing is rasterised on this side. What crosses
|
||||
//! into CPU memory for a range layer is five floats and a matrix.
|
||||
//!
|
||||
//! # The label field
|
||||
//!
|
||||
//! Region masks index a compacted label field uploaded once per segmentation.
|
||||
@@ -31,9 +42,10 @@
|
||||
//! same place and cadence the region adjacency graph is already built at.
|
||||
|
||||
use dr_pipeline::mask::{MaskSource, MaskStack, Stroke, MAX_LAYERS};
|
||||
|
||||
use wgpu::util::DeviceExt;
|
||||
|
||||
use crate::{GpuContext, GpuError};
|
||||
use crate::{DemosaicedImage, GpuContext, GpuError};
|
||||
|
||||
/// Modes understood by `mask.wgsl`. Kept beside the shader's `switch`.
|
||||
const MODE_REGIONS: u32 = 0;
|
||||
@@ -43,6 +55,10 @@ const MODE_SUBJECT: u32 = 3;
|
||||
/// Brush layers go through their own entry points rather than the `switch`, so
|
||||
/// this is only ever read by a person looking at a captured frame.
|
||||
const MODE_BRUSH: u32 = 4;
|
||||
/// TRACES: FR-DEV-10
|
||||
const MODE_LUMINANCE: u32 = 5;
|
||||
/// TRACES: FR-DEV-10
|
||||
const MODE_COLOUR: u32 = 6;
|
||||
|
||||
/// Six vertices — two triangles — per stroke. See `vs_brush`.
|
||||
const VERTICES_PER_STROKE: u32 = 6;
|
||||
@@ -66,7 +82,19 @@ struct MaskParams {
|
||||
axis: [f32; 2],
|
||||
softness: f32,
|
||||
angle: f32,
|
||||
_pad1: [f32; 2],
|
||||
/// TRACES: FR-DEV-10
|
||||
/// Source texels per mask texel, per axis. See `image_value` in the
|
||||
/// shader for why a range averages its footprint rather than sampling it.
|
||||
source_step: [f32; 2],
|
||||
|
||||
/// Camera RGB → linear sRGB, one row per `vec4` because that is the
|
||||
/// alignment a uniform gives a three-component vector anyway. Only a
|
||||
/// range mask reads them.
|
||||
cam_to_srgb: [[f32; 4]; 3],
|
||||
/// `rgb`: as-shot white balance. `w`: non-zero for a gamma-encoded source.
|
||||
/// The same packing the generated adjust shader uses, so the two agree by
|
||||
/// construction rather than by inspection.
|
||||
as_shot_wb: [f32; 4],
|
||||
}
|
||||
|
||||
/// One stroke, as `mask.wgsl`'s `StrokeHeader` expects it.
|
||||
@@ -371,6 +399,14 @@ pub struct MaskPass {
|
||||
/// label slots even when rasterising a gradient. A placeholder is cheaper
|
||||
/// and far simpler than two pipelines differing only in what they ignore.
|
||||
placeholder: LabelField,
|
||||
/// TRACES: FR-DEV-10
|
||||
/// Bound at the image slot for every mask that is not a range.
|
||||
///
|
||||
/// Never sampled by those modes, so its contents do not matter — but it is
|
||||
/// cleared rather than left undefined, because a placeholder whose value
|
||||
/// is arbitrary is one that makes a binding mistake look like a mask that
|
||||
/// nearly works.
|
||||
empty_image: wgpu::TextureView,
|
||||
}
|
||||
|
||||
impl MaskPass {
|
||||
@@ -405,6 +441,20 @@ impl MaskPass {
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
// TRACES: FR-DEV-10
|
||||
// The photograph, for a range mask. Unfilterable for the
|
||||
// same reason the field above is: every read is a
|
||||
// `textureLoad`, and this pipeline binds no sampler.
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
binding: 6,
|
||||
visibility: wgpu::ShaderStages::FRAGMENT,
|
||||
ty: wgpu::BindingType::Texture {
|
||||
sample_type: wgpu::TextureSampleType::Float { filterable: false },
|
||||
view_dimension: wgpu::TextureViewDimension::D2,
|
||||
multisampled: false,
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
@@ -515,6 +565,7 @@ impl MaskPass {
|
||||
}
|
||||
|
||||
let placeholder = LabelField::upload(ctx, &[0], 1, 1, 0)?;
|
||||
let empty_image = empty_image(ctx);
|
||||
// Everywhere outside, so a layer that somehow reaches this masks
|
||||
// nothing rather than everything.
|
||||
let empty_subject = SubjectMasks::upload(ctx, &[&[-1.0f32][..]], 1, 1)?;
|
||||
@@ -530,6 +581,7 @@ impl MaskPass {
|
||||
allocations: 0,
|
||||
placeholder,
|
||||
empty_subject,
|
||||
empty_image,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -538,11 +590,17 @@ impl MaskPass {
|
||||
/// `labels` may be `None` when no layer is a region mask; a region layer
|
||||
/// without one is skipped rather than drawn wrong, since a mask that
|
||||
/// silently covers the whole frame would apply an edit everywhere.
|
||||
///
|
||||
/// `source` is the photograph a range layer measures (FR-DEV-10), and it
|
||||
/// is skipped on the same rule for the same reason: without it the shader
|
||||
/// would read a blank placeholder, and a band that happens to contain
|
||||
/// black would then cover the whole frame.
|
||||
pub fn render(
|
||||
&mut self,
|
||||
stack: &MaskStack,
|
||||
labels: Option<&LabelField>,
|
||||
subjects: Option<&SubjectMasks>,
|
||||
source: Option<&DemosaicedImage>,
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> Result<&MaskArray, GpuError> {
|
||||
@@ -597,7 +655,24 @@ impl MaskPass {
|
||||
_ => (&self.empty_subject, 0),
|
||||
};
|
||||
|
||||
let params = self.params(layer, field, width, height);
|
||||
// TRACES: FR-DEV-10
|
||||
// A range layer with no photograph bound is skipped rather than
|
||||
// drawn against the placeholder, on exactly the rule the two
|
||||
// cases above follow: an absent mask that defaults to "everything"
|
||||
// takes a local adjustment global, which is a far quieter failure
|
||||
// than a layer that visibly did not render.
|
||||
let image = match (&layer.source, source) {
|
||||
(s, None) if s.is_range() => {
|
||||
log::warn!(
|
||||
"mask layer {} selects a range with no image loaded; skipping",
|
||||
layer.id
|
||||
);
|
||||
continue;
|
||||
}
|
||||
(_, image) => image,
|
||||
};
|
||||
|
||||
let params = self.params(layer, field, image, width, height);
|
||||
match &layer.source {
|
||||
MaskSource::Brush { strokes } => {
|
||||
self.draw_brush(&mut encoder, slot as u32, ¶ms, strokes, width, height)
|
||||
@@ -611,6 +686,7 @@ impl MaskPass {
|
||||
field,
|
||||
&selected,
|
||||
subject,
|
||||
image,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -634,9 +710,34 @@ impl MaskPass {
|
||||
&self,
|
||||
layer: &dr_pipeline::mask::MaskLayer,
|
||||
field: &LabelField,
|
||||
source: Option<&DemosaicedImage>,
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> MaskParams {
|
||||
// TRACES: FR-DEV-10
|
||||
// How much of the photograph one mask texel covers. One when there is
|
||||
// no image bound, which is a value nothing reads — the range modes are
|
||||
// the only readers and they are skipped in that case.
|
||||
let source_step = match source {
|
||||
Some(image) => {
|
||||
let (sw, sh) = image.size();
|
||||
[
|
||||
sw as f32 / width.max(1) as f32,
|
||||
sh as f32 / height.max(1) as f32,
|
||||
]
|
||||
}
|
||||
None => [1.0, 1.0],
|
||||
};
|
||||
// Row-major nine, widened to three `vec4`s. Identity where there is no
|
||||
// image, so a range that somehow reached the shader without one would
|
||||
// read camera values rather than nothing — the same defensive choice
|
||||
// the demosaicer makes for an uncalibrated body.
|
||||
let m = source.map_or([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0], |i| {
|
||||
i.color_matrix()
|
||||
});
|
||||
let wb = source.map_or([1.0, 1.0, 1.0], |i| i.as_shot_wb());
|
||||
let non_linear = source.is_some_and(|i| i.is_non_linear());
|
||||
|
||||
let base = MaskParams {
|
||||
width,
|
||||
height,
|
||||
@@ -650,7 +751,13 @@ impl MaskPass {
|
||||
axis: [1.0, 0.0],
|
||||
softness: 0.0,
|
||||
angle: 0.0,
|
||||
_pad1: [0.0, 0.0],
|
||||
source_step,
|
||||
cam_to_srgb: [
|
||||
[m[0], m[1], m[2], 0.0],
|
||||
[m[3], m[4], m[5], 0.0],
|
||||
[m[6], m[7], m[8], 0.0],
|
||||
],
|
||||
as_shot_wb: [wb[0], wb[1], wb[2], if non_linear { 1.0 } else { 0.0 }],
|
||||
};
|
||||
|
||||
match &layer.source {
|
||||
@@ -719,6 +826,33 @@ impl MaskPass {
|
||||
mode: MODE_BRUSH,
|
||||
..base
|
||||
},
|
||||
// TRACES: FR-DEV-10
|
||||
// A band, carried in the fields the gradients measure geometry
|
||||
// in. Reused rather than given their own, and it is not a
|
||||
// shortcut: `centre` and `axis` are two pairs of floats whose
|
||||
// meaning has always been the mode's to decide, and a range that
|
||||
// added four more would grow the uniform every other mask pays
|
||||
// for. What matters is that nothing here is a *coordinate* — a
|
||||
// range is not a function of position at all.
|
||||
MaskSource::Luminance { lo, hi, softness } => MaskParams {
|
||||
mode: MODE_LUMINANCE,
|
||||
axis: [*lo, *hi],
|
||||
softness: *softness,
|
||||
..base
|
||||
},
|
||||
MaskSource::Colour {
|
||||
hue,
|
||||
hue_width,
|
||||
chroma_lo,
|
||||
chroma_hi,
|
||||
softness,
|
||||
} => MaskParams {
|
||||
mode: MODE_COLOUR,
|
||||
centre: [*hue, *hue_width],
|
||||
axis: [*chroma_lo, *chroma_hi],
|
||||
softness: *softness,
|
||||
..base
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -887,6 +1021,7 @@ impl MaskPass {
|
||||
field: &LabelField,
|
||||
selected: &wgpu::Buffer,
|
||||
subject: (&SubjectMasks, usize),
|
||||
source: Option<&DemosaicedImage>,
|
||||
) {
|
||||
let params_buf = self
|
||||
.ctx
|
||||
@@ -924,6 +1059,13 @@ impl MaskPass {
|
||||
}),
|
||||
),
|
||||
},
|
||||
// TRACES: FR-DEV-10
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 6,
|
||||
resource: wgpu::BindingResource::TextureView(
|
||||
source.map_or(&self.empty_image, |i| i.view()),
|
||||
),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
@@ -1005,6 +1147,38 @@ impl MaskPass {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-10
|
||||
/// A single black texel, bound at the image slot for a mask that is not a
|
||||
/// range.
|
||||
///
|
||||
/// Written rather than merely allocated. Undefined contents would be read by
|
||||
/// nothing today, but a binding mistake in a range mask would then produce
|
||||
/// whatever the driver left in memory — a mask that flickers between builds
|
||||
/// and machines, which is the hardest shape of bug this pass could have.
|
||||
fn empty_image(ctx: &GpuContext) -> wgpu::TextureView {
|
||||
let texture = ctx.device.create_texture_with_data(
|
||||
&ctx.queue,
|
||||
&wgpu::TextureDescriptor {
|
||||
label: Some("mask-empty-image"),
|
||||
size: wgpu::Extent3d {
|
||||
width: 1,
|
||||
height: 1,
|
||||
depth_or_array_layers: 1,
|
||||
},
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format: DemosaicedImage::FORMAT,
|
||||
usage: wgpu::TextureUsages::TEXTURE_BINDING,
|
||||
view_formats: &[],
|
||||
},
|
||||
wgpu::util::TextureDataOrder::LayerMajor,
|
||||
// Four half-floats of zero. Rgba16Float, so eight bytes.
|
||||
&[0u8; 8],
|
||||
);
|
||||
texture.create_view(&wgpu::TextureViewDescriptor::default())
|
||||
}
|
||||
|
||||
/// The shorter edge of the space the mask is rasterised in.
|
||||
///
|
||||
/// Feather and morphology are stored as fractions of it, so the same edit is
|
||||
@@ -1131,7 +1305,7 @@ mod tests {
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
let array = pass
|
||||
.render(&stack, Some(&field), None, w, h)
|
||||
.render(&stack, Some(&field), None, None, w, h)
|
||||
.expect("render");
|
||||
assert_eq!(array.size(), (w, h));
|
||||
assert_eq!(array.layers(), 1);
|
||||
@@ -1153,7 +1327,7 @@ mod tests {
|
||||
}));
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
assert!(pass.render(&stack, None, None, 8, 8).is_ok());
|
||||
assert!(pass.render(&stack, None, None, None, 8, 8).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -1176,7 +1350,9 @@ mod tests {
|
||||
}));
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
let array = pass.render(&stack, None, None, 16, 16).expect("render");
|
||||
let array = pass
|
||||
.render(&stack, None, None, None, 16, 16)
|
||||
.expect("render");
|
||||
assert_eq!(array.layers(), 2, "one slice per active layer");
|
||||
}
|
||||
|
||||
@@ -1208,7 +1384,9 @@ mod tests {
|
||||
stack.push(painted(&[(false, 0.1, vec![(0.2, 0.2), (0.8, 0.8)])]));
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
let array = pass.render(&stack, None, None, 32, 32).expect("render");
|
||||
let array = pass
|
||||
.render(&stack, None, None, None, 32, 32)
|
||||
.expect("render");
|
||||
assert_eq!(array.layers(), 1);
|
||||
}
|
||||
|
||||
@@ -1258,7 +1436,7 @@ mod tests {
|
||||
};
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
let array = pass
|
||||
.render(&MaskStack::new(), None, None, 8, 8)
|
||||
.render(&MaskStack::new(), None, None, None, 8, 8)
|
||||
.expect("render");
|
||||
assert_eq!(
|
||||
array.layers(),
|
||||
@@ -1281,17 +1459,20 @@ mod tests {
|
||||
}));
|
||||
|
||||
let mut pass = MaskPass::new(&ctx).expect("mask pass");
|
||||
pass.render(&stack, None, None, 32, 32).expect("render");
|
||||
pass.render(&stack, None, None, None, 32, 32)
|
||||
.expect("render");
|
||||
assert_eq!(pass.allocations(), 1);
|
||||
|
||||
pass.render(&stack, None, None, 32, 32).expect("render");
|
||||
pass.render(&stack, None, None, None, 32, 32)
|
||||
.expect("render");
|
||||
assert_eq!(
|
||||
pass.allocations(),
|
||||
1,
|
||||
"same size and layer count should not reallocate"
|
||||
);
|
||||
|
||||
pass.render(&stack, None, None, 64, 64).expect("render");
|
||||
pass.render(&stack, None, None, None, 64, 64)
|
||||
.expect("render");
|
||||
assert_eq!(pass.allocations(), 2, "a resize must reallocate");
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user