FR-DEV-19a said a part is added to the mask or taken out of it, and mask-editing.md listed Intersect as an M2 item with nothing built. Both now say what shipped: the requirement names the third join and why it is a product rather than a minimum, and how old and new sidecars read across it; the plan marks the blend-table row built, names the tests that hold the GPU to the definition, and splits M2 into what is done and what is still outstanding (joining non-painted parts from the panel, per-part distance fields, folding two layers).
799 lines
39 KiB
Markdown
799 lines
39 KiB
Markdown
# Editing a mask
|
||
|
||
**Status:** Draft · 2026-09-06
|
||
**Companion to:** [requirements.md](requirements.md) §3.3 FR-DEV-3, FR-DEV-10 ·
|
||
[segmentation.md](segmentation.md) · [architecture.md](architecture.md) §5.4 ·
|
||
[spot-removal.md](spot-removal.md) (the pattern a canvas tool follows here)
|
||
|
||
The model finds a subject in a second and the photographer cannot then change
|
||
it by a single pixel. This document specifies the tools that close that gap —
|
||
paint, erase, push, and combining one selection with another — and the
|
||
interface they are driven through.
|
||
|
||
---
|
||
|
||
## 1. The gap, precisely
|
||
|
||
The core is further along than the interface, and it is worth being exact
|
||
about which half is missing, because it changes the size of the work.
|
||
|
||
**Painting exists everywhere except where a finger is.**
|
||
[`MaskSource::Brush`](../../core/dr-pipeline/src/mask.rs), [`Stroke`], the
|
||
simplification and the point budgets, the sidecar's `stroke = …` line and its
|
||
parser, the GPU's per-stroke bounding-box draw with add and erase blend
|
||
states — all of it is written, tested, and reachable from no control in the
|
||
application. [`toolrail.slint:138`](../../ui/dr-ui/ui/toolrail.slint#L138) says so
|
||
in as many words: *"what is missing is the canvas interaction"*.
|
||
|
||
**A mask has exactly one source.** A layer is one `MaskSource` and a shaping
|
||
of its edge. There is no way to say *this subject **and** that one*, *the sky
|
||
**except** the branches*, or *the subject **only where** it is bright* — the
|
||
last being an intersection of a model mask with a range mask (FR-DEV-10), two
|
||
features that already exist and cannot meet.
|
||
|
||
**The edge moves all at once or not at all.** `Morphology` grows or shrinks
|
||
the *whole* boundary. When the model's coverage stops two pixels inside the
|
||
shoulder and leaks four pixels into the hair, no global number fixes both, and
|
||
that is the ordinary case rather than a corner one.
|
||
|
||
**And you cannot see the mask.** *(Built — see §6.)* The overlay on the canvas
|
||
was [`overlay_rgba`](../../ui/dr-ui/src/segmentation.rs) and nothing else — a
|
||
CPU-built false-colour picture of *what the model detected*, at proxy
|
||
resolution. It is not the layer's alpha: it knows nothing of the layer's
|
||
feather, its falloff, its morphology, its invert, or its opacity. Nobody can
|
||
refine an edge they are not being shown.
|
||
|
||
That one turned out to be load-bearing for the other three rather than the
|
||
last of four. With no way to see a mask, choosing a category produced a layer
|
||
whose extent was invisible and whose adjustment had not been touched yet — so
|
||
the correct behaviour and the broken one look identical, and "the segmentation
|
||
does not make masks" is what it reads as from the outside.
|
||
|
||
Those four are one feature, and this is its specification.
|
||
|
||
## 2. Non-goals
|
||
|
||
- **Not pixel layers.** §1.3 of the requirements excludes them. Everything
|
||
below stores geometry and parameters; no rasterised mask is ever written to
|
||
a file, and no mask ever exists in CPU memory (ARCH §5.4).
|
||
- **Not a second segmentation UI.** Choosing *which* subject or category the
|
||
model offers stays exactly as it is. This is about what happens after.
|
||
- **Not per-layer neighbourhood adjustments.** `layer_chain()` excludes detail
|
||
and optics operations for reasons that have not changed.
|
||
- **Not automatic refinement.** No "improve this mask" button that silently
|
||
redraws a boundary the photographer approved. The tools here move an edge
|
||
only where a hand is.
|
||
- **Not a mask library.** Saving a mask and applying it to another photograph
|
||
is a reasonable later feature and depends on none of this.
|
||
|
||
## 3. The five tools
|
||
|
||
Named as a photographer would say them, because these are the words the
|
||
interface will use.
|
||
|
||
| Tool | What it does | New machinery |
|
||
|------|--------------|---------------|
|
||
| **Paint** | Adds to the mask under the brush | none — `brush_add` exists |
|
||
| **Erase** | Takes away under the brush | none — `brush_erase` exists |
|
||
| **Push** | Drags the boundary itself: outward fills behind it, inward empties behind it | a warp pass |
|
||
| **Combine** | Joins another selection to this mask — add, subtract, or keep only the overlap | three blend states |
|
||
| **Show** | Draws the mask that actually results, live | two uniforms in the composed shader |
|
||
|
||
Push is the one worth defining carefully, because "tug the edge" can mean two
|
||
different operations and only one of them is right here — §5.3.
|
||
|
||
## 4. The model: a mask is a stack of parts
|
||
|
||
### 4.1 A part
|
||
|
||
The change that carries all four gaps at once is that a layer stops holding a
|
||
source and starts holding an ordered list of them.
|
||
|
||
```rust
|
||
/// One selection joined into a layer's mask.
|
||
pub struct MaskPart {
|
||
/// Stable identity, for the sidecar and for merge (FR-NC-9).
|
||
pub id: String,
|
||
/// How this part enters the mask built so far. Ignored on the first part,
|
||
/// which *is* the mask so far.
|
||
pub join: Join,
|
||
pub source: MaskSource,
|
||
/// Shaping, moved down from the layer: two parts of one mask routinely
|
||
/// want different edges — a model's soft coverage joined to a hand-painted
|
||
/// correction that must be exactly where it was painted.
|
||
pub invert: bool,
|
||
pub feather: f32,
|
||
pub falloff: Falloff,
|
||
pub morphology: Morphology,
|
||
pub morph_radius: f32,
|
||
pub refine: f32,
|
||
/// The model raster behind a Subject or Category source. Per part now,
|
||
/// for the same reason it was per layer: it materialises *this* selection.
|
||
pub coverage: Option<Arc<Coverage>>,
|
||
}
|
||
|
||
/// How a part joins the mask before it.
|
||
pub enum Join {
|
||
/// Everything either has. The default, and what "add a brush" means.
|
||
Union,
|
||
/// What the mask had, minus this. "Subtract".
|
||
Subtract,
|
||
/// Only where both agree. Where a subject meets a luminance band.
|
||
Intersect,
|
||
/// Not a set operation: this part's strokes *move* the mask under it.
|
||
/// Only ever a painted part carrying push strokes — see §5.3.
|
||
Warp,
|
||
}
|
||
```
|
||
|
||
A layer's mask is then `parts.fold(empty, join)`, and every one of §1's gaps
|
||
becomes an ordinary use of it:
|
||
|
||
- **Paint on an auto mask** — append a `Union` part whose source is painted.
|
||
- **Erase from one** — strokes inside that part carry `Erase`, or the part
|
||
itself is a `Subtract`. Both work; the panel offers the first, because a
|
||
photographer alternating add and erase over one area is drawing one
|
||
correction, not two.
|
||
- **Merge two selections** — two `Union` parts.
|
||
- **Cut one out of another** — a `Subtract` part.
|
||
- **The bright part of the sky** — a `Category` part, then an `Intersect`
|
||
luminance part.
|
||
- **Tug an edge** — a `Warp` part.
|
||
|
||
**Why a list on the layer rather than a boolean tree.** A tree expresses more
|
||
and no photographer has ever wanted the extra. A flat ordered fold is what
|
||
Lightroom, Capture One and darktable all present, it reads top to bottom in a
|
||
panel with no parentheses to draw, and — decisively here — it merges under
|
||
FR-NC-9 as a sequence of independently-keyed blocks, where a tree would merge
|
||
as a shape whose two halves can be individually won by different devices and
|
||
recombined into something neither ever had.
|
||
|
||
### 4.2 A stroke gains a mode
|
||
|
||
```rust
|
||
pub enum StrokeMode { Add, Erase, Push }
|
||
|
||
pub struct Stroke {
|
||
pub mode: StrokeMode, // was: erase: bool
|
||
pub radius: f32,
|
||
pub hardness: f32,
|
||
pub flow: f32,
|
||
/// How strongly the deposit clings to the picture's own edges: 0 paints
|
||
/// anywhere, 1 paints only what matches the colour under the point the
|
||
/// stroke began. See §5.6.
|
||
pub cling: f32,
|
||
pub points: Vec<(f32, f32)>,
|
||
}
|
||
```
|
||
|
||
`erase: bool` becomes a three-valued mode. Everything else about `Stroke` —
|
||
the grid snapping, `simplify`, `MIN_STEP_FRACTION`, `MAX_STROKE_POINTS` and
|
||
its continuation rule — is unchanged and applies to a push stroke exactly as
|
||
it does to a painted one.
|
||
|
||
### 4.3 What the layer keeps
|
||
|
||
```rust
|
||
pub struct MaskLayer {
|
||
pub id: String,
|
||
pub name: String,
|
||
pub enabled: bool,
|
||
/// Invert and opacity stay here: they are the two uniforms the generated
|
||
/// shader already reads per layer (LAYER_UNIFORM_FIELDS), they apply to
|
||
/// the finished mask, and moving them would change the composed shader.
|
||
pub invert: bool,
|
||
pub opacity: f32,
|
||
/// Never empty. `parts[0]` is the base.
|
||
pub parts: Vec<MaskPart>,
|
||
pub ops: Vec<Box<dyn Operation>>,
|
||
}
|
||
```
|
||
|
||
`MaskSource::Brush { strokes }` becomes `MaskSource::Painted { strokes }` —
|
||
the same variant under a name that no longer implies it is the only thing a
|
||
brush can touch. `MaskLayer::begin_stroke` / `extend_stroke` / `end_stroke`
|
||
keep their signatures and route to the layer's *active* part, which the panel
|
||
sets; that is the only change their callers see.
|
||
|
||
**Budgets.** `MAX_LAYER_POINTS` (4096) becomes a budget across all of a
|
||
layer's parts rather than one part's, so a layer's worst-case sidecar size and
|
||
worst-case rasterisation cost are unchanged. `MAX_PARTS = 8` per layer, on the
|
||
argument `MAX_LAYERS` makes: past that it is not a selection any more, and a
|
||
bound the panel can show is better than one a file discovers.
|
||
|
||
### 4.4 The sidecar, and why no existing file changes
|
||
|
||
Blocks are `[mask <version> <layer-id>]` today. Parts are their own blocks:
|
||
|
||
```
|
||
[mask default m1]
|
||
name = Sky
|
||
source = category
|
||
signature = 4711
|
||
category = sky
|
||
feather = 0.004
|
||
falloff = smooth
|
||
opacity = 1
|
||
|
||
[part default m1 p2]
|
||
join = subtract
|
||
source = painted
|
||
stroke = add 0.05 0.5 1 0.6 0.31,0.42 0.33,0.44 …
|
||
```
|
||
|
||
Three compatibility rules, and together they mean **every sidecar written by
|
||
every build so far loads into this one unchanged, and a layer this build
|
||
writes with one part is byte-identical to what it writes today**:
|
||
|
||
1. A `[mask …]` block with no `[part …]` blocks after it is one part. Its
|
||
source keys and its shaping keys build `parts[0]`; nothing is missing and
|
||
nothing needs a default invented for it.
|
||
2. A layer with exactly one part writes the old shape — source and shaping
|
||
inside `[mask …]`, no part block. The format grows only when the feature is
|
||
used.
|
||
3. `stroke = …` lines keep their position and their meaning. The mode token
|
||
gains `push`, and `cling` is a fourth number written only when non-zero.
|
||
An older build meeting either drops *that stroke and only that stroke*,
|
||
which is the rule [`parse_stroke`](../../core/dr-pipeline/src/sidecar.rs)
|
||
already documents and already implements.
|
||
|
||
**Merge (FR-NC-9).** A part is a block with an id, so two devices that added
|
||
different parts to the same layer merge to a layer with both, and the same
|
||
part edited on both is a conflict over that part rather than over the mask.
|
||
Part *order* is the one thing that is not per-field: it is stored as the block
|
||
order and resolved the way the layer order already is.
|
||
|
||
## 5. On the device
|
||
|
||
### 5.1 The order a layer rasterises in
|
||
|
||
Per active layer, into that layer's slice of the existing mask array:
|
||
|
||
1. **Part 0** draws as it does today — one full-screen pass through the
|
||
`switch` in `mask.wgsl`, or per-stroke bounding boxes for a painted one.
|
||
2. **Each further part** draws the same way, with the blend state its `join`
|
||
names (§5.2). No intermediate texture: the accumulator *is* the slice.
|
||
3. **A `Warp` part** copies the slice aside and redraws it displaced (§5.3).
|
||
4. `invert` and `opacity` are unchanged — two uniforms, read in the composed
|
||
shader, applied to the finished mask.
|
||
|
||
`MAX_LAYERS` and the array's memory are untouched: parts collapse into one
|
||
slice, so eight layers still cost eight channels.
|
||
|
||
### 5.2 Combining costs three blend states and no new texture
|
||
|
||
The set operations are already expressible in fixed-function blending over
|
||
`r8unorm`, which is why this is the cheap half of the feature:
|
||
|
||
| Join | `src_factor`, `dst_factor`, op | Result | Built |
|
||
|------|-------------------------------|--------|-------|
|
||
| Union | `One`, `One`, `Max` | `max(dst, src)` | M1 |
|
||
| Subtract | `Zero`, `OneMinusSrc`, `Add` | `dst · (1 − src)` | M1 |
|
||
| Intersect | `Zero`, `Src`, `Add` | `dst · src` | M2 (built 2026-09-24) |
|
||
|
||
The middle row is `brush_erase`, already constructed in
|
||
[`MaskPass::new`](../../core/dr-gpu/src/mask.rs). The other two are the same
|
||
three vertices with a different `BlendState`, and nothing is read back.
|
||
|
||
**One correction to the first draft of this section, found in the building.**
|
||
It claimed no second texture was needed, because a part could be blended
|
||
straight onto the layer's slice. That is wrong, and the reason is the erase
|
||
stroke: an erase inside a part means *a hole in that part*, not a hole in the
|
||
mask. Drawn straight onto the accumulator it takes away whatever the parts
|
||
before it had put there — so tidying the edge of a correction punches through
|
||
the subject underneath, and the failure reads as the model's mask having holes
|
||
in it.
|
||
|
||
So a part is drawn into one scratch texture (proxy-sized, `r8unorm`, allocated
|
||
the first time any layer has more than one part) and blended from there. A
|
||
layer of one part still takes the old path exactly — straight into its slice,
|
||
no scratch, no combine pass — which is what keeps every existing mask
|
||
rendering as it did. `an_erase_stroke_holes_its_own_part_and_not_the_mask` in
|
||
[`local_adjustments.rs`](../../core/dr-gpu/tests/local_adjustments.rs) is the test
|
||
that holds this in place.
|
||
|
||
Intersection, as built, is exactly the table's row: a third pipeline beside
|
||
the other two in `MaskPass::new`, `Join::apply` stating the three on the CPU,
|
||
and `a_part_unioned_subtracted_and_intersected_gives_the_three_fields` and
|
||
`the_joins_match_their_definition` in `local_adjustments.rs` holding the GPU
|
||
to it. It is a product, not a minimum: the two agree wherever either side is
|
||
fully in or out, and between two soft edges the product is the softer reading,
|
||
which is the right way for an overlap of two partial selections to be wrong.
|
||
Its blend is `Add`, not `Max`, so it adds nothing to the Android question
|
||
below.
|
||
|
||
`Max` blending on `r8unorm` is core WGPU and universally supported on the
|
||
desktop backends; **verify it on the Android adapter before M2 lands**, since
|
||
that is the platform where a blend mode is most likely to be quietly emulated
|
||
or absent. If it is missing, union is `One, OneMinusSrc, Add` — a screen blend
|
||
— which differs from `max` only where both parts are partially covered, and
|
||
never by more than the softness of their two edges.
|
||
|
||
### 5.3 Push is a warp, not a local morphology
|
||
|
||
Two mechanisms fit "drag the edge", and the choice matters.
|
||
|
||
**Local morphology** — offset the distance threshold inside the brush — is
|
||
exact, but it exists only where there is a signed distance field, which is
|
||
`Subject` and `Category` and nothing else. A push tool that works on a
|
||
model mask and does nothing on a gradient, a range or a painted part is a tool
|
||
the photographer cannot trust.
|
||
|
||
**A warp** works on any mask, because it never asks what the mask is made of.
|
||
Each dab of a push stroke contributes a displacement, and the mask is resampled
|
||
through the sum of them:
|
||
|
||
```
|
||
D(p) = Σ over dabs (b − a) · w(|p − seg(a,b)| / radius)
|
||
mask'(p) = mask(p − D(p))
|
||
```
|
||
|
||
Drag from inside the mask outward and the sample point moves back into the
|
||
interior, so the boundary follows the finger and **the area behind it fills
|
||
in**. Drag from outside inward and the vacated area samples from outside, so
|
||
**the area behind it empties**. That is exactly the pair of behaviours asked
|
||
for, from one gesture, with the direction supplied by the drag rather than by
|
||
a mode switch.
|
||
|
||
Three honest costs:
|
||
|
||
- It needs the mask it is sampling, and a pass cannot sample the target it
|
||
writes: a warp part costs one texture copy of the slice, plus one draw over
|
||
the strokes' bounding box. One scratch texture, proxy-sized, `r8unorm`,
|
||
allocated on first use and reused by every layer, since layers rasterise in
|
||
sequence.
|
||
- The displacement is a **sum**, not a sequential resample. Two crossing
|
||
push strokes therefore compose approximately rather than exactly. Bound
|
||
`|D|` at one brush radius per stroke: past that a warp tears rather than
|
||
drags, and no photographer means the difference.
|
||
- A warp moves what is under it, including detail the photographer painted by
|
||
hand. That is what it is for, and it is why push is a part in the list —
|
||
it can be removed later without disturbing the parts beneath it.
|
||
|
||
### 5.4 Painting has to be incremental
|
||
|
||
Today [`MaskPass::render`](../../core/dr-gpu/src/mask.rs) clears each slice and
|
||
redraws every stroke of the layer. That is right when a shape changes and
|
||
wrong while a finger is down: at 120 reports a second, a layer holding 4096
|
||
points redraws all of them per dab, and the cost of a stroke grows as it is
|
||
painted — the failure mode `MAX_STROKE_POINTS` already names for one stroke,
|
||
here across the layer.
|
||
|
||
**While a stroke is live, only the new segment is drawn.** The slice already
|
||
holds everything up to the previous dab, the blend states are the same ones
|
||
that would have been used in a full redraw, and the segments are drawn once
|
||
each in the same order — so the incremental result is identical to the
|
||
rebuilt one rather than an approximation of it. `MaskPass` keeps, per layer,
|
||
the `(part count, stroke count, point count)` it last drew; anything else
|
||
changing falls back to the full rebuild it does now.
|
||
|
||
This is required, not an optimisation to schedule later: it is what decides
|
||
whether painting is usable on the phone, and it is the specific failure
|
||
[`mask.rs`'s module docs](../../core/dr-pipeline/src/mask.rs) say this whole
|
||
design exists to avoid.
|
||
|
||
### 5.5 Distance fields become per part
|
||
|
||
[`SubjectMasks`](../../core/dr-gpu/src/mask.rs) uploads one signed distance field
|
||
**per active layer, in stack order**, and
|
||
[`DevelopSession`](../../ui/dr-ui/src/develop.rs) builds them on the same
|
||
indexing. With parts, a field belongs to the part that shaped it: the upload
|
||
becomes one field per *model-backed part*, flattened in `(layer, part)` order,
|
||
and the rasteriser indexes it by a running counter rather than by `slot`.
|
||
|
||
This is the most invasive change in the document — it touches the field
|
||
builder, the upload, the key that decides when to rebuild, and the shader's
|
||
`subject` binding index — and it is mechanical. It is also the reason M2 is
|
||
its own milestone rather than a rider on M1.
|
||
|
||
### 5.6 Cling: paint that stops at the picture's edge
|
||
|
||
A brush that respects the photograph's own boundaries is the difference
|
||
between refining a mask and colouring it in, and the ingredients are already
|
||
bound to this pass: the demosaiced source (for range masks) and, when there is
|
||
one, the compacted label field.
|
||
|
||
At each dab, `cling > 0` multiplies the deposit by agreement with the pixel
|
||
under the **stroke's first point**: colour distance in the same linear-sRGB
|
||
space `colour_mask` already works in, falling off over a tolerance set by
|
||
`cling`. Where a label field exists, agreement is 1 inside the same region and
|
||
falls to the colour test outside it, so the brush stops dead at a watershed
|
||
boundary and softly at a colour one.
|
||
|
||
Roughly fifteen lines of WGSL reusing `image_value` and `hue_of`, one number
|
||
in the sidecar, and it is the single control that makes a 5%-radius brush
|
||
usable along hair.
|
||
|
||
## 6. Seeing the mask
|
||
|
||
**Status: built.** `MaskStack::rendered`, `Reveal` as a list of
|
||
`(layer, colour)`, `RevealStyle`, `EditGraph::compose_revealing`,
|
||
`MaskPass::render_revealing`; on the panel, an eye and a colour per row and one
|
||
"Show masks as" strip above the stack.
|
||
|
||
The mask array is **already bound to the composed adjust shader**, so this is
|
||
almost free, and it is the first thing to build because every other tool here
|
||
is unusable without it.
|
||
|
||
### 6.1 One correction to this section, found in the building
|
||
|
||
The draft said "two uniforms — which layer to reveal (−1 for none) and which
|
||
style — always emitted and guarded by the uniform, so switching the overlay on
|
||
is a uniform write rather than a shader recompile". The second half of that is
|
||
not available, and the reason is the case the feature exists for.
|
||
|
||
A uniform can *select* a slot. It cannot conjure one. A layer with no
|
||
adjustment on it changes no pixel, so it is not `is_active`, so it occupies no
|
||
slice of the mask array and the rasteriser never draws it — and that is
|
||
precisely the layer a photographer wants to look at, for the whole of the time
|
||
between choosing a subject and deciding what to do to it. Revealing it means
|
||
*rendering* it, which changes the sequence of layers, which changes the
|
||
uniform block. The composition moves either way.
|
||
|
||
So the slot and the style are written into the source, and turning the reveal
|
||
on, off, or onto another layer recompiles the fused shader. That is a button
|
||
press rather than a frame, and the uniform would only have added a branch per
|
||
pixel on top of a recomposition that was happening anyway.
|
||
|
||
`MaskStack::rendered(reveal)` is the one sequence this rests on: `active()`
|
||
plus the layer being looked at. The rasteriser, the composer and the distance
|
||
field builder all index by position in it, so all three must be given the same
|
||
`reveal` — two of them disagreeing shows as an adjustment applied through
|
||
another layer's mask, which is why they take it as an argument rather than
|
||
reading a flag.
|
||
|
||
### 6.2 Where the block runs, and why not with the others
|
||
|
||
After the output transform, immediately before the clip and the encode — not
|
||
among the layer blocks. Everything there runs on scene-referred colour in the
|
||
working space, where a flat tint would be pushed through the base curve and
|
||
the camera matrix and arrive as some other colour, and an alpha's white on
|
||
black would arrive as neither.
|
||
|
||
### 6.3 Not on the graph
|
||
|
||
The reveal is an argument to `EditGraph::compose_revealing`, and
|
||
`compose_for` — which the exporter, the thumbnail and the neutral probe all
|
||
call — has no way to ask for one. A flag on the graph would have been fewer
|
||
parameters, would have type-checked, and would have been one forgotten reset
|
||
away from a red tint baked into an exported file.
|
||
|
||
Three styles, all read from the same alpha:
|
||
|
||
- **Tint** — the mask over the picture in its colour at ~50%. The default,
|
||
and what every editor's photographers already expect. The colour is the
|
||
mask's own, chosen from the swatches on its row — which is what answers a
|
||
red tint over a red dress.
|
||
- **Alpha** — the mask alone, white on black. For judging an edge, where a
|
||
tint over a busy picture cannot be read.
|
||
- **Edge** — the boundary outlined over the untouched picture. For checking
|
||
registration against detail the other two hide, and the same reasoning the
|
||
region overlay's white outline already carries.
|
||
|
||
**Per mask, not per selection.** The first build of this showed the *selected*
|
||
layer's mask in one global style, and it answered the wrong question. What a
|
||
photographer asks of two masks is how they meet — where the sky's edge sits
|
||
against the building's — and that needs both on screen at once, in colours that
|
||
can be told apart. So each row of the stack has an eye, and each mask a colour
|
||
from a six-entry palette (`MASK_COLOURS` in `develop.rs`); the eye is drawn in
|
||
that colour so the row says which shape on the picture is its. The style is
|
||
the one thing that stays global, because a tint beside an outline beside an
|
||
alpha would be three pictures that cannot be read against each other. Alpha
|
||
therefore draws every shown mask, each in its colour, on black.
|
||
|
||
**When it appears.** A new layer arrives with its eye open, in the first colour
|
||
nothing else is using — making a mask is asking what it selected, and for a
|
||
subject or a category that question has no other answer on screen. Arming
|
||
Paint or Erase opens the selected layer's eye if it was closed, on the same
|
||
argument `on_part_added` makes: a stroke into an invisible mask is
|
||
indistinguishable from a tool that did nothing. Pressing a swatch opens the
|
||
eye too, since colouring a mask nobody can see would change no pixel.
|
||
|
||
Only ever *this* layer's eye, and only on an explicit action. Every other eye
|
||
keeps whatever it was set to, and nothing re-arms in the background. The
|
||
existing `overlay-hidden` property is the precedent and the trap it documents
|
||
applies unchanged: an automatic reveal that re-arms a switch somebody turned
|
||
off is worse than no automatic reveal at all.
|
||
|
||
Viewing state and not edit state: eyes and colours are on the session, not on
|
||
the layer, and a photograph reopened has every eye closed.
|
||
|
||
The ~1s reveal after a shaping slider is released, from the draft, is not
|
||
built. It is a timer rather than a decision.
|
||
|
||
The region overlay stays exactly what it is — a picture of what the model
|
||
detected — and gains a name in the interface that says so, because two
|
||
overlays that look alike and mean different things is worse than either.
|
||
|
||
## 7. The interface
|
||
|
||
### 7.1 Where the tools live
|
||
|
||
**In the Local panel, not the tool rail.** The rail's entries arm a canvas
|
||
gesture for the whole photograph; a brush is meaningless without a layer to
|
||
paint into, and a rail entry that silently created one — or that lit up and
|
||
did nothing with no layer selected — is precisely the kind of surprise the
|
||
rail's own notes argue against. The tool strip sits in the Local panel's
|
||
header, enabled only when a part is selected:
|
||
|
||
```
|
||
┌ Local ─────────────────────────────┐
|
||
│ [Select] [Paint] [Erase] [Push] ◉ │ ◉ = show mask
|
||
├────────────────────────────────────┤
|
||
│ ▸ Sky ● ⌄ │
|
||
│ Category · sky │
|
||
│ ├ Subtract · Painted ✕ │
|
||
│ └ Intersect · Luminance ✕ │
|
||
│ [+ Add ⌄] [− Subtract ⌄] [∩ ⌄] │
|
||
├────────────────────────────────────┤
|
||
│ Brush size ──●─── 0.05 │
|
||
│ hardness ──●── 0.5 │
|
||
│ flow ─────● 1.0 │
|
||
│ cling ──●─── 0.4 │
|
||
└────────────────────────────────────┘
|
||
```
|
||
|
||
`toolrail.slint`'s note about `MaskSource::Brush` being the next rail entry is
|
||
superseded by this and should be replaced with the reasoning, not deleted —
|
||
the file is where somebody will next look for it.
|
||
|
||
### 7.2 The part list
|
||
|
||
Each layer row gains its parts as indented rows. A part row carries its join
|
||
(a chip that cycles add / subtract / intersect), its source name, a delete,
|
||
and selection — selecting a part is what points the brush and the shaping
|
||
controls at it. The layer row keeps invert, opacity and enable, which are the
|
||
layer's.
|
||
|
||
`[+ Add]`, `[− Subtract]` and `[∩]` each open the same menu of sources the
|
||
"new layer" buttons already offer: a gradient, a range, a subject, a category,
|
||
or painted. One code path, three joins.
|
||
|
||
**Folding two layers into one** uses the multi-selection
|
||
[`masks_ui.rs`](../../ui/dr-ui/src/masks_ui.rs) already supports: with two layers
|
||
selected, "Combine" appends the second's parts to the first and removes it.
|
||
Offered only when the second layer's adjustments are neutral, and otherwise
|
||
offered with a warning that names what will be lost — quietly discarding an
|
||
edit the user made is not a combine.
|
||
|
||
### 7.3 The brush and the cursor
|
||
|
||
Radius, hardness, flow and cling are sliders in the panel and all four are
|
||
live on the canvas as a cursor: an outer ring at the radius, an inner ring at
|
||
the hardness, and — this matters on a phone — the ring drawn at the *touch
|
||
point offset above the finger*, since the thing being painted is under the
|
||
hand that is painting it.
|
||
|
||
Radius is stored per tool, not per stroke and not per layer: a photographer
|
||
who sets a small eraser expects it to still be small the next time they erase.
|
||
|
||
### 7.4 Gestures
|
||
|
||
Each of these needs a `GESTURE:` block beside its implementation — that is the
|
||
only place [gestures.md](../gestures.md) can be written from.
|
||
|
||
| Gesture | Touch | Pointer | Keyboard |
|
||
|---------|-------|---------|----------|
|
||
| Paint into the selected part | Drag on the picture | Drag | — |
|
||
| Erase instead of paint | Hold the Erase tool | Alt-drag | — |
|
||
| Push the boundary | Drag with Push armed | Drag | — |
|
||
| Change the brush size | Drag the size slider | Scroll with Alt over the picture | `[` `]` |
|
||
| See the mask | Press the eye | Press the eye | `\` while held |
|
||
| Undo one stroke | The history list | Ctrl+Z | Ctrl+Z |
|
||
| Add a part | `[+ Add]`, pick a source | Same | — |
|
||
| Fold two layers | Select both, Combine | Same | — |
|
||
|
||
The `why` each block needs is mostly one sentence — *a stroke is a decision
|
||
and a decision is one undo step* — except for Alt-drag, which needs to say
|
||
that a modifier is the only way to alternate paint and erase without leaving
|
||
the stroke, and that touch cannot have it, which is why the tool strip is a
|
||
strip and not a single toggle.
|
||
|
||
### 7.5 The Slint hazards this walks into
|
||
|
||
Named because all of them compile:
|
||
|
||
- **The paint `TouchArea` must be declared in front of
|
||
`ScaleRotateGestureHandler`** to receive the press at all, exactly as the
|
||
region picker and the repair placer are. The consequence is that a
|
||
two-finger pinch may not reach the handler behind it while paint is armed.
|
||
Repair has the same arrangement today, so **check what repair mode actually
|
||
does with a pinch before designing around it**; if pinch is lost, the answer
|
||
is geometry — arm the paint area over the picture only — not z-order.
|
||
- **A drag must be measured in the parent frame**, never in the coordinates of
|
||
something the drag moves. `GradientHandles` in `masks.slint` is the
|
||
reference.
|
||
- **The part rows are a repeater inside the develop column**, so their chips
|
||
go through `ChipGrid` or `Segmented { columns: 3 }` — a non-wrapping chip
|
||
row sets the width of the whole sidebar.
|
||
- **None of the above is caught by the test suite.** A screenshot is the only
|
||
check; see the project's notes on capturing one under XWayland.
|
||
|
||
## 8. History, undo, labels
|
||
|
||
A stroke is one step, recorded on release: `Edit::Action`, not
|
||
`Edit::Control`. `Control` is the right variant for a dragged control and the
|
||
wrong one here precisely because it coalesces — two strokes painted a second
|
||
apart are two decisions, and coalescing them under one key would make the
|
||
second untakeable back. A push stroke follows the same rule, as does adding,
|
||
removing or re-joining a part.
|
||
|
||
The mask stack is already snapshotted per step and shared by `Arc` when a step
|
||
does not touch it, so the cost of an undoable stroke is a clone of one layer's
|
||
parts, not of the picture. New keys in
|
||
[`labels.rs`](../../ui/dr-ui/src/labels.rs):
|
||
|
||
```
|
||
history.mask_painted "Paint Mask"
|
||
history.mask_erased "Erase Mask"
|
||
history.mask_pushed "Push Mask Edge"
|
||
history.mask_part_added "Add To Mask"
|
||
history.mask_part_removed "Remove From Mask"
|
||
history.mask_joined "Change How Mask Joins"
|
||
history.masks_combined "Combine Masks"
|
||
```
|
||
|
||
## 9. Sync and merge
|
||
|
||
Nothing here stores pixels, so nothing here changes what sync carries beyond
|
||
size. A painted correction is bounded by `MAX_LAYER_POINTS` at roughly 50 kB
|
||
of text in the worst case and a few hundred bytes in the ordinary one. A model
|
||
part still carries its run-length coded `coverage` line, unchanged and still
|
||
outside `PartialEq`.
|
||
|
||
Two conflicts are new, and both resolve per block: two devices adding
|
||
different parts to one layer (both are kept, in block order), and two devices
|
||
painting the same part (a conflict over that part, arbitrated the way a layer
|
||
already is). A device that has never run a model still renders every part
|
||
correctly, because coverage travels with the part exactly as it travelled with
|
||
the layer.
|
||
|
||
## 10. Performance and budgets
|
||
|
||
The number that matters is the cost of one dab while the finger is down, and
|
||
with §5.4 it is bounded by the dab's own bounding box rather than by the
|
||
stroke's history: `(2r)²` pixels × one segment. At the default 5% radius on a
|
||
1600×1067 proxy that is about 25 000 pixels — a fraction of a millisecond, and
|
||
independent of how long the stroke has been going.
|
||
|
||
Everything else runs on a shape change, not per frame:
|
||
|
||
| Work | When | Rough cost at 1600×1067 |
|
||
|------|------|------------------------|
|
||
| Full layer rebuild | part added, source changed, resize | one draw per part |
|
||
| Warp | a push part exists and something below it changed | one copy + one bbox draw |
|
||
| Distance field | a model part's morphology changed | as today, CPU, per part |
|
||
| Overlay | never — it is two uniforms in a shader that already runs | — |
|
||
|
||
The scratch texture for warp is one proxy-sized `r8unorm` — under 2 MB — and
|
||
is allocated on first use, so a photographer who never pushes an edge never
|
||
pays for it.
|
||
|
||
Measure against [frame-budget.md](frame-budget.md), and measure it on an idle
|
||
machine: the 16 ms guard is load-sensitive enough that a busy build makes it
|
||
fail and an idle one makes it pass, so a single run proves nothing either way.
|
||
|
||
## 11. Order of work
|
||
|
||
**M1 — See it and paint it.** The overlay (§6), `parts` on the layer with
|
||
`Union` and `Subtract`, `Painted` parts, the tool strip, the brush HUD and
|
||
cursor, the canvas gestures, incremental raster (§5.4). No new shader code for
|
||
the mask pass beyond the blend variants; no change to distance fields, because
|
||
a painted part needs none. *This is the whole of the user-visible ask except
|
||
push and intersect,* and it is deliberately the milestone that stands alone.
|
||
|
||
Done: parts with `Union` and `Subtract`, painted parts, the tool strip, the
|
||
canvas gesture, and the overlay (§6). Outstanding: the brush HUD and cursor —
|
||
radius, hardness and flow are sliders with no ring drawn on the photograph,
|
||
so the size of the brush is a number rather than a thing you can see — and
|
||
the incremental raster of §5.4, without which a layer's whole stroke history
|
||
is redrawn per dab.
|
||
|
||
One lesson from the order it was actually built in, since §6 said it and the
|
||
build did not listen: the overlay is not the last quarter of M1, it is the
|
||
first. Parts and painting shipped without it and the result was a feature
|
||
nobody could tell was working — the panel listed a mask, the photograph showed
|
||
nothing, and every report of it came back as "the masks do not work".
|
||
|
||
**M2 — Combine properly.** `Intersect`, model and gradient and range parts,
|
||
per-part distance fields (§5.5), the part list with its join chips, folding
|
||
two layers.
|
||
|
||
Done (2026-09-24): `Intersect` — the join, its blend state, the sidecar word
|
||
`intersect`, the part row's chip cycling + / − / ∩, and an "∩ Intersect"
|
||
button beside Add and Subtract (§7.1). The part list with its chips and a
|
||
per-part eye was already there from M1. Outstanding: joining a model,
|
||
gradient or range part from the panel (the buttons still join a painted
|
||
part, so an intersection is painted where the mask should survive), per-part
|
||
distance fields, and folding two layers.
|
||
|
||
**M3 — Push, and cling.** The warp pass (§5.3) and edge-aware deposit (§5.6).
|
||
Both are refinements of a tool that already works, which is the right place
|
||
for the two riskiest pieces.
|
||
|
||
**M4 — What the use of it asks for.** Left open on purpose. Likely candidates:
|
||
a "select the subject under the pointer" brush, pressure from a stylus, and a
|
||
per-part opacity.
|
||
|
||
## 12. Requirements to add
|
||
|
||
Proposed text for [requirements.md](requirements.md) §3.3, in the shape the
|
||
neighbouring entries take:
|
||
|
||
> **FR-DEV-19 — Mask editing.** A mask layer's coverage shall be editable by
|
||
> hand after it is created, by painting into it, erasing from it, dragging its
|
||
> boundary, and joining further selections to it. Every edit is stored as
|
||
> geometry and parameters in the edit graph; no rasterised mask is written to
|
||
> a file and none exists in CPU memory.
|
||
>
|
||
> **FR-DEV-19a — Mask composition.** A layer's mask is an ordered list of
|
||
> parts, each naming a source and how it joins the mask before it — union,
|
||
> subtraction, or intersection. A layer of one part is exactly the layer of
|
||
> today, and reads and writes the same sidecar.
|
||
>
|
||
> **FR-DEV-19b — Hand correction.** A part may be painted, with add, erase and
|
||
> push strokes, at a radius, hardness, flow and edge-clinging the photographer
|
||
> sets. Strokes are stored as normalised coordinates and rasterised on the
|
||
> device.
|
||
>
|
||
> **FR-DEV-19c — Boundary push.** A push stroke displaces the mask beneath it
|
||
> along the drag, filling behind an outward drag and emptying behind an inward
|
||
> one, on any mask source rather than only those with a distance field.
|
||
>
|
||
> **FR-DEV-19d — Mask visualisation.** The mask a layer actually produces —
|
||
> after its parts, its shaping, its inversion and its opacity — shall be
|
||
> displayable over the photograph as a tint, as an alpha, or as an outline,
|
||
> and shall appear automatically while a mask is being edited.
|
||
|
||
Each needs `TRACES:` comments at the implementation and a regenerated
|
||
[traceability.md](traceability.md) **in the same commit**, since that file
|
||
carries line numbers.
|
||
|
||
## 13. Verification
|
||
|
||
Testable without a GPU, and therefore not optional:
|
||
|
||
- `Stroke` mode round-trips through the sidecar, including a push stroke and a
|
||
non-zero cling; an unknown mode costs one stroke and not the layer.
|
||
- A one-part layer writes the byte-identical sidecar it writes today, and
|
||
every existing fixture in `mask_sidecar.rs` still loads.
|
||
- Parts merge per block: different parts added on two devices produce a layer
|
||
with both; the same part edited on both is one conflict.
|
||
- Point budgets hold across parts, and painting past them refuses rather than
|
||
dropping the oldest strokes.
|
||
- The panel model builds part rows from a hand-made stack with no device
|
||
present, the way `rows_from` does for adjustments.
|
||
- History: a stroke is one step, undo restores the parts, and a step that
|
||
leaves the masks alone still shares the `Arc`.
|
||
|
||
Needing a device, in `dr-gpu/tests`:
|
||
|
||
- A truth table for the joins: a half-covering part unioned, subtracted and
|
||
intersected with a known base gives the three expected fields.
|
||
- A warp of a known step edge moves it by the drag, and the area behind it
|
||
fills — asserted on a readback of the mask array, which is a test reading
|
||
back, not the application.
|
||
- Incremental drawing equals a full rebuild, dab for dab, on a stroke of a few
|
||
hundred points. This is the test that keeps §5.4 honest.
|
||
- `Max` blending behaves on every adapter CI runs on.
|
||
|
||
And a screenshot, because §7.5 is not reachable by any of the above. Run
|
||
`dr-ui` tests single-threaded; parallel runs segfault in this crate for
|
||
unrelated reasons.
|
||
|
||
## 14. Decisions I need
|
||
|
||
1. **Parts now, or strokes first?** M1 as written introduces `parts` and pays
|
||
the migration once. The cheaper alternative is a `strokes: Vec<Stroke>`
|
||
field on the layer, brushed over whatever the source produced, and parts
|
||
later — half the code, and it makes "subtract this from that" a second
|
||
mechanism invented afterwards. I would take the migration now.
|
||
2. **Push as a warp, or local morphology on model masks only?** §5.3 argues
|
||
the warp. It is the more expensive of the two and the only one that works
|
||
on every source.
|
||
3. **Tool strip in the Local panel, or an entry in the rail?**
|
||
`toolrail.slint` predicts a rail entry; §7.1 argues against it. The rail is
|
||
easier and worse.
|
||
4. **Do the FRs in §12 go into `requirements.md` now,** or stay proposed here
|
||
until the first milestone lands?
|