fb1b44ce47e250f173e2e9b9e02c01b5e12a7c04
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  dtourolleandClaude Sonnet 5
|
b3641b5307 |
Let a subject's mask be refined, and let several masks be edited at once
Two related gaps in the mask panel, from the same conversation: a subject's outline is only ever as sharp as the whole-frame pass that found it, and a change meant for several layers had to be dragged once per layer. ## Refine mask A "Refine mask" button on a subject layer re-runs detection on a padded crop around that instance's own box instead of the whole frame — the subject reaches the model at its own size rather than squeezed into the model's fixed 640x640 window alongside everything else in the photograph. `RefineJob` mirrors `SegmentationJob`'s split (built on the session, run off it, adopted back), and the crop itself is rendered through `Framing::set_view` — the same ephemeral viewport the interactive zoom already uses to render a region above proxy resolution, so no new render path and no change to the model's own input size was needed. `dr-segment` is untouched: `Tiling::Whole` already treats whatever buffer it is handed as the one window. The result is still downsampled onto the shared proxy grid every instance's mask lives on, but from a sharper source than the whole-frame pass ever saw for that subject, which is what the edge actually reads out of. ## Multi-select `active_mask: Option<String>` is now `active_masks: Vec<String>`. A plain click still replaces the selection; a control- or command-click toggles one layer in or out of it. `set_param` and `reset_op` fan out to every selected layer, each set to the exact value the slider now shows rather than offset by however far it already was — one slider, one reading, applied everywhere selected. Dragging a gradient's on-canvas handle is deliberately not extended to multi-select: several gradients have no single geometry a shared handle could move, so `gradient_handles` stays empty unless exactly one layer is selected. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
7fcb8dc107 |
Give the develop column room, and the mask a way out of the way
Three faults reported together, and they share a shape: each is something the panel decided on the photographer's behalf. **The column was 280px on every screen.** That width was chosen for a tablet, where the column is a large fraction of the display and every pixel of it is taken from the photograph. On a desktop window the mode strip alone — two modes, a separator, "All", and a chip per attribute the operation set declares — does not fit, so it scrolled sideways. A control you have to pan to reach is one you do not know is there. 380px when the window is classed expanded, 280px when it is not; driven off `layout-class` because reading the window width inside the layout that sets it is a binding loop. **The overlay could not be hidden.** An earlier "Overlay" button was removed for a good reason — it *armed* the overlay, so local mode could be entered and still show nothing. Hiding is the opposite need and was never served: a mask is judged against the photograph beneath it, and that photograph is exactly what the overlay covers. `overlay-hidden` is kept separate from `overlay-on` so a recompute cannot switch the overlay back on under someone who just turned it off. **The masks were coarse because the model saw the subject small.** The graph's input is a fixed 640x640 and every frame is letterboxed into it, so a bird 200px across in a 1600px proxy reaches the model at 80px. `Tiling::Grid` has been implemented and tested since the segmentation spike and defaulted off, because it costs one inference per tile — 2.8s for a 3x2 grid against 470ms. Now offered as "Look closer (slower)", which says what it costs, rather than spending it on every image or on none. The tiling choice enters the segmentation signature. A mask stores the signature its region ids index into, and a tiled run finds different instances in a different order; sharing a signature would silently reinterpret a layer built against the coarse pass — a wrong mask rather than a stale one, and nothing announces it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
96a7b405c2 |
Say which photograph the sliders are pointed at
Selecting a mask layer silently re-points about thirty controls at that layer's chain. Same panel, same order, same sliders, different meaning — and the only thing that said so was a sentence in the panel above, which a photographer reaching for the exposure slider has no reason to read. An exposure change lands on the whole frame when it was meant for a face, or the reverse; both are silent, and both are discovered later. `ui-navigation.md` §1.1 calls it the dangerous one and it is: the others in that document cost time, this one costs work. The remedy is the classic one for a modal fault — make the mode visible — and the application already had the pattern. Crop arms a canvas interaction, draws an overlay, gives the column one job and is left by the control that entered it. Local masking is the same animal built as a peer panel, and that is what created the ambiguity. So `crop-mode` stops being a bare boolean and becomes one value of a three-state mode, which is the point: two modes could both be on before, and now that is not a state the interface can be in rather than one it is tested against. **One strip, not two.** The mode control was going to sit beside the group strip that filters the adjustments, which is two controls above one column answering the same question — what am I working on. They are one control now, `Crop · Local │ All · Light · Colour`, which is the shape Lightroom Mobile's bottom strip has for the same reason. The two halves are different kinds of state and are drawn differently: a mode is a chip that fills with the accent when it is on, a group is a word with a rule under it. That difference is what lets both be read at once, which they routinely are — picking Light while a mask is selected filters *that layer's* chain and does not leave the mode. Dropping the scope on a group press would be the same fault coming back from the other end, and would make Light mean two things depending on where it was pressed. The strip stays pinned above the develop column rather than moving to the top of the canvas as the document proposed. The half that filters the column belongs to the column, and the photograph is the subject. The canvas keeps one button, which now names the mode it leaves rather than saying "Done" — that was unambiguous with one mode and would not be with two — because the column can be closed on a narrow window and no mode may be inescapable. Entering a mode is a side effect, so Rust owns it rather than the strip writing the property: crop drops the zoom, local turns the overlay on, and leaving clears the selection. That last one is the fix. The "Overlay" and "Select" toggles are gone because they armed things that are simply what the mode *is* — a mode that has to be switched on separately is one you can enter and have do nothing. Escape and the Android back gesture join `back_step` as one `LeaveMode` rather than a second exit concept, and the mode is left before the zoom is: it was entered later, and it is the bigger step back. The heading is where the scope goes. Not a caption beside the panel, the heading *of* the panel that changed — `ADJUST` becomes the layer's name, the same string the selected row in the stack shows. That is the difference between describing a hazard and removing it. **Handles on the photograph.** A linear or radial mask could be created and then not moved, so a radial sat at the centre of the frame at its default size for ever. Three faults stood in the way of drawing one. The first is that a gradient did not render at all until the model had run. The rasteriser was built on the way out of `segment` and the array's size was read *off* the segmentation, so a gradient added to an unsegmented photograph produced nothing — silently, in the same way exports and thumbnails once did: the shader still emits the layer's block and the empty placeholder multiplies it by zero. The proxy size is a property of the photograph. Both are derived from it now, and deliberately at the same size rather than by coincidence, because a subject's distance field is sampled against that array. The second is hit-testing. A handle is drawn in output coordinates and stored in source ones, and between them lie the crop, the zoom, the pan, the straightening and the turns. `Framing::source_at` is `wgsl_prologue` evaluated on the CPU, kept in that file beside it so that keeping the two in step is one file's problem — a handle mapped through anything less drifts off the mask the moment the view moves, which is exactly what masks are rasterised in source space to avoid. The third is that a drag is a displacement, not a destination. Each handle answers to the movement of the pointer since the press, applied to where the mask was when the press landed. Snapping the handle to the pointer instead jerks it by up to half a touch target on the first press, and the target is finger-sized because a tablet has no hover to reveal a control and no modifier to qualify it. A ramp gets three handles — centre, width, angle. An ellipse gets three too: centre and one per semi-axis, the major one carrying the direction as well as the length, because where an axis is put says both. It had a fourth, and it is gone: standing off the shape by a fixed distance, the rotation arm began outside the photograph at the size a new radial is created at, so the first thing anyone saw was a control they could not reach without first shrinking the mask. Two faults here were found by looking at the screen rather than at the source, both of the kind that cannot be found any other way. A `1px` rule with a size and no position is *centred* by Slint, so the seam between the photograph and the column was a hairline down the middle of the panel, through the histogram and every slider under it — twice, once in `app.slint` and once in `AdjustPanel`. And handing Slint a fresh model for the handles on every pointer event made the repeater rebuild its items, taking the `TouchArea` holding the gesture with them: the handle jumped once and then went dead under a finger that was still down. `develop.rs` carries the same warning about the parameter rows, where it broke slider drags; the model is rewritten in place now. The tests worth having are the ones about ambiguity and about the map. That the same row reads the frame's value, then the layer's, then the frame's again is §1.1 in one assertion. That dragging a handle onto another gradient's matching handle *produces* that gradient closes the loop between the two directions of the framing map, through a view that is cropped, zoomed, panned, straightened and quarter-turned at once — a one-legged map is invisible when the framing is neutral, because then both legs are the identity. Not done here: the histogram still reports the whole frame while the sliders edit a layer. That disagreement is real and is N3's, which this unblocks. The strip has room for a Brush entry beside Crop and Local when the painted masks land in the core, and it needs nothing here but the canvas interaction. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
37f63edbf9 |
Take the watershed out of the product path
It does not work on a photograph, so nothing should offer it. `Segmentation` is now one model pass and what it recognised: no region field, no merge tree, no label upload, no granularity slider, and no readback of the whole proxy to build a graph that collapses. A click means "the object under the cursor". The region-selection path went with the hierarchy it indexed — including the shift-click add/subtract, which has no meaning for a whole object and would have been a modifier that silently did nothing. The passes, the hierarchy and the semantic prior stay in `dr-gpu` and `dr-segment`, tested and documented. It is the *merge criterion* that fails — the saddle is the minimum gradient along a boundary, so one weak pixel merges two regions and real gradient noise puts a weak pixel on every boundary. That is one function to replace, and the evidence for replacing it is worth keeping. What is gone is the wiring, the option, and the control that offered a user a choice with no outcome. `MaskSource::Regions` remains in the pipeline: it is tested, it round-trips through the sidecar, and a stored layer that names regions must still load and be reported stale rather than failing to parse. |
||
|
dtourolle
|
6163b63895 |
Put the edge controls where the mask is
Feather, falloff, grow/shrink/close/open and their amount, on the selected layer. All four read the one distance field, so all four are live — nothing recomputes except a compound morphology, and the session keys on that separately so a feather drag rebuilds nothing. Shown only for sources that go through the distance field. A gradient carries its own falloff in its geometry, and offering a second one would be two controls fighting over the same edge. Picking an operation seeds a small amount if none is set. Selecting "Grow" and seeing nothing happen would read as a broken control rather than as a radius of zero. |
||
|
dtourolle
|
ee10097435 |
Mask the subject the model found, not the regions underneath it
The watershed hierarchy does not survive a photograph, so local masking stops depending on it. A layer can now be one recognised object, and the object's own coverage is the mask. `Options::watershed` defaults off. It costs ~80 ms plus a full-resolution readback to produce a ladder that collapses, and paying that on every photograph buys a control that misleads. Kept switchable rather than deleted: the passes and the hierarchy are correct in themselves and it is the merge criterion that fails, which is a change to one function. Masks now rasterise in **source** space at proxy resolution and are sampled by the composed shader after the framing map. That fixes a real bug: they were rasterised in output space, so zooming slid the photograph underneath a mask that stayed pinned to the viewport, and cropping moved every adjustment to a different part of the picture. Doing it this way also leaves the framing map in exactly one place — a second copy in the mask shader would have been a second thing to keep in step, failing only when straightened. A subject is stored as identity, not pixels: the mask is megabytes and is reproducible by running the same model over the same image, so the sidecar carries the index, the class and the score, and the session carries the pixels. The class is there to be checked — if instance 3 comes back a "car" where it was a "dog", something changed and the layer is stale rather than silently masking the wrong thing. The overlay now draws instances and is transparent everywhere else. The region version covered every pixel and so hid the photograph it was drawn over; the question it exists to answer is whether an outline follows the subject, which you can only answer by seeing both. `examples/local.rs` is the worked example: subject in colour with the rest monochrome, and the subject lifted out of its background. Run on a 5472x3648 CR2 it finds two people and two cars, and the colour-pop keeps her hat and hair while the wall and grass behind go grey. |
||
|
dtourolle
|
b1433ad4a9 |
Give a mask an edge treatment, and find out the watershed has none worth having
Two things, and the second is why the first matters more than expected. Mask layers gain a feather, a falloff curve and a morphology, all defined against a signed distance from the boundary rather than as separate features — one exact distance field answers "how soft" and "how far" at once, so dilation is a threshold at -r, erosion one at +r, and closing and opening are one of each in sequence. The compound pair costs a second distance field, which is why they are named rather than presented as a radius that happens to be signed. Types, defaults and sidecar round-trip only; the field itself is next. `edge-feather` and `edge-falloff`, not `feather` and `falloff`, because a radial mask already writes `feather` for the fraction of its radius it ramps over. Same word, different quantity, different units — sharing the key would have made an existing file ambiguous. The diagnostic that provoked this is committed as an ignored test, because "does the ladder land on things a person means" is the question S15 exists to answer and it should not depend on whoever still has the script. On bus.jpg it answers badly: 35,075 regions at blur 2 over an 810x1080 frame, and cutting that to 400 gives *one* region covering nearly the whole picture plus 399 noise specks. Not over-segmentation — collapse. Almost every saddle is near zero, so the merge order joins everything meaningful before it joins anything spurious, and a global cut spends its entire budget on grain. So the granularity ladder does not currently work on a photograph, and the region masks built on it inherit that. Recorded rather than worked around: the next commits move local masking onto the model's instances, where the edge treatment above is what makes a quarter-resolution mask usable. |
||
|
dtourolle
|
12d320cf33 |
Record what the spec got wrong about the model that exists
docs/segmentation.md §4 priced arm B as costing a C dependency under the NDK and treated that as most of the difference between the arms. It is not a cost that has to be paid: `ort`'s `alternative-backend` disables its linking entirely and `ort-tract` supplies the API from tract, which is pure Rust. D13's "largest exception the policy would tolerate" turns out not to be needed, and the answer generalises to the face pipeline — so D13's runtime half is now answered and only its licensing half is open. Three findings contradict §4 outright and are recorded as F4-F6 rather than quietly designed around. There is no ADE20K-trained YOLO, so the shipped vocabulary selects subjects and not stuff — "select the sky" comes from the watershed or from nowhere. It is instance segmentation, so it partitions nothing and two people come back as two instances. And tract cannot parse a dynamic-shape export, which fixes the input at 640 square and makes tiling the only route to more semantic resolution. Arm C ships, but §8's criteria are not what decided it, and saying so matters more than claiming the process worked. §8 asked for a two- interaction margin over arm A on a traced corpus. That comparison was never run: F4 and F5 changed what the arms are, and a model that recognises subjects but has no word for sky cannot be a selection tool alone, while a watershed cannot tell a person from the wall behind them. They stopped being candidates and became complements. What is *not* done is written down as plainly: the 24-image corpus is untraced, so M1-M4 have no numbers and "this feels right" has not become one. M5 is answered on one device only, and region ids now reach the sidecar — so a cross-vendor divergence would mean a mask written on the desktop meaning something else on Android. F3 stands. |
||
|
dtourolle
|
9b4f0815e5 |
Show the regions, click one, and adjust it
The local panel sits above the adjust panel because it decides what those sliders act on; below it, a photographer would set an exposure and only then discover which scope it landed in. Selecting a layer re-scopes the existing controls to that layer's chain — there is no second set of sliders, and there must not be, or every operation added to `ops/` would need a local twin. The overlay is drawn over the canvas rather than blended into the render, because it is a diagnostic and not an edit: it must not reach the histogram, an export, or the texture handed to the compositor. Nearest- neighbour always — the map's values are *names*, so smoothing between region 4 and region 9 invents a colour belonging to neither and softens exactly the edge the overlay exists to show. Picking gets its own touch area above the pan handler. Panning wants press-drag-release and picking wants a click; interleaving them in one handler is how a drag ends up selecting a region the user was scrolling past. Shift is tracked as window state because a TouchArea's click carries no modifiers. Three states a layer can be in are worth distinguishing, and each has a different remedy: stale needs re-segmenting, "no adjustment yet" needs a slider moved, and the ordinary case needs nothing said. A bare selection renders nothing and looks identical to a broken mask, which is the first thing a new user will hit. Known rough edge, commented where it happens: segmentation blocks the UI thread for about half a second. Moving it to a worker needs the develop session — GPU resources behind a RefCell shared with every callback — to be reachable from another thread, which is a restructuring rather than a change to the call. The button says "Finding regions…" first so the stall is announced rather than looking like a hang. |