Compare commits

...
160 Commits
Author SHA1 Message Date
dtourolle cf84cec96f Release 0.15.0
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m19s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 49s
Build and test / Android (aarch64) (push) Successful in 31m25s
Build and test / android-image (push) Successful in 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / Desktop (Linux) (push) Successful in 48m26s
Build and test / windows-image (push) Successful in 5s
🐳 Windows image / Build and push (push) Successful in 4s
Build and test / Layer separation (push) Successful in 34s
Build and test / Windows (x86_64, cross) (push) Successful in 19m57s
Build and test / Publish the release (push) Successful in 1m6s
2026-09-25 07:51:31 -04:00
dtourolle f0e7b8e11c Re-record the manual on the keyboard layout, and stop the zoom caption promising blocks
Every scene was recorded again on a build of this branch rebased onto the
keyboard work and TD-1, since nearly every scene depends on files those
changed: the develop top bar now carries a star strip and Pick/Reject, the
roll shows flags and stars, and the grid's selection bar gains Label and
Flag. `--changed` could not be trusted to find them, because the rebase
made each picture's commit newer than the sources it was recorded from.

The develop-zoom caption said the wheel goes on "until the pixels are
blocks". On Xvfb the deepest frames come out smooth even though the app
draws past 1:1 nearest-neighbour on a real display (confirmed by eye on
the desktop), and a GIF shrunk to 960 wide could not show 3-pixel blocks
anyway. The caption now says what the clip shows; the prose above it,
which describes what the app does, stays.
2026-09-25 07:26:37 -04:00
dtourolle 90c7cb65d3 Regenerate the manual page after the rebase onto the keyboard work
The rebase combined this branch's README additions with master's, and
index.html is generated from the README; manual-check passes on the
regenerated page.
2026-09-25 07:26:37 -04:00
dtourolle 70583b9b2f Fail CI when the manual shows a picture no scene makes
The traceability job now runs `tools/manual/record.sh --check`: every
picture docs/manual/README.md shows must be made by a scene in
tools/manual/scenes.py, and every picture a scene makes must be shown.
It reads the two files and nothing else, so it needs no app, display or
LFS pull.

--changed now dates a scene by the newest commit among its pictures
rather than each picture alone. A scene that also makes a picture which
re-records byte for byte (panorama-aligned beside panorama.gif) no
longer stays listed for ever. A scene all of whose pictures come out
identical (launch) stays listed until one differs, which costs one
harmless re-run.
2026-09-25 07:26:37 -04:00
dtourolle f426bb903a Picture this round's features in the manual
Four scenes, recorded and looked at frame by frame:
- library-labels: 6, 7, 8 and 9 over four New York frames, 7 again to
  take one off, then the Green chip narrowing the grid and back.
- compose-perspective: two towers shot from below, stood upright with
  Vertical at about +58, then held against Before.
- crop-orphan: a stroke in the top-left corner, a crop that leaves it
  outside, the notice with Undo crop and Keep crop, and Undo crop.
- local-intersect: a linear gradient over the lower half, then
  Intersect and two strokes that survive only where it is.

develop-zoom already ends on hard-edged pixels (previous commit). The
text added is a sentence or two under the existing headings, so the
other branch's structure and anchors are left alone.
2026-09-25 07:26:36 -04:00
dtourolle c41f99ea52 Record the manual's scenes by name, and each against what it depends on
scenes.py aimed every press at window pixels, and the develop column had
already moved under it: Compose now sits above Adjust, so the old
exposure coordinate lands on a straighten slider. Every scene now names
what it presses by its accessible label through the automation hook,
places points on the photograph relative to the canvas, and opens its
photographs by file name. Each starts from a known place and undoes what
it did, so one can be recorded alone; the few that continue another's
state name it, and running one runs that first into a scratch folder.

Each scene also declares the pictures it makes and the sources they
depend on. `record.sh --check` fails when the manual shows a picture no
scene makes, or a scene makes one it does not show; it reads two files.
`record.sh --changed` re-records the scenes whose sources, or own code,
changed since the commit that last touched their pictures. record.sh
builds with the automation feature, restores the library from
DR_LIBRARY_SNAPSHOT, starts from a fresh profile and pins inference to
the CPU; the launch screen is recorded from an empty profile of its own.

Re-recorded with the ported scenes, and looked at frame by frame. What
differs from the pictures they replace:
- develop, presets, settings, local, compose, film, wb, light: the
  current develop column (Compose with Vertical and Horizontal above
  Adjust, the Label button), otherwise the same moments.
- library pictures: the filter bar's colour-label chips; no collection
  left over from an earlier run in the sidebar; library-selection is the
  twelve alpine frames rather than eight of them and four New York ones.
- library-rating rates two frames nobody had rated, so the stars are set
  and not cleared.
- develop-zoom goes on past 1:1 with the wheel and ends on the file's
  pixels as hard-edged blocks.
- repair covers a real mark on the road, with a size that fits it; film
  is shown on the Chinatown frame instead of the road.
- panorama tries Perspective, Spherical and Cylindrical before filling.
- launch, launch-folder and panorama-aligned came out byte-identical.
2026-09-25 07:26:36 -04:00
dtourolle a6ea6ba83f Let the manual's scripts find a control by its name
Every scene in tools/manual aimed at window pixels written in by hand, so
a panel that gained a row moved every slider under it and the recording
went on dragging where the slider used to be. The develop column has
already moved that way (Compose now sits above Adjust), and nothing said.

A build with the `automation` feature listens on the Unix socket named
by DR_AUTOMATION and answers where an element is: by its accessible
label, the name a screen reader reads, or by its markup id for the few
things that are not controls (the canvas, the crop rectangle). It uses
Slint's element queries, which need the compiler's debug tables, so the
feature also turns those on in build.rs. It only answers questions; the
input is still xdotool's real pointer. No default build has the feature,
and one that has it listens only when the variable is set.

drive.py gains click-on, drag-on, hold-on, wait-for, wait-gone, labels
and ids. The grid's cells are now named by their file, each rating star
by its value, the sidebar's + as "New collection", and the Adjust
heading's reset as "Reset all adjustments" - controls a screen reader
could not reach before either.
2026-09-25 07:26:36 -04:00
dtourolle b480de5bff Record TD-1 as paid off, checked by eye on the tablet
The pre-rotation patches landed in the previous four commits. This
records how the debt was paid, by a fourth route its own list missed:
patching wgpu-hal and Slint's Skia surface locally rather than waiting
for either upstream. It also updates architecture.md §1 and §6.1, which
still said Android draws with OpenGL behind a readback.

The verification is stated as what it was: the user found the
release-signed build clean on the tablet in portrait. No dumpsys
composition or bufferTransform readings were taken, because adb would
not hold the device that morning, and no frame times were measured.
2026-09-25 07:26:00 -04:00
dtourolle a8043e6827 Hand Android's develop frame to the compositor as a texture again
With Skia drawing pre-rotated on wgpu's Vulkan swapchain, Android no
longer needs to draw with Skia over OpenGL, which was the only reason
the develop view read its frame back through memory (TD-1).

So `unstable-wgpu-29` moves back to the common slint dependency. The
android-activity backend then builds `SkiaRenderer::default_wgpu_29`,
and `shared_gpu` loses its Android arm. The one wgpu device is handed to
Slint through `BackendSelector::require_wgpu_29` on both platforms.
`slint::android::init_with_event_listener` runs before `dr_ui::run`, so
the selector reaches the Android adapter before its window exists.
`renderer-femtovg-wgpu` stays desktop-only, since Android has no FemtoVG.

The two `#[cfg(target_os = "android")]` readbacks in `develop::render`
(the frame through `export_pixels` and the focus overlay through
`read_overlay`) are gone. `read_overlay` stays for the tests that check
what the overlay marks.

Built for arm64 and release-signed. Not yet run on the tablet.
2026-09-25 04:20:19 -04:00
dtourolle 4b4c9e2e6d Pre-rotate Slint's Skia drawing on the wgpu swapchain on Android
The other half of the wgpu-hal patch: that one lets a caller promise a
pre-rotated swapchain, and this is the caller keeping the promise.

On configure, `WGPUSurface` reads the surface's `currentTransform`,
sizes the swapchain in the panel's orientation (swapped for a quarter
turn), tells wgpu-hal to use that transform, and before each frame
concatenates the matching rotation onto the Skia canvas. Everything
Slint draws goes through that one matrix, so an imported wgpu texture is
rotated with the rest of the window. Input is not rotated, and must not
be, because Android delivers it in window coordinates.

Three details that would each have been a visible bug:

- `resize_event` compared the new size against the swapchain's. The
  swapchain is transposed while a quarter turn is in effect, so the
  comparison now uses the window's size, kept beside it.
- A half turn, landscape to reverse landscape, changes the transform
  without resizing the window, and wgpu-hal hides the SUBOPTIMAL that
  would report it. So the transform is re-read before every frame. That
  costs one query into the native window.
- The item renderer snapped the origin to the pixel grid only when the
  canvas matrix was a pure translation. Under a rotation that is never
  true, so portrait would have lost pixel alignment everywhere. The check
  now accepts right-angle rotations and flips without scaling.

The direction of each rotation follows the Vulkan spec's reading of
preTransform (the image is drawn already rotated clockwise by the
transform). It has not been confirmed on the device yet.
2026-09-25 04:19:05 -04:00
dtourolle dc9da52651 Let a wgpu-hal caller choose the Vulkan swapchain's preTransform
wgpu-hal creates every swapchain with `preTransform = IDENTITY` (#3345).
On a tablet whose panel is mounted landscape, a portrait window then
hands Android an unrotated buffer: SurfaceFlinger falls back to rotating
it on the GPU (composition CLIENT), and on this device those frames tear.
That is why Android draws with Skia over OpenGL today, and why the
develop view pays a readback (TD-1).

The field cannot just be set to `currentTransform` inside wgpu. It is a
promise that the image is already drawn rotated and sized in the panel's
orientation, and only the renderer above wgpu can keep it. So the patch
is the smallest thing that lets that renderer ask:
`vulkan::Surface::current_transform` reads the surface's transform, and
`set_pre_transform` makes the next swapchain use it. The default stays
IDENTITY, so desktop and any caller that does not opt in behave exactly
as upstream.
2026-09-25 04:18:39 -04:00
dtourolle dc1add9dbb Vendor wgpu-hal 29.0.4 and i-slint-renderer-skia 1.17.1, unmodified
The Android develop view reads its frame back through memory (TD-1)
because wgpu's Vulkan swapchain never pre-rotates, and a portrait window
on this tablet's landscape panel then tears. The fix is a small patch to
each of these two crates, and this commit is only the ground it lands on:
both are byte-for-byte the crates.io sources the lockfile already
resolved, so the commits that follow are the patch and nothing else.

third_party/ is excluded from the workspace, or every path dependency
under the root would become a member and `--workspace` would test and
lint upstream code as ours. The README says how to carry the patches
across a Slint or wgpu bump, which matters because a stale version here
does not fail the build — cargo just warns and uses the unpatched crate.
2026-09-25 04:18:39 -04:00
dtourolle b5ae5c2be1 Record FR-DEV-16 as met and where FR-UI-5 stands
FR-DEV-16's promise that the gesture book cannot describe a binding the
application lacks is now enforced by the gestures gate in both directions,
and every binding it names is bound and tagged, so it is marked met, with
what develop answers beyond the list.

FR-UI-5 is not met in full: its keyboard half and the 2026-09-19 amendment
are, but no develop slider takes the scroll wheel, so its status says that
rather than rounding up.
2026-09-24 23:42:28 -04:00
dtourolle aa2a88f655 Show each frame's flag and stars on the develop roll
FR-UI-5's 2026-09-19 amendment asks for the rating and flag wherever they
can be set and on the roll's cells, so that stepping along a set in develop
shows what has been judged. The roll drew thumbnails only, so the keys that
now judge the open photograph left no trace on its neighbours.

Each roll cell carries a small badge with a tick or a cross and a star
count, drawn only when there is something to show, in shapes and a number
rather than colours (NFR-A11Y-3).
2026-09-24 23:42:28 -04:00
dtourolle f8737e3fda Close the help sheet with Escape, and keep keys from acting behind it
F1 opens the help sheet from the grid, and nothing on the keyboard closed
it: Escape fell through to the shell, and every other key went on judging,
labelling and keywording the photographs hidden behind the sheet.

Escape and Back now close the sheet first, ahead of the grid's other
sheets, since it is drawn over all of them. While it is open the grid's
handler declines every other key, so a stray P or Ctrl+K changes nothing
the reader cannot see.
2026-09-24 23:42:27 -04:00
dtourolle d489a34190 Drive develop, the grid, the sidebar and People from the keyboard
An audit of every action by view against the keys the handlers bind left
develop without zoom, pan, fit or a way back to the grid, the grid without
select-none, thumbnail size or keywording, People with no key at all, and
the export and copy sheets without Enter. It also found the reverse gap
FR-UI-5 forbids: pick and reject had no route but P, X and U, and the
2026-09-19 amendment's judging in develop had not been built.

Develop: Ctrl+= and Ctrl+Plus zoom in and Ctrl+- out about the middle of the
view, Ctrl+0 fits and Ctrl+1 goes to 1:1, Shift and an arrow pan a magnified
view, G goes back to the grid, Ctrl+Y redoes, and Enter keeps a crop that hid
a mask. 0-5, P, X and U rate and flag the open photograph without moving on,
with stars and Pick/Reject in the top bar as the pointer and touch route.
= and - nudge the control last moved by a hundredth of its travel; the
framing sliders, perspective included, now count as "last moved", so R puts
them back as well. J turns the selected mask part's join chip.

Grid: Ctrl+D and Ctrl+Shift+A clear the selection, = and - resize the
thumbnails, Ctrl+K opens keywording, and Flag in the selection bar gives
pick and reject a pointer and touch route. Sidebar: Enter commits a
collection's name, and Enter or Escape hands the keyboard back to the grid,
where it used to go nowhere until something was clicked. People: Up and Down
walk the rail, F2 puts the name field under the keys, and Escape or Back now
leave the screen the way its back button does instead of doing nothing.
Sheets: Enter does what the export or copy sheet's button does.

The choices follow Lightroom where it has one. No new key steals typing: the
grid's and People's keys live on focus holders that are not ancestors of any
text field, and the sheets' Enter comes after a focused field has had it.
Every binding is tagged beside its handler, and the gate added in the
previous commit holds the two to each other.
2026-09-24 23:42:26 -04:00
dtourolle 9d1e31ffbb Fail CI when a key is bound but not in the gesture book, or listed but not bound
The gesture book is generated from GESTURE tags, so it could not describe a
gesture nobody tagged, but nothing made anyone tag one. The arrow keys, Enter,
P, X, U, Delete, F1 and F2 all worked in the grid with no line in the help
sheet, and a tag could name a key whose handler had gone.

Key handlers now compare one canonical string, Keys.chord(event) == "Ctrl+Z",
instead of reading event.text and the modifiers themselves. keys.slint folds
the key and its modifiers into that spelling, so the literal in the handler is
the whole binding and the checker reads exactly what the handler dispatches
on. Each handler carries a KEYMAP comment naming the gesture-book section its
keys belong to, and a tag's keys field names its keys between backticks.
gestures-check now fails when a handler binds a key no tag in that section
names, when a tag names a key no handler there binds, when any .slint file
other than keys.slint reads event.text, when a compared literal is not
canonical, and when keys.slint's named keys drift from the Rust list.

Spellings are normalised in one place, chord.rs: Ctrl+z, Control+Z and
LeftArrow all mean what the handler's "Ctrl+Z" and "Left" mean. Shift and Alt
count only for letters and named keys, because on the French layout every
digit needs shift and a 6 has to be a 6 however it was typed.

A Rust keymap that both dispatched and was read by the generator was the
alternative. It would have moved the handlers' decisions away from the Slint
state they depend on, and a window that forgot to install it would have had
no working keys at all.

The keys that were already bound and undocumented are now tagged.
2026-09-24 23:42:25 -04:00
dtourolle 150e53e878 Link fifty gestures on the help sheet to the manual section that shows them
The sheet could now offer "See it", but no gesture said where to look.

Every GESTURE tag whose move the manual describes names that section:
the white-balance picker, zoom and pan, masks, undo and snapshots,
export, colour labels and ratings, selection, collections, the People
page, thumbnail size. Fifty of the fifty-one; the one left, putting a
single control back to its default, has no section and is too small to
earn one.

Three important gestures had nowhere to land, so the manual gains three
short sections, without pictures for now: Bursts (opening a folded
burst, choosing the frame it shows, and the eyes-open filter), Moving
between photographs (the roll, the arrows and A/D in develop) and
Copying settings (Copy, Paste, Paste to N, and choosing what a copy
carries). The page, the gesture book and docs/gestures.md are
regenerated from them.
2026-09-24 23:25:37 -04:00
dtourolle 7591738c73 Let a gesture name the manual section that shows it
The help sheet says which move does a thing, and the manual has a
picture of the thing being done, but nothing joined the two: a user
reading "Pinch it with two fingers" had no way from there to the GIF of
it.

A GESTURE tag takes an optional `manual:` field naming a heading of
docs/manual/README.md by its anchor. The scan checks every one against
the anchors the bundled page is rendered with and fails when the manual
has no such heading, so renaming a section cannot leave the sheet
linking to the top of the page; gestures-check carries the same failure
into CI. The anchor goes into gesture_book.rs as a new field, and into
docs/gestures.md as a "See it" link to manual/README.md#anchor. The help
sheet draws a "See it" button beside the title of each gesture that has
one, which opens the bundled manual at that section.

The field is additive: a tag without it is unchanged, and no gesture
carries one yet.
2026-09-24 23:24:17 -04:00
dtourolle 352e59498b Open the bundled manual from Help and from Settings
The packages now carry the manual, but nothing in the application opened
it: the help sheet listed gestures and stopped there.

The help sheet gains a Manual button beside Done, and Settings a Manual
row under About beside the version. Both go through dr_ui::manual, which
finds the installed page through dr_plat::system_data_dirs (the package's
share directory on Linux, the executable's directory on Windows), and a
development build also in the checkout it was compiled from. A copy with
no manual says so on the status line rather than doing nothing.

On the desktop the page goes to the system browser. A section is a URL
fragment, and xdg-open's generic mode and Windows' FileProtocolHandler
both turn a file: URL into a path and drop the fragment, so a section is
opened through a one-line redirect page written to the data directory:
the opener gets a plain path, which every opener keeps, and the browser
follows the redirect to index.html#section itself. The launcher behind
the sign-in's open_in_browser is split out so both share it; the https
check stays with the sign-in.

Android has no path to give a browser: an asset is not a file, a copy in
private storage is unreadable to other apps, a file: URI across apps is
refused, and a content: URI leaves the browser resolving every picture
against the provider. So ManualActivity, a WebView reading
file:///android_asset/manual/index.html straight out of the APK, shows
it, started by class name with the section as an extra. JavaScript is
off, links off the page go to the browser, and the theme is day-night so
the page's own light and dark follow the system. A test checks that the
manifest, the Java class and dr_ui agree on the name and the extra.
2026-09-24 22:56:09 -04:00
dtourolle d8f26fb5cd Ship the manual with the Arch package, the Windows installer and the APK
The rendered manual was in the repository and nowhere else, so an
installed application still had nothing to open.

Each packager now carries docs/manual/index.html and its pictures, to
where the application will look for them: /usr/share/darkroom/manual on
Arch, manual\ beside darkroom.exe on Windows (where the models already
are, and where dr_plat::system_data_dirs points), and assets/manual in
the APK, stored rather than deflated since a GIF or PNG is already
compressed. The manual is about 27 MB, which the APK and the installer
both grow by; the pictures are 1600x1100 screenshots and short GIFs,
and against an APK that already carries 170 MB of inference runtime and
70 MB of models they are not worth re-encoding for.

The pictures are LFS objects, so each packager refuses a pointer where a
picture should be, as it already does for the models: shipped, a pointer
is a manual of broken images that nothing reports. The Android and
Windows CI legs therefore fetch docs/manual/media, which they excluded
while nothing they built read it, and the installer smoke test checks
that the page and every picture were installed.
2026-09-24 22:33:41 -04:00
dtourolle 10216355c1 Render the manual as one HTML page the application can carry
The manual existed only as docs/manual/README.md, which the forge renders
and nothing else does. An installed copy of the application, on a laptop
with no network or on a tablet, had no manual it could open.

`traces manual` renders the README to docs/manual/index.html with
pulldown-cmark (already in the tree as Slint's Markdown parser, so this
adds a dependency edge and no crate). The page is one file with an inline
stylesheet that follows the system's light or dark preference, a
contents list of every section and subsection, and the pictures by their
relative media/ paths. Each heading carries the id the forge gives it, so
README.md#rating-and-flagging and index.html#rating-and-flagging are the
same link. A picture alone in its paragraph becomes a figure whose alt
text is shown as the caption, and every picture reserves its 16:11 box
before it loads, so a jump into the middle of the page lands where it
aimed rather than a screenful above. Links to design documents, which the
installed page has no copy of, point at the forge.

The page is committed rather than rendered at build time, as the gesture
book is: it is user-facing text reviewed in the diff, and the three
packagers then only copy it. `traces manual-check` fails in CI when the
committed page is not the render of the README, and the pre-commit hook
regenerates it when the README is staged.
2026-09-24 22:32:30 -04:00
dtourolle d8e031888e Regenerate the gesture book, which four rebases left with conflict markers
docs/gestures.md on master carried 162 lines of <<<<<<< / ======= / >>>>>>>
from 4642c77, ade627a, c3d1f83 and 46f5b95. Their branches were rebased
onto each other, the generated files conflicted, and the resolution
regenerated the requirements matrix with `traceability -- report` and then
staged gestures.md as it stood, on the assumption that the same command
writes it. It does not: the gesture book has its own `gestures` mode. The
source tags were never in conflict, so nothing is lost; this is the file
regenerated from them, and `gestures-check` passes on it.
2026-09-24 22:24:41 -04:00
dtourolle 114d979397 Add Vertical and Horizontal perspective sliders to Compose
The keystone existed in framing but nothing in develop could reach it:
framing is presented by its own Compose panel rather than generated, so
new framing parameters get no control until the panel names them.

Compose now has Vertical and Horizontal sliders under Straighten,
mirrored from the session like the angle, recorded as parameter steps
("Vertical Perspective" in the history), cleared by the Compose reset
and by opening the next photograph. Releasing either slider refits the
crop the way releasing the straighten slider does: a keystone alone
needs no crop, but it moves the empty corners of a straightened frame,
so the crop that avoided them before may not after, or may have room
to grow back.
2026-09-24 22:13:12 -04:00
dtourolle 5a500118ae Correct converging verticals with a keystone in framing
There was no perspective transform anywhere in the pipeline: framing
offered a ±45° straighten, quarter turns and flips, and a building shot
looking up kept its leaning walls.

Framing gains a vertical and a horizontal keystone (-100..100). They are
parameters of framing rather than a new stage, so they carry its Compose
attribute, persist in the sidecar under framing, and are withheld from a
default paste exactly as the crop is. In the prologue the keystone runs
after the crop and the straightening and before the stored orientation
and the lens warp, so "vertical" is the photograph's displayed height and
the lens still sees its whole frame.

The map takes the output frame onto a trapezoid inside the source, built
as a homography from four corners and uploaded as three columns in the
framing uniform block (which grows from two vec4s to five). A keystone on
its own therefore never exposes an empty corner and leaves any crop valid.
Combined with a straightening angle the empty area is a pulled-back
quadrilateral the closed-form inscribed rectangle cannot describe, so
max_inscribed_crop searches for the largest centred rectangle whose
corners all have a source pixel behind them. source_at and output_at
apply the same map, so masks, gradients and spot handles follow it.
2026-09-24 22:13:11 -04:00
dtourolle ff89a4fa21 Specify perspective correction as FR-DEV-20
Issue #13 asks for a vertical and horizontal keystone, and its number was
renumbered from FR-DEV-19 when the spec gave that to mask editing. The
clause was never written into the register, so the work had nothing to
trace to.

It is written as part of framing: after the crop and the straightening,
before the stored orientation and the lens warp, carrying framing's
Compose attribute, and with the inscribed crop accounting for it.
2026-09-24 22:13:11 -04:00
dtourolle 04495afbfd Regenerate the traceability matrix for the colour-label work
The pre-commit hook left the matrix as it stood on two of the commits before this one, so it named neither the new test file's NFR-A11Y-3 tag nor the line numbers the label code moved. Regenerated from the tree as it now is.
2026-09-24 21:52:23 -04:00
dtourolle 96f1d5c896 Say in the manual and the register that colour labels exist
The manual's library section described rating and flagging only, and
the outstanding register still said colour labels were "set and shown
nowhere" and that three NFR-A11Y-3 clauses had no test. Both are now
untrue: the manual gives the keys, the Label button and the chips, and
the register names the test file and what it can and cannot vouch for.
2026-09-24 21:52:23 -04:00
dtourolle f1db919b9d Fail a test when a star, a flag or a label differs by colour alone
NFR-A11Y-3 was argued in comments beside the rating strip, the
pick/reject mark and the focus-peaking chips, and nothing would have
failed if an edit made a set star differ from an unset one only in tint.
Only the clipping readout had a test.

These read the markup, as the accessibility-name tests do, since a
rendered window cannot be asked what a colour-blind reader sees. The
star and the flag must choose their glyph from their state, and the
glyphs they choose must be different drawings in icons.slint. The peaking
chips must be distinct words that reach the screen as text. Each colour
label must carry its own letter and the name the catalog's code stands
for, the mark must draw the letter, and the grid cell, the filter chips
and develop must draw the mark or the name. Breaking any of these by
hand makes the matching test fail.
2026-09-24 21:52:23 -04:00
dtourolle 46f5b95828 Show and set colour labels in the grid and develop, and filter by them
Colour labels could be read from a Lightroom sidecar and queried by the
selector, but nothing drew one or set one, so the only labels a library
held were ones another program had written.

Every mark carries its label's initial on its colour — R, Y, G, B, P —
so a label is read without telling red from green, which is what
NFR-A11Y-3 asks of colour labels by name. A grid cell shows the mark
before its filename. In the grid, 6, 7, 8 and 9 set red, yellow, green and
blue as Lightroom's keys do, on the photograph under the pointer or on
the selection by the rule the star keys follow; the same key again takes
the label off, and over a mixed selection it sets it on all. The
selection bar gains Label, which opens the six choices — each a mark and
a name — and purple, which has no key, is there. In develop the top bar
says "Label: Green" beside the mark, opens the same choices, and 6-9
label the open photograph.

Each gesture is one catalog transaction, then the grid, the counts and
both sidecars are written as a rating's are. The filter bar gains a chip
per label, its mark and its name with a count, one at a time; the filter
is one SQL term, travels in the place record, and "All" clears it.
2026-09-24 21:52:23 -04:00
dtourolle d748527a4c Keep colour labels in the sidecar so they survive and travel
A rating and a flag are written to DarkRoom's sidecar as well as the
catalog, because the catalog is a disposable index and the sidecar is
how a judgement reaches the photographer's other devices. A label had no
place there, so once labels could be set, one would have lived only in
the catalog of the device it was set on and gone with it.

The sidecar version now carries `label` (0 none, 1-5 as the catalog
codes it), written only when set. It merges under the rating's rule, so a
device that never labelled a frame cannot clear another device's label,
and a code this build does not know reads as none rather than as some
other colour. A judgement write carries the catalog's label with the
stars, and the scan takes a sidecar's label into the catalog when it has
one. An older build keeps the line as an unknown key and writes it back.
2026-09-24 21:52:23 -04:00
dtourolle 89859d39d1 Let the catalog set colour labels, toggle them, and count them
Colour labels reached `versions.label` only from an XMP sidecar: nothing
in the catalog could set one, clear one, or read it back alongside the
stars, so there was nothing for an interface to call.

`set_label` and `set_label_many` write it the way ratings are written,
the bulk form in one transaction so a key over a selection is one commit.
`toggled_label` holds Lightroom's rule for a label key: it clears only
when every image already carries that label, and otherwise sets it on all
of them, so a half-red selection comes out red rather than inverted.
`Judgement` carries the label, so the grid's one window query brings it
with the stars, and `label_histogram` counts each label in one grouped
statement for the filter chips. A label does not make a frame "judged":
it is a pile of the photographer's own, not a cull decision.

The doc comment for `default_version_id` had been stranded above
`label_code` when that was inserted; it is back on its function.
2026-09-24 21:52:23 -04:00
dtourolle ade627a5d0 Fade the draft into the sharp frame when a drag settles
When a gesture stopped, the half-resolution draft was replaced by the
full-resolution frame in one step, a visible jump from soft to sharp.
FR-DSP-4 asks for a refinement that is smooth, not a jarring swap.

The canvas now keeps the last draft frame (`canvas-previous`) and draws
it over the sharp one, fading it out over 150 ms when the draft flag
clears. The fade costs no render: the draft is a refcount on the texture
it was drawn into, and the adjust pass ping-pongs between two output
targets, so the sharp frame is written into the other one. While a
gesture is drafting the layer is hidden and snapped opaque, so a new
drag shows its draft at once; past the fade it is hidden again, and a
settled canvas composites one image as before.
2026-09-24 21:52:23 -04:00
dtourolle 4642c77e18 Dim the histogram while the canvas shows a draft
The histogram is measured on settled frames only, so during a drag it
describes the frame from before the gesture while the canvas shows
something newer, and nothing said so. The draft flag stopped at the
render closure.

It now reaches the interface: `canvas-draft` on the window and
`Levels.provisional` for the readouts, both set on every canvas render
from the flag that chose the frame's resolution, and cleared when a
render fails. The histogram panel dims its display reading to half
while a draft is up and brings it back when the frame settles. Dimmed
rather than captioned, because a caption appearing on every drag would
move the column; the raw reading has no frame to lag and is left alone.
2026-09-24 21:52:23 -04:00
dtourolle 7d0870c3fb Keep a drag in draft until it stops, then render sharp once
During any drag longer than 120 ms the canvas rendered a full-resolution
frame every 128 ms under the finger. The settle timer was armed by the
first draft of a burst and not re-armed by later ones, so it counted from
the start of the gesture rather than from its last movement, fired
mid-drag, and the next coalesced event armed it again. Each of those
frames is the most expensive one the canvas draws, landing where the
frame budget is tightest.

The draft/sharp decision now lives in `refine::Refine`, apart from the
timers that carry it out. Every draft frame arms a settle timer carrying
a generation token and only the newest token is honoured, so the sharp
frame lands SETTLE_DELAY after the last movement. A request arriving
while a settle is still owed also counts as part of the gesture, so a
slow stretch of a drag (one event per frame, nothing to coalesce) no
longer renders sharp between drafts. A timer that fires with a render
already posted defers to it.

The tests drive the state machine through simulated timelines; the
long-drag case reproduced the four mid-drag sharp frames before the fix.
2026-09-24 21:52:23 -04:00
dtourolle c3d1f83b96 Say so when a crop leaves a mask outside the frame
Cropping tighter past a mask layer made it invisible without a word:
the layer stayed in the panel and the sidecar, and its adjustment went
on landing on pixels nobody would see again.

When a crop is let go, develop now measures what the gesture did to the
mask stack (dr_pipeline::orphan) and, if any layer is now entirely or
mostly outside the frame, shows a notice over the photograph: how many
layers, their names, "Undo crop" and "Keep crop". The crop is already
applied and nothing waits on the answer.

The crop overlay gains a release callback carrying the rect the press
began from, so the measurement runs once per gesture and never on the
drag's per-frame changes. Choosing a ratio is measured the same way,
being a crop committed in one click.

"Undo crop" is the ordinary undo, and the notice is tied to the history
revision it was raised at: the redraw that follows any history move
clears it, so the crop and its warning go back as one step. A second
drag folded into the same step is measured from where that step began.
A crop that strands nothing shows nothing.
2026-09-24 21:52:03 -04:00
dtourolle fc0ea8824d Format the crop-orphan measurement
rustfmt wraps two tuples in dr_pipeline::orphan that the previous commit
left on one line past the width limit. No change in behaviour.
2026-09-24 21:52:02 -04:00
dtourolle 9772785f81 Measure which mask layers a crop takes out of the frame
Mask geometry is stored in source coordinates, so re-cropping tighter
never destroys a layer. It makes it invisible: the layer stays in the
panel and the sidecar, its adjustment lands on pixels nobody will see,
and nothing says so. The spec had no clause for this; FR-DEV-17 now
states it, under the ID issue #10 reserved.

dr_pipeline::orphan samples each layer's mask on a 64x64 lattice over
the source, with the gradient, radial, brush and model-raster geometry
the mask shader uses, folds the parts by their joins and inversions, and
maps the samples through the framing to see how much of the coverage
the crop keeps. `hidden_by_crop` reports the layers whose share fell
below a tenth, and only those the change newly hid, so an already
stranded layer is not announced again on every later adjustment.

Ranges follow the picture and region selections need a label map this
crate does not hold, so a layer that adds either is never reported: a
false alarm on the common path would teach the notice to be dismissed
unread.
2026-09-24 21:52:02 -04:00
dtourolle 733a033274 Test that a stub decoder reaches the scan, the ladder and export
FR-RAW-2's "without changing callers" needs a test that would fail if a
caller named the concrete decoder; passing a real RAW through rawler
cannot tell the two apart, because both routes give the same answer.

The decoder_seam tests hand a stub decoder, for a container no real
decoder reads, to the catalog scan (read_metadata_only over a folder
backend), the preview ladder (the remote two-stage fetch, an import's
thumbnail and the viewer's no-GPU fallback) and export (open_for_export,
skipped without an adapter). Each assertion is on something only the
stub produces: its camera and date, a header fetched at its 64-byte
budget rather than HEADER_BYTES, preview and sensor sizes turned by its
orientation. Switching collect_metadata or make_thumbnail back to the
free functions fails two of the three tests.

The develop test_support module is widened to the crate so the export
test shares the one headless GPU context the other tests use. The
requirements note for FR-RAW-2 now records the trait as built and the
second decoder as not.
2026-09-24 21:33:14 -04:00
dtourolle 414094bd38 Route dr-ui's decoding through the Decoder trait
With the trait in place the claim still meant nothing while every caller
named dr_decode's free functions: a second decoder would have had to be
threaded through the scan, the thumbnail ladder, import, the viewer,
export, merge and repairs at the moment it arrived.

Each of those now takes a &dyn Decoder and reads headers, previews,
orientation and sensor data through it, including the header budget a
remote fetch asks for (header_bytes) and where it finds the embedded
preview (locate_preview). Only the places that start a job name
dr_decode::default(): the thumbnail, sweep and thumbnail-sweep threads,
the viewer's open handlers, and the request structs a job is handed
(BatchRequest, MergeRequest, the import Request, the repairs Toolkit),
so a caller can be given another decoder by changing what it is handed.

The default is rawler through the same free functions as before, so
nothing a user sees changes. The trait gains Debug as a supertrait so
request structs that derive Debug can carry one.
2026-09-24 21:33:14 -04:00
dtourolle d8fb382ce9 Put the RAW decoder behind a Decoder trait
FR-RAW-2 says a second decoder may be added for broader camera coverage
without changing callers, and D2 names LibRaw as that second decoder.
Nothing tested the claim: dr_decode was one decoder reached through free
functions, so adding another would have meant editing every caller at
the moment there was most pressure not to.

Decoder is an object-safe trait over bytes: header_bytes, metadata,
orientation, locate_preview, preview and decode. Rawler implements it by
delegating to the existing free functions, so behaviour is unchanged,
and dr_decode::default() hands it out as a &'static dyn Decoder, which
is what the places that start work will name. JPEG recognition, decoding
and completeness checks stay free functions: they are not a RAW
decoder's to vary.

Nothing in the trait takes a path or a SourceRef; the decoder states how
much of a file it needs and where its preview sits, and the caller's
storage fetches that.
2026-09-24 21:26:17 -04:00
dtourolle e8f68a92f8 Record intersection as built in the requirement and the mask plan
FR-DEV-19a said a part is added to the mask or taken out of it, and
mask-editing.md listed Intersect as an M2 item with nothing built. Both
now say what shipped: the requirement names the third join and why it is
a product rather than a minimum, and how old and new sidecars read across
it; the plan marks the blend-table row built, names the tests that hold
the GPU to the definition, and splits M2 into what is done and what is
still outstanding (joining non-painted parts from the panel, per-part
distance fields, folding two layers).
2026-09-24 21:25:48 -04:00
dtourolle 8f3df7b68d Format the intersect panel test as rustfmt lays it out
The chained lookup in the_intersect_button_joins_a_part_that_intersects was
one line past rustfmt's width, so fmt --check failed on the branch. Split
as rustfmt wants it; no behaviour changes.
2026-09-24 21:25:48 -04:00
dtourolle 5f0b7ac799 Offer intersection in the mask panel: an Intersect button and a third chip state
The pipeline could now keep only where two selections agree, but the panel
had no way to ask for it: the part row's chip flipped between + and -, and
the buttons under the parts joined an added or a subtracted correction.

An "∩ Intersect" button joins a painted part that intersects, and the chip
on a part row cycles + -> - -> ∩ and round, so an existing part can be
turned into an intersection without being repainted. Both go through the
same session calls as before, indexing Join::ALL, whose first two entries
kept their places. The chip is now a tagged gesture, so it is in the
gesture book.
2026-09-24 21:25:48 -04:00
dtourolle 8cdad3863d Keep only where two selections agree, as a third way to join a mask part
A layer's parts could be added to the mask or taken out of it, and nothing
else. The selections that need composing most are the ones that are
neither: the sky that is also bright, the subject that is also skin. With
union and subtract alone, "this and that" had to be spelled as "this minus
everything that is not that", which needs a second part that selects the
complement and rarely exists.

Join gains Intersect, stored as "intersect" in the part block of a sidecar.
It is the product of the two coverages, dst * src, which is one more
fixed-function blend state beside union's max and subtract's
dst * (1 - src) (mask-editing.md 5.2): the same scratch texture, the same
three vertices, no shader arithmetic. The product equals the minimum
wherever either side is fully in or out, and is the softer reading where
two soft edges overlap. Join::apply spells the three operations on the CPU
so the GPU tests can be held to one definition.

A layer that intersects with a part covering nothing now reports that it
covers nothing, so it is not rasterised as an empty slice. Old sidecars
never contain the word, so they read as before; a build from before this
reads "intersect" as a union, the existing unknown-join fallback, which
keeps the part visible rather than dropping it. Join::ALL keeps union and
subtract at indices 0 and 1 so a stored panel index still means the same
join.
2026-09-24 21:25:48 -04:00
dtourolle 229def0afc Show the file's own pixels at 1:1 and beyond
Zoomed to 1:1 or past it, the develop canvas showed a smoothed blur
rather than the photograph's pixels, so focus and noise could not be
judged at the magnification meant for judging them.

Two things caused it. The canvas only switched to nearest-neighbour
strictly past 1:1, with a margin, so the 1:1 inspection itself stayed
smooth. And the switch mostly had nothing to act on: the pipeline
rendered a viewport-sized frame at every zoom, so past 1:1 it was the
pipeline doing the enlarging - bilinearly whenever a straightening angle
or lens correction was in the chain - and the detail stage then sharpened
and denoised those invented pixels at radii scaled up to match. The
texture reached the canvas already blurred and was presented 1:1.

Now, from 1:1 on, the visible region is rendered at the source's own
resolution (render::render_size) and the canvas enlarges it with
nearest-neighbour, so the blocks on screen are the pixels an export would
have; it is also less shading. The decision lives in two small
functions, render::magnification and render::shows_source_pixels,
measured in physical pixels like one_to_one_zoom, with a half-percent
tolerance so the inspection zoom counts as 1:1 even where fit() rounded
the other edge. Below 1:1 the render and the smooth filter are unchanged.
2026-09-24 21:24:57 -04:00
dtourolle 5569a066ff Upgrade accounts saved as http:// to https on launch
Benchmarks / CPU and I/O (per commit) (push) Successful in 1m53s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 45m7s
Build and test / Layer separation (push) Successful in 41s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 40s
Build and test / Android (aarch64) (push) Successful in 28m49s
Build and test / Windows (x86_64, cross) (push) Successful in 17m9s
Build and test / Publish the release (push) Skipped
Before the previous commit, browser sign-in could store an account as
http://, and the client now refuses to send to one. Left alone, such a
library would fail to open with a configuration error, so its stored
endpoint is rewritten before anything reads it.

The endpoint is half of two keys, and both are handled:

- The keyring entry is filed under it. Rewriting only the record would
  strand the app password under the old key and sign the user out, so
  AccountStore::move_endpoint copies the secret across first, rewrites
  the record in place (the last record is the one resumed), and deletes
  the old entry only once nothing refers to it.
- namespace() is built from it and names the catalog directory. For an
  http to https rewrite it does not change, because the namespace strips
  either scheme. A move that would change it is refused, not performed,
  so no later rewrite can abandon a catalog either.

The rewrite is a new BackendProvider::upgrade_endpoint hook, which does
nothing by default, and not a second call to normalise_endpoint. The
folder connector's normalise_endpoint canonicalises the path and needs
it to exist, so running it on every launch would fail a library on an
unplugged disk, or rename one whose path now resolves differently. Only
Nextcloud implements the hook.

If the move fails (for example, a locked keyring), it is logged, the
account is left as it was, and the move is tried again on the next
launch.

Closes #65.
2026-09-24 20:44:19 -04:00
dtourolle ea31791388 Keep the typed server after browser sign-in, and open only https
Login Flow v2 saved the account under the `server` field of the poll
response, not under the address the person typed. That field is the
server's idea of its own URL. Behind a TLS-terminating proxy without
`overwriteprotocol` (a common setup) it says http://, and the account then
sent its app password in the clear on every request after that. The
typed address, already upgraded to https by normalise_endpoint, has just
carried the whole flow, so it is the one kept.

The flow's other two URLs come from the server as well, and are now
upgraded from http to https, and refused if they use any other scheme:

- The login URL is handed to the OS to open. On Windows that is
  `rundll32 url.dll,FileProtocolHandler`, which runs a file: or UNC path
  rather than showing a web page, so a hostile server could launch a
  program when the user starts signing in. open_in_browser also refuses
  anything that is not https, as the last check before a process starts.
- The poll endpoint is where the app password comes back from.

The host is not checked. A server reached by its LAN address can answer
with its public name, and refusing that would break a working setup
without protecting anything: the account is stored under the typed
address whatever the server says.

Part of #65.
2026-09-24 20:44:19 -04:00
dtourolle adade27de4 Refuse plain http in the Nextcloud client, below every URL it sends
NFR-SEC-3 held only for the address a person types: normalise_endpoint
upgrades it to https, and nothing else was checked. The login flow's poll
endpoint, an account an older build saved and a redirect all come from
somewhere else, and any of them naming http:// would send the app
password in Basic auth in the clear.

http_client now sets https_only. reqwest checks it before connecting and
again on each redirect, so a refused request never opens a socket, which
the new test checks with a listener that nothing may reach.

A refused scheme is reported as a Configuration error, not Network. The
request never left the process, and Network puts the app into offline
mode over a connection that is working. Other builder errors (a URL that
does not parse) go the same way, for the same reason.

Part of #65.
2026-09-24 20:44:19 -04:00
dtourolle a3f3e188e1 Move the people tray's ticks in place instead of rebuilding it per press
Benchmarks / CPU and I/O (per commit) (push) Successful in 1m52s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 45m4s
Build and test / Layer separation (push) Successful in 41s
Traceability / Requirement traces (push) Successful in 29s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 29m25s
Build and test / Windows (x86_64, cross) (push) Successful in 34m4s
Build and test / Publish the release (push) Skipped
Filtering the grid by a face crawled on the reference library. The SQL
is not it — the person predicate counts in ~20 ms, the eyes-open term in
~60 — but every press on the tray ran `push_people_chips`, which read the
whole people table (26,362 rows, nearly all empty groups a regrouping
pass left behind) and then called `push_people_roster`, which read it
again and replaced the roster model. The roster is every person holding
a face, 1,581 chips, in a row Slint does not virtualise: a new model
tore down and re-created all of them and laid the row out again, to
move one tick.

A press now walks the roster model and sets `picked` on the rows whose
tick changed; the roster is built only when the tray opens. Both reads
use `people_in_use` (2,140 rows) rather than `people`. A picked person
the in-use query leaves out — emptied by a split while the filter held
them — still gets a chip, since a term with no chip cannot be removed,
and without one the in-place update would fall back to a rebuild on
every press.
2026-09-24 20:21:50 -04:00
dtourolle 031315bdb6 Run cargo fmt over the develop shortcuts and the star-range filter
Benchmarks / CPU and I/O (per commit) (push) Successful in 2m8s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 33s
Build and test / Android (aarch64) (push) Successful in 14m21s
Build and test / android-image (push) Successful in 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / Desktop (Linux) (push) Successful in 47m57s
Build and test / windows-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / Layer separation (push) Successful in 51s
Build and test / Windows (x86_64, cross) (push) Successful in 34m3s
Build and test / Publish the release (push) Successful in 1m3s
bddf325 and 00c028c went in unformatted, so the Desktop job's
`cargo fmt --check` step failed on master (run 1693) and the release job
that needs it was skipped. Whitespace only.
2026-09-24 20:05:02 -04:00
dtourolle 00c028c8c8 Rate under the pointer, filter a star range, and name Help as help
Benchmarks / CPU and I/O (per commit) (push) Successful in 1m53s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 2m50s
Build and test / Layer separation (push) Successful in 33s
Traceability / Requirement traces (push) Successful in 40s
🐳 Android image / Build and push (push) Successful in 4s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 4s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 14m19s
Build and test / Windows (x86_64, cross) (push) Successful in 17m50s
Build and test / Publish the release (push) Skipped
Rating keys in the grid follow darktable's rule: with the pointer over a
photograph outside the selection, 0-5, P, X and U judge that photograph
alone; over one inside it, the whole selection, as before; off the grid,
the selection. The hover is cleared when the grid scrolls, so a key after
a wheel turn cannot judge whatever used to be under the pointer.

Holding F and tapping digits filters by stars: one digit for exactly
that many, two for everything between them, F alone to show every
rating again. The filter gains a ceiling to do it (`max_rating`, one
BETWEEN in the query). The place record carries it, and a record from an
older build reads as having none. The star chips light across a capped
range and the bar says "2-3★ only" beside them.

The grid also takes Ctrl+E and Ctrl+Shift+E for the selection, Ctrl+V to
paste onto it and Ctrl+A to select all. The "Gestures" button is now
"Help", its sheet "Controls and shortcuts", and F1 opens it.
2026-09-24 05:11:36 +02:00
dtourolle bddf3250c5 Add Lightroom's export and copy shortcuts to develop
Ctrl+E opens an export sheet: the export defaults on their own, over the
photograph, with an Export button. Ctrl+Shift+E exports straight away on
those defaults. There is no per-export copy of the settings, so what is
chosen in the sheet is saved as it is on the settings page, and the next
Ctrl+Shift+E uses it.

To make that one set of controls in two places, the export options move
out of the settings page into export.slint: an `ExportOptions` global
that Rust writes once, and two panels that read it. The window no longer
forwards forty `settings-*` properties to the page.

Ctrl+Shift+C opens a copy sheet with the edit-kind chips the preset
sheet already uses and a Copy button, which is how a paste leaves each
photograph's crop and rotation alone (Compose off). A and D step along
the roll beside the arrows. While either sheet is up the develop keys
stand down, so A cannot change the photograph behind the form, and
Escape closes it.
2026-09-24 05:11:16 +02:00
dtourolle 41486bd59b Step to the next photograph from the keyboard in a library's develop
The arrow keys and space in develop called `next-image` and
`prev-image`, which walk the files given on the command line. A
photograph opened from a library leaves that list empty, so the keys did
nothing and only a click on the photo roll moved on.

With a library open the step now goes through the roll: it opens the
neighbouring frame exactly as clicking it would, and saves the outgoing
edit the same way.
2026-09-24 05:09:39 +02:00
dtourolle d6d27fb062 Publish a Gitea Release from CI on every v* tag
Benchmarks / CPU and I/O (per commit) (push) Successful in 2m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 45m9s
Build and test / Layer separation (push) Successful in 38s
Traceability / Requirement traces (push) Successful in 44s
🐳 Android image / Build and push (push) Successful in 5s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 5s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 14m25s
Build and test / Windows (x86_64, cross) (push) Successful in 34m30s
Build and test / Publish the release (push) Skipped
Nothing made a release. CI built the APK and the installer on the master
push and kept them as workflow artefacts, the Linux binary was not kept
at all, and most tags went out with no downloads until they were
attached by hand.

build-and-test now also runs on v* tags. On a tag the desktop job keeps
its release binary, and a release job that needs desktop, Android and
Windows collects the three, names them with the version and runs
tools/publish-release.sh. The script titles and describes the release
from the annotated tag's message as the server holds it, writes
SHA256SUMS, and attaches what is not already there, so a re-run after
an interrupted upload finishes the job instead of duplicating it. The
same script is how a release is made or finished by hand.

Tried on v0.14.1, whose release was made by hand with the same files:
it found the release, reported all four files attached, and changed
nothing.
2026-09-24 03:43:42 +02:00
dtourolle 317a2f40bd Release 0.14.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m19s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 43m6s
Build and test / Layer separation (push) Successful in 37s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 3s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Traceability / Requirement traces (push) Successful in 46s
Build and test / Android (aarch64) (push) Successful in 29m7s
Build and test / Windows (x86_64, cross) (push) Successful in 34m31s
2026-09-23 19:07:22 -04:00
dtourolle 949fe40d5b Pin "Export N" to the right of the library's selection bar
It was the last of a dozen buttons in a row that scrolls sideways once
it outgrows the window, which at a desktop width it does. The button
sat past the right edge and nothing says the row scrolls to a mouse,
so batch export of a selection looked like a feature the library did
not have. The rest of the row still scrolls; Export, which is also the
cancel for a running batch, now sits beside it and is always visible.
2026-09-23 19:05:42 -04:00
dtourolle 2be80d4203 Show "Paste to N" on every selection, disabled until something is copied
It appeared only once settings had been copied this session, so with an
empty clipboard nothing on the selection bar said pasting onto a
selection was possible. It is one of the things a selection can have
done to it, like filing it in a collection, and now sits with them
beside Presets.
2026-09-22 21:37:41 -04:00
dtourolle 9ebaa15099 Move copy, paste and presets into the develop top bar
They sat in the develop column under a "SETTINGS" heading, which read as
application settings, and went away with the panel toggle and in the
mask and spot modes. The strip is where undo already is for the same
reason: these act on the whole edit, not on any one panel.

The paste button still names what it would apply. The TransferPanel
component is gone; the Transfer global and its Rust wiring are
unchanged.
2026-09-22 21:37:31 -04:00
dtourolle aee355fada Make the in-flight claim test wait for the waiter to park
a_second_claim_waits_for_the_first_to_be_released failed on CI: the
second claim came back Some. The waiter thread signalled the main thread
before calling claim, so the main thread could drop the first guard
before the waiter reached the lock. The path was free by then, and the
waiter claimed it outright.

The registry now keeps a test-only count of threads parked in claim,
bumped under the lock just before the condvar wait. The test spins until
that count is one before releasing. The release needs the same lock, so
it can only reach a waiter that is already waiting. Passed 500 runs in a
row.
2026-09-22 21:12:45 -04:00
dtourolle 7fa3176f88 Release 0.14.0
Benchmarks / Frame budget (on demand) (push) Skipped
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m25s
Build and test / Desktop (Linux) (push) Failing after 26m11s
Build and test / Layer separation (push) Successful in 33s
🐳 Android image / Build and push (push) Successful in 10m41s
Build and test / android-image (push) Successful in 10m42s
🐳 Windows image / Build and push (push) Successful in 4m17s
Build and test / windows-image (push) Successful in 4m17s
Traceability / Requirement traces (push) Successful in 59s
Build and test / Android (aarch64) (push) Successful in 40m53s
Build and test / Windows (x86_64, cross) (push) Successful in 46m51s
2026-09-21 11:32:41 +02:00
dtourolle af162dd010 Merge: origin's sync ordering and adoption work, its scan-complete trigger ported into library_ui/ 2026-09-21 11:32:05 +02:00
dtourolle f5300a9f43 Merge: the UI crate restructured so a feature owns a function, not a region
Nine agent branches, merged one at a time on ui-wiring and gated at each
step: develop.rs, library.rs, collections_ui.rs and library_ui.rs become
module directories; every screen's wire() and lib.rs::run() become lists
of named functions, with the develop screen's callbacks in develop_ui.rs;
the six view booleans become View and Page enums; the collections sidebar
and the library grid get their own Slint globals, taking 155 members off
AppWindow. No behaviour change: a multiset audit of code lines over every
moved region lost nothing, the workspace gate is clean, and the manual
recorded from this build and from master's from the same library snapshot
matches picture for picture, apart from a panorama stall that master
shows too when a merge starts during the engine's TensorRT compile queue.
2026-09-20 23:41:25 +02:00
dtourolle b19d470189 Merge: the library grid on its own Slint global, the last of the screen state off the root 2026-09-20 22:43:09 +02:00
dtourolleandClaude Opus 5 bfadd9c409 Ship the border filler in the Windows installer, and count what is staged
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m2s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 25m17s
Build and test / Layer separation (push) Failing after 0s
Traceability / Requirement traces (push) Failing after 0s
🐳 Android image / Build and push (push) Failing after 0s
Build and test / android-image (push) Failing after 0s
Build and test / Android (aarch64) (push) Skipped
🐳 Windows image / Build and push (push) Failing after 0s
Build and test / windows-image (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Skipped
The installer smoke test asserted seven model files, the number on the
day it was written; models/face has since gained the eye-state trio's
companions and the int8 detector forms, and the run on f71d7ba failed
with thirteen installed. The test now expects as many files as
package.sh's directories hold, so the next model needs no edit here.

package.sh also stages models/inpaint, which the APK and the Arch
package already carry and the Windows build did not: without
migan-512.onnx the panorama's border fill has no model on Windows.
xfeat needs nothing, it is embedded in the binary. windows.md §5.2
lists the result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 22:35:04 +02:00
dtourolle 050b2a3914 Collapse the blank runs the library global's move left behind
Moving each group of properties and callbacks out of AppWindow left
several two- and three-line gaps where a removed block's neighbours no
longer needed separating. No declarations changed.
2026-09-20 22:32:47 +02:00
dtourolleandClaude Opus 5 195388b2e3 Adopt faces a hundred images per commit, and hold one generation per image in the shards
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 4h42m34s
Build and test / Layer separation (push) Successful in 59s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The shard import recorded each adopted image in its own transaction:
fourteen thousand commits, and fourteen thousand turns at the write lock
that every read on the UI thread queued behind — the sync was felt as a
laggy grid and as "database is locked" from whichever writer lost the
wait. `record_detections_within` takes the caller's transaction, and the
import commits every hundred images.

The store carried every detector generation of an image — 24,123 entries
for 19,089 images on the reference library, a third of its 293 MB — when
only the strongest is ever adopted. A put now skips a pass a held one
outranks, and retires the passes it outranks from the index; sealed
shards keep their bytes, but nothing is written twice from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 22:31:47 +02:00
dtourolle e166b64ee9 Format fill.rs, which the last release left past the width limit 2026-09-20 22:29:42 +02:00
dtourolle a773ad5c27 Merge: the developer docs under docs/dev, and the folder indexed for users first 2026-09-20 22:27:48 +02:00
dtourolle 1e472fd251 Move the library's routes and status onto its own Slint global
The scan/opening/status/error lines, offline mode and its retry, pinning a
collection offline, the sync and thumbnail-sweep state, and the callbacks
that route the grid to a rescan, a sync, another library, a panorama merge
or an export — the last of what AppWindow still carried under the
library- prefix — move onto the `Library` global started earlier on this
branch. Rust reaches them through window.global::<Library>() rather than
window.set_/get_/on_/invoke_ on the root.

library-visible is the one name that stays: it is computed from
active-page and active-view, the shell's own routing state, which a
global cannot read. AppWindow now declares no other library- property or
callback.
2026-09-20 22:22:55 +02:00
dtourolle 6bf67cefc4 Move the filter bar and the timeline onto the library's Slint global
The date-range fields and their band on the capture-time axis, the
timeline's bars, labels, scrub/pinch/pan/zoom callbacks and its
sweep/current-bucket/anchored state, and the filter bar's people chips,
mode, eyes-open toggle and gesture reference move from AppWindow onto the
`Library` global. Rust reaches them through window.global::<Library>()
rather than window.set_/get_/on_ on the root, as the earlier commits on
this branch did for the grid's cells, selection, ratings and keywords.
2026-09-20 22:07:12 +02:00
dtourolle 00e2fe6aaf Move ratings, flags and keywording onto the library's Slint global
The keywording sheet's rows and its open/assign/unassign callbacks, the
star and flag callbacks a cell click or a judgement key fires, the burst
toggle and representative-chosen callbacks, the trash-selection shortcut,
and the rating/unjudged/flag filter chips with their rating-counts model
move from AppWindow onto the `Library` global started in the previous
commit. Rust reaches them through window.global::<Library>() rather than
window.set_/get_/on_ on the root.
2026-09-20 21:57:40 +02:00
dtourolle 402dcdc24c Move the library grid's cells and selection onto their own Slint global
AppWindow carried the grid's loaded window of cells, the keyboard cursor,
drag and drop, the held-row long-press state, columns and cell size, the
scroll and viewport bookkeeping, the photo roll's pick and centre-request,
and the local-only/reorder/collection-filing gestures that act on a
selection, as properties and callbacks on the root component. That state now
lives in the `Library` global declared in library.slint, next to the structs
(LibraryCell, TimelineBar, KeywordRow, PersonChip) it and the grid's other
components already share; Rust reaches it through
window.global::<Library>() instead of window.set_/get_/on_/invoke_ on the
root, the same change collections.slint's `Collections` global made for the
sidebar.

library-visible stays on AppWindow: it is computed from active-page and
active-view, the shell's own routing state, which a global cannot read.
Everything else still prefixed library- — the timeline, the filter bar,
ratings and flags, keywording, and the routes and status lines — stays on
the window for now and moves in the commits that follow.
2026-09-20 21:42:48 +02:00
dtourolle 94b39410bc Tidy the ported drag block, and format fill.rs as master left it 2026-09-20 21:40:20 +02:00
dtourolle 2014c80e62 Merge: master at 0.13.6, with the drag-ghost file and the shared model lookup ported into the split modules 2026-09-20 21:32:21 +02:00
dtourolleandClaude Opus 5 6fd342680b Format set_indexed_at's signature
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m15s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Build and test / Desktop (Linux) (push) Successful in 1h8m29s
🐳 Windows image / Build and push (push) Successful in 5s
Build and test / windows-image (push) Successful in 5s
Build and test / Layer separation (push) Successful in 34s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
baed1c4 landed it over rustfmt's width; `cargo fmt --check` is the
first gate the Desktop job runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:31:07 +02:00
dtourolleandClaude Opus 5 78acc73dad Regenerate the traceability matrix for the sync commits
f71d7ba and 34ac2f1 added tags without re-running the report, which
the Traceability job's "is it committed" step rejects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:28:55 +02:00
dtourolleandClaude Opus 5 954246b969 Register the inference requirements, and format the fill test
Two CI gates have failed on every push since 0.13.4 and both are
fixed here.

The traceability gate rejected `FR-INF-1` as an orphan: settings.slint
and dr-ui tag it, but the four register entries inference.md §12 wrote
were never carried into requirements.md, which is the only file the
extractor reads. §3.12 and §4.10 now hold FR-INF-1..3 and NFR-INF-1
verbatim, with the acceptance milestones pointed back at inference.md.
The matrix is regenerated (188 defined, 155 covered) and the README's
"where it stands" line, which the 0.13.6 release commit skipped, says
0.13.6 and the new figures.

`cargo fmt --check` failed on the `fill_border` call in dr-pano's
padding test, which is the first thing the Desktop job runs after
installing the toolchain and why it failed within a minute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:28:55 +02:00
dtourolleandClaude Opus 5 baed1c4782 Keep the peer's run marker on adopted faces, so they are not re-exported as ours
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 32s
Build and test / Layer separation (push) Successful in 29s
Traceability / Requirement traces (push) Failing after 42s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
Adopting an image from a peer's face shard stamped its run marker as now,
and the export reads a catalog marker newer than the shard's as a
re-index. So every adopted image went straight back out under this
device's client id: 14,100 adopted, 15,457 "newly indexed" on the next
pass, twenty-two shards of a peer's faces uploaded a second time.

merge_shard now carries the peer's indexed_at into the local index, and
the import writes that marker into face_index; where an older peer's shard
carries none, the store takes the catalog's, so the two agree either way
and the export finds nothing to send.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:25:53 +02:00
dtourolle fbfa891296 Regenerate the traceability matrix after the rebase 2026-09-20 21:23:23 +02:00
dtourolle aee62dc7f2 Wrap the line the docs move pushed past the width limit 2026-09-20 21:16:03 +02:00
dtourolle 84fade99ec Put the developer docs under docs/dev and index the folder for users first
docs/ had 26 developer documents flat beside the manual, and the two
audiences are very differently sized: most readers want the manual and
the gesture reference, a few want the register, the designs and the
measurements. The manual and gestures.md stay at the top; everything for
someone changing the code moves to docs/dev/, and the two documents that
name their own successors — the v0.1 milestone and the UI-refinement plan
— go to docs/dev/archive/ rather than being deleted, since both are still
cited. docs/README.md is the index, users first.

Every reference follows: code comments, Cargo manifests, the workflows,
the pre-commit hook, the bench and traceability tools (which locate the
repo root by docs/dev/requirements.md now), packaging, the Docker READMEs,
CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level
deeper and is regenerated. Links out of the moved documents into the tree
gain a level; a link checker over every Markdown file finds none broken.
2026-09-20 21:16:03 +02:00
dtourolle 3bfa73d1e1 Merge: the collections sidebar on its own Slint global 2026-09-20 21:14:00 +02:00
dtourolle 4616cb0a23 Merge: library_ui split into a module directory 2026-09-20 21:12:36 +02:00
dtourolle cc73ea3153 Split library_ui.rs into a module directory by area of behaviour
controller holds LibraryController and the window-sizing constants every
other module reads and writes through pub(super) fields, the same shape
collections_ui and develop already use. open is the launch-to-scan cycle and
the worker that checks the catalog file before either touches it. offline is
what of a collection is on this device and the prompt that offers to change
it. window fills the grid model from the catalog and drains the thumbnail
fetch, which is the piece the catalog-reads-are-proportional-to-what-changed
rule (docs/catalog.md §1) bears on most directly. sync is the background
passes that reach beyond the loaded window: the metadata sweep, the
whole-library thumbnail pass, and the exchange with the server.
ratings_keywords applies a judgement or a keyword to a selection and queues
the sidecar and XMP writes behind it. timeline is the capture-time sidebar
and the photographer's place together, kept in one file because a restored
place ends by moving the timeline marker and a scrub is a restore of one
instant, so most calls between the two would otherwise cross a module
boundary. grid wires the grid's own callbacks — the keyboard cursor,
cell-size zoom, the routes into and out of develop — and filter_bar wires
the rating, people, date and offline-scope filters, calling back into
whichever of the above owns the work a filter change triggers.

Extracted by item rather than by line range, so every doc comment and
TRACES/GESTURE annotation stayed attached to the code it describes; the
sorted set of TRACES/GESTURE lines in the new directory is identical to the
original file's. Tests moved with the code they exercise, including the
handful of fixtures — settle, model_of, with_catalog, zoom_cell, pinch_step
— that only one target module needed and so were not worth sharing through
a test_support module the way the other splits use one. Items that crossed
a new module boundary were widened from private to pub(super), narrower
than the whole-file access the original gave them; a few items already
pub(crate) for recovery_ui or presets stayed there rather than being
narrowed, since nothing needed them tightened further.

mod.rs re-exports the same surface library_ui:: callers used before, so
lib.rs and every other caller needed no change.
2026-09-20 21:10:02 +02:00
dtourolle f6ff5eabd9 Give the collections sidebar its own Slint global
AppWindow carried the sidebar's tree, its row menu, renaming, drag and
drop between rows, the trash row, and the membership sheet as ~40
properties and callbacks on the root component, in the pattern CH-1
describes and the develop screen's globals (Adjustments, Framing, Steps,
...) already replaced. Collections.* in collections.slint now holds that
state, declared next to the MembershipRow struct it and the membership
sheet both use; Rust reaches it through window.global::<Collections>()
instead of window.set_/get_/on_/invoke_ on the root.

collection-selected, collection-select, and collection-offline-menu stay
on AppWindow: library_ui.rs invokes collection-select directly and
registers collection-offline-menu's handler, and lib.rs reads
collection-selected for back-navigation, so moving them would have meant
editing library_ui.rs, which another change on this branch is splitting
into a module directory. collections-visible stays too — it is
lib.rs's panel-layout state, seeded from the saved layout before the
sidebar exists, and collections_ui never touches it. Everything prefixed
library- (the grid's drag, selection and keyword state that the sidebar's
Rust also wires for cross-feature gestures like filing a selection into
a collection) stays on the window as well, since it belongs to the
library screen, not the sidebar.
2026-09-20 21:07:25 +02:00
dtourolleandClaude Opus 5 34ac2f14d3 Sync faces and the catalog before thumbnails, so a fresh device sees its names and collections first
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 4s
Build and test / Desktop (Linux) (push) Failing after 35s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 30s
Traceability / Requirement traces (push) Failing after 48s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The thumbnail stage ran first and, on a device that had just adopted its
peers' shards, spent its time re-uploading hundreds of megabytes under its
own client id while faces, people, collections and dates waited behind it.
Faces go first — the catalog merge assigns identities to faces this device
holds — then the catalog, then thumbnails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:04:10 +02:00
dtourolleandClaude Opus 5 f71d7bacc6 Take the server's shards and dates when the scan completes, not after the sweep
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 36s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 43m11s
Build and test / Windows (x86_64, cross) (push) Failing after 41m4s
The derived sync fired only after the metadata sweep, so a fresh device
re-derived every thumbnail it scrolled past, re-detected faces and re-read
every header for hours before adopting the shards and snapshot that held
all of it. It now fires as soon as the scan completes — the first moment
the rows the merges key on exist — and the sweep starts behind it. In
steady state that pass is one listing.

The catalog merge gains a fourth half: capture metadata (captured_at,
offset, camera, lens, ISO) for images still at metadata_state < 2, matched
by oc:fileid from a remote row at 2. A date is a fact about the file's
bytes, not local state, and the snapshot already carried it. The sweep's
per-chunk query then finds nothing left, and the timeline is whole on a
fresh device without a header fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 20:37:59 +02:00
dtourolle 14ed1dc410 Merge: View and Page enums in place of the six view booleans 2026-09-20 20:32:41 +02:00
dtourolle 38819da222 Replace the six view booleans with View and Page enums
app.slint carried show-launch, show-library, show-identity, show-settings,
show-import and show-merge as separate booleans, so the root component chose
what to draw with five- and six-term conjunctions and nothing stopped two of
them being true at once. Replaced with two enums: View { develop, library,
identity, launch } for which top-level screen is showing, and Page { none,
settings, import, merge } for which page, if any, is drawn over it.

Two values rather than one, because the two questions are genuinely
different. Settings, Import and Merge are reachable from more than one View
and are drawn outermost without touching it — closing one has to return to
whichever View was already current, and today that works because the
underlying property is left alone while the page sits over it. A single
View with five or more variants would need a second field remembering what
to return to; Page needs nothing to remember, since View was never
overwritten in the first place. Identity, by contrast, genuinely replaces
the window the way Launch and Library do (see the existing "like the launch
screen" comment on its `if`), so it is a View variant, not a Page.

Every `if` chain in app.slint that used to compare four, five or six
booleans now compares active-view and active-page to at most one variant
each. library-visible collapsed from a six-term conjunction to
`active-page == Page.none && active-view == View.library`.

The Rust side follows: every set_show_*/get_show_* call in library_ui.rs,
identity_ui.rs, settings_ui.rs, merge_ui.rs, import_ui.rs, launch_ui.rs and
lib.rs now reads or writes active-view or active-page instead, including
lib.rs's startup match (View.launch vs View.develop, since a Startup that
skips the launch screen used to leave both old booleans false and fall
through the chain to develop) and identity_ui's close handler, which now
writes View.library or View.develop in one call where it used to write
show-library then show-identity separately.

back_one_step needed one deliberate adjustment beyond the mechanical
rename. Identity was never represented in NavState: back had nothing to do
when Identity was opened from the library (show-library stayed true,
unread by IdentityScreen's own condition) and could only reach ToLibrary
when opened from develop, which likewise wrote a property IdentityScreen
never read — so escaping out of Identity was invisible in both cases before
this change. With a single active-view, falling into the general case
would instead overwrite the value IdentityScreen's `if` does read and close
it as an unintended side effect. back_one_step now swallows the gesture
while View.identity is current, reproducing the same "nothing visible
happens" outcome for both origins without threading identity_ui's private
came-from-library state through lib.rs for one screen.

Verified with tools/manual/drive.py against a private Xvfb and the debug
build: launch screen to library, Settings opened and closed, Identity
opened and closed (including Escape doing nothing while it is open),
develop opened from a cell and closed both by the back button and by
Escape. Screenshots under verify/.
2026-09-20 20:30:56 +02:00
dtourolle d6e9c7dc94 Merge: collections_ui split into a module directory 2026-09-20 20:30:12 +02:00
dtourolle 9c8f21b754 Split collections_ui.rs into a module directory by area of behaviour
collections_ui.rs had grown to 4,591 lines covering the sidebar controller,
the click/drag selection policy, tree refresh, the drag gesture, the trash
worker, twelve wiring functions, and the row's rename/create/context menu,
all in one file. Split into collections_ui/ with one module per area, the
way develop/ and library/ were already split on this branch:

- controller.rs: CollectionsController and the pure drop/delete/release
  decisions (decide_drop, decide_delete, decide_release, menu_detail,
  delete_warning) that a test can drive without a window.
- press.rs: PressUndo and the click-and-release selection policy
  (apply_press, select_row, commit_press, cancel_press).
- tree_sync.rs: rebuilding the sidebar from the catalog and pushing
  catalog-derived state into the grid (refresh_tree, offline_state,
  sync_lifted/sync_selection/sync_reorderable/sync_badges,
  refresh_membership, direct_holdings).
- drag.rs: the cursor bitmap (compose_drag_image, blit_scaled) and the
  hold/spring timers (arm_hold, arm_spring, should_spring,
  collapse_spring_opened) plus their delay constants.
- trash.rs: the soft delete (start_trash, start_restore, drain_trash,
  stop_trash, refresh_trash, format_bytes).
- wiring_grid.rs / wiring_tree.rs: the wire() entry point and its twelve
  wire_* functions, split in two because together they were the largest
  single piece (grid-facing selection/drag/trash vs. sidebar-facing
  navigation/create/rename/row-drag/menu/membership).
- rename_menu.rs: creating, naming and renaming collections, and the row's
  context menu (apply_rename, create_child, unique_name, open_row_menu,
  close_row_menu, close_rename).

mod.rs carries the module's own top-level doc comment, the `pub use`
re-exports for the eight items the rest of the crate reaches by
`collections_ui::` path (CollectionsController, wire, refresh_tree,
sync_badges, sync_selection, select_row, commit_press, cancel_press), and a
shared `test_support` for the one fixture (`ids`) more than one file's
tests needed. Every item that only crossed a boundary within this module,
not out of it, was narrowed to `pub(super)` rather than kept at the
crate-wide `pub` a single file gave it for free.

Extracted with a brace-aware pass that kept each item's own leading doc
comment and attributes attached to it, and tests moved with the code they
exercise; every TRACES/GESTURE comment lands on the same code it did
before. No file outside the new directory changed — lib.rs's `mod
collections_ui;` resolves to the directory automatically, and every
outside caller's `collections_ui::` path still resolves through mod.rs's
re-exports.
2026-09-20 20:29:21 +02:00
dtourolle bd7d75522d Merge: the format fix from docs-layout 2026-09-20 20:26:56 +02:00
dtourolle 4ef1b74f2f Wrap the line the docs move pushed past the width limit 2026-09-20 20:26:54 +02:00
dtourolle 681486196e Release 0.13.6
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m17s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Build and test / Desktop (Linux) (push) Failing after 29s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 42s
Build and test / Android (aarch64) (push) Failing after 2m48s
Build and test / Windows (x86_64, cross) (push) Failing after 4m22s
2026-09-20 20:19:48 +02:00
dtourolle f5d0d57574 Regenerate the traceability matrix after the rebase 2026-09-20 20:19:43 +02:00
dtourolle c96e670356 Re-record the panorama for the trained filler; the scene waits for the preview and for the DNG instead of guessing 2026-09-20 20:19:01 +02:00
dtourolle 9c556364fa Pad an open void's canvas to a tile: the merge page's preview is shorter than one, and filled nothing 2026-09-20 20:19:01 +02:00
dtourolle 8d72cabff5 Ship the border filler trained against MI-GAN's own discriminator: texture in the deep bands, level with stock on LPIPS 2026-09-20 20:19:01 +02:00
dtourolleandClaude Opus 5 8a90d888d5 Probe and compile for a package's models, not only the user's
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 53s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m19s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
`inference::init` listed the models from the user's shared directory
alone, while the app loads them from there or from the package's
`/usr/share/darkroom/models`. On a fresh package install the probe found
"no model to probe with", stayed on the CPU, and compiled nothing. Both
now resolve each file with the same search, `library::shared_model`.

The PKGBUILD names the ONNX Runtime packages as optional dependencies,
since the app loads one from /usr/lib if present.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 19:39:05 +02:00
dtourolle e86edef47c Merge: run() reduced to construction and startup, the develop wiring in its own module 2026-09-20 19:28:23 +02:00
dtourolle 0aff5e6c8c Merge: the collections, identity, merge, launch and import wiring lifted into named functions 2026-09-20 19:28:12 +02:00
dtourolle 5458314083 Split import_ui::wire into one function per section
The 188-line wire() registered the import page's callbacks in four
comment-delimited sections. Lift each into its own fn: wire_opening_and_closing,
wire_choosing_a_source, wire_options, wire_running. context stays generic
over C on each of the three functions that use it, matching how survey()
and start() already take it (impl Fn, implicitly Sized) rather than coercing
it to a trait object, which would have needed ?Sized added to those two
unrelated functions for no benefit.
2026-09-20 19:23:21 +02:00
dtourolleandClaude Opus 5 39a22875b1 Add the MIGraphX rung for AMD GPUs
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m20s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 45s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 46s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m19s
Build and test / Windows (x86_64, cross) (push) Failing after 3m2s
Measured on a Radeon RX 7900 XT against Arch's onnxruntime-rocm 1.29
(docs/inference.md §1.3): MIGraphX fp16 runs the detectors at 2.4–3.4 ms
against 10–58 ms on the CPU provider, the inpainter at 8 ms against 514,
with a 15–135 s compile per graph the first time and under a second from
its cache after. A compiling rung on TensorRT's terms, wired the same way.

The ROCm execution provider is gone (removed in ONNX Runtime 1.23), so the
AMD ladder is MIGraphX then the CPU, with no non-compiling rung between.

MIGraphX is registered through the runtime's generic key/value entry
point rather than ort's builder: 1.29 reads the legacy options struct for
its precision flags only, and the compiled-program cache directory
(`migraphx_model_cache_dir`) only travels the generic way. The provider's
cache key omits the precision, so f32 and fp16 programs get their own
directories. The probe fingerprint now includes the provider libraries
beside the runtime and the ROCm version, since a distribution's CPU and
ROCm builds are the same file at the same path.

`status().failed` reports only the rungs above the selection, so an AMD
desktop's About line says why MIGraphX won rather than that the NVIDIA
providers are not in the build.

Two examples: `ep_probe` times each provider cold and from cache, and
`ladder` drives `init` as the app does to watch the first-run sequence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 19:23:00 +02:00
dtourolle cb7b9d71c4 Split lib.rs::run into named construction and wiring functions
run() was 2,724 lines that built every controller, owned the develop
session and its render loop, and registered every develop-screen callback
inline — the state CH-1 in docs/dev/code-health.md describes. This is the
mechanical split CH-1 calls for, done in one pass rather than section by
section since develop_ui.rs only compiles once lib.rs stops registering
those callbacks itself.

develop_ui.rs is new: a DevelopWiring struct holding the session, the rows
model, the redraw/render closures and the other controllers' handles, and
one wire_* function per section run() used to contain — presets, export,
the adjustment panel, film, undo/redo, zoom/pan/crop, rotation/flips/
straightening, navigation and peaking — called in the order run()
registered them. window travels as its own parameter throughout rather
than living on the struct, because the generated AppWindow type is not
Clone; every other field is an owned clone so each function's body could
be pasted from run() unchanged.

lib.rs::run is now under 300 lines: construction and startup only, calling
named functions for the window and its diagnostics/inference wiring, the
launch/library/collections/identity/settings screens, import and merge,
the render loop (render_now/redraw/show), the remote open path, the
window chrome (resize, layout class, panel toggles, back gesture), and
develop_ui::wire for the rest. Every TRACES and GESTURE comment moved with
the code it annotates.

Two latent type errors surfaced while restructuring rather than being
introduced by it: activity and display were being passed around as bare
ActivityLog/DisplayWatch instead of the Rc<...> their constructors
actually return, which only worked before because nothing needed to name
the type explicitly.
2026-09-20 19:17:22 +02:00
dtourolle beb7a5eac0 Split launch_ui::wire into one function per section
The 225-line wire() registered the launch screen's callbacks in nine
comment-delimited sections. Lift them into fn's, merging a few adjacent
ones that were only a handful of lines each: wire_sign_in covers both the
browser flow and the app-password fallback (the same button in two forms),
wire_choose_folder_and_open covers opening the folder browser and the
final "Open library" press, since both are short and sit back to back.
wire_use_folder, wire_sign_out, wire_formats, wire_folder_picker_navigation
and wire_copy_url stay as they were sectioned. wire() calls each in the
original order and keeps the closing render() call, which every section
relies on having run once at startup.
2026-09-20 19:11:43 +02:00
dtourolle 262ed2553c Split merge_ui::wire into one function per section
The 312-line wire() registered the panorama page's callbacks in three
comment-delimited sections. Lift each into its own fn: wire_start (the
"Merge to panorama" press, its fetch, and the DARKROOM_START_MERGE dev
entry point — all three only ever used together), wire_decision (confirm,
projection and border chips, the fill knobs), and wire_stop_and_leave
(abandon, close). wire() itself now just calls the three in order; S, C
and F stay generic on wire_start since sources/context/on_done are used
nowhere else.
2026-09-20 19:00:31 +02:00
dtourolle 8d08ffd7b7 Split identity_ui::wire into one function per feature
The 837-line wire() had almost no section comments, unlike its siblings, so
the seams had to be found by reading it rather than following markers. Lift
each into its own fn: wire_dials (the two grouping sliders), wire_navigation
(open/close/switch person — needs models() too, for the missing-model
banner), wire_rename_and_merge (a rename and the namesake offer it can
raise), wire_face_actions (pick/confirm/reject/split, the grid's own
actions), wire_grouping_preview, wire_recluster, wire_indexing (the shared
launcher behind Index/Re-index plus Stop), and wire_coverage_and_ignore.
wire() keeps the generic-to-trait-object coercions and the eyes_available
closure, since most of the above need it, and calls each function in the
original order. The reload! macro moved from inside wire() to module scope,
dedented, since macro_rules is scoped textually and every extracted function
uses it.
2026-09-20 18:49:43 +02:00
dtourolle 8540518022 Split collections_ui::wire into one function per section
The 1,457-line wire() registered every collections-sidebar and grid-drag
callback in one function, sectioned only by comment. Lift each section
into its own fn: wire_selection (split further into wire_selection and
wire_selection_filing, since the original section ran to 375 lines),
wire_drag, wire_trash, wire_trash_from_grid, wire_tree_navigation,
wire_create, wire_rename, wire_remove, wire_row_drag (the tree row's own
hold-drag, which the original "remove" comment's span covered but which
is really a separate feature), wire_row_menu, and wire_membership.
wire() itself now just coerces the shared closures to trait objects and
calls each in the original order. visible_ids is coerced to
Rc<dyn Fn() -> Vec<ImageId>> at the top, alongside on_scope_changed and
session, so the new functions take plain trait objects instead of
threading a generic parameter through every one of them.
2026-09-20 18:36:18 +02:00
dtourolle b952f5976a Merge: the wire sections lifted into named functions, beside the module splits 2026-09-20 18:26:24 +02:00
dtourolle a1d511fd4b Split library.rs into library/ by area of behaviour
library.rs was 7,729 lines wiring together everything "open a remote
library" touches: scanning, pulling other devices' judgements out of
sidecars found along the way, writing local edits back out to the
sidecar outbox, pushing/reloading XMP by hand, fetching and prefetching
thumbnails and originals, generating thumbnails locally, the metadata
and thumbnail background sweeps, on-disk paths for the catalog and
model files, and reading the grid's cells, spans and rating filter.
Same motivation as the develop.rs split (docs/dev/code-health.md CH-1):
a pure, no-behaviour-change move into one file per area, each under
about 1,500 lines.

Tracing actual call sites rather than trusting the file's physical
layout mattered here: `persist`, `load_folder_etags`, `pull_sidecars`,
`load_sidecar_etags`, `record_sidecar_read` and `apply_judgement` sit
textually beside the XMP push/reload functions but are called only
from `run_scan` (pulling a device's own past judgements out of the
sidecars a scan just walked), so they went to scan.rs and not xmp.rs.
`cells` came out at over 1,800 lines once its tests moved with it and
split further into cells.rs (windowed reads, trash, ordinals) and
spans.rs (collection scope, manual reordering, the capture-time
histogram) -- ten submodules rather than the nine first planned.

Previously-private items reached from a sibling module became
`pub(super)`, narrower than the whole-crate reachability one file gave
them. Tests moved with the code they test; the two test fixtures used
across more than one file (`scanned`, and develop.rs's
`session_with_a_left_half_subject` in the matching commit) joined the
shared `test_support` module alongside the existing `entry`/
`with_images`/`image_ids` helpers. `mod.rs` re-exports every module's
public items under `library::`, including the `pub(crate)`
`test_support` module `repairs.rs` reads its fixtures from, so no file
outside `library` needed a change.

The previous commit split develop.rs the same way; taken alone it left
dr-ui without library.rs, so that intermediate commit does not build on
its own. This one restores it.
2026-09-20 18:21:43 +02:00
dtourolle 050c2c9d16 Split develop.rs into develop/ by area of behaviour
develop.rs had grown to 9,327 lines covering everything the develop
session does: opening a photograph, the parameter-row and curve-widget
panel model, mask viewing and editing, mask creation and the rasteriser
that turns a mask stack into GPU arrays, spot repairs, scene
segmentation, framing and zoom, white-balance sampling, rendering and
film choice, and the undo/snapshot history. docs/dev/code-health.md
CH-1 names dr-ui's lack of a view layer as the reason every feature
kept landing in a handful of files; this is the first of the two pure
splits it recommends as easy, no-behaviour-change wins independent of
that larger rework.

The boundaries follow the file's own sections (several were already
marked off with comment headers) and the seams a full read turned up
underneath them -- mask storage/rasterisation turned out to be a
distinct concern from mask viewing and editing, and rows/tabs/curves
from each other, so those split further than the headers alone
suggested. Each module stays under about 1,500 lines. Struct fields
and the handful of helper methods now called from a sibling module
became `pub(super)`, which is strictly narrower than the whole-crate
reachability a single file gave them; nothing gained visibility outside
`develop`. Tests moved with the code they test, including the few
cases where a helper one file's tests needed was itself only defined
in another's -- those became shared fixtures in `mod.rs` alongside the
`headless`/`read_back`/`grey_session` helpers that already worked that
way. `mod.rs` re-exports every item `develop::` callers outside this
module used before, so lib.rs, masks_ui.rs and the rest needed no
changes.
2026-09-20 18:21:26 +02:00
dtourolle bd3b993b90 Split library_ui::wire into one function per section
wire() registered every grid callback in one 1,214-line function behind
four section comments, two of which were themselves far over 300 lines
with no further markers. Each fenced section becomes its own function,
called from wire() in the original order with the section's own comment
kept as its doc comment:

- "the keyboard cursor (FR-CULL-4)" (496 lines) splits at its own topic
  breaks into wire_grid_cursor_and_zoom (cursor movement, cell zoom and
  pinch), wire_grid_sync_and_load (explicit sync/thumbnail requests and
  the reloads a changed viewport, column count or scroll position
  trigger), wire_timeline (the capture-time sidebar) and wire_grid_routes
  (grid/launch/develop navigation and a manual rescan).
- "ratings and flags" and "keywords" were already under 300 lines and
  become one function each.
- "the filter bar" (447 lines) splits into wire_filter_ratings_and_people
  (which keeps the section's own comment), wire_filter_dates and
  wire_filter_scope_and_offline.

The three callbacks registered before the first section comment
(on_settings_xmp_reload, on_library_cell_clicked, on_library_roll_pick)
and the trailing crate::recovery_ui::wire call stay directly in wire(),
since neither is inside a fenced section.
2026-09-20 17:25:00 +02:00
dtourolle 59605f9fbb Split masks_ui::wire into one function per section
wire() registered every mask-panel callback in one 778-line function
behind section comments. Each of the seven fenced sections (computing
the region map, refining a subject's mask, dragging a gradient,
selecting on the photograph, the stack, the edge treatment, adding
layers) becomes its own function, called from wire() in the original
order with the section's own comment kept as its doc comment. The
"adding layers" section was itself over 300 lines and had no further
section markers inside it, so it is split at its own natural seam
between painting/viewing a mask (wire_layers_paint) and working the
parts and add-mask buttons (wire_layers_parts); the second half gets an
introductory doc line since there was no comment of its own to reuse.
Locals declared just for one section's closures (running, refining,
dragging) move into that section's function instead of staying in
wire().
2026-09-20 17:24:42 +02:00
dtourolle 04949741c1 Split settings_ui::wire into one function per section
wire() registered every settings-page callback in one 416-line function,
fenced only by section comments. Each fenced section (opening and
closing, cache, faces, export, reset) is now its own private function
that wire() calls in the same order, with the section's own comment kept
as its doc comment. on_budget_changed and on_open are coerced to trait
objects at the top of wire() so the new functions take a plain
Rc<dyn Fn> rather than needing their own generic parameter, with no
change in the closures registered or the order they are registered in.
2026-09-20 17:24:28 +02:00
dtourolle 5b4ad11853 Manual: nested collections, and the ghost drawn as it should be
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 42s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 38s
Build and test / Android (aarch64) (push) Failing after 2m20s
Build and test / Windows (x86_64, cross) (push) Failing after 3m3s
A scene that makes a parent, nests two collections in it by drag and by
the menu, files frames into a child and opens the parent to see it count
both; stills of the tree and of the menu. The collections recording is
re-made now that the bitmap under the cursor is the photograph.

drive.py grows a multi-leg drag: a diagonal with much vertical in it is
taken by the grid's Flickable as a scroll before the DragArea can claim
it, so a drag to the sidebar goes sideways first.
2026-09-20 16:27:18 +02:00
dtourolle 6b1aac477d Put the developer docs under docs/dev and index the folder for users first
docs/ had 26 developer documents flat beside the manual, and the two
audiences are very differently sized: most readers want the manual and
the gesture reference, a few want the register, the designs and the
measurements. The manual and gestures.md stay at the top; everything for
someone changing the code moves to docs/dev/, and the two documents that
name their own successors — the v0.1 milestone and the UI-refinement plan
— go to docs/dev/archive/ rather than being deleted, since both are still
cited. docs/README.md is the index, users first.

Every reference follows: code comments, Cargo manifests, the workflows,
the pre-commit hook, the bench and traceability tools (which locate the
repo root by docs/dev/requirements.md now), packaging, the Docker READMEs,
CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level
deeper and is regenerated. Links out of the moved documents into the tree
gain a level; a link checker over every Markdown file finds none broken.
2026-09-20 16:20:15 +02:00
dtourolle 2afc2a7890 Report the grid's column count on creation, not only on change
`changed columns` fires on a change, and a first evaluation is not one:
a grid built after the window had settled at its size never said how
wide it was, so Rust placed month headings for the one column it was
told about at start-up — every month began a row, and was announced
wherever its first cell fell, mid-row included. Opening a collection
showed "October 2025" stranded over a row of August.
2026-09-20 15:58:44 +02:00
dtourolle 6507593715 Hand the drag ghost to the renderer through a file, so it draws
The bitmap under the cursor was a solid red rectangle. Slint's drag
overlay uploads the image as a texture, draws it and drops the texture in
one call; with the wgpu FemtoVG renderer the drop is immediate and the
draw is deferred to the flush, so the frame binds femtovg's placeholder —
which is red. An image with a cache key survives in the texture cache
until after the flush, and only a path gives one. So the composite goes
to the data directory's scratch as a PNG and comes back through
load_from_path; one file per drag, removed when the drag ends. A
workaround for Slint 1.17.1, written up as one beside the code.
2026-09-20 15:58:44 +02:00
dtourolle 08727cff5a Release 0.13.5
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 45s
Build and test / Layer separation (push) Successful in 25s
Traceability / Requirement traces (push) Failing after 38s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m20s
Build and test / Windows (x86_64, cross) (push) Failing after 3m1s
2026-09-20 15:24:47 +02:00
dtourolle f4c3f425dd Show the picker correcting something in the manual
The recording sampled a red brick wall and moved the sliders by three
units, which at GIF size is a click that does nothing. The scene now
drags the frame cold first and picks a white air conditioner, so the
correction is visible and the picker's being absolute - set from the
photograph, not from where the sliders were - is what the picture
shows. The text says so, and says a blown highlight is refused.
2026-09-20 15:24:27 +02:00
dtourolle 12f8990e09 Average a patch under the white balance picker, not one photosite
The probe's comment said a 192px render "averages a small neighbourhood
into each of its pixels". It does not: the composed shader fetches the
source at one position per output pixel - nearest for an unrotated
frame, four photosites blended otherwise - so the probe was a point
sample of a noisy sensor, and two painted-white air conditioners on the
same wall answered +37 and -50.

The tap is now narrowed to the patch of the canvas around the click, a
couple of percent of its width and square on screen, and rendered at
64x64 with interpolation forced on, which puts a sample on every sensor
pixel under it at any ordinary zoom. The samples are averaged, with the
void and clipped ones left out rather than allowed to pull the mean, and
fewer than half surviving is refused. compose_camera_probe takes the
patch; the merge's compose_camera_linear keeps its nearest sampling. The
readback shrinks from six megabytes to sixty-four kilobytes.

A frame of alternating warm and cool columns, averaging neutral, moves
the controls by at most two units; a point sample swung them to sixty.
2026-09-20 15:24:25 +02:00
dtourolle 8a897bbc01 Release 0.13.4
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m22s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 29s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m22s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
2026-09-20 14:03:58 +02:00
dtourolle a03e082fe2 Point the manual's white balance scene at "pick", and record it again
The scene clicked 40px to the right of "pick", on "reset", so the
recording showed a neutral group being reset and a click on the wall
that panned. Re-recorded with the picker fixed: the word lights, the
sample moves temperature and tint, and Before shows what it corrected.
2026-09-20 13:41:17 +02:00
dtourolle 4576499c3b Refuse a clipped highlight as a neutral
Sampling the overcast sky on a Canon 6D frame set tint to -100 and
temperature to -15 for a patch the canvas showed as pure white. A clipped
photosite is sensor white, not a colour: every channel stopped counting,
so what the tap hands back is the as-shot multipliers themselves, which
are strongly magenta, and the solver dutifully drove green to its stop.
The display shader already fades such a pixel to a neutral of the same
brightness before any operation runs, so the picker was balancing against
something the photographer could not see.

The probe now refuses a sample with any channel at or above the onset the
shader fades from, the way the solver already refuses black. The threshold
is one constant, CLIP_ONSET, formatted into the shader and read by the
probe, so the two cannot drift apart.
2026-09-20 13:41:17 +02:00
dtourolle 2f47087223 Measure the white balance probe in camera RGB, where the gains multiply
Pressing "pick" and clicking a near-neutral wall on a Canon 6D frame set
tint to -77 and turned the whole photograph green. The white balance
operation runs first in the chain, on camera RGB, before the body's base
curve and colour matrix; the probe was read off a display render after
all three, and the solve treated that sRGB triple as if the gains
multiplied it directly. On a JPEG the two spaces coincide, which is why
the existing tests passed while the picker was broken on every raw file.

The probe now reads the camera-space tap a merge stitches from, composed
under the edit's own framing so a fraction of the canvas is a fraction of
the probe, and puts the as-shot balance on itself - exactly the value the
operation's gains are about to multiply. No operations run in the tap, so
nothing has to be stripped and restored, and the display target is left
alone, so a sample that found nothing usable no longer needs a redraw.

A raw-frame test with the 6D's matrix and a typical as-shot balance
samples a warm grey and asserts the rendered pixel comes back neutral; it
fails on the previous probe.
2026-09-20 13:41:11 +02:00
dtourolle 764ad55ead Stand each person in the grouping pass by at most 100 references
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m23s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 55s
Build and test / Layer separation (push) Successful in 27s
Traceability / Requirement traces (push) Failing after 54s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Every face the user has ruled on entered the pass as an anchor, and the
scan is exhaustive by design (`dr_face::neighbours`), so a person with
750 confirmed faces cost 750 comparisons against every other face in
the library — and the cost of a library grew with how well it was
named. Most of those comparisons said nothing new: thirty frames from
one afternoon are one point of view, not thirty, and a face that
matches one of them matches the rest.

Each person now enters through at most 100 of their anchored faces
(`dr_face::references`). Eligible are those whose raw embedding is at
least 15 long — one above the gallery floor, since a reference speaks
for someone rather than merely being admitted — with an unmeasured
length admitted as it is everywhere else. From those, the set spanning
the greatest volume is chosen greedily: the longest vector first, then
at each step the face with the largest component orthogonal to the
chosen so far. That is pivoted Gram–Schmidt, and the product of the
residuals it picks is the Gram determinant, so the greedy step is the
exact greedy on the objective. A near-duplicate of a chosen face has
no residual and is passed over; the one profile shot among two hundred
frontal frames is taken early; faces inside the span of the chosen add
no volume and are not taken to fill the cap.

The faces not chosen keep their confirmations and are not touched by
the pass — they stay in the anchor map, so it never releases them —
they are simply not compared. A person none of whose faces is long
enough is still stood for, by their longest, rather than losing their
anchor and having their next face filed as a stranger. Under the cap
nothing changes: every eligible face stands, and the short ones stay
in as the probes they were.

At the reference library's 3,851 confirmations the scan shrinks by
about a fifth; at 15,000 it is a fifth of what it was.
2026-09-20 13:29:16 +02:00
dtourolle 301e6f3828 Release 0.13.3
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m21s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m8s
2026-09-20 13:01:41 +02:00
dtourolle a437363bd6 Schema V20: put the mis-spelled run markers right
The markers the previous commit stops writing are already in the
catalogs — 2 on the desktop, 429 on the tablet — and in the shards
both have exchanged. Renaming them to the faces' own id with a fresh
time is what makes the export send each image again, under an entry
newer than the empty one `held_model` would otherwise pick. Where the
old write had inserted its marker beside the right one, the wrong one
goes and the right one is refreshed for the same reason: its entry in
the shards is older than the empty one.

Images V14 left with faces and no marker are not touched. That state is
the quality pass's cue, and the fixed write marks them correctly when
it reaches them.

Checked against copies of both real catalogs: the desktop renames 2,
the tablet deletes 429, both in under 200 ms.
2026-09-20 13:00:59 +02:00
dtourolle 065872bec5 Keep a run marker under the detector that found the faces
`record_updates` — the write behind the quality, eye and crop passes —
re-marked the image as indexed under the pipeline the pass ran as, and
left the faces it had updated under the id of the detector that found
them. On a desktop set to Thorough that put `scrfd_10g+w600k_mbf` over
faces spelled `w600k_mbf`; on the tablet, `scrfd_10g_i8+w600k_mbf` over
faces it had adopted from the desktop's thorough pass.

Every reader takes the marker and the faces to agree. `marker_under`
reads the marker as the detector having examined the image, so the
upgrade repair never revisits it. The shard store keys each face by
its pipeline id, so `export_to_shards` selects an image's faces by the
marker's id, finds none, and sends an entry that says the thorough
detector looked and found nothing — over photographs with named faces
on them. The desktop's shard index holds 54 such entries beside real
faces; the tablet's eye pass over the faces it had adopted made 430
more, and both devices have exchanged them. `held_model` takes the
newest entry for an image, which is the empty one. Nothing has been
lost yet only because the two spellings of the thorough detector rank
equal and neither side adopts the other's; a third device, or either
one after a reinstall, would adopt "nothing here" for 484 images. And
the desktop's eye pass is 4,739 images from doing the same to every
face from before V14 — which are the ones that only exist on the
desktop, and would then never reach anywhere.

The marker now takes the id the faces carry; the pass's own id is used
only when it dropped the last of them and there is no detector left to
name. A stale marker under another spelling of the same embedder is
removed in the same transaction, so one embedder has one marker.
2026-09-20 13:00:58 +02:00
dtourolle 0ed38ada28 Adopt a peer's unmeasured faces instead of refusing them
The tablet showed a fraction of each person: 681 of the desktop's 3,851
confirmations, and none of Ian's 746, Catherine's 626 or my own 480.
Every face that existed on both devices agreed on who it was, and the
people rows were identical — the merge was fine. The missing 3,170
confirmations were on faces the tablet did not hold at all: the
desktop's 16,080 faces from the original detector, on 4,310 images,
detected before schema V14 kept the quality reading.

Those faces were in shards the tablet had already downloaded, in
August's export. `import_from_shards` looked at them on every sync pass
and declined each one, because a face without a quality reading was
"work this device cannot finish": adopting it would write the run
marker, and the marker was what stopped an image being looked at again.
That was true when it was written and has not been since the quality
repair existed — that pass lists its work by `f.quality IS NULL`, not by
the marker, exactly as the eye pass does, and faces without an eye
reading were already adopted on that reasoning.

The refusal had no exit. V14 had deleted the markers of every image
holding such faces so the quality pass would find them, and
`export_to_shards` walks the markers, so the desktop never re-exported
them either; the unmeasured August copies were the only ones there
would ever be. The tablet's answer was to queue all 17,727 images for a
re-detection of its own, a fetch of the whole library, while holding
the faces on disk.

Adopt them. The receiving device's quality pass measures them when it
reaches them, and the desktop's confirmations match onto them by box
overlap on the next catalog merge. The test that asserted the refusal
now asserts the adoption and that the image is still owed to the pass.
2026-09-20 12:58:41 +02:00
dtourolle 695d5ec304 Correct four claims in the README against the tree
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m26s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Eighteen declared operations, not fifteen; JPEG XL is an export format;
the grid does not filter by keyword, only the catalog's query can; and a
panorama's provenance is a sidecar beside the composite, not a history
step in it.
2026-09-20 11:57:17 +02:00
dtourolle ef1afc254d Release 0.13.2
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m36s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 34s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m26s
Build and test / Windows (x86_64, cross) (push) Failing after 3m7s
2026-09-20 11:06:54 +02:00
dtourolle 103c6e7fc0 Rewrite the README for someone arriving, not someone already here
Benchmarks / CPU and I/O (per commit) (push) Failing after 30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 53s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 41s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m25s
Build and test / Windows (x86_64, cross) (push) Failing after 3m11s
It said 0.9.0 against a 0.13.1 tree, listed focus peaking and burst
grouping as unbuilt when both have shipped, and opened with a page of
prose about the display path before saying what the application does.
Lead with what it is and a picture of it, say how to get it on each
platform and what state each channel is in, keep the honest account of
what is missing, and put the manual first in the documentation table.
2026-09-20 11:06:10 +02:00
dtourolle e9398de9c1 Ship the fine-tuned border filler: MI-GAN 512 trained on projection-shaped voids from the maintainer's library 2026-09-20 10:56:38 +02:00
dtourolle b1c99b5796 Let the fill show the model an open void: mirror depth 0 means no ring and nothing known beyond the band 2026-09-20 10:56:38 +02:00
dtourolle 3de109fbd1 Write down the catalog and screen-refresh patterns the Identity fixes exposed
A CLAUDE.md at the root, for anyone changing this code: the ways a redraw
and a catalog read came to cost half a second per click, what each fix
looked like, and how to measure the next one against a copy of a real
catalog.
2026-09-20 10:56:37 +02:00
dtourolle 388bda6af3 Count the outstanding repairs from the faces, on partial indexes
"How many images still owe a quality reading" was a correlated EXISTS per
image over `faces`, and the face row is 8 KB of embedding and crop before
the column it looks at, so each count opened every row. Six such counts
run on every open of the Identity screen and at the end of every sweep:
160 ms on the reference library.

V19 adds three partial indexes holding only the faces still owing each
pass, keyed on the image and carrying the model id the predicate reads,
and replaces `faces_image` with `(image_id, model_id)` so "does this image
hold this embedder's faces" is answered from the index too. The planner
takes a partial index when the count is driven from `faces` and ignores it
inside the EXISTS, so `Needs::Face` carries the per-face fragment and
`repairs::count` spells the query from the faces' side; the list and the
per-image check keep the EXISTS. A test holds the two spellings to the
same answer for every repair.
2026-09-20 10:56:37 +02:00
dtourolle 73845d8a77 Ask the server about a collection once per job, not once per file
Every `move_to` guaranteed its destination's parent with a `MKCOL` for
each ancestor down from the account root, and a trash folder under a
library root several levels deep meant three round trips answering
`405 Method Not Allowed` before the one `MOVE` that did anything — for
every image of a delete, on a connection built for that job.

The backend now records the collections it has confirmed exist and asks
about each once. It lives for one job, so a folder another client removes
mid-batch is the one case this misses, and the `MOVE` then reports the
`409` rather than hiding it.
2026-09-20 10:56:37 +02:00
dtourolle b4821ee1ab Filter the people rail in the query, and count the unassigned faces
`faces::people` grouped `face_person` after a LEFT JOIN over every person
and sorted the lot by name; the rail then discarded the empty, unnamed
groups a regrouping pass leaves behind — 17,000 of 19,000 rows on the
reference library. `people_in_use` filters them in the WHERE and joins
`people` to face counts aggregated first (2,000 groups), so the sort sees
only the rows that will be drawn. `count_unassigned` replaces fetching
2,400 ids to take their length. `load_people` 22 ms → 10 ms.
2026-09-20 10:56:36 +02:00
dtourolle 9d1aa5735b Confirm a group, and split one, in one transaction
`confirm_all` called `faces::confirm` per face, and `split_off` called
`reject` then `confirm` per face: each opens and commits its own
transaction, so a click on a group of several hundred was several hundred
commits. `faces::confirm_all` is two statements — clear the rejections the
confirmations override, then flip the rows — and `faces::reassign` does a
split's reject-and-confirm for every face under one commit. 16 ms → 2 ms
and 22 ms → 4 ms on the largest group.
2026-09-20 10:56:36 +02:00
dtourolle 97a854833d Reuse the face grid's decoded crops across a redraw
A confirm or a reject changes one row and redraws the whole grid, and the
redraw re-read every crop blob of the selected person (4 MB for the
largest) and decoded every one — 316 ms per click on the reference
library's 754-face person, to arrive at the pixels already on screen.

`load_faces` now takes the crops the previous load decoded, keyed by face,
and moves each into its new cell; the blob read is skipped when every face
is already in hand. `refresh` drains the old cells into it rather than
cloning them. The redraw is 2.6 ms.
2026-09-20 10:56:36 +02:00
dtourolle e0e193efb4 Do not recount face coverage on every confirm, and count it without listing
Every click on the Identity screen's face grid — confirm, reject, split,
rename, merge — redrew the whole screen, and the redraw recomputed the
coverage line. That line lists every repair's outstanding images to count
them: six scans of the images table with a correlated EXISTS over the
8 KB face rows, an ORDER BY the job's visiting order, a Target with its
path per row, and a thumbnail-index query per image with faces. On the
reference library (24k images, 19k faces) that was ~200 ms of the
~540 ms each click cost, spent computing a figure a confirm cannot change.

`refresh` now takes what changed: `Changed::Identities` re-reads the rail
and the grid and leaves the coverage line alone; `Changed::Library` — an
open, a sweep ending or stopped, the face data deleted — re-reads it too.

For the times it does run, `repairs::counts` counts instead of building
and dropping the lists, and the thumbnail store is read once
(`ThumbStore::held`) rather than probed once per image in the audit, the
outstanding list and the proxy repair.

`identity_bench` is the measurement: the reads a click performs and the
batch writes, timed against a copy of a real catalog.
2026-09-20 10:56:36 +02:00
dtourolle d790961b28 Add the manual: every feature pictured from the application itself
Benchmarks / CPU and I/O (per commit) (push) Failing after 30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 47s
Build and test / Layer separation (push) Successful in 27s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 4s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Traceability / Requirement traces (push) Failing after 38s
Build and test / Android (aarch64) (push) Failing after 2m24s
Build and test / Windows (x86_64, cross) (push) Failing after 3m21s
docs/manual/README.md is a tour for a photographer opening DarkRoom for
the first time — one picture per thing, moving where movement is the
point. tools/manual/ is how the pictures are made: drive.py puppeteers the
desktop build on a private Xvfb (launch, click, drag, type, screenshot,
record), scenes.py is each picture as a script, and record.sh runs them
all over a folder and writes the results into docs/manual/media/.

The media is in LFS, with the CI pulls excluding it as they exclude the
fixtures; a screenshot changes wholesale when the interface does.

Nothing in the pictures shows a person, by design: the demo library is
seventy urban and alpine frames, chosen from the catalog's rows that face
detection found nobody in.

The traceability matrix is regenerated here after the rebase that
brought this branch up to master.
2026-09-20 00:26:00 +02:00
dtourolle 14ac41bee0 Give the keyword sheet's field focus, and the empty trash its own words
Without focus the keys typed into the keywords sheet went to the grid
behind the scrim, and the Return meant for the keyword opened a
photograph. The naming sheet already takes focus on open; do the same.

An empty trash said "No images found — check the library folder and which
formats are ticked", which sends someone off to fix a library that is
fine.
2026-09-20 00:21:14 +02:00
dtourolle 86410def88 Title the gesture book for the whole application, not the grid 2026-09-20 00:21:14 +02:00
dtourolle df8be10c7d Stop promising to ask for an export folder
An empty device destination read "Ask each time" on the settings page,
and nothing asks: an export made with the field blank is refused with
"no export folder is set". Say what will happen.
2026-09-20 00:21:14 +02:00
dtourolle f176043632 Report a merge worker that dies rather than leaving the page on Stop
wgpu reports a device out of memory by panicking, and a twelve-frame
merge on a GPU another process is using is where that happens. The panic
unwound the worker, the sender went with it, and the page sat on "Stop"
with every control disabled and nothing to say why — the crash record on
disk was the only sign. Catch the panic and send it as a failure, and
treat a closed channel with no final event as a dead worker too.
2026-09-20 00:21:14 +02:00
dtourolle 9c2cd73337 Show a mask's tint only while masking
The eyes are per layer and outlive the mode, so a photographer coming
back finds the layers they were looking at still lit. But the tint is a
way of looking at a mask, and outside Local there is no mask being looked
at: the sky stayed red through Repair and back in Photo, a mode that had
been left leaving its overlay behind — the fault ui-navigation.md D-N1
exists to prevent.
2026-09-20 00:21:14 +02:00
dtourolle e3acbdf4a3 Wrap the falloff and edge chips so a category mask cannot widen the column
Five chips in one row declare 440px, and the develop column takes the
widest panel's request — so selecting a category mask levered the sidebar
past the window's edge, clipping the histogram, the group strip and the
subject list. The same trap ChipGrid's comment records for film formats.
2026-09-20 00:21:14 +02:00
dtourolle eddaa44cd3 Announce a month at the next row it opens, not only if it begins one
A heading was drawn only on a cell that both began a month and began a
row, so at seven columns most months were never named, and the one
heading on screen — always on the window's first cell — was wrong about
every row below it. Worse, two headings drawn on the same cell overprinted
each other. Now a row carries a heading whenever its first cell's month is
not the one last announced: a month starting mid-row is named on the next
row it opens, one row late and right about everything under it.
2026-09-20 00:21:14 +02:00
dtourolle 7fba28f7d8 Upload a snapshot of a thumbnail shard, never the live file
Every shard is in WAL mode and every put opens its own connection, so
while thumbnails are being generated on several threads — which is when
the first sync pass runs — the log is never checkpointed and the main
file holds whatever the last quiet moment left in it. For a shard created
seconds earlier that is nothing: a zero-byte file with the schema still
in the log. The sync read that file and uploaded it, and every other
device merging it failed with "no such table: thumbs" on every pass.

Copy the shard through SQLite's backup API into scratch first, which
serialises against writers and carries the log, and upload that.
2026-09-20 00:21:14 +02:00
dtourolle 0fa9003e54 Let the top level be chosen as the library root
Confirming "/" in the folder picker set an empty root, which the launch
model read as no root at all: "Open library" stayed disabled after the
question had plainly been answered, and a folder library — whose folder
is the whole library — could never be opened without first descending
into a subfolder of it. The empty string was carrying two meanings.

Record the choice as its own fact on the account (`root_chosen`, defaulted
so existing configuration loads unchanged), treat a folder endpoint as
chosen by definition, and let the launch screen say so: a folder is shown
as a LIBRARY rather than an ACCOUNT, the second question becomes an
optional "scan only a subfolder", and the library header names the folder
instead of calling it "· whole account".
2026-09-20 00:21:13 +02:00
dtourolle e750bcdb8c Date the composite at the mean of its frames' capture times
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 31m52s
Build and test / Layer separation (push) Successful in 47s
Traceability / Requirement traces (push) Failing after 55s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Benchmarks / CPU and I/O (per commit) (push) Failing after 32s
Build and test / Android (aarch64) (push) Failing after 2m37s
Build and test / Windows (x86_64, cross) (push) Failing after 3m10s
It carried the first frame's, so it sorted before the sweep it was made
from. The middle of the sweep puts it among them.
2026-09-19 22:36:09 +02:00
dtourolle 48c4b403d2 Date a DNG whose IFDs follow its pixels: read the head and the tail
The scan reads the first 256 KB of a file for its metadata. A camera
writes its IFDs at the front, so that is the whole structure; the linear
DNG a merge writes puts its first IFD after the pixels, and rawler,
given the head alone, finds no decoder in it. The composite was
catalogued without a date and sorted to the very end of the grid, after
every dated photograph — which is where a panorama merged on the tablet
went unfound.

dr-decode's own TIFF reader now reads through a head and a tail at a
known offset; trailing_ifd says where the tail starts and
metadata_split reads the two together. The scan, when the head fails
and points beyond itself, fetches from the IFD to the end — kilobytes —
and dates the file from both. Tested against the writer's own output.
2026-09-19 22:35:58 +02:00
dtourolle 9d04ff2154 Release 0.13.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 12m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 5s
Build and test / Desktop (Linux) (push) Failing after 31m25s
🐳 Windows image / Build and push (push) Successful in 4s
Build and test / windows-image (push) Successful in 4s
Build and test / Layer separation (push) Successful in 46s
Build and test / Android (aarch64) (push) Failing after 6h25m37s
Build and test / Windows (x86_64, cross) (push) Failing after 3m17s
Traceability / Requirement traces (push) Failing after 48s
2026-09-19 22:06:06 +02:00
dtourolle c6cfb2a02a Put the -1 on the greens along the chroma axis, not across it
The Malvar "R at green in R row" kernel weights the two greens two
sites away along the row at -1 and the pair up and down the column at
+1/2. The shader had the two swapped, in the comment as well as the
code, so the transcription checked against itself. Both sum to zero
and reconstruct a flat patch exactly, which is all the tests fed it.

On an edge the correction at green sites is half strength and the
false colour doubles: 0.375 against 0.19 on a grey step, and a
blue/yellow zipper around every clipped highlight at 1:1. The other
three kernels and the CFA tables were right.

A grey vertical step now runs through the pass; the transposed kernel
fails it at 0.375.
2026-09-19 22:05:39 +02:00
dtourolle b83f192847 Package release 2 of 0.13.0: the inference engine and the user runtime directory 2026-09-19 21:20:53 +02:00
dtourolle ecb648818b Search the user's own runtime directory before the system library
The reference desktop's only system ONNX Runtime is Arch's
onnxruntime-opt-cuda: 1.29, built without TensorRT and against cuDNN 8
on a cuDNN 9 machine. The probe rejects both providers correctly and
the app runs on the CPU provider, which is right and not what anyone
wants. runtime/ beside the models is now searched ahead of /usr/lib,
tools/fetch-desktop-runtime.sh fills it with the four libraries from
the current onnxruntime-gpu wheel (cuDNN 9, TensorRT 10), and the
About caption lists every rung that lost and why, not only the first.
Verified: the app selects TensorRT from that directory with no
environment variable set.
2026-09-19 21:15:55 +02:00
dtourolle 5fbf8944d7 Count the filler in the APK's bundled-model array
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m35s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 31m59s
Build and test / Layer separation (push) Successful in 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Failing after 57s
Build and test / Android (aarch64) (push) Failing after 54m6s
Build and test / Windows (x86_64, cross) (push) Failing after 1h4m55s
The unpack list gained migan-512.onnx without its length following;
nothing on the desktop compiles that crate, and the first Android build
of 0.13.0 stopped there.
2026-09-19 20:55:38 +02:00
418 changed files with 113370 additions and 34138 deletions
+9
View File
@@ -20,3 +20,12 @@
# reasoning as the models, with the opposite default: the model is not
# optional and the fixtures are.
fixtures/** filter=lfs diff=lfs merge=lfs -text
# The manual's pictures live in LFS for the same reason the models do: a
# screenshot or a GIF changes wholesale when the interface it shows changes,
# and every re-recording would otherwise stay in every clone for good. The
# desktop and benchmark legs exclude the directory, since nothing they build
# or test reads it; the Android and Windows legs fetch it, because the APK
# and the installer carry the manual (docs/manual/index.html) with its
# pictures, and their packagers refuse a pointer.
docs/manual/media/** filter=lfs diff=lfs merge=lfs -text
+4 -4
View File
@@ -1,6 +1,6 @@
name: Benchmarks
# The suite docs/requirements.md §8 has been promising since it was written:
# The suite docs/dev/requirements.md §8 has been promising since it was written:
# "an automated benchmark suite against a synthetic 50k catalog, run per-commit
# … A regression beyond stated tolerance fails the build."
#
@@ -29,7 +29,7 @@ name: Benchmarks
# every commit to establish, every time, that this runner has no GPU. It
# runs on demand (Actions → Run workflow) so that a runner that *does*
# have one can be pointed at it, and the numbers it produces belong in
# docs/frame-budget.md by hand, as they already are.
# docs/dev/frame-budget.md by hand, as they already are.
on:
push:
@@ -148,7 +148,7 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
git lfs pull --exclude="fixtures/**"
git lfs pull --exclude="fixtures/**,docs/manual/media/**"
- name: Cache cargo
uses: actions/cache@v4
@@ -183,7 +183,7 @@ jobs:
- name: Frame budget (FR-DSP-3)
run: cargo test --release -p dr-gpu --test frame_budget -- --nocapture
# The instrument behind docs/frame-budget.md. It exits non-zero with no
# The instrument behind docs/dev/frame-budget.md. It exits non-zero with no
# adapter, which is right for a tool a person runs deliberately and wrong
# for a job that usually has none — hence continue-on-error. Its table is
# in the log for whoever asked for this run; the committed numbers are
+77 -3
View File
@@ -7,6 +7,10 @@ name: Build and test
on:
push:
branches: [main, master, develop]
# A release tag builds again and publishes what it built (the `release`
# job at the end). The master push of the same commit has usually filled
# the caches, so the second run is the warm one.
tags: ['v*']
pull_request:
branches: [main, master, develop]
@@ -96,6 +100,8 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
# The manual's pictures too: the APK carries the manual, and
# assemble-apk.sh refuses a pointer where a picture should be.
git lfs pull --exclude="fixtures/**"
ls -lR models/
@@ -154,6 +160,16 @@ jobs:
- name: Build
run: cargo build --workspace --release
# Only on a release tag: the binary is 150 MB and nothing but the
# release job wants it.
- name: Upload the desktop binary
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/upload-artifact@v3
with:
name: darkroom-desktop-x86_64-linux
path: target/release/darkroom-desktop
if-no-files-found: error
- name: Disk after
if: always()
run: df -h /workspace 2>/dev/null || df -h .
@@ -213,6 +229,8 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
# The manual's pictures too: the APK carries the manual, and
# assemble-apk.sh refuses a pointer where a picture should be.
git lfs pull --exclude="fixtures/**"
ls -lR models/
@@ -322,7 +340,7 @@ jobs:
env:
CARGO_TARGET_DIR: target-android
# Absent secrets mean a debug signature, which is what a fork or a
# branch build should get. Set all three (see docs/android-signing.md)
# branch build should get. Set all three (see docs/dev/android-signing.md)
# and the same job produces a release-signed APK instead.
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
KEYSTORE_PASS: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
@@ -370,7 +388,7 @@ jobs:
# TRACES: FR-PLAT-WIN-3
# The Windows executable and its installer, cross-built from Linux
# (docs/windows.md §7). No Windows machine anywhere in this job: what it
# (docs/dev/windows.md §7). No Windows machine anywhere in this job: what it
# can prove is that the binary links, is a Windows executable with no
# MinGW runtime imports, starts under Wine, and that the installer installs
# and uninstalls under Wine. What it cannot prove — a Vulkan device, a
@@ -406,6 +424,8 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
# The manual's pictures too: the installer carries the manual, and
# package.sh refuses a pointer where a picture should be.
git lfs pull --exclude="fixtures/**"
ls -l models/face models/scene
@@ -454,7 +474,17 @@ jobs:
wine "$SETUP" /S 2>/dev/null
INST=$(echo "$HOME"/.wine/drive_c/users/*/AppData/Local/Programs/DarkRoom)
ls "$INST"
[ "$(ls "$INST/models" | wc -l)" = 7 ] || { echo "FAIL: expected 7 model files"; exit 1; }
# As many files as package.sh stages: everything but the READMEs in
# the directories it copies. A literal here went stale the first
# time a model was added.
WANT=$(find models/face models/scene models/inpaint -maxdepth 1 -type f ! -name README.md | wc -l)
GOT=$(ls "$INST/models" | wc -l)
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT model files, installed $GOT"; exit 1; }
# The manual, and every picture it shows, counted the same way.
[ -f "$INST/manual/index.html" ] || { echo "FAIL: no manual installed"; exit 1; }
WANT=$(ls docs/manual/media | wc -l)
GOT=$(ls "$INST/manual/media" | wc -l)
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT manual pictures, installed $GOT"; exit 1; }
wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \
| grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; }
wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \
@@ -514,3 +544,47 @@ jobs:
fi
done
exit $FAILED
# A v* tag becomes a Gitea Release carrying the three builds and their
# SHA256SUMS, titled and described by the tag's message. Until this job
# existed every release was made by hand, and most tags never got one.
#
# It needs all three platform jobs, so a tag whose tests fail publishes
# nothing; re-run the failed job and this one follows. The work is
# tools/publish-release.sh, which is also how a release is finished by hand.
release:
if: startsWith(github.ref, 'refs/tags/v')
needs: [desktop, android, windows]
runs-on: linux/amd64
name: Publish the release
container:
image: catthehacker/ubuntu:act-latest
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Fetch the builds
uses: actions/download-artifact@v3
with:
path: dist
# Named for the download page, with the version in each name the way
# the hand-made releases had them. The installer already carries its
# version from package.sh.
- name: Publish
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN || github.token }}
TAG: ${{ github.ref_name }}
run: |
set -e
V="${TAG#v}"
ls -lR dist
mkdir -p out
cp dist/darkroom-arm64-v8a-apk/darkroom.apk "out/darkroom-${V}-arm64-v8a.apk"
cp dist/darkroom-desktop-x86_64-linux/darkroom-desktop "out/darkroom-desktop-${V}-x86_64-linux"
chmod +x "out/darkroom-desktop-${V}-x86_64-linux"
cp dist/darkroom-windows-x86_64-setup/DarkRoom-${V}-x86_64-setup.exe out/
bash tools/publish-release.sh "$TAG" out/*
+21 -6
View File
@@ -7,7 +7,7 @@ name: Traceability
# fail its own threshold. Two rules follow, and the extractor's own tests
# enforce both:
#
# 1. Denominators are parsed from docs/requirements.md at run time.
# 1. Denominators are parsed from docs/dev/requirements.md at run time.
# 2. Coverage is |traced ∩ defined| / |defined|, never a raw traced count.
#
# This job is static analysis of source comments plus markdown parsing, so it
@@ -67,6 +67,12 @@ jobs:
# threshold: zero requirements parsed, zero files scanned, a ratio above
# 100%, or any orphan tag all fail the build. A misconfigured run must not
# report a plausible-looking 0%.
# Every picture the manual shows is made by a scene in
# tools/manual/scenes.py, and every picture a scene makes is shown.
# Two files read; no app, no display.
- name: Manual pictures have scenes
run: tools/manual/record.sh --check
- name: Traceability gate
run: cargo run -q -p traceability -- check
@@ -74,11 +80,11 @@ jobs:
run: |
set -e
cargo run -q -p traceability -- report
if ! git diff --quiet docs/traceability.md; then
if ! git diff --quiet docs/dev/traceability.md; then
echo ""
echo "docs/traceability.md is out of date."
echo "docs/dev/traceability.md is out of date."
echo "Run: cargo run -p traceability -- report"
git diff --stat docs/traceability.md
git diff --stat docs/dev/traceability.md
exit 1
fi
@@ -91,10 +97,19 @@ jobs:
# they will conclude the application is broken rather than the page.
#
# This also fails on a malformed tag, so a typo costs a gesture its
# desktop half loudly rather than silently.
# desktop half loudly rather than silently — and on a key a Slint
# handler binds that no tag names, or a key a tag names that no handler
# binds (tools/traceability/src/keymap.rs).
- name: Regenerate the gesture vocabulary and check it is committed
run: cargo run -q -p traceability -- gestures-check
# The manual's page, which the packages carry and the help sheet links
# into. Blocking for the gesture book's reason: it is shown to the user,
# and a page that disagrees with the README is a manual describing an
# application that no longer exists.
- name: Regenerate the manual page and check it is committed
run: cargo run -q -p traceability -- manual-check
# Advisory, not blocking: not every file implements a requirement, and a
# tag on every function is noise that rots faster than it helps. Tag the
# unit that decides.
@@ -127,4 +142,4 @@ jobs:
- name: Summary
if: always()
run: head -30 docs/traceability.md || true
run: head -30 docs/dev/traceability.md || true
+18 -4
View File
@@ -26,7 +26,7 @@ fi
# The artefacts are generated from the tree, so regenerating them because one
# was itself edited would be circular.
case "$(tr -d '[:space:]' <<< "${staged}")" in
docs/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs)
docs/dev/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs | docs/manual/index.html)
exit 0
;;
esac
@@ -41,9 +41,9 @@ if ! cargo run -q -p traceability -- report >/dev/null 2>&1; then
exit 0
fi
if ! git diff --quiet -- docs/traceability.md; then
git add docs/traceability.md
echo "pre-commit: regenerated docs/traceability.md and staged it"
if ! git diff --quiet -- docs/dev/traceability.md; then
git add docs/dev/traceability.md
echo "pre-commit: regenerated docs/dev/traceability.md and staged it"
fi
# The gesture vocabulary, same discipline.
@@ -65,3 +65,17 @@ for f in docs/gestures.md ui/dr-ui/src/gesture_book.rs; do
echo "pre-commit: regenerated ${f} and staged it"
fi
done
# The manual's page, when its source is part of the commit. Rendered from
# nothing but the README, so there is no reason to pay for it otherwise.
if grep -qx 'docs/manual/README.md' <<< "${staged}"; then
if ! out="$(cargo run -q -p traceability -- manual 2>&1)"; then
echo "pre-commit: the manual would not render" >&2
echo "${out}" >&2
exit 1
fi
if ! git diff --quiet -- docs/manual/index.html; then
git add docs/manual/index.html
echo "pre-commit: regenerated docs/manual/index.html and staged it"
fi
fi
+132
View File
@@ -0,0 +1,132 @@
# Working in this repository
Notes for anyone — person or agent — changing this code. They record what
went wrong once and what the fix looked like, so the same shape is not
written again. Requirements live in `docs/dev/requirements.md`; this file is
about habits, not features.
## Catalog reads: work is proportional to what changed, never to library size
`docs/dev/catalog.md §1` states the rule. These are the ways it was broken on
the Identity screen, found when every confirm click cost half a second on a
24k-image library (2026-09-19), and what each fix looked like.
**A redraw must know what changed.** A click handler that calls "refresh
everything" pays for everything. `identity_ui::refresh` takes a `Changed`:
a confirm re-reads the rail and the grid and *not* the coverage line,
because moving a face between people cannot alter how many images are
indexed. Before adding a read to a shared refresh, ask which events can
change its answer, and gate it on those.
**Count with `COUNT(*)`, never with `.len()` on a list you then drop.**
`repairs::counts` used to build every repair's work list — a `Target` with
its path per row, sorted into visiting order — to report its length. Six
repairs, 350 ms, nothing kept. If the caller wants a number, the query
returns a number.
**One query, not one per row.** `ThumbStore::contains` in a filter over
5,000 rows is 5,000 prepared statements; `ThumbStore::held(size)` reads the
index once into a set. The same applies to any `query_row` inside a loop
over a result set — including `deep_count` per sidebar row, which is fine
at sidebar scale and would not be at grid scale. Aggregate in one
statement and look up in memory.
**Filter and aggregate in SQL, and aggregate the small side first.**
`faces::people` read 19,000 rows, grouped, sorted them by name, and the
screen threw 17,000 away (empty unnamed groups). `people_in_use` filters in
the `WHERE`, and joins `people` to a pre-aggregated `face_person` (2,000
groups) rather than grouping after a `LEFT JOIN` over every person. The
sort then sees only the rows that will be drawn.
**Wide rows make "just check one column" a table scan.** A `faces` row is
~8 KB (a 1 KB embedding and a ~5 KB crop, then the columns added later).
Any predicate that reads `quality`, `crop` or an eye column for every face
reads every row. V17 learned this for the eye filter; V19 applies it to the
repair counts with partial indexes (`faces_owed_*`) that hold only the rows
still owing, keyed on what the predicate joins on and carrying `model_id`
because the predicate reads it. Two things to know about them:
- **Drive the count from the small side.** SQLite uses a partial index
when the query starts from `faces` (`repairs::count`, `Needs::Face`) and
ignores it inside a correlated `EXISTS (... WHERE f.image_id = i.id ...)`.
That is why `Needs::Face` carries the per-face fragment and spells it two
ways.
- **Spell the predicate as the index's `WHERE` is spelled.** `NEEDS_EYES`
is `(f.eye_right IS NULL OR f.landmarks_dense IS NULL)` because
`faces_owed_eyes` is `WHERE eye_right IS NULL OR landmarks_dense IS NULL`.
Change one, change both, and `counts_are_the_sizes_of_the_lists` will
tell you if they drift.
Check a query's plan with `EXPLAIN QUERY PLAN` against a copy of a real
catalog before trusting an index exists for it: "SEARCH ... USING COVERING
INDEX" is the answer you want, "SEARCH f USING INDEX faces_image" on a wide
table means every probe opens a row.
## Catalog writes: one transaction per user action
`faces::confirm` opens a transaction. Calling it in a loop over a group is
a commit per face; `faces::confirm_all` is two statements and one commit,
`faces::reassign` one transaction for a whole split. When a UI action
touches N rows, give the catalog a function that takes the N, not a loop
that calls the one-row function N times — `unchecked_transaction` cannot
nest, so this has to be designed in at the catalog layer, not wrapped
from above.
## Screens: keep what is already decoded
`identity::load_faces` takes the crops the grid is currently showing and
hands them back into the new cells. Before that, a click re-read 4 MB of
crop blobs and decoded 700 JPEGs to produce the pixels already on screen.
When a redraw replaces a model, the expensive parts of the old model — a
decoded image, a cut portrait — are the first thing to reuse; only the row
that changed needs new work. Drain the old cells rather than cloning them.
## Remote calls: one round trip per file, not one per ancestor
`NextcloudBackend::move_to` guaranteed its destination's parent with a
`MKCOL` per ancestor from the account root, on every file of a batch —
three `405`s before each `MOVE`. The backend now remembers the collections
it has confirmed (`known_dirs`) for its lifetime, which is one job. When a
per-file operation has a per-batch precondition, satisfy it once.
## Providers: read the runtime's source for the version on disk, not the binding
Two things the MIGraphX rung (2026-09-20) got wrong before it was measured
right, both because `ort`'s builder was trusted to mean what its method
names say.
**A binding's option builder may fill a struct the runtime no longer
reads.** `ep::MIGraphX::with_save_model` sets fields of the legacy
`OrtMIGraphXProviderOptions`; ONNX Runtime 1.29 reads that struct for the
precision flags and ignores the rest, so every session compiled for 40 s
and the cache directory went nowhere. The option that works
(`migraphx_model_cache_dir`) exists only in the generic key/value
registration, which `session::migraphx` calls on the API table directly.
Before wiring a provider option, fetch the provider's source at the
runtime's exact version and find where the option is *read*.
**A provider's cache key may leave out what you are varying.** MIGraphX
keys a compiled program on graph, GPU and its own version — not precision.
The first fp16 measurement built in 0.3 s and matched f32 to the tenth of a
millisecond, because it had loaded the f32 program. A "from cache" build
that is suspiciously fast on the first run of a new configuration is a key
collision, not a fast provider; give each precision its own directory (the
engine does) and check the cache directory gained a file.
## Measuring
`cargo run --release -p dr-ui --example identity_bench -- CATALOG THUMBS`
times what one click on the Identity screen reads and what the batch
operations write. Run it against a **copy** of a real catalog (it writes),
never the library's own file; `sqlite3 catalog.sqlite ".backup copy.sqlite"`
takes a consistent one while the app runs. Compare the `cpu` column when
other builds are running on the machine — the wall clock doubles under
load, the CPU figure does not. Keep the binary from before the change and
run both back to back rather than trusting numbers taken an hour apart.
Reference figures from the 2026-09-19 fixes, largest person (754 faces),
24k images, 19k faces, before → after. What one click read: `load_people`
22 ms → 12 ms, `load_faces` 316 ms → 2.4 ms, `audit` 190 ms → not run
(66 ms when it is, on open and at the end of a sweep). What one click
wrote: `confirm_all` 16 ms → 2 ms, `split_off` 23 ms → 4.5 ms. A click on
the face grid went from ~530 ms of catalog work to ~15 ms.
+11 -11
View File
@@ -90,18 +90,18 @@ break it by accident:
cargo run --release -p dr-bench -- check
```
That is the benchmark suite (`docs/requirements.md` §8), which builds a
That is the benchmark suite (`docs/dev/requirements.md` §8), which builds a
synthetic 50,000-image catalog and fails the build if a performance target is
missed or a measurement has drifted past its tolerance. It runs on every push in
its own workflow. [`docs/benchmarks.md`](docs/benchmarks.md) says what it
its own workflow. [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) says what it
measures, what it deliberately does not, and how to read a failure. If you have
touched the catalog, the decoder, the thumbnail store or the exporter, run it
before you send.
## Requirements and traceability
[`requirements.md`](docs/requirements.md) is the register of record.
[`traceability.md`](docs/traceability.md) is generated from `TRACES:` tags in
[`requirements.md`](docs/dev/requirements.md) is the register of record.
[`traceability.md`](docs/dev/traceability.md) is generated from `TRACES:` tags in
the source and must never be hand-edited:
```rust
@@ -124,7 +124,7 @@ Note that it tracks line numbers, so a change that only moves code still moves
the matrix. Never regenerate it with a stale prebuilt binary.
**One convention that the tooling cannot enforce.** A tag proves that a tag
exists, not that the code under it does the thing — `docs/code-health.md`
exists, not that the code under it does the thing — `docs/dev/code-health.md`
CH-4 has the details, and two requirements currently read as covered on the
strength of plumbing a future feature would use. So: **close a requirement
with a test that would fail if the behaviour were removed.** Coverage that
@@ -163,12 +163,12 @@ One commit per change. If you fixed two things, that is two commits.
| Document | Read it when |
|---|---|
| [`core/dr-pipeline/ops/README.md`](core/dr-pipeline/ops/README.md) | Adding or changing a develop operation — start here regardless |
| [`docs/architecture.md`](docs/architecture.md) | Anything touching the render path, catalog or sync |
| [`docs/code-health.md`](docs/code-health.md) | Deciding what to work on; grades each seam by what it costs |
| [`docs/benchmarks.md`](docs/benchmarks.md) | A change that could plausibly cost time or memory |
| [`docs/technical-debt.md`](docs/technical-debt.md) | Something looks wrong — check it was not chosen |
| [`docs/distribution.md`](docs/distribution.md) | Packaging a build, or adding a permission to one |
| [`docs/requirements.md`](docs/requirements.md) | Reference, not reading |
| [`docs/dev/architecture.md`](docs/dev/architecture.md) | Anything touching the render path, catalog or sync |
| [`docs/dev/code-health.md`](docs/dev/code-health.md) | Deciding what to work on; grades each seam by what it costs |
| [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) | A change that could plausibly cost time or memory |
| [`docs/dev/technical-debt.md`](docs/dev/technical-debt.md) | Something looks wrong — check it was not chosen |
| [`docs/dev/distribution.md`](docs/dev/distribution.md) | Packaging a build, or adding a permission to one |
| [`docs/dev/requirements.md`](docs/dev/requirements.md) | Reference, not reading |
`technical-debt.md` is the one to check before "fixing" anything surprising.
It records compromises that were deliberate, each with the reasoning and a
Generated
+42 -29
View File
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "darkroom-android"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"android_logger",
"dr-plat",
@@ -1234,7 +1234,7 @@ dependencies = [
[[package]]
name = "darkroom-desktop"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"anyhow",
"dr-plat",
@@ -1408,7 +1408,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]]
name = "dr-bench"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"anyhow",
"dr-catalog",
@@ -1425,7 +1425,7 @@ dependencies = [
[[package]]
name = "dr-catalog"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-face",
"dr-plat",
@@ -1440,7 +1440,7 @@ dependencies = [
[[package]]
name = "dr-decode"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-types",
"env_logger",
@@ -1454,7 +1454,7 @@ dependencies = [
[[package]]
name = "dr-export"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-decode",
"dr-gpu",
@@ -1473,7 +1473,7 @@ dependencies = [
[[package]]
name = "dr-face"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]]
name = "dr-film"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"log",
"serde",
@@ -1495,7 +1495,7 @@ dependencies = [
[[package]]
name = "dr-gpu"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"bytemuck",
"dr-decode",
@@ -1513,8 +1513,9 @@ dependencies = [
[[package]]
name = "dr-inference-engine"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"env_logger",
"libloading",
"log",
"ort",
@@ -1527,7 +1528,7 @@ dependencies = [
[[package]]
name = "dr-ingest"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-plat",
"dr-types",
@@ -1539,7 +1540,7 @@ dependencies = [
[[package]]
name = "dr-lens"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"lensfun",
"log",
@@ -1547,7 +1548,7 @@ dependencies = [
[[package]]
name = "dr-pano"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-decode",
"dr-inference-engine",
@@ -1561,7 +1562,7 @@ dependencies = [
[[package]]
name = "dr-pipeline"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-types",
"log",
@@ -1570,7 +1571,7 @@ dependencies = [
[[package]]
name = "dr-plat"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"android-native-keyring-store",
"dr-types",
@@ -1586,7 +1587,7 @@ dependencies = [
[[package]]
name = "dr-preset-xmp"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-pipeline",
"log",
@@ -1596,7 +1597,7 @@ dependencies = [
[[package]]
name = "dr-segment"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1609,7 +1610,7 @@ dependencies = [
[[package]]
name = "dr-sync"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"async-trait",
"dr-plat",
@@ -1623,7 +1624,7 @@ dependencies = [
[[package]]
name = "dr-sync-folder"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"async-trait",
"dr-sync",
@@ -1635,7 +1636,7 @@ dependencies = [
[[package]]
name = "dr-sync-nextcloud"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"async-trait",
"dr-decode",
@@ -1657,7 +1658,7 @@ dependencies = [
[[package]]
name = "dr-thumbs"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-types",
"jpeg-encoder",
@@ -1669,7 +1670,7 @@ dependencies = [
[[package]]
name = "dr-types"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"serde",
"serde_json",
@@ -1678,7 +1679,7 @@ dependencies = [
[[package]]
name = "dr-ui"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"anyhow",
"async-trait",
@@ -1703,9 +1704,11 @@ dependencies = [
"dr-types",
"dr-xmp",
"env_logger",
"i-slint-backend-testing",
"jni 0.22.4",
"log",
"ndk-context",
"png",
"pollster",
"reqwest",
"rusqlite",
@@ -1715,12 +1718,13 @@ dependencies = [
"slint-build",
"thiserror 2.0.20",
"tokio",
"url",
"wgpu",
]
[[package]]
name = "dr-xmp"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"dr-types",
"log",
@@ -2778,6 +2782,18 @@ dependencies = [
"i-slint-renderer-skia",
]
[[package]]
name = "i-slint-backend-testing"
version = "1.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521e901e3d47ab829c0ef500c63155776208707cd93259e6a7803ed627fa2786"
dependencies = [
"cfg_aliases",
"i-slint-common",
"i-slint-core",
"vtable",
]
[[package]]
name = "i-slint-backend-winit"
version = "1.17.1"
@@ -2958,8 +2974,6 @@ dependencies = [
[[package]]
name = "i-slint-renderer-skia"
version = "1.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b6eed7f3f0a9a3d3ca6e8b9d4ca233371d989351fdb2a7ab88ec368b99e7b57"
dependencies = [
"ash",
"bytemuck",
@@ -7021,9 +7035,10 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "traceability"
version = "0.13.0"
version = "0.15.0"
dependencies = [
"anyhow",
"pulldown-cmark",
"serde",
"serde_json",
]
@@ -7921,8 +7936,6 @@ dependencies = [
[[package]]
name = "wgpu-hal"
version = "29.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "97ace1c17727311c22a46e4e3faf56ea6de81af99dcc839bdfb54857b94d448d"
dependencies = [
"android_system_properties",
"arrayvec",
+18 -2
View File
@@ -27,9 +27,12 @@ members = [
"tools/bench",
"tools/traceability",
]
# Patched copies of upstream crates, not our code: see third_party/README.md.
# Excluded so `--workspace` does not test, lint or format them as ours.
exclude = ["third_party"]
[workspace.package]
version = "0.13.0"
version = "0.15.0"
edition = "2021"
rust-version = "1.92"
license = "GPL-3.0-or-later"
@@ -48,7 +51,7 @@ dr-export = { path = "core/dr-export" }
dr-face = { path = "core/dr-face", default-features = false }
dr-film = { path = "core/dr-film" }
# `tract` on by default so a test binary can open a session with nothing
# installed; the apps add `native` to look for a runtime file (docs/inference.md §3).
# installed; the apps add `native` to look for a runtime file (docs/dev/inference.md §3).
dr-inference-engine = { path = "core/dr-inference-engine" }
dr-ingest = { path = "core/dr-ingest" }
dr-gpu = { path = "core/dr-gpu" }
@@ -127,6 +130,10 @@ url = "2.5"
async-trait = "0.1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
# The manual's HTML rendering (tools/traceability). Already in the tree as
# Slint's Markdown parser, so this adds a dependency edge and no crate; only
# the HTML writer is needed, not the command-line front end.
pulldown-cmark = { version = "0.13", default-features = false, features = ["html"] }
base64 = "0.23"
# Display-server clients, for FR-DSP-8's per-display profile acquisition.
@@ -262,3 +269,12 @@ opt-level = 0
[profile.release]
lto = "thin"
codegen-units = 1
# Two upstream crates carry a local patch so that the Android build can draw
# with wgpu on a rotated display (technical-debt.md TD-1). Both are exact
# copies of the version the lockfile already resolves, plus that patch;
# third_party/README.md says what was changed and how to carry it forward
# when Slint or wgpu moves.
[patch.crates-io]
wgpu-hal = { path = "third_party/wgpu-hal-29.0.4" }
i-slint-renderer-skia = { path = "third_party/i-slint-renderer-skia-1.17.1" }
+109 -59
View File
@@ -1,84 +1,134 @@
# DarkRoom
A cross-platform, non-destructive RAW photo editor for Linux and Android.
A non-destructive RAW photo editor and library for Linux and Android, with a
GPU develop pipeline, a catalog that syncs between devices, and no account,
no telemetry and no cloud of its own.
**Status:** 0.9.0, and no longer a spike. A library opens, culls, develops and
exports on both platforms, across eight tagged releases. What is *not*
built is written down rather than merely absent — see
[docs/outstanding.md](docs/outstanding.md) for the requirements that have no
implementation and why, and [docs/technical-debt.md](docs/technical-debt.md)
for the compromises that were chosen.
[![The library: seventy frames, the timeline beside them, the filter bar above](docs/manual/media/library.png)](docs/manual/README.md)
## Documentation
**[The manual](docs/manual/README.md)** shows every feature, pictured from
the application itself. This page says what it is, how to get it, and what
is still missing.
| Document | Contents |
|---|---|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
| [requirements.md](docs/requirements.md) | What the software must do — 179 numbered requirements |
| [architecture.md](docs/architecture.md) | How it is built — crates, GPU pipeline, data model, sync |
| [technical-debt.md](docs/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
| [outstanding.md](docs/outstanding.md) | What is not built, and whether that is a decision or a gap |
| [code-health.md](docs/code-health.md) | What a contribution costs, per seam, measured |
| [traceability.md](docs/traceability.md) | Generated: which requirement is claimed by which file |
| [faces.md](docs/faces.md) | Face detection and identity — the models, the licence problem, and what S14 measured |
## What it does
## Building
**A library.** Point it at a folder — on this machine, on a network mount,
or one a Nextcloud client keeps in virtual-files mode, where a placeholder
is treated as the photograph rather than as a one-byte file — or at a
Nextcloud account directly. The grid is virtualised, ordered by capture
time with a timeline beside it, and filtered by rating, flag, person and
whether the file is here. Ratings, keywords, collections and a
trash that survives a crash mid-operation. Card ingest. Bursts fold. Face
detection and identity, with the index syncing between devices.
Desktop:
**Developing.** Eighteen declared operations fused into one compute
dispatch, plus the neighbourhood work that cannot be: clarity, texture,
capture sharpening, noise reduction, lens correction, spectral film
simulation. Crop and straighten, spot repair, and local adjustments over
masks the model draws — click a subject or a category, then paint, subtract
a gradient, grow or shrink the edge. Focus peaking and a raw histogram for
judging what is recoverable. Named presets; XMP sidecars other editors read.
[![Segmenting an urban scene and choosing the sky as a mask](docs/manual/media/local-segment.png)](docs/manual/README.md#local-adjustments)
**Panoramas.** Select the frames, align, choose a projection, fill the
ragged border rather than crop it, and the composite lands beside its
sources as a DNG, with a sidecar recording what it was merged from.
[![Twelve hand-held frames aligned on a cylinder](docs/manual/media/panorama-aligned.png)](docs/manual/README.md#merging-a-panorama)
**Export.** JPEG, PNG, AVIF, JPEG XL, 8- and 16-bit TIFF, with resize, output
sharpening, a naming template and a colour space — to a folder here or back
into the library.
**On both platforms.** The same core runs on a desktop and a 12-inch
tablet; the interface is one layout, tuned for a wide viewport with touch
targets throughout. On desktop the develop view draws the compute pass's
texture directly — no readback between the GPU and the screen.
## Getting it
| Platform | How | State |
|---|---|---|
| Arch Linux | [`packaging/PKGBUILD`](packaging/PKGBUILD) — `makepkg -si` | Built from every release |
| Android | The APK from each CI run, or `./docker/android/package.sh --install` | Runs on a tablet; F-Droid not yet submitted |
| Windows | `DarkRoom-<version>-x86_64-setup.exe`, cross-built by CI ([windows.md](docs/dev/windows.md)) | Verified under Wine only; unsigned |
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; choosing a library does not yet work in the sandbox |
Or build it. Git LFS is required for the model weights, and the toolchain
pins itself to 1.92.0:
```bash
cargo run -p darkroom-desktop
git lfs install && git lfs pull
cargo run --release -p darkroom-desktop
```
Android (containerised toolchain, see [docker/android](docker/android/README.md)):
Android, through the containerised toolchain ([docker/android](docker/android/README.md)):
```bash
./docker/android/build.sh cargo ndk -t arm64-v8a build --release
```
Git LFS is required for the model weights, and the toolchain pins itself.
[CONTRIBUTING.md](CONTRIBUTING.md) has the details and the four commands CI
will run against what you send.
[CONTRIBUTING.md](CONTRIBUTING.md) has the system packages, the four
commands CI runs against what you send, and the shortest useful
contribution — a develop operation is one YAML file, and it arrives with its
controls, its place in the chain and its tests.
## Current state
## Where it stands
**Working.** A catalog over a local folder, a Nextcloud account, or a folder a
sync client keeps in virtual-files mode — where a placeholder is treated as the
photograph rather than as a one-byte file. A virtualised library grid with a
capture-time timeline, ratings, labels, keywords, collections and a trash that
survives a crash mid-operation. Card ingest. Face detection and identity, with
the index syncing between devices. A develop pipeline of fifteen declared
operations fused into a single compute dispatch, plus the neighbourhood
operations that cannot be — clarity, texture, capture sharpening, noise
reduction, lens correction, spectral film simulation. Crop, straighten, spot
removal, gradient and subject-segmentation masks, named presets, and a
generated panel that no operation in `ui/` is allowed to name. Export to JPEG,
PNG and 8- or 16-bit TIFF with resize and output sharpening.
**0.15.0**, twenty-three tagged releases in. 190 numbered requirements in
scope, 84% of them claimed by code and [traced to it](docs/dev/traceability.md);
the rest are written down rather than merely absent.
**The zero-copy display path works on desktop.** The compute pass writes a
texture that Slint composites directly, which is what
[ARCH §6.1](docs/architecture.md) requires; the readback it forbids costs 96%
of frame time at 4K, and
**Not built:** plugins (post-v1, [D12](docs/dev/requirements.md)), compare and
survey culling, AI denoise, tiled rendering, HDR merge and
focus stacking, most of the Android platform integration beyond running,
and the Flatpak's library chooser. The performance targets are half
verified: the per-commit benchmark suite §8 requires exists for everything
that does not need a frame — the catalog, the scan, the thumbnails — and
not yet for the render path, so a regression there fails nothing.
[outstanding.md](docs/dev/outstanding.md) is the list, with the reasoning for
each.
```bash
cargo run -p dr-gpu --example bench --features readback
```
**The one deliberate compromise worth knowing about before reading
anything else:** the Android develop view reads its frame back through the
CPU, because zero-copy there needs wgpu's Vulkan swapchain and that tears a
portrait window on a tablet whose panel is mounted landscape. It is debt,
not a revision of the rule — [technical-debt.md TD-1](docs/dev/technical-debt.md)
has the measurements and the three things any one of which would remove it.
still reproduces that measurement. **The one exception is the Android develop
view**, which reads the frame back through the CPU because zero-copy there
needs wgpu's Vulkan swapchain, and that tears a portrait window on a tablet
whose panel is mounted landscape. It is debt, not a revision of the rule: the
reasoning, the on-device measurements that forced it, and the three separate
things any one of which would remove it are in
[technical-debt.md TD-1](docs/technical-debt.md).
## Documentation
**Not built.** Plugins, compare and survey culling, focus peaking, burst
grouping, AI denoise, tiled and progressive rendering, and most of the Android
platform integration beyond running. The performance targets in §4.1 are
unverified rather than unmet — the per-commit benchmark suite §8 requires does
not exist, so nothing fails a build on a regression.
[docs/outstanding.md](docs/outstanding.md) is the list, with the reasoning.
[docs/README.md](docs/README.md) is the index. The short version, for someone using it:
| | |
|---|---|
| [manual](docs/manual/README.md) | Every feature, pictured |
| [gestures.md](docs/gestures.md) | How it is driven — generated from the code, so it cannot describe a gesture that does not exist |
For someone changing it:
| | |
|---|---|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
| [requirements.md](docs/dev/requirements.md) | What the software must do — the numbered register, and the decisions |
| [architecture.md](docs/dev/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
| [technical-debt.md](docs/dev/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
| [outstanding.md](docs/dev/outstanding.md) | What is not built, and whether that is a decision or a gap |
| [code-health.md](docs/dev/code-health.md) | What a contribution costs, per seam, measured |
| [traceability.md](docs/dev/traceability.md) | Generated: which requirement is claimed by which file |
Designs, one per subsystem:
[segmentation](docs/dev/segmentation.md) and [mask editing](docs/dev/mask-editing.md) ·
[spot removal](docs/dev/spot-removal.md) · [panorama](docs/dev/panorama.md) ·
[faces](docs/dev/faces.md) · [inference](docs/dev/inference.md) ·
[storage and sync](docs/dev/storage.md) · [catalog](docs/dev/catalog.md) ·
[display and extension](docs/dev/display-and-extension.md) ·
[navigation](docs/dev/ui-navigation.md) · [distribution](docs/dev/distribution.md) ·
[windows](docs/dev/windows.md) · [benchmarks](docs/dev/benchmarks.md).
## Licence
GPL-3.0-or-later.
GPL-3.0-or-later. The photographs in the manual and the test fixtures are
the author's and are there to show and test this project, nothing else.
The model weights carry their own licences — [models/LICENCE.md](models/LICENCE.md).
@@ -141,6 +141,23 @@
</intent-filter>
</activity>
<!-- The manual (dr_ui::manual): a WebView over the copy the APK
carries in assets/manual. See ManualActivity.java for why it is
not the browser.
Not exported: nothing outside this app has a reason to start it,
and dr_ui starts it by class name, which needs no intent filter.
Its own task entry is not wanted either — it is a page over the
app, and Back returns to the photograph it was opened from.
configChanges so a rotation reflows the page rather than
reloading it at the top. -->
<activity
android:name="paris.tourolle.darkroom.ManualActivity"
android:exported="false"
android:label="DarkRoom manual"
android:theme="@style/ManualTheme"
android:configChanges="orientation|keyboardHidden|screenSize|screenLayout|uiMode" />
<!-- FR-PLAT-AND-6, outbound. Android has refused file:// URIs
between apps since API 24 — handing one out raises
FileUriExposedException in *this* process — so an exported JPEG
@@ -0,0 +1,105 @@
package paris.tourolle.darkroom;
import android.app.Activity;
import android.content.ActivityNotFoundException;
import android.content.Intent;
import android.net.Uri;
import android.os.Bundle;
import android.webkit.WebResourceRequest;
import android.webkit.WebSettings;
import android.webkit.WebView;
import android.webkit.WebViewClient;
/**
* The manual that ships in the APK, shown in a WebView.
*
* <h2>Why an activity of our own rather than the browser</h2>
*
* <p>The desktop hands the manual to the system browser. Android leaves no
* way to do the same: the page is an asset inside the APK, which is not a
* file; an unpacked copy in app-private storage is a file no browser may
* read; a {@code file:} URI handed to another app is refused since API 24;
* and a {@code content:} URI serves the page but leaves the browser to fetch
* every picture by a relative URL against the provider, which browsers do not
* reliably do. A WebView reads {@code file:///android_asset/} straight from
* the APK, pictures and section anchor included, and nothing is unpacked.
*
* <h2>What it is not</h2>
*
* <p>A browser. JavaScript stays off (the page has none), and a link that
* leaves the manual — the design documents are on the forge — goes to the
* user's browser rather than opening inside this view, so the only thing ever
* shown here is the page the APK carries.
*
* <p>Started by {@code dr_ui::manual} with {@code Intent.setClassName}, so the
* name here and there must agree; a test in lib.rs checks the manifest
* declares it.
*/
public final class ManualActivity extends Activity {
/** The section to open at, a heading's anchor. Absent opens the top. */
public static final String EXTRA_ANCHOR = "anchor";
private static final String PAGE = "file:///android_asset/manual/index.html";
private WebView web;
@Override
protected void onCreate(Bundle saved) {
super.onCreate(saved);
setTitle("DarkRoom manual");
web = new WebView(this);
WebSettings settings = web.getSettings();
settings.setJavaScriptEnabled(false);
// Pinch to zoom into a screenshot, which is 1600 pixels wide and drawn
// at the width of a phone.
settings.setBuiltInZoomControls(true);
settings.setDisplayZoomControls(false);
web.setWebViewClient(new WebViewClient() {
@Override
public boolean shouldOverrideUrlLoading(WebView view, WebResourceRequest request) {
Uri uri = request.getUrl();
if ("file".equals(uri.getScheme())) {
return false;
}
try {
startActivity(new Intent(Intent.ACTION_VIEW, uri));
} catch (ActivityNotFoundException e) {
// No browser on the device: the link does nothing, which
// is all it could do.
}
return true;
}
});
setContentView(web);
if (saved != null) {
web.restoreState(saved);
} else {
String anchor = getIntent().getStringExtra(EXTRA_ANCHOR);
web.loadUrl(anchor == null || anchor.isEmpty() ? PAGE : PAGE + "#" + anchor);
}
}
@Override
protected void onSaveInstanceState(Bundle out) {
super.onSaveInstanceState(out);
web.saveState(out);
}
/** Back walks back through the sections visited, then leaves. */
@Override
public void onBackPressed() {
if (web.canGoBack()) {
web.goBack();
} else {
super.onBackPressed();
}
}
@Override
protected void onDestroy() {
web.destroy();
super.onDestroy();
}
}
@@ -0,0 +1,5 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Day or night as the system is; see values/themes.xml. -->
<resources>
<style name="ManualTheme" parent="@android:style/Theme.DeviceDefault.DayNight" />
</resources>
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
The manual's theme (ManualActivity). Light below API 29, which has no
day-night theme in the platform; values-v29 follows the system from there.
The WebView takes prefers-color-scheme from whether this theme is light, and
the manual's stylesheet takes its colours from that.
-->
<resources>
<style name="ManualTheme" parent="@android:style/Theme.DeviceDefault.Light" />
</resources>
+37 -6
View File
@@ -240,7 +240,7 @@ fn android_main(app: slint::android::AndroidApp) {
///
/// **Face weights are absent from the repository by design.** The InsightFace
/// grant is research-only and incompatible with this project's licence
/// (docs/faces.md §2), so a desktop user fetches them, runs
/// (docs/dev/faces.md §2), so a desktop user fetches them, runs
/// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build
/// that carries none is the ordinary case and face indexing simply stays off.
///
@@ -321,19 +321,19 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// before it reports the tab available.
//
// Three detectors, because which one runs is a setting
// (`FaceDetector`, docs/faces.md §12.3) and a tablet has no other way to
// (`FaceDetector`, docs/dev/faces.md §12.3) and a tablet has no other way to
// obtain the one it was not shipped with. Twenty megabytes of APK for
// the choice; the embedder is the same for all three.
//
// Then the three eye-state models (docs/faces.md §17): landmarks, open
// Then the three eye-state models (docs/dev/faces.md §17): landmarks, open
// or closed, sunglasses. The app indexes without them; with them the
// eyes-open filter has something to read, and a tablet has no other way
// to get them either.
//
// The int8 forms beside the three detectors are what the Hexagon runs
// (docs/inference.md §5); the engine loads the sibling when the probe
// (docs/dev/inference.md §5); the engine loads the sibling when the probe
// chose that rung and ignores it otherwise.
const BUNDLED: [(&std::ffi::CStr, &str); 13] = [
const BUNDLED: [(&std::ffi::CStr, &str); 14] = [
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
(
c"models/scrfd_500m_640.int8.onnx",
@@ -420,7 +420,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// on a first launch they were not on disk until this line. The runtime
// is in the APK's native library directory beside `libdarkroom.so`,
// which is also where Qualcomm's DSP loader has to be pointed for the
// Hexagon skel (docs/inference.md §3, §8).
// Hexagon skel (docs/dev/inference.md §3, §8).
dr_ui::inference::init(native_library_dir().into_iter().collect());
}
@@ -581,6 +581,37 @@ mod tests {
);
}
/// `dr_ui::manual` starts the manual by class name. A name the manifest
/// does not declare is an `ActivityNotFoundException` on the device and a
/// Manual button that does nothing, so the three spellings — dr_ui's, the
/// manifest's and the Java file's — are checked to be one.
#[test]
fn the_manual_activity_dr_ui_starts_is_declared() {
let manifest = manifest();
let wanted = dr_ui::manual::ANDROID_ACTIVITY;
let element = manifest
.split("<activity")
.skip(1)
.find(|a| attribute(a, "android:name").as_deref() == Some(wanted))
.unwrap_or_else(|| panic!("the manifest declares no activity {wanted}"));
assert_eq!(
attribute(element, "android:exported").as_deref(),
Some("false"),
"the manual activity has no reason to be startable by another app"
);
let java = include_str!("../android/java/paris/tourolle/darkroom/ManualActivity.java");
let (package, class) = wanted.rsplit_once('.').expect("unqualified class name");
assert!(java.contains(&format!("package {package};")));
assert!(java.contains(&format!("class {class} ")));
assert!(
java.contains(&format!(
"EXTRA_ANCHOR = \"{}\"",
dr_ui::manual::ANDROID_EXTRA_ANCHOR
)),
"ManualActivity reads the section from a different extra than dr_ui writes"
);
}
#[test]
fn the_provider_hands_out_one_file_at_a_time_and_nothing_by_itself() {
let manifest = manifest();
+3
View File
@@ -25,3 +25,6 @@ winresource = "0.1"
[features]
default = []
# The manual's recording hook (dr-ui's `automation`); tools/manual/record.sh
# builds with it, nothing else does.
automation = ["dr-ui/automation"]
+12 -7
View File
@@ -21,7 +21,7 @@ fn main() -> anyhow::Result<()> {
// build made on a machine that cannot run the application — the Linux CI
// producing the Windows binary, checked under Wine — has an exit that
// proves the executable starts without opening a window or touching the
// user's directories (docs/windows.md §6).
// user's directories (docs/dev/windows.md §6).
if std::env::args().nth(1).as_deref() == Some("--version") {
println!("darkroom-desktop {}", env!("CARGO_PKG_VERSION"));
return Ok(());
@@ -63,7 +63,7 @@ fn main() -> anyhow::Result<()> {
// Before the window: the probe runs on its own thread and the first
// frame does not wait for it, but the models a background job asks for
// should already know where the runtime is (docs/inference.md §4).
// should already know where the runtime is (docs/dev/inference.md §4).
dr_ui::inference::init(runtime_dirs());
dr_ui::run(paths)?;
@@ -78,15 +78,19 @@ fn main() -> anyhow::Result<()> {
/// Where a desktop package may have put `libonnxruntime`, most specific
/// first. None of these existing is the tract build, which is a complete
/// application and not an error (docs/inference.md §3).
/// application and not an error (docs/dev/inference.md §3).
///
/// `DARKROOM_ORT_DIR` is for a developer pointing at a runtime that is not
/// installed — the wheel's `capi` directory, say. Then beside the executable
/// and in the package's private library directory, for a package that
/// bundles its own; then the Flatpak prefix; then the system library
/// directory, for a distribution that ships ONNX Runtime as a package of its
/// own. A system copy whose GPU providers do not load is not a problem: the
/// probe builds a real session before believing a provider.
/// bundles its own; then the user's own `runtime/` beside the models, where
/// `tools/fetch-desktop-runtime.sh` puts one; then the Flatpak prefix; then
/// the system library directory, for a distribution that ships ONNX Runtime
/// as a package of its own. The user's copy outranks the system's because
/// the system's is the one most likely to be built without the GPU
/// providers, or against the wrong cuDNN — and a system copy whose providers
/// do not load is not a problem, only a slower app: the probe builds a real
/// session before believing a provider.
fn runtime_dirs() -> Vec<PathBuf> {
let mut dirs = Vec::new();
if let Some(dir) = std::env::var_os("DARKROOM_ORT_DIR") {
@@ -98,6 +102,7 @@ fn runtime_dirs() -> Vec<PathBuf> {
dirs.push(bin.join("../lib/darkroom"));
}
}
dirs.push(dr_ui::inference::user_runtime_dir());
#[cfg(target_os = "linux")]
dirs.extend([
PathBuf::from("/app/lib/darkroom"),
+1 -1
View File
@@ -315,7 +315,7 @@ fn full_library(
// The three phases, separately, because "a regroup takes n seconds" does
// not tell anyone which half to optimise — and the answer differs between
// a desktop and a tablet (docs/faces.md §9).
// a desktop and a tablet (docs/dev/faces.md §9).
{
let dim = candidates.first().map(|c| c.embedding.len()).unwrap_or(0);
let flat: Vec<f32> = candidates
+1 -1
View File
@@ -82,7 +82,7 @@
//! Grouping has no natural `subject_id`: it is a property of a *run* of frames,
//! so a per-image job would rebuild the world once per photograph. It is
//! therefore a debounced library-level pass, for exactly the reasons
//! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
//! docs/dev/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
//! of it — one ordered walk, no per-pair comparison beyond adjacent frames.
//!
//! # Grouping is not hiding
+210 -33
View File
@@ -2,7 +2,7 @@
//! Face data as sealed shards, so a second device does not re-index the library.
//!
//! Indexing a 23,500-image library is on the order of two hours of CPU
//! (docs/faces.md §12.2). It is also **byte-identical on every device**: the
//! (docs/dev/faces.md §12.2). It is also **byte-identical on every device**: the
//! same model over the same proxy produces the same embedding. Paying for it
//! once per account rather than once per device is the whole point of this
//! module, and it is the same bargain the thumbnail store already makes.
@@ -178,6 +178,67 @@ impl FaceShardStore {
.flatten()
}
/// The other pipelines this file is held under that share `model_id`'s
/// embedder — the generations a put of `model_id` may supersede.
fn siblings(&self, file_id: u64, model_id: &str) -> Vec<String> {
let mut stmt = match self.index.prepare(&format!(
"SELECT model_id FROM entries
WHERE file_id = ?1 AND model_id != ?2 AND {} = ?3",
crate::faces::embedder_sql("model_id")
)) {
Ok(s) => s,
Err(_) => return Vec::new(),
};
stmt.query_map(
rusqlite::params![
file_id as i64,
model_id,
crate::faces::embedder_of(model_id)
],
|r| r.get::<_, String>(0),
)
.map(|rows| rows.filter_map(|r| r.ok()).collect())
.unwrap_or_default()
}
/// Whether a pass this file is already held under outranks `model_id`,
/// so a put of `model_id` would add a generation nobody would adopt.
pub fn outranked(&self, file_id: u64, model_id: &str) -> bool {
use dr_types::FaceDetector;
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
return false;
};
self.siblings(file_id, model_id)
.iter()
.filter_map(|m| FaceDetector::for_model_id(m))
.any(|held| held.outranks(incoming))
}
/// Forget the index entries for generations of this file that `model_id`
/// outranks. The bytes stay where they are — a sealed shard is
/// immutable — but the store stops offering them, and a later export or
/// merge writes nothing for them again.
fn supersede(&self, file_id: u64, model_id: &str) -> Result<(), CatalogError> {
use dr_types::FaceDetector;
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
return Ok(());
};
for held in self.siblings(file_id, model_id) {
let weaker = FaceDetector::for_model_id(&held).is_some_and(|h| incoming.outranks(h));
if weaker {
self.index.execute(
"DELETE FROM entries WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, held],
)?;
self.index.execute(
"DELETE FROM faces_meta WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, held],
)?;
}
}
Ok(())
}
pub fn contains(&self, file_id: u64, model_id: &str) -> bool {
self.index
.query_row(
@@ -233,6 +294,15 @@ impl FaceShardStore {
faces: &[SharedFace],
indexed_at: Option<i64>,
) -> Result<u32, CatalogError> {
// One generation per image per embedder. A store carried every pass
// — 24,123 entries for 19,089 images on the reference library, a
// third of its 293 MB — and only the strongest was ever adopted.
// A weaker pass arriving after a stronger one is not written; a
// stronger one arriving retires the weaker from the index.
if self.outranked(file_id, model_id) {
return Ok(0);
}
self.supersede(file_id, model_id)?;
let incoming = faces
.iter()
.map(|f| BYTES_PER_FACE + if f.crop.is_empty() { 0 } else { BYTES_PER_CROP })
@@ -474,15 +544,28 @@ impl FaceShardStore {
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX,
)?;
let mut q =
src.prepare("SELECT file_id, model_id, faces_found, source_edge FROM indexed")?;
let images: Vec<(i64, String, i64, i64)> = q
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))?
// The peer's marker travels with the image: it is what lets
// `import_from_shards` record the adoption under the time the peer
// indexed it, and so what keeps `export_to_shards` from reading the
// adoption as a re-index and sending the peer's faces back out under
// this device's name. A shard from before the column has none.
let mut q = src.prepare(&format!(
"SELECT file_id, model_id, faces_found, source_edge, {} FROM indexed",
match has_column(&src, "indexed", "indexed_at") {
Ok(true) => "indexed_at",
_ => "NULL",
}
))?;
let images: Vec<(i64, String, i64, i64, Option<i64>)> = q
.query_map([], |r| {
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
})?
.collect::<Result<_, _>>()?;
let mut adopted = 0;
for (file_id, model_id, _found, edge) in images {
if self.contains(file_id as u64, &model_id) {
for (file_id, model_id, _found, edge, indexed_at) in images {
if self.contains(file_id as u64, &model_id) || self.outranked(file_id as u64, &model_id)
{
continue;
}
let mut fq = src.prepare(&format!(
@@ -501,12 +584,27 @@ impl FaceShardStore {
let faces: Vec<SharedFace> = fq
.query_map(rusqlite::params![file_id, &model_id], read_shared_face)?
.collect::<Result<_, _>>()?;
self.put_image(file_id as u64, &model_id, edge as u32, &faces)?;
self.put_image_at(file_id as u64, &model_id, edge as u32, &faces, indexed_at)?;
adopted += 1;
}
Ok(adopted)
}
/// Record when the catalog indexed a held image, for an entry that
/// arrived without a marker — a peer's shard from before the column.
pub fn set_indexed_at(
&self,
file_id: u64,
model_id: &str,
at: i64,
) -> Result<(), CatalogError> {
self.index.execute(
"UPDATE entries SET indexed_at = ?3 WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, model_id, at],
)?;
Ok(())
}
/// Read back everything held for one image.
pub fn get_image(
&self,
@@ -798,8 +896,21 @@ pub fn import_from_shards(
})?
.collect::<Result<_, _>>()?;
/// Images per write transaction. Large enough that fourteen thousand
/// adoptions are a hundred and forty commits rather than fourteen
/// thousand; small enough that a read on the UI thread, queued behind
/// the lock, waits a fraction of a second and not the whole import.
const CHUNK: usize = 100;
let mut adopted = 0;
let mut tx = conn.unchecked_transaction()?;
let mut in_chunk = 0;
for (file_id, image_id, local) in candidates {
if in_chunk == CHUNK {
tx.commit()?;
tx = conn.unchecked_transaction()?;
in_chunk = 0;
}
let Some(held) = store.held_model(file_id as u64, model_id) else {
continue;
};
@@ -820,20 +931,22 @@ pub fn import_from_shards(
let Some((faces, edge)) = store.get_image(file_id as u64, &held)? else {
continue;
};
// A peer that embedded before the quality was kept has done work this
// device cannot finish: the number exists only at embedding time, and
// adopting the faces would write the run marker that keeps them from
// ever being measured (schema V14). Left for this device's own pass —
// or for the peer's, whose re-export replaces these.
// A face the peer embedded before its quality was kept (schema V14)
// is adopted with the reading missing, exactly as one without an eye
// reading is. The measuring passes find their work by the NULL
// column, not by the run marker (`dr_ui::repairs`, `faces_needing`),
// so adopting costs the reading nothing and this device's own pass
// fills it.
//
// A missing *eye* reading is not the same case and is adopted. The
// measuring pass finds those by the NULL, not by the marker, so
// adopting the faces costs the reading nothing (schema V16) — and a
// peer that has no eye models may be the only one that has done the
// detection at all.
if faces.iter().any(|f| f.quality.is_none()) {
continue;
}
// This used to refuse such faces, on the reasoning that the marker
// would stop them ever being measured — true before the quality
// repair existed, and wrong after. What it cost: V14 had dropped the
// markers of every image holding such faces, so the peer never
// re-exported them, and the only copies in the shards were the
// unmeasured ones. A tablet holding shards with 4,310 of the
// desktop's images and 3,170 of its confirmations declined every one
// of them, showed a fraction of each person, and queued the whole
// library for a re-detection of its own instead.
let local: Vec<crate::faces::DetectedFace> = faces
.into_iter()
.map(|f| crate::faces::DetectedFace {
@@ -856,15 +969,42 @@ pub fn import_from_shards(
})
.collect();
crate::faces::record_detections(
conn,
crate::faces::record_detections_within(
&tx,
dr_types::ImageId(image_id as u64),
&held,
edge,
&local,
)?;
// The peer's marker, not this moment. `record_detections` stamps the
// run as now, and `export_to_shards` reads a marker newer than the
// shard's as a re-index — so every adopted image went straight back
// out as this device's own work: 14,100 adopted, 15,457 "newly
// indexed" on the next pass, and twenty-two shards of a peer's faces
// uploaded again under a second name. Where the peer's shard carried
// no marker, the store takes the catalog's, so the two agree either
// way and the export sees nothing to send.
match store.indexed_at(file_id as u64, &held) {
Some(theirs) => {
tx.execute(
"UPDATE face_index SET indexed_at = ?3
WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id, held, theirs],
)?;
}
None => {
let ours: i64 = tx.query_row(
"SELECT indexed_at FROM face_index WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id, held],
|r| r.get(0),
)?;
store.set_indexed_at(file_id as u64, &held, ours)?;
}
}
adopted += 1;
in_chunk += 1;
}
tx.commit()?;
Ok(adopted)
}
@@ -1100,6 +1240,32 @@ mod tests {
assert!(!s.contains(1, "lvface"));
}
/// One generation per image per embedder: a stronger detector's pass
/// retires a weaker one from the index, and a weaker pass arriving after
/// a stronger is not written at all.
#[test]
fn a_stronger_pass_retires_a_weaker_one_and_a_weaker_is_not_added() {
let dir = tempdir();
let mut s = FaceShardStore::open(&dir).unwrap();
s.put_image(1, "w600k_mbf", 1024, &[face(1, 1)]).unwrap();
s.put_image(1, "scrfd_10g+w600k_mbf", 1024, &[face(1, 2)])
.unwrap();
assert!(s.contains(1, "scrfd_10g+w600k_mbf"));
assert!(!s.contains(1, "w600k_mbf"), "the fast pass was not retired");
assert_eq!(s.len(), 1, "faces_meta still counts the retired pass");
s.put_image(1, "scrfd_2.5g+w600k_mbf", 1024, &[face(1, 3)])
.unwrap();
assert!(
!s.contains(1, "scrfd_2.5g+w600k_mbf"),
"a weaker pass was added"
);
assert_eq!(
s.held_model(1, "w600k_mbf").as_deref(),
Some("scrfd_10g+w600k_mbf")
);
}
#[test]
fn re_storing_an_image_replaces_rather_than_doubling_it() {
let dir = tempdir();
@@ -1320,6 +1486,11 @@ mod catalog_round_trip {
assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5);
let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
assert!(emb.iter().any(|e| e.embedding[0] == 1));
// And what B adopted is not B's work: its next export sends nothing.
// Adopting used to stamp the run as now, so every adopted image went
// back out under B's name as a re-index.
assert_eq!(export_to_shards(&b, &mut store_b, "w600k_mbf").unwrap(), 0);
}
/// The desktop switched to a stronger detector part-way through the
@@ -1365,11 +1536,12 @@ mod catalog_round_trip {
);
}
/// A face a peer embedded without measuring it is work this device
/// cannot finish, and adopting it would write the marker that stops it
/// ever being measured. The image stays outstanding instead.
/// A face a peer embedded without measuring it is adopted all the same,
/// and left on this device's quality pass by its missing reading. Refusing
/// it was what stranded every confirmation the desktop had made on faces
/// from before V14: the tablet held the shards and would not use them.
#[test]
fn a_peers_unmeasured_faces_are_left_for_this_device_to_index() {
fn a_peers_unmeasured_faces_are_adopted_and_left_for_the_quality_pass() {
let b = device(&[(90, 5001), (91, 5002)]);
let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap();
let shared = |file_id: u64, quality: Option<f32>| SharedFace {
@@ -1395,13 +1567,18 @@ mod catalog_round_trip {
.put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))])
.unwrap();
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 1);
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 2);
let cov = faces::coverage(&b, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1);
assert_eq!(cov.outstanding(), 1, "the unmeasured image was adopted");
assert!(faces::for_image(&b, dr_types::ImageId(90))
.unwrap()
.is_empty());
assert_eq!(cov.indexed, 2);
assert_eq!(cov.outstanding(), 0, "the unmeasured image was refused");
let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap();
assert_eq!(got.len(), 1);
assert_eq!(got[0].quality, None, "a reading was invented");
// Still owed to the measuring pass, which lists by the column.
assert_eq!(
faces::count_needing(&b, "w600k_mbf", "f.quality IS NULL").unwrap(),
1
);
}
#[test]
+252 -19
View File
@@ -1,7 +1,7 @@
//! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
//! People and faces: what was detected, who it is, and who said so.
//!
//! The storage half of docs/faces.md. `dr-face` finds faces and turns them into
//! The storage half of docs/dev/faces.md. `dr-face` finds faces and turns them into
//! 512 numbers; this module is where those numbers acquire an identity, and
//! where the user's corrections outrank the model's guesses.
//!
@@ -110,7 +110,7 @@ pub struct DetectedFace {
/// Raw rather than unit length, so the length ([`Self::quality`]) is in
/// the blob and not only beside it. Readers re-normalise on load.
pub embedding: Vec<u8>,
/// Source pixels across the aligned crop (docs/faces.md §7).
/// Source pixels across the aligned crop (docs/dev/faces.md §7).
pub crop_px: f32,
/// Length of the raw embedding before normalisation — the model's own
/// reading of how recognisable the crop was, and the gate on whether
@@ -211,7 +211,7 @@ pub use dr_face::Calibration;
/// the same face in the same photograph, for carrying an identity across a
/// re-detection.
///
/// Set at the reference library's P≈0.95 line (docs/faces.md §9's table:
/// Set at the reference library's P≈0.95 line (docs/dev/faces.md §9's table:
/// 0.449), which is far above anything two different people in one frame
/// reach and below what one face re-embedded from a better crop of itself
/// does. The number is only ever asked about *overlapping* boxes on *one*
@@ -279,7 +279,25 @@ pub fn record_detections(
faces: &[DetectedFace],
) -> Result<Vec<FaceId>, CatalogError> {
let tx = conn.unchecked_transaction()?;
let ids = record_detections_within(&tx, image_id, model_id, source_edge, faces)?;
tx.commit()?;
Ok(ids)
}
/// [`record_detections`] inside a transaction the caller owns.
///
/// For a caller recording many images at once — the shard import adopts
/// fourteen thousand in one pass — where a commit per image is fourteen
/// thousand fsyncs and fourteen thousand turns at the write lock that every
/// read on the UI thread queues behind. `unchecked_transaction` cannot nest,
/// so the batching has to be offered here rather than wrapped from above.
pub fn record_detections_within(
tx: &Connection,
image_id: ImageId,
model_id: &str,
source_edge: u32,
faces: &[DetectedFace],
) -> Result<Vec<FaceId>, CatalogError> {
// Everything the old faces knew, so it can be carried across the
// replacement. Read only when there is something to carry it onto: a
// pass that found nothing has nothing to match, and decoding a vector
@@ -287,7 +305,7 @@ pub fn record_detections(
let prior = if faces.is_empty() {
Vec::new()
} else {
read_priors(&tx, image_id)?
read_priors(tx, image_id)?
};
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
@@ -389,7 +407,6 @@ pub fn record_detections(
],
)?;
tx.commit()?;
Ok(ids)
}
@@ -556,6 +573,19 @@ impl FaceUpdate {
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how what was written
/// here reaches the other devices.
///
/// It is re-written under the pipeline id the **faces carry**, not the one
/// this pass ran as. `model_id` names the pass only through its embedder;
/// the detector half of a marker is a statement about who drew the boxes,
/// and this pass drew none. Every reader takes the two to agree: the export
/// selects an image's faces by the marker's id, `marker_under` takes a
/// marker as proof the detector has been over the image, and the shard
/// store keys each face by it. When the marker was written as
/// `scrfd_10g+w600k_mbf` over faces still spelled `w600k_mbf`, the export
/// found no faces under it and sent the other devices an entry saying the
/// thorough detector had looked and found nothing — over photographs with
/// named faces on them. With no faces left, the pass's own id is the only
/// one there is, and the marker says so.
pub fn record_updates(
conn: &Connection,
image_id: ImageId,
@@ -602,13 +632,25 @@ pub fn record_updates(
for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
}
let remaining: i64 = tx.query_row(
let (remaining, found_by): (i64, Option<String>) = tx.query_row(
&format!(
"SELECT COUNT(*) FROM faces WHERE image_id = ?1 AND {} = ?2",
"SELECT COUNT(*), MIN(model_id) FROM faces WHERE image_id = ?1 AND {} = ?2",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, embedder_of(model_id)],
|r| r.get(0),
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let marker = found_by.as_deref().unwrap_or(model_id);
// One marker per embedder: a stale one under another spelling would
// keep saying that detector had been here, which is the claim the
// faces' own id is now making in its place.
tx.execute(
&format!(
"DELETE FROM face_index
WHERE image_id = ?1 AND model_id != ?2 AND {} = ?3",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, marker, embedder_of(model_id)],
)?;
tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
@@ -619,7 +661,7 @@ pub fn record_updates(
source_edge = excluded.source_edge",
rusqlite::params![
image_id.0 as i64,
model_id,
marker,
now_secs(),
remaining,
source_edge as i64,
@@ -838,6 +880,22 @@ pub fn unassigned(conn: &Connection, model_id: &str) -> Result<Vec<FaceId>, Cata
rows.collect::<Result<_, _>>().map_err(Into::into)
}
/// How many faces have no identity yet — [`unassigned`] counted rather than
/// listed, for a screen that only shows the number.
pub fn count_unassigned(conn: &Connection, model_id: &str) -> Result<u64, CatalogError> {
let n: i64 = conn.query_row(
&format!(
"SELECT COUNT(*) FROM faces f
LEFT JOIN face_person fp ON fp.face_id = f.id
WHERE fp.face_id IS NULL AND {} = ?1",
embedder_sql("f.model_id")
),
[embedder_of(model_id)],
|r| r.get(0),
)?;
Ok(n as u64)
}
/// One face's stored embedding, as the clustering pass consumes it.
///
/// A struct rather than a tuple because it crosses a crate boundary and "the
@@ -910,17 +968,46 @@ pub fn rename_person(conn: &Connection, person: PersonId, name: &str) -> Result<
/// Merged-away people are excluded: they exist as redirects so a sync does not
/// resurrect them, not as entries in a list.
pub fn people(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
let mut q = conn.prepare(
people_where(conn, false)
}
/// Everyone who holds a face, carries a name, or was set aside — the people a
/// screen has a row for.
///
/// The rest are the empty, unnamed groups a regrouping pass leaves behind
/// (`prune_empty_unnamed`), and on the reference library they were 17,000 of
/// 19,000 rows: read, counted, sorted by name and then thrown away by the
/// caller on every redraw. Filtered here, in the query, they are never
/// sorted. The filter is SQL's `trim`, which strips spaces and not every
/// whitespace character, so a name that is only a tab is listed rather than
/// hidden — the safe direction for a row the user typed something into.
pub fn people_in_use(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
people_where(conn, true)
}
/// [`people`], with face counts aggregated once per person *before* the join
/// rather than grouped after it: the face table is joined to the 2,000
/// people it names, not the 19,000 rows of the people table.
fn people_where(conn: &Connection, in_use: bool) -> Result<Vec<Person>, CatalogError> {
let filter = if in_use {
"AND (c.person_id IS NOT NULL OR trim(p.name) <> '' OR p.ignored)"
} else {
""
};
let mut q = conn.prepare(&format!(
"SELECT p.id, p.uuid, p.name,
COALESCE(SUM(fp.confirmed = 1), 0),
COALESCE(SUM(fp.confirmed = 0), 0),
COALESCE(c.confirmed, 0),
COALESCE(c.suggested, 0),
p.ignored
FROM people p
LEFT JOIN face_person fp ON fp.person_id = p.id
WHERE p.merged_into IS NULL
GROUP BY p.id
ORDER BY 4 DESC, 5 DESC, p.name",
)?;
LEFT JOIN (SELECT person_id,
SUM(confirmed = 1) AS confirmed,
SUM(confirmed = 0) AS suggested
FROM face_person
GROUP BY person_id) c ON c.person_id = p.id
WHERE p.merged_into IS NULL {filter}
ORDER BY 4 DESC, 5 DESC, p.name"
))?;
let rows = q.query_map([], |r| {
Ok(Person {
id: PersonId(r.get::<_, i64>(0)? as u64),
@@ -1057,6 +1144,72 @@ pub fn confirm(conn: &Connection, face: FaceId, person: PersonId) -> Result<(),
Ok(())
}
/// The user says every suggested face of this person is right.
///
/// What "Confirm all" runs, and the reason it is not a loop over [`confirm`]:
/// that is a transaction per face, and on a group of several hundred it was
/// several hundred commits for one click. Two statements, one commit, and the
/// same two rules `confirm` applies face by face — an earlier rejection of
/// the pair is overridden, and a face already confirmed is left alone.
///
/// Returns how many suggestions became confirmations.
pub fn confirm_all(conn: &Connection, person: PersonId) -> Result<u64, CatalogError> {
let tx = conn.unchecked_transaction()?;
tx.execute(
"DELETE FROM face_person_rejected
WHERE person_id = ?1
AND face_id IN (SELECT face_id FROM face_person
WHERE person_id = ?1 AND confirmed = 0)",
[person.0 as i64],
)?;
let n = tx.execute(
"UPDATE face_person SET confirmed = 1, probability = 1.0
WHERE person_id = ?1 AND confirmed = 0",
[person.0 as i64],
)?;
tx.commit()?;
Ok(n as u64)
}
/// The user says these faces are `to`, not `from`.
///
/// [`reject`] from one and [`confirm`] onto the other, for every face, in one
/// transaction — what a split commits. Rejecting first is what stops the
/// split being undone: without it the next pass sees a face that looks like
/// `from` and suggests it straight back. Confirmed rather than suggested on
/// `to`, because the user has just asserted these belong together.
pub fn reassign(
conn: &Connection,
faces: &[FaceId],
from: PersonId,
to: PersonId,
) -> Result<(), CatalogError> {
let tx = conn.unchecked_transaction()?;
{
let mut reject = tx.prepare(
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
VALUES (?1, ?2)",
)?;
let mut unrejected =
tx.prepare("DELETE FROM face_person_rejected WHERE face_id = ?1 AND person_id = ?2")?;
let mut confirm = tx.prepare(
"INSERT INTO face_person (face_id, person_id, probability, confirmed)
VALUES (?1, ?2, 1.0, 1)
ON CONFLICT(face_id) DO UPDATE SET
person_id = excluded.person_id,
probability = 1.0,
confirmed = 1",
)?;
for face in faces {
reject.execute(rusqlite::params![face.0 as i64, from.0 as i64])?;
unrejected.execute(rusqlite::params![face.0 as i64, to.0 as i64])?;
confirm.execute(rusqlite::params![face.0 as i64, to.0 as i64])?;
}
}
tx.commit()?;
Ok(())
}
/// The user says this face is **not** this person.
///
/// Stored rather than implied by removal, so the next clustering pass does not
@@ -1446,7 +1599,7 @@ fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
}
}
fn now_secs() -> i64 {
pub(crate) fn now_secs() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -1779,6 +1932,86 @@ mod tests {
assert!(at >= marked_at, "the marker was not refreshed");
}
/// The marker a per-face pass leaves names the detector that drew the
/// boxes, whatever pipeline the pass itself ran as. A marker under the
/// pass's id over faces spelled another way is one the export finds no
/// faces under — and it sent every other device "nothing here".
#[test]
fn an_update_keeps_the_marker_under_the_detector_that_found_the_faces() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(
&c,
img,
"w600k_mbf",
1024,
&[DetectedFace {
quality: None,
..face(1)
}],
)
.unwrap();
// The state V14 leaves: the faces, and no marker at all.
c.execute("DELETE FROM face_index", []).unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let markers: Vec<(String, i64)> = c
.prepare("SELECT model_id, faces_found FROM face_index")
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(markers, vec![("w600k_mbf".to_string(), 1)]);
// A marker already there under the pass's own id is replaced, not
// kept beside the right one.
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (1, 'scrfd_10g+w600k_mbf', 0, 0, 6000)",
[],
)
.unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(vec![1, 2, 3]),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let n: i64 = c
.query_row("SELECT COUNT(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 1, "a second marker survived");
// With every face dropped there is no detector left to name, and
// the pass's own id records that it looked.
record_updates(&c, img, "scrfd_10g+w600k_mbf", 6000, &[], &ids).unwrap();
let marker: (String, i64) = c
.query_row("SELECT model_id, faces_found FROM face_index", [], |r| {
Ok((r.get(0)?, r.get(1)?))
})
.unwrap();
assert_eq!(marker, ("scrfd_10g+w600k_mbf".to_string(), 0));
}
/// Re-detection is coalesced per image, so it must replace rather than
/// append — otherwise every re-index doubles the library's face count.
#[test]
@@ -2253,7 +2486,7 @@ mod tests {
}
/// The reference implementation's fitted MBF curve puts the P=0.5 boundary
/// at cosine 0.267 (docs/faces.md §1). Our own first end-to-end run scored
/// at cosine 0.267 (docs/dev/faces.md §1). Our own first end-to-end run scored
/// 0.596 between distinct photographs of one person and 0.05 between
/// different people, so those two must land either side.
#[test]
+112 -1
View File
@@ -119,6 +119,8 @@ pub struct MergeReport {
pub keywords_fused: usize,
/// Keyword assignments taken from the remote.
pub keywords_assigned: usize,
/// Images whose capture metadata was taken from the remote.
pub metadata_adopted: usize,
}
impl MergeReport {
@@ -133,6 +135,7 @@ impl MergeReport {
|| self.keywords_deleted > 0
|| self.keywords_fused > 0
|| self.keywords_assigned > 0
|| self.metadata_adopted > 0
}
/// Whether the local catalog holds anything the remote did not, and so
@@ -193,10 +196,67 @@ pub fn merge_all(conn: &Connection) -> Result<MergeReport, CatalogError> {
merge_collections_within(&tx, &mut report)?;
merge_keywords_within(&tx, &mut report)?;
merge_people_within(&tx, &mut report)?;
merge_metadata_within(&tx, &mut report)?;
tx.commit()?;
Ok(report)
}
/// Adopt capture metadata from an attached catalog, on its own.
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
let tx = conn.unchecked_transaction()?;
let mut report = MergeReport::default();
merge_metadata_within(&tx, &mut report)?;
tx.commit()?;
Ok(report)
}
/// Capture metadata a peer's sweep already read, for images this device has
/// not dated yet.
///
/// The `images` table is local state and the merge leaves it alone — except
/// for these columns, which are not: a capture time, an offset, a camera, a
/// lens and an ISO are facts about the file's bytes, identical on every
/// device, and read by fetching a header per image across the whole library
/// (`dr_ui::library::spawn_sweep`). A fresh device inherits its peers'
/// thumbnails and faces from the shards and then spent hours re-reading
/// every header for the timeline; the snapshot it had just merged held
/// every one of those dates.
///
/// Matched by `oc:fileid`, as collection membership is. Only rows still at
/// `metadata_state < 2` take anything, and only from a remote row at 2: a
/// date this device read for itself is never overwritten, and a peer that
/// has not read one has nothing to give. The sweep's own query
/// (`metadata_state < 2`) then finds nothing left to do for them.
const METADATA_BY_FILE_ID: &str = "
UPDATE main.images
SET captured_at = r.captured_at,
captured_offset = coalesce(main.images.captured_offset, r.captured_offset),
camera = coalesce(main.images.camera, r.camera),
lens = coalesce(main.images.lens, r.lens),
iso = coalesce(main.images.iso, r.iso),
metadata_state = 2
FROM (SELECT lr.image_id, ri.captured_at, ri.captured_offset,
ri.camera, ri.lens, ri.iso
FROM remote_cat.images ri
JOIN remote_cat.remote rr ON rr.image_id = ri.id
JOIN main.remote lr ON lr.file_id = rr.file_id
WHERE ri.metadata_state >= 2 AND ri.captured_at IS NOT NULL) AS r
WHERE main.images.id = r.image_id
AND main.images.metadata_state < 2";
fn merge_metadata_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
// A snapshot from before these columns, or from a library with no server
// behind it, has nothing to join on.
if !remote_has(tx, "remote")?
|| !remote_has_column(tx, "images", "metadata_state")?
|| !remote_has_column(tx, "images", "captured_offset")?
{
return Ok(());
}
report.metadata_adopted = tx.execute(METADATA_BY_FILE_ID, [])?;
Ok(())
}
/// Merge people and identity judgements from an attached catalog.
///
/// The people half of [`merge_all`], on its own, for the same reason the other
@@ -729,7 +789,7 @@ fn attached_has_table(conn: &Connection, schema: &str, table: &str) -> Result<bo
/// # What travels, and what is recomputed
///
/// The rule this module already follows for the rest of the catalog: user
/// judgements travel, inference is rebuilt. Concretely (docs/faces.md, and the
/// judgements travel, inference is rebuilt. Concretely (docs/dev/faces.md, and the
/// asymmetry `crate::faces` opens with):
///
/// - **People** — uuid, name, and whether the user set them aside. Merged by
@@ -1168,6 +1228,57 @@ mod tests {
// ---- integration over two real catalogs ------------------------------
/// A fresh device takes the capture dates a peer's sweep read, matched by
/// `oc:fileid`, and never overwrites a date it read for itself.
#[test]
fn capture_metadata_arrives_for_undated_images_only() {
let c = two_catalogs();
// Three photographs on both devices: 1 undated here and dated there;
// 2 dated on both, differently; 3 undated on both.
for id in 1..=3 {
add_image_without_hash(&c, "main", id);
add_image_without_hash(&c, "remote_cat", id + 10);
add_remote_id(&c, "main", id, 100 + id);
add_remote_id(&c, "remote_cat", id + 10, 100 + id);
}
c.execute(
"UPDATE remote_cat.images
SET captured_at = 1000, captured_offset = 60, camera = 'X', metadata_state = 2
WHERE id = 11",
[],
)
.unwrap();
c.execute(
"UPDATE remote_cat.images SET captured_at = 2000, metadata_state = 2 WHERE id = 12",
[],
)
.unwrap();
c.execute(
"UPDATE main.images SET captured_at = 2222, metadata_state = 2 WHERE id = 2",
[],
)
.unwrap();
let report = merge_metadata(&c).unwrap();
assert_eq!(report.metadata_adopted, 1);
let row = |id: i64| -> (Option<i64>, Option<i64>, Option<String>, i64) {
c.query_row(
"SELECT captured_at, captured_offset, camera, metadata_state
FROM main.images WHERE id = ?1",
[id],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
)
.unwrap()
};
assert_eq!(row(1), (Some(1000), Some(60), Some("X".into()), 2));
assert_eq!(row(2), (Some(2222), None, None, 2));
assert_eq!(row(3), (None, None, None, 0));
// Idempotent: a second pass finds nothing left to take.
assert_eq!(merge_metadata(&c).unwrap().metadata_adopted, 0);
}
fn two_catalogs() -> Connection {
attached_remote(schema::for_attached("remote_cat"))
}
+171 -14
View File
@@ -49,6 +49,12 @@ pub struct Judgement {
/// 0..=5. Zero means *unrated*, which is a state in its own right.
pub rating: u8,
pub flag: FlagState,
/// TRACES: FR-CAT-5
/// The colour label, or `None`. Not part of [`Judgement::is_judged`]:
/// a label sorts photographs into piles of the photographer's own
/// meaning — "to print", "send to Anna" — and says nothing about whether
/// a frame has been culled, which is the question "unjudged" asks.
pub label: Option<ColourLabel>,
}
impl Judgement {
@@ -267,14 +273,6 @@ pub fn align_default_version_uuids(conn: &Connection) -> Result<usize, CatalogEr
Ok(moved)
}
/// The default version's row id for an image, creating one if it has none.
///
/// Every write path goes through this rather than assuming a version exists.
/// An image can arrive without one in two ways that are not worth trying to
/// prevent: a row inserted by a build predating this module, and a scan whose
/// version pass was interrupted between the image insert and the commit.
/// Failing a rating because of either would be the wrong answer — the user
/// pressed a key and expects a star.
/// TRACES: FR-CAT-13
/// How `versions.label` encodes a colour label, and back.
///
@@ -304,6 +302,14 @@ pub fn label_from_code(code: Option<i64>) -> Option<ColourLabel> {
})
}
/// The default version's row id for an image, creating one if it has none.
///
/// Every write path goes through this rather than assuming a version exists.
/// An image can arrive without one in two ways that are not worth trying to
/// prevent: a row inserted by a build predating this module, and a scan whose
/// version pass was interrupted between the image insert and the commit.
/// Failing a rating because of either would be the wrong answer — the user
/// pressed a key and expects a star.
pub fn default_version_id(conn: &Connection, image: ImageId) -> Result<i64, CatalogError> {
let existing: Option<i64> = conn
.query_row(
@@ -387,6 +393,88 @@ pub fn set_flag_many(
apply_many(conn, images, |conn, id| set_flag(conn, id, flag))
}
/// TRACES: FR-CAT-5
/// Set or clear the colour label for one image.
pub fn set_label(
conn: &Connection,
image: ImageId,
label: Option<ColourLabel>,
) -> Result<(), CatalogError> {
let version = default_version_id(conn, image)?;
conn.execute(
"UPDATE versions SET label = ?2 WHERE id = ?1",
rusqlite::params![version, label.map(label_code)],
)?;
Ok(())
}
/// TRACES: FR-CAT-5
/// Set or clear a label on many images in one transaction — one keystroke
/// over a selection is one commit, as for [`set_rating_many`].
pub fn set_label_many(
conn: &Connection,
images: &[ImageId],
label: Option<ColourLabel>,
) -> Result<usize, CatalogError> {
apply_many(conn, images, |conn, id| set_label(conn, id, label))
}
/// TRACES: FR-CAT-5
/// What a label key does to a set of images: Lightroom's toggle.
///
/// Pressing the key for the label every one of them already carries takes it
/// off; otherwise every one of them gets it. Decided over the whole set
/// rather than per image, so a selection that was half red comes out all red
/// rather than inverted — the photographer pressed "red", and a key that
/// turned half of them red and the other half plain would be two answers to
/// one question.
pub fn toggled_label(
current: impl IntoIterator<Item = Option<ColourLabel>>,
pressed: ColourLabel,
) -> Option<ColourLabel> {
let mut any = false;
for label in current {
any = true;
if label != Some(pressed) {
return Some(pressed);
}
}
if any {
None
} else {
Some(pressed)
}
}
/// TRACES: FR-CAT-5 | FR-CAT-6
/// How the library divides by colour label, for the filter chips' counts.
///
/// Index 0 is unlabelled and index `n` the label whose code is `n`. One
/// grouped statement — the same shape as [`rating_histogram`], and for the
/// same reason it LEFT JOINs: an image without a version row is unlabelled,
/// not missing.
pub fn label_histogram(conn: &Connection) -> Result<[usize; 6], CatalogError> {
let mut out = [0usize; 6];
let mut stmt = conn.prepare(
"SELECT coalesce(v.label, 0) AS l, count(*)
FROM images i
LEFT JOIN versions v ON v.image_id = i.id AND v.is_default = 1
GROUP BY l",
)?;
let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?;
for (code, count) in rows.flatten() {
// A code this build does not know counts as unlabelled, which is how
// `label_from_code` reads it everywhere else.
let slot = if label_from_code(Some(code)).is_some() {
code as usize
} else {
0
};
out[slot] += count as usize;
}
Ok(out)
}
/// Shared bulk wrapper, so the two axes cannot drift in their commit
/// behaviour — a partially-committed rating and a fully-committed flag from
/// the same keystroke would be hard to explain and harder to notice.
@@ -411,21 +499,22 @@ fn apply_many(
/// An image with no version reads as unrated and unflagged rather than as an
/// error: that is exactly what it is.
pub fn judgement(conn: &Connection, image: ImageId) -> Result<Judgement, CatalogError> {
let row: Option<(i64, i64)> = conn
let row: Option<(i64, i64, Option<i64>)> = conn
.query_row(
"SELECT rating, flag FROM versions
"SELECT rating, flag, label FROM versions
WHERE image_id = ?1
ORDER BY is_default DESC, id ASC
LIMIT 1",
[image.0 as i64],
|r| Ok((r.get(0)?, r.get(1)?)),
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
)
.optional()?;
Ok(match row {
Some((rating, flag)) => Judgement {
Some((rating, flag, label)) => Judgement {
rating: rating.clamp(0, MAX_RATING as i64) as u8,
flag: flag_from_code(flag),
label: label_from_code(label),
},
None => Judgement::default(),
})
@@ -452,7 +541,7 @@ pub fn judgements(
.collect::<Vec<_>>()
.join(",");
let sql = format!(
"SELECT image_id, rating, flag FROM versions
"SELECT image_id, rating, flag, label FROM versions
WHERE image_id IN ({placeholders}) AND is_default = 1"
);
@@ -467,15 +556,17 @@ pub fn judgements(
r.get::<_, i64>(0)?,
r.get::<_, i64>(1)?,
r.get::<_, i64>(2)?,
r.get::<_, Option<i64>>(3)?,
))
})?;
for (image, rating, flag) in rows.flatten() {
for (image, rating, flag, label) in rows.flatten() {
out.insert(
ImageId(image as u64),
Judgement {
rating: rating.clamp(0, MAX_RATING as i64) as u8,
flag: flag_from_code(flag),
label: label_from_code(label),
},
);
}
@@ -686,6 +777,72 @@ mod tests {
assert_eq!(distinct, 200);
}
#[test]
fn a_label_round_trips_and_clears() {
// TRACES: FR-CAT-5
let cat = with_images(1);
let id = ids(&cat)[0];
set_label(cat.connection(), id, Some(ColourLabel::Green)).unwrap();
assert_eq!(
judgement(cat.connection(), id).unwrap().label,
Some(ColourLabel::Green)
);
set_label(cat.connection(), id, None).unwrap();
assert_eq!(judgement(cat.connection(), id).unwrap().label, None);
}
#[test]
fn a_label_is_not_a_judgement() {
// "Unjudged" is the cull's resume point; a label is a pile of the
// photographer's own, and labelling a frame must not hide it there.
let cat = with_images(1);
let id = ids(&cat)[0];
set_label(cat.connection(), id, Some(ColourLabel::Red)).unwrap();
assert!(!judgement(cat.connection(), id).unwrap().is_judged());
}
#[test]
fn labelling_a_selection_is_one_commit_and_reaches_every_image() {
// TRACES: FR-CAT-5
let cat = with_images(4);
let all = ids(&cat);
assert_eq!(
set_label_many(cat.connection(), &all, Some(ColourLabel::Blue)).unwrap(),
4
);
let found = judgements(cat.connection(), &all).unwrap();
assert!(all
.iter()
.all(|id| found[id].label == Some(ColourLabel::Blue)));
assert_eq!(
label_histogram(cat.connection()).unwrap(),
[0, 0, 0, 0, 4, 0]
);
}
#[test]
fn a_label_key_toggles_only_when_every_image_already_has_it() {
// TRACES: FR-CAT-5
use ColourLabel::*;
assert_eq!(toggled_label([Some(Red), Some(Red)], Red), None);
assert_eq!(toggled_label([Some(Red), None], Red), Some(Red));
assert_eq!(toggled_label([Some(Blue)], Red), Some(Red));
assert_eq!(toggled_label([], Red), Some(Red));
}
#[test]
fn the_label_histogram_sums_to_the_library() {
// TRACES: FR-CAT-6
// Images without a version row count as unlabelled rather than
// vanishing, as the rating histogram's do.
let cat = with_images(3);
let first = ids(&cat)[0];
set_label(cat.connection(), first, Some(ColourLabel::Purple)).unwrap();
let h = label_histogram(cat.connection()).unwrap();
assert_eq!(h, [2, 0, 0, 0, 0, 1]);
assert_eq!(h.iter().sum::<usize>(), 3);
}
#[test]
fn a_rating_round_trips() {
let cat = with_images(1);
+2 -2
View File
@@ -16,7 +16,7 @@
//! # The one thing a rebuild does not recover
//!
//! **Collections.** A manual collection is a set of images the user assembled
//! by hand and nothing in the filesystem records it (`docs/catalog.md` §8.1) —
//! by hand and nothing in the filesystem records it (`docs/dev/catalog.md` §8.1) —
//! which is the whole reason the catalog file itself syncs. So the two offers
//! are not interchangeable, and the interface must not present them as if they
//! were: a restore keeps the user's collections, a rebuild does not.
@@ -570,7 +570,7 @@ mod tests {
// The first NFR-R6 branch, asserted on the thing that distinguishes it
// from the second: a collection exists nowhere but the catalog, so it
// is the evidence that the *contents* came back and not merely a
// readable file (docs/catalog.md §8.1).
// readable file (docs/dev/catalog.md §8.1).
let dir = tempdir("restore");
let path = dir.join("catalog.sqlite");
fixture(&path, 500);
+221 -3
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 18;
pub const SCHEMA_VERSION: i64 = 20;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -181,9 +181,105 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?;
}
if from < 19 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V19)?;
tx.pragma_update(None, "user_version", 19)?;
tx.commit()?;
}
if from < 20 {
let tx = conn.unchecked_transaction()?;
v20_markers_name_the_detector_that_found_the_faces(&tx)?;
tx.pragma_update(None, "user_version", 20)?;
tx.commit()?;
}
Ok(from)
}
// V20 -- TRACES: FR-CAT-7
//
// Run markers that named the wrong detector, put right.
//
// `faces::record_updates` -- the write behind the quality, eye and crop
// passes -- re-marked an image under the pipeline the pass ran as, while
// the faces it had updated kept the id of the detector that found them.
// A marker of `scrfd_10g+w600k_mbf` over faces spelled `w600k_mbf` reads,
// to every consumer, as the thorough detector having examined the image:
// the upgrade repair skips it, and `face_shard::export_to_shards` selects
// its faces by the marker's id, finds none, and tells every other device
// that the thorough detector found nothing there. The desktop's shard index
// held 54 such entries over photographs with named faces, and the tablet's
// eye pass over faces it had adopted from the desktop had made 430 more.
//
// The write is fixed to keep the marker under the faces' own id. This puts
// the markers already written right, with a fresh time so the export sends
// each image again under an entry newer than the empty one -- which is what
// `held_model` orders by. Where the right marker is still there beside the
// wrong one (the old write inserted rather than replaced), the wrong one
// goes and the right one is refreshed for the same reason: its entry in
// the shards is older than the empty one, and a device that has neither
// would take the empty one. An image V14 left with faces and no marker at
// all is not touched: that state is the quality pass's cue, and the fixed
// write marks it correctly when the pass reaches it.
//
// Restated in Rust rather than SQL because the embedder half of a pipeline
// id is `faces::embedder_sql`, which this must agree with.
fn v20_markers_name_the_detector_that_found_the_faces(tx: &Connection) -> Result<(), CatalogError> {
let fi = crate::faces::embedder_sql("face_index.model_id");
let f = crate::faces::embedder_sql("f.model_id");
// A marker is wrong when the image holds faces of its embedder under
// another id. First the wrong ones that sit beside a right one -- the
// update below would collide with it -- then the rest are renamed.
let wrong = format!(
"EXISTS (SELECT 1 FROM faces f
WHERE f.image_id = face_index.image_id
AND {f} = {fi}
AND f.model_id != face_index.model_id)"
);
let found_by = format!(
"(SELECT MIN(f.model_id) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi})"
);
let now = crate::faces::now_secs();
tx.execute(
&format!(
"UPDATE face_index
SET indexed_at = ?1
WHERE model_id = {found_by}
AND EXISTS (SELECT 1 FROM face_index w
WHERE w.image_id = face_index.image_id
AND w.model_id != face_index.model_id
AND {} = {fi})",
crate::faces::embedder_sql("w.model_id")
),
[now],
)?;
tx.execute(
&format!(
"DELETE FROM face_index
WHERE {wrong}
AND EXISTS (SELECT 1 FROM face_index o
WHERE o.image_id = face_index.image_id
AND o.model_id = {found_by})"
),
[],
)?;
tx.execute(
&format!(
"UPDATE face_index
SET model_id = {found_by},
faces_found = (SELECT COUNT(*) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi}),
indexed_at = ?1
WHERE {wrong}"
),
[now],
)?;
Ok(())
}
/// The seven columns V16 adds to `faces`, in the order the readers name them.
///
/// Named once because three places have to agree on them: this migration,
@@ -766,6 +862,44 @@ CREATE TABLE IF NOT EXISTS xmp_conflicts (
);
"#;
// V19 -- TRACES: NFR-P9
//
// The indexes the repair counts are served from, and V17's lesson applied
// to the rest of the face columns.
//
// "How many images still owe a quality reading" was answered per image: a
// correlated EXISTS over `faces` that had to open each face's row to look
// at one nullable column -- the row being eight kilobytes of embedding and
// crop. Six such counts run every time the Identity screen opens and every
// time a sweep ends, 160 ms of them on the reference library. Three
// partial indexes hold only the faces still owing each pass, keyed by the
// image and carrying the model id the predicate also reads, so the count
// walks a few thousand index entries and touches no row at all -- and each
// index shrinks to nothing as its pass completes. The planner takes them
// when the count is driven from `faces` (`repairs::count`) and ignores
// them inside the per-image EXISTS, which is why that function has two
// spellings of the same predicate.
//
// `faces_image_model` replaces `faces_image`: the same key with the model
// id beside it, so "does this image hold this embedder's faces" -- asked in
// the audit, the proxy repair and the outstanding-detection count -- is an
// index-only probe where it used to read the row for the model id. Every
// lookup that used `faces_image` is served by its prefix.
//
// Not applied to attached catalogs, like V7 and V17: an index is a local
// concern, and a merge never runs these queries across an attachment.
const V19: &str = r#"
CREATE INDEX IF NOT EXISTS faces_image_model ON faces(image_id, model_id);
DROP INDEX IF EXISTS faces_image;
CREATE INDEX IF NOT EXISTS faces_owed_quality ON faces(image_id, model_id)
WHERE quality IS NULL;
CREATE INDEX IF NOT EXISTS faces_owed_crop ON faces(image_id, model_id)
WHERE crop IS NULL;
CREATE INDEX IF NOT EXISTS faces_owed_eyes ON faces(image_id, model_id)
WHERE eye_right IS NULL OR landmarks_dense IS NULL;
"#;
// V18 -- TRACES: FR-CULL-8a | FR-CULL-12
//
// The 106 dense landmarks the eye pass reads its eye boxes from, kept beside
@@ -875,7 +1009,7 @@ CREATE INDEX face_index_model ON face_index(model_id);
const V8: &str = r#"
-- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
-- People and faces (docs/faces.md, docs/catalog.md §10).
-- People and faces (docs/dev/faces.md, docs/dev/catalog.md §10).
--
-- Everything here is **derived data** except one column. Faces, landmarks,
-- embeddings, cluster assignments and suggestions are all reproducible by
@@ -912,7 +1046,7 @@ CREATE TABLE faces (
landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise
detector_confidence REAL NOT NULL,
embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since
-- Source pixels across the aligned 112x112 crop (docs/faces.md §7).
-- Source pixels across the aligned 112x112 crop (docs/dev/faces.md §7).
--
-- Not cosmetic: it is the honest quality signal for the UI, a feature in
-- the §8 calibration -- FR-CULL-9 names face size as an axis along which an
@@ -1809,6 +1943,90 @@ mod tests {
assert_eq!(faces, 2);
}
#[test]
fn v20_renames_markers_to_the_detector_that_found_the_faces() {
let c = mem();
c.pragma_update(None, "user_version", 0).unwrap();
migrate(&c).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'test')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1,1,'a',0),(2,1,'b',0),(3,1,'c',0),(4,1,'d',0),(5,1,'e',0)",
[],
)
.unwrap();
// 1: the desktop's case -- old faces, re-marked as thorough.
// 2: the tablet's case -- adopted thorough faces, re-marked int8,
// and the right marker still beside it (refreshed, so it is
// exported again over the empty entry).
// 3: right already. 4: examined and empty. 5: V14's state, faces
// and no marker.
for (image, model) in [
(1, "scrfd_10g+w600k_mbf"),
(2, "scrfd_10g_i8+w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(4, "scrfd_10g+w600k_mbf"),
] {
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, 100, 0, 6000)",
rusqlite::params![image, model],
)
.unwrap();
}
for (image, model) in [
(1, "w600k_mbf"),
(1, "w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(5, "w600k_mbf"),
] {
c.execute(
"INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, embedding,
crop_px, model_id, detected_at)
VALUES (?1, 0.1, 0.1, 0.2, 0.2, X'00', 0.9, X'00', 180.0, ?2, 0)",
rusqlite::params![image, model],
)
.unwrap();
}
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let markers: Vec<(i64, String, i64, bool)> = c
.prepare(
"SELECT image_id, model_id, faces_found, indexed_at > 100
FROM face_index ORDER BY image_id, model_id",
)
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(
markers,
vec![
(1, "w600k_mbf".to_string(), 2, true),
(2, "scrfd_10g+w600k_mbf".to_string(), 0, true),
(3, "scrfd_10g+w600k_mbf".to_string(), 0, false),
(4, "scrfd_10g+w600k_mbf".to_string(), 0, false),
]
);
// Re-enterable: nothing left to rename.
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let n: i64 = c
.query_row("SELECT count(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 4);
}
#[test]
fn job_uniqueness_coalesces_rather_than_duplicating() {
let c = mem();
+121
View File
@@ -0,0 +1,121 @@
//! TRACES: FR-RAW-2
//! The seam a second decoder plugs into.
//!
//! D2 keeps LibRaw as the fallback for bodies rawler does not cover. Adding
//! it later should be a new `impl Decoder`, not an edit to every caller that
//! reads a header, cuts a thumbnail or opens a photograph for export — which
//! is what the free functions alone would have made it. So the callers take a
//! `&dyn Decoder`, and only the places that start a job name [`default`].
//!
//! Bytes in, always. Nothing here takes a path or a `SourceRef`: resolving a
//! file to bytes is `Storage`'s job at the caller, so the same decoder serves a
//! local file, an Android document and a range fetched from Nextcloud. The
//! decoder's part in that is to say how much of a file it needs
//! ([`Decoder::header_bytes`]) and where its preview sits
//! ([`Decoder::locate_preview`]); the storage layer fetches exactly that.
//!
//! What stays a free function is what is not a decoder's to vary: recognising
//! a JPEG ([`crate::probe`]), decoding one ([`crate::decode_jpeg`]) and
//! checking one is whole ([`crate::is_complete_jpeg`]). A second RAW decoder
//! would not read a JPEG differently.
use dr_types::Orientation;
use crate::{DecodeError, Metadata, Preview, PreviewLocation, PreviewSize, RawImage};
/// TRACES: FR-RAW-2
/// A RAW decoder, over bytes.
///
/// Object-safe so a caller can hold `&dyn Decoder` without becoming generic,
/// `Send + Sync` because the callers that need one most — the thumbnail
/// lanes, the export worker — run off the UI thread, and `Debug` so a job
/// description that carries one can still be printed.
pub trait Decoder: Send + Sync + std::fmt::Debug {
/// How much of the start of a file [`Self::metadata`] and
/// [`Self::locate_preview`] need. A caller reading over a network fetches
/// this range and no more.
fn header_bytes(&self) -> u64;
/// Capture metadata, from a header or a whole file, without touching
/// sensor data.
fn metadata(&self, bytes: &[u8]) -> Result<Metadata, DecodeError>;
/// How the stored pixels are turned, from a header. `None` where the file
/// does not say, which callers take as upright.
fn orientation(&self, header: &[u8]) -> Option<Orientation>;
/// Where the embedded preview best suited to a thumbnail sits in the file,
/// from its header, so a remote caller can fetch that range alone.
fn locate_preview(&self, header: &[u8], file_len: u64) -> Option<PreviewLocation>;
/// The embedded preview at the size asked for, falling through the ladder
/// to the next size where the file lacks it.
fn preview(&self, bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError>;
/// Sensor data, for develop and export. The expensive path.
fn decode(&self, bytes: &[u8]) -> Result<RawImage, DecodeError>;
}
/// TRACES: FR-RAW-2
/// The decoder the application ships: rawler for sensor data and the
/// previews it knows, DarkRoom's own container walk for headers and ranges.
///
/// Its methods are the crate's free functions, unchanged. They stay public
/// for the tools and examples that read one file and have no caller to keep
/// decoder-agnostic.
#[derive(Debug, Clone, Copy, Default)]
pub struct Rawler;
impl Decoder for Rawler {
fn header_bytes(&self) -> u64 {
crate::HEADER_BYTES
}
fn metadata(&self, bytes: &[u8]) -> Result<Metadata, DecodeError> {
crate::metadata(bytes)
}
fn orientation(&self, header: &[u8]) -> Option<Orientation> {
crate::orientation(header)
}
fn locate_preview(&self, header: &[u8], file_len: u64) -> Option<PreviewLocation> {
crate::locate_preview(header, file_len)
}
fn preview(&self, bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError> {
crate::extract_preview(bytes, size)
}
fn decode(&self, bytes: &[u8]) -> Result<RawImage, DecodeError> {
crate::decode(bytes)
}
}
/// TRACES: FR-RAW-2
/// The decoder a job uses unless it was handed another.
///
/// Named by the places that start work — a thread, a UI handler — and by
/// nothing below them. Returning `&'static dyn Decoder` rather than `Rawler`
/// is the point: a caller that only has this cannot reach past the trait.
pub fn default() -> &'static dyn Decoder {
static RAWLER: Rawler = Rawler;
&RAWLER
}
#[cfg(test)]
mod tests {
use super::*;
/// The default is the shipped decoder, reached through the trait: same
/// header budget, and the same answer to bytes neither can read.
#[test]
fn the_default_is_rawler_behind_the_trait() {
let d = default();
assert_eq!(d.header_bytes(), crate::HEADER_BYTES);
let junk = [0u8; 64];
assert_eq!(d.metadata(&junk).is_err(), crate::metadata(&junk).is_err());
assert!(d.decode(&junk).is_err());
assert_eq!(d.orientation(&junk), crate::orientation(&junk));
}
}
+26
View File
@@ -11,14 +11,20 @@
//!
//! Fusing them would force a full decode where a header read suffices, which
//! is exactly why Lightroom stalls ~2 s per image during culling.
//!
//! Callers reach these through the [`Decoder`] trait rather than by name, so a
//! second decoder can be put behind them without changing any of them
//! (FR-RAW-2). [`Rawler`] is the one that ships; [`default`] hands it out.
pub mod base_curve;
mod decoder;
mod error;
mod locate;
mod preview;
pub mod profile;
pub use base_curve::BaseCurve;
pub use decoder::{default, Decoder, Rawler};
pub use error::DecodeError;
pub use locate::{
defects, is_complete_jpeg, jpeg_metadata, locate_preview, tiff_metadata, BadLine, BadPixel,
@@ -304,6 +310,26 @@ pub fn metadata(bytes: &[u8]) -> Result<Metadata, DecodeError> {
error::guarded("metadata", || metadata_unguarded(bytes))
}
/// TRACES: FR-CAT-5
/// Where a TIFF-shaped file keeps its first IFD, when the head handed to
/// [`metadata`] does not reach it — the linear DNG a merge writes puts its
/// IFDs after the pixels, and rawler, given the head alone, finds no
/// decoder in it. The caller fetches from this offset to the end and
/// reads the two ranges with [`metadata_split`].
pub fn trailing_ifd(head: &[u8]) -> Option<u64> {
locate::trailing_ifd(head)
}
/// TRACES: FR-CAT-5
/// [`metadata`] for a file read in two ranges: `head` from offset 0 and
/// `tail` from `tail_at`. The EXIF sub-IFD such a file wrote before its
/// pixels is in the head; the first IFD and its values are in the tail.
pub fn metadata_split(head: &[u8], tail: &[u8], tail_at: u64) -> Result<Metadata, DecodeError> {
error::guarded("metadata", || {
locate::tiff_metadata_split(head, tail, tail_at)
})
}
fn metadata_unguarded(bytes: &[u8]) -> Result<Metadata, DecodeError> {
use rawler::rawsource::RawSource;
+82 -13
View File
@@ -183,22 +183,65 @@ struct Entry {
value: u32,
}
/// The bytes a [`TiffReader`] reads: the head of a file, and optionally a
/// second range from further in, at a known offset.
///
/// A camera writes its IFDs at the front, so the first 256 KB of a file
/// is the whole structure. A file written strip by strip — the linear
/// DNG a merge produces — has its first IFD at the *end*, after the
/// pixels, and a reader that only has the head sees a pointer into
/// nothing. Rather than fetch 800 MB to read a date, the caller fetches
/// the head, asks [`crate::trailing_ifd`] where the IFD is, fetches that
/// tail, and reads through both. Offsets are the file's own throughout;
/// a read that falls in neither range is simply absent.
#[derive(Clone, Copy)]
struct Src<'a> {
head: &'a [u8],
tail: &'a [u8],
/// Where `tail` starts in the file.
tail_at: usize,
}
impl<'a> Src<'a> {
fn whole(data: &'a [u8]) -> Self {
Src {
head: data,
tail: &[],
tail_at: 0,
}
}
fn get(&self, start: usize, len: usize) -> Option<&'a [u8]> {
let end = start.checked_add(len)?;
if let Some(b) = self.head.get(start..end) {
return Some(b);
}
let s = start.checked_sub(self.tail_at)?;
self.tail.get(s..s.checked_add(len)?)
}
}
/// A minimal TIFF structure reader.
///
/// Deliberately not a general TIFF parser: it reads the IFD chain and entry
/// values and nothing else, because that is all locating a preview needs.
struct TiffReader<'a> {
data: &'a [u8],
data: Src<'a>,
little_endian: bool,
first_ifd: u32,
}
impl<'a> TiffReader<'a> {
fn new(data: &'a [u8]) -> Option<Self> {
if data.len() < 8 {
Self::over(Src::whole(data))
}
fn over(data: Src<'a>) -> Option<Self> {
let head = data.head;
if head.len() < 8 {
return None;
}
let little_endian = match &data[0..2] {
let little_endian = match &head[0..2] {
b"II" => true,
b"MM" => false,
_ => return None,
@@ -362,9 +405,7 @@ impl<'a> TiffReader<'a> {
};
raw[..len.min(4)].to_vec()
} else {
self.data
.get(e.value as usize..e.value as usize + len)?
.to_vec()
self.data.get(e.value as usize, len)?.to_vec()
};
let s = String::from_utf8_lossy(&bytes);
@@ -396,8 +437,7 @@ impl<'a> TiffReader<'a> {
// same way to recover the original byte order.
return None;
}
let start = e.value as usize;
self.data.get(start..start.checked_add(len)?)
self.data.get(e.value as usize, len)
}
fn offsets(&self, e: &Entry) -> Vec<u32> {
@@ -420,8 +460,8 @@ impl<'a> TiffReader<'a> {
}
}
fn read_u16(data: &[u8], at: usize, le: bool) -> Option<u16> {
let b = data.get(at..at + 2)?;
fn read_u16(data: Src<'_>, at: usize, le: bool) -> Option<u16> {
let b = data.get(at, 2)?;
Some(if le {
u16::from_le_bytes([b[0], b[1]])
} else {
@@ -429,8 +469,8 @@ fn read_u16(data: &[u8], at: usize, le: bool) -> Option<u16> {
})
}
fn read_u32(data: &[u8], at: usize, le: bool) -> Option<u32> {
let b = data.get(at..at + 4)?;
fn read_u32(data: Src<'_>, at: usize, le: bool) -> Option<u32> {
let b = data.get(at, 4)?;
Some(if le {
u32::from_le_bytes([b[0], b[1], b[2], b[3]])
} else {
@@ -460,7 +500,36 @@ pub fn jpeg_metadata(bytes: &[u8]) -> Result<crate::Metadata, crate::DecodeError
/// no `DateTimeOriginal` for some DNGs whose tag sits plainly at byte 826 —
/// and without this fallback those images are silently undated.
pub fn tiff_metadata(tiff_data: &[u8]) -> Result<crate::Metadata, crate::DecodeError> {
let reader = TiffReader::new(tiff_data)
tiff_metadata_over(Src::whole(tiff_data))
}
/// Where a TIFF-shaped file's first IFD is, when the head does not reach
/// it: the offset to fetch from, so [`tiff_metadata_split`] can read it.
/// `None` for a file that is not TIFF, or whose IFD the head already holds.
pub fn trailing_ifd(head: &[u8]) -> Option<u64> {
let r = TiffReader::new(head)?;
let at = r.first_ifd as u64;
(at >= head.len() as u64).then_some(at)
}
/// [`tiff_metadata`] over a head and a tail fetched separately: the head
/// from offset 0, the tail from `tail_at`. For the file whose IFDs follow
/// its pixels.
pub fn tiff_metadata_split(
head: &[u8],
tail: &[u8],
tail_at: u64,
) -> Result<crate::Metadata, crate::DecodeError> {
tiff_metadata_over(Src {
head,
tail,
tail_at: usize::try_from(tail_at)
.map_err(|_| crate::DecodeError::Metadata("tail offset out of range".into()))?,
})
}
fn tiff_metadata_over(src: Src<'_>) -> Result<crate::Metadata, crate::DecodeError> {
let reader = TiffReader::over(src)
.ok_or_else(|| crate::DecodeError::Metadata("malformed EXIF header".into()))?;
let mut md = crate::Metadata::default();
+28
View File
@@ -241,6 +241,8 @@ mod tests {
let source = SourceMetadata {
make: Some("Canon".into()),
model: Some("Canon EOS 6D".into()),
captured_at: Some(1_754_398_664),
captured_offset: Some(120),
..Default::default()
};
write_linear_dng(
@@ -301,6 +303,32 @@ mod tests {
assert_eq!((crop.p.x, crop.p.y, crop.d.w, crop.d.h), (2, 1, 15, 10));
}
#[test]
fn the_catalog_reads_the_date_from_a_head_and_a_tail() {
// TRACES: FR-CAT-5
// The IFDs follow the pixels, so a scan that has the first bytes of
// the file has a pointer into nothing; rawler finds no decoder in
// that, and the composite would sit undated at the end of the grid.
// The scan's second range — from the first IFD to the end — with
// the head is enough to date it, and to name the camera.
let bytes = write(640, 400, 64);
let head = &bytes[..4096];
assert!(
dr_decode::metadata(head).is_err(),
"the head alone must not read"
);
let at = dr_decode::trailing_ifd(head).expect("the IFD is beyond the head");
assert!(at as usize > head.len());
let tail = &bytes[at as usize..];
assert!(tail.len() < 4096, "the tail is the IFD, not the pixels");
let md = dr_decode::metadata_split(head, tail, at).expect("read from two ranges");
assert_eq!(md.captured_at, Some(1_754_398_664));
assert_eq!(md.captured_offset, Some(120));
assert_eq!(md.model.as_deref(), Some("Canon EOS 6D"));
// A head that holds everything is not a trailing-IFD file.
assert_eq!(dr_decode::trailing_ifd(&bytes), None);
}
#[test]
fn a_strip_of_the_wrong_length_is_refused() {
let mut bytes = std::io::Cursor::new(Vec::new());
+2 -2
View File
@@ -11,7 +11,7 @@ log.workspace = true
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
# provider the device has (docs/inference.md). This crate never names either.
# provider the device has (docs/dev/inference.md). This crate never names either.
ort = { workspace = true, optional = true }
dr-inference-engine = { workspace = true, optional = true }
ndarray = { workspace = true, optional = true }
@@ -37,7 +37,7 @@ required-features = ["inference"]
[features]
# Nothing on by default, and in particular **no `embedded-model`**: the weights
# are not a build input and never become one (docs/faces.md §2.2). A feature
# are not a build input and never become one (docs/dev/faces.md §2.2). A feature
# flag that *could* embed them is a flag someone eventually sets in a packaging
# script, and the InsightFace grant does not survive that.
default = []
+1 -1
View File
@@ -1,4 +1,4 @@
//! Detect the faces in a JPEG and read each one's eyes (docs/faces.md §17).
//! Detect the faces in a JPEG and read each one's eyes (docs/dev/faces.md §17).
//!
//! The thing worth looking at is whether the eye boxes land on eyes and
//! whether soft ones are refused — so with `--dump DIR` the crops the
+1 -1
View File
@@ -7,7 +7,7 @@
//! DET.onnx EMB.onnx photo.jpg [photo.jpg ...]
//!
//! The models must have had their input dims frozen first; see
//! `tools/fix-face-model-shapes.sh` and docs/faces.md §12 M1.
//! `tools/fix-face-model-shapes.sh` and docs/dev/faces.md §12 M1.
use std::time::Instant;
+1 -1
View File
@@ -1,4 +1,4 @@
//! M1 (docs/faces.md §12) — will tract load these graphs at all?
//! M1 (docs/dev/faces.md §12) — will tract load these graphs at all?
//!
//! The one measurement everything else in the face subsystem is conditional
//! on. `det_500m.onnx` has a dynamic H/W input, which is exactly what tract
+1 -1
View File
@@ -9,7 +9,7 @@
//!
//! # What it is for
//!
//! docs/faces.md §9 has the desktop numbers and the question they leave open:
//! docs/dev/faces.md §9 has the desktop numbers and the question they leave open:
//! a GPU GEMM is worth roughly 1.5× of a regroup on a twenty-core desktop,
//! because the scan is under a third of the pass there. On a tablet the CPU is
//! several times slower and the GPU is not, so the same optimisation is worth
+5 -5
View File
@@ -1,4 +1,4 @@
//! Five-point face alignment (docs/faces.md §5).
//! Five-point face alignment (docs/dev/faces.md §5).
//!
//! ArcFace embeddings are trained on faces warped to a canonical 112×112
//! arrangement. Feeding the model a plain bounding-box crop *works* — it
@@ -208,7 +208,7 @@ impl Similarity {
///
/// # Why least squares and not RANSAC
///
/// The reference C++ implementation (docs/faces.md §1.1) fits this with
/// The reference C++ implementation (docs/dev/faces.md §1.1) fits this with
/// OpenCV's `estimateAffinePartial2D` under RANSAC. RANSAC over five points is
/// a strange fit: the minimal sample for a similarity is two, so it can discard
/// landmarks it judges outliers and solve from a subset — and on a profile face
@@ -427,7 +427,7 @@ fn sample_window(
// ── eyes ──────────────────────────────────────────────────────────────────
/// Width of an eye crop as the classifier reads it, in pixels. Fixed by the
/// OCEC input (`docs/faces.md` §17): 40 wide, 24 high.
/// OCEC input (`docs/dev/faces.md` §17): 40 wide, 24 high.
pub const EYE_PATCH_WIDTH: usize = 40;
/// Height of an eye crop as the classifier reads it, in pixels.
pub const EYE_PATCH_HEIGHT: usize = 24;
@@ -438,7 +438,7 @@ pub const EYE_PATCH_HEIGHT: usize = 24;
/// The classifier was trained on a whole-body detector's *eye* boxes — tight
/// round the palpebral fissure — and measured on 25 open-eyed faces from the
/// reference library, a tight box is what it wants: 22 of 25 read open at
/// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/faces.md §17.2). A tenth, so a
/// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/dev/faces.md §17.2). A tenth, so a
/// contour landing a pixel short of the lashes still holds them.
pub const EYE_BOX_MARGIN: f32 = 0.1;
@@ -571,7 +571,7 @@ pub const SUNGLASSES_EDGE: usize = 48;
/// clear glasses, at 0.68. Erring towards "sunglasses" is the safe direction
/// for what this feeds: a face called sunglasses is left alone by the
/// eyes-open filter, where a pair of sunglasses missed hands the eye
/// classifier a lens to guess at (docs/faces.md §17).
/// classifier a lens to guess at (docs/dev/faces.md §17).
pub const SUNGLASSES_WINDOWS: [(f32, f32, f32, f32); 2] =
[(0.0, 0.0, 112.0, 112.0), (-5.0, -14.0, 122.0, 122.0)];
+3 -3
View File
@@ -1,4 +1,4 @@
//! Cosine to probability (docs/faces.md §8, FR-CULL-9).
//! Cosine to probability (docs/dev/faces.md §8, FR-CULL-9).
//!
//! FR-CULL-9 is a hard requirement rather than an implementation detail: no
//! code path may threshold a bare cosine, every threshold in the subsystem is
@@ -28,7 +28,7 @@
//! calibration to the belief it was supposed to test — and that is the whole
//! of the alternative.
//!
//! docs/faces.md §8.1 names one more that would cost no labelling at all: two
//! docs/dev/faces.md §8.1 names one more that would cost no labelling at all: two
//! faces in adjacent frames of one burst are near-certainly the same person,
//! and FR-CULL-5's grouping is sitting there. Nothing draws on it. This crate
//! cannot see a catalog, let alone the bursts in one — it is handed cosines by
@@ -83,7 +83,7 @@ pub struct Calibration {
}
impl Default for Calibration {
/// The reference implementation's fitted MBF curve (docs/faces.md §1):
/// The reference implementation's fitted MBF curve (docs/dev/faces.md §1):
/// steepness 16.2, P=0.5 at cosine 0.267.
///
/// **`valid` is false**, and that is the point. It is a documented
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-CULL-8a
//! The two small classifiers behind a face's eye state (docs/faces.md §17).
//! The two small classifiers behind a face's eye state (docs/dev/faces.md §17).
//!
//! **OCEC** — *open closed eyes classification*, Hyodo 2025 — reads one
//! 40×24 eye and answers P(open). **SGC** — *sunglasses classification*,
+1 -1
View File
@@ -1,4 +1,4 @@
//! Grouping faces into people (docs/faces.md §9, FR-CULL-10).
//! Grouping faces into people (docs/dev/faces.md §9, FR-CULL-10).
//!
//! Model-free: this is arithmetic over embeddings, and it is where the
//! subsystem's accuracy actually lives, so it is testable with no weights on
+4 -4
View File
@@ -1,4 +1,4 @@
//! SCRFD face detection (docs/faces.md §4).
//! SCRFD face detection (docs/dev/faces.md §4).
//!
//! One forward pass produces a box, a confidence and **five landmarks** per
//! face — the landmarks being the reason for this detector rather than a
@@ -138,7 +138,7 @@ impl Detection {
pub struct Detector {
session: Model,
/// f32 or int8 — the int8 form finds a different set of faces and is a
/// different detector in `model_id` (docs/inference.md §7).
/// different detector in `model_id` (docs/dev/inference.md §7).
form: Form,
/// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}.
///
@@ -346,7 +346,7 @@ fn iou(a: &(f32, f32, f32, f32), b: &(f32, f32, f32, f32)) -> f32 {
/// How the image is fitted into the graph's fixed square input.
///
/// The forward and inverse mappings live in one struct on purpose:
/// docs/faces.md §4.1 notes that what matters is not *where* the padding goes
/// docs/dev/faces.md §4.1 notes that what matters is not *where* the padding goes
/// but that the two agree. A mismatch offsets every box and landmark by the
/// padding, producing detections that look plausible and embeddings that
/// quietly cluster badly three stages later.
@@ -372,7 +372,7 @@ impl Letterbox {
///
/// `(x·255 − 127.5) / 128` — note `/128`, not `/127.5`. The reference
/// implementation this is ported from uses `/128` for both models, and
/// every measured number in docs/faces.md §1 came from it.
/// every measured number in docs/dev/faces.md §1 came from it.
///
/// Padding is grey, matching the reference's `114`: the value the network
/// reads least as an edge, where black would draw a hard border across the
+2 -2
View File
@@ -1,4 +1,4 @@
//! ArcFace / MobileFaceNet inference (docs/faces.md §6).
//! ArcFace / MobileFaceNet inference (docs/dev/faces.md §6).
//!
//! Takes an aligned crop and returns 512 L2-normalised floats. The alignment is
//! not optional and cannot be skipped by accident: [`Embedder::embed`] takes an
@@ -66,7 +66,7 @@ impl Embedder {
pub fn from_bytes(bytes: &[u8], model: ModelId) -> Result<Self, FaceError> {
// Always the f32 form: an embedding must compare across devices
// (docs/inference.md §7), and the engine pins this role to it.
// (docs/dev/inference.md §7), and the engine pins this role to it.
let loaded = dr_inference_engine::open(Role::Embedder, Form::F32, bytes)?;
let acquired = loaded.acquire()?;
let session = acquired.lock();
+2 -2
View File
@@ -1,4 +1,4 @@
//! What an embedder produces, and how it is stored (docs/faces.md §6).
//! What an embedder produces, and how it is stored (docs/dev/faces.md §6).
//!
//! Deliberately **model-free**: the vector, its identity, its comparison and
//! its storage encoding are arithmetic, and `calibrate` and `cluster` are built
@@ -273,7 +273,7 @@ mod tests {
);
}
/// The claim docs/faces.md §6 makes about the storage format: the f16
/// The claim docs/dev/faces.md §6 makes about the storage format: the f16
/// round-trip costs ~1e-3 of cosine, three orders below the separation
/// between a match and a non-match.
#[test]
+3 -3
View File
@@ -67,14 +67,14 @@ pub const SUNGLASSES_THRESHOLD: f32 = 0.5;
/// The classifier was trained on eyes down to about a dozen pixels wide
/// (its reference footage averaged 15–21); below that the 40-pixel patch is
/// an interpolation of nothing, and the answer is noise that reads as
/// "closed". docs/faces.md §17.3 has the measurement behind the number.
/// "closed". docs/dev/faces.md §17.3 has the measurement behind the number.
pub const MIN_EYE_PX: f32 = 12.0;
/// Least [`Eye::sharpness`] for the eye to be read.
///
/// The same measure as the face's `min_sharpness`, over the eye patch, and
/// chosen the same way: the value under which the open-eyed faces of the
/// reference sample were being called closed. docs/faces.md §17.3.
/// reference sample were being called closed. docs/dev/faces.md §17.3.
pub const MIN_EYE_SHARPNESS: f32 = 0.02;
/// An eye narrower than this fraction of its partner is the far eye of a
@@ -82,7 +82,7 @@ pub const MIN_EYE_SHARPNESS: f32 = 0.02;
///
/// A landmark model's contour for a hidden eye collapses towards the nose.
/// Measured on twenty native renders of the reference library
/// (docs/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near
/// (docs/dev/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near
/// one, two three-quarter faces whose far eye read closed sat at 0.54, and
/// every face looking at the camera — winks included, since a shut eye's
/// box keeps its width — sat at 0.78 or more. 0.6 splits the gap.
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-CULL-8a
//! Dense facial landmarks — InsightFace's `2d106det` (docs/faces.md §17.2).
//! Dense facial landmarks — InsightFace's `2d106det` (docs/dev/faces.md §17.2).
//!
//! SCRFD's five points place a face; they do not place an eye. Its eye
//! point is loose enough that a window centred on it left the eye in a
+5 -4
View File
@@ -1,4 +1,4 @@
//! Faces and identity (S14, docs/faces.md).
//! Faces and identity (S14, docs/dev/faces.md).
//!
//! Two models, run over the native render, producing per face a box, five
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
@@ -21,9 +21,9 @@
//! for a packaging script to switch on. The application obtains a model at
//! runtime; this crate takes bytes and never fetches anything.
//!
//! docs/faces.md §2 is the full reading, including what would have to change
//! docs/dev/faces.md §2 is the full reading, including what would have to change
//! for that to stop being true. The eye-state models are the exception: MIT,
//! weights and all, and shipped in `models/face/` (docs/faces.md §17).
//! weights and all, and shipped in `models/face/` (docs/dev/faces.md §17).
//!
//! # Why the runtime is split behind a feature
//!
@@ -50,11 +50,12 @@ pub mod eyes;
pub mod landmarks;
pub mod naming;
pub mod neighbours;
pub mod references;
/// Smallest long edge a face crop may be sampled from.
///
/// **A floor on the crop source, not on the detector input.** The distinction
/// is the whole of FR-CULL-8 and `docs/faces.md` §7: detection letterboxes
/// is the whole of FR-CULL-8 and `docs/dev/faces.md` §7: detection letterboxes
/// every buffer into 640×640, so its input resolution decides nothing, while
/// [`warp`] samples the 112×112 the embedder sees and so converts source
/// resolution directly into embedding quality. FR-CULL-8 requires that crop to
+1 -1
View File
@@ -115,7 +115,7 @@ pub struct Faces<'a> {
/// Source pixels across the aligned crop, for the calibration's size term.
pub crop_px: &'a [f32],
/// Which photograph each face came from. Two faces in one frame are not
/// the same person, so those pairs are never returned (docs/faces.md §9).
/// the same person, so those pairs are never returned (docs/dev/faces.md §9).
pub images: &'a [u64],
/// Which faces may be compared *against* — the gallery
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]).
+232
View File
@@ -0,0 +1,232 @@
//! TRACES: FR-CULL-10 | NFR-P9
//! Which of a person's faces stand for them in a grouping pass.
//!
//! # Why not all of them
//!
//! Every face the user has ruled on enters [`crate::cluster`] as an anchor,
//! and the pass compares every face against every other
//! ([`crate::neighbours`] is exhaustive by design). So a person with 750
//! confirmed faces costs 750 comparisons against each of the library's other
//! faces, and the cost of naming a library well grows with how well it is
//! named: a fully confirmed library of 25,000 faces spends almost the whole
//! scan re-comparing faces whose identity is already settled against each
//! other.
//!
//! Most of those comparisons say nothing new. A person's confirmed faces are
//! heavily redundant — thirty frames from one afternoon are one point of
//! view, not thirty — and a new face that matches one of them matches the
//! others too. What a new face needs to be measured against is the person's
//! *range*: the angles, ages and lights they have been photographed in, each
//! represented once.
//!
//! # The choice: the most diverse of the good ones
//!
//! Two rules, in order.
//!
//! **Good enough to vouch.** Only faces whose raw embedding was at least
//! [`MIN_REFERENCE_QUALITY`] long are eligible — a stricter floor than the
//! gallery's ([`crate::embedding::MIN_GALLERY_QUALITY`]), because a reference
//! is asked to speak *for* a person rather than merely be admitted to the
//! comparison. A face whose length was never recorded is admitted, as it is
//! everywhere else: a rule that cannot be checked admits rather than excludes.
//!
//! **As far apart as possible.** From the eligible pool, up to
//! [`MAX_REFERENCES`] faces are chosen to maximise the volume they span —
//! the determinant of their Gram matrix — greedily: start from the longest
//! vector, and at each step add the face with the largest component
//! orthogonal to everything chosen so far. That is Gram–Schmidt with a
//! pivot, and the product of the squared residuals it picks *is* the
//! determinant, so the greedy step is the exact greedy on the objective.
//! The effect is that a near-duplicate of a chosen face has almost no
//! residual and is passed over, while the one profile shot among two
//! hundred frontal frames is taken early.
//!
//! What is not chosen still belongs to the person. Those faces keep their
//! confirmations and are not touched by the pass; they are simply not
//! compared, which is the whole saving.
/// The most faces that stand for one person.
///
/// A hundred is far more points of view than a person has. What it bounds
/// is the cost: with every person at the cap, a scan against the named part
/// of a library is `people × 100` comparisons per face rather than
/// `confirmations`, and the two part company as soon as a library is used.
pub const MAX_REFERENCES: usize = 100;
/// The shortest raw embedding that may stand for a person.
///
/// One above the gallery floor: a reference vouches for someone, and the
/// margin keeps the faces that only just cleared the gallery — the ones
/// nearest the middle of the sphere — out of the set that speaks for a
/// person.
pub const MIN_REFERENCE_QUALITY: f32 = 15.0;
/// Whether a face of this quality may stand for a person.
///
/// `None` is "never measured" and is admitted, as in
/// [`crate::embedding::in_gallery`].
pub fn eligible(quality: Option<f32>) -> bool {
quality.is_none_or(|q| q >= MIN_REFERENCE_QUALITY)
}
/// Choose which of one person's faces stand for them.
///
/// `embeddings` and `quality` are one entry per face, the embeddings unit
/// length and all of one dimension. Returns the indices chosen, in the order
/// chosen — the first is the longest eligible vector, and each after it is
/// the one furthest from the span of those before. Every eligible face is
/// returned when there are `max` or fewer of them, so a person under the
/// cap loses nothing.
///
/// Deterministic: equal residuals break on the longer vector, then the lower
/// index, so two devices holding the same faces choose the same references
/// and group the same way (`cluster::clustering_is_deterministic`).
pub fn select(embeddings: &[&[f32]], quality: &[Option<f32>], max: usize) -> Vec<usize> {
debug_assert_eq!(embeddings.len(), quality.len());
let mut pool: Vec<usize> = (0..embeddings.len())
.filter(|&i| eligible(quality[i]))
.collect();
if pool.len() <= max {
return pool;
}
// Longest first, so the seed is the pool's front and a tie on residual
// resolves to the earlier position. A missing reading ranks below any
// measured one for this purpose only: it is admitted, but a face that
// was measured and found long is the better seed.
pool.sort_by(|&a, &b| {
let qa = quality[a].unwrap_or(0.0);
let qb = quality[b].unwrap_or(0.0);
qb.total_cmp(&qa).then(a.cmp(&b))
});
// Residuals: what remains of each pool vector outside the span of the
// chosen ones. Copied, since they are rewritten in place.
let mut residual: Vec<Vec<f32>> = pool.iter().map(|&i| embeddings[i].to_vec()).collect();
let mut taken = vec![false; pool.len()];
let mut chosen = Vec::with_capacity(max);
while chosen.len() < max {
// The face with the most left outside the span. The seed is the
// pool's front by construction: every unit vector has the same
// residual before anything is chosen, up to rounding, and rounding
// is not a reason to prefer one. After that `> best` and not `>=`,
// so a genuine tie keeps the earlier (longer) candidate.
let mut pick = None;
let mut best = 0.0_f32;
if chosen.is_empty() {
pick = Some(0);
best = residual[0].iter().map(|x| x * x).sum();
} else {
for (k, r) in residual.iter().enumerate() {
if taken[k] {
continue;
}
let n2: f32 = r.iter().map(|x| x * x).sum();
if n2 > best {
best = n2;
pick = Some(k);
}
}
}
// Nothing left outside the span: every remaining face is a
// combination of the chosen ones and adds no volume.
let Some(k) = pick.filter(|_| best > 1e-6) else {
break;
};
taken[k] = true;
chosen.push(pool[k]);
// Project the chosen direction out of every remaining residual.
let inv = best.sqrt().recip();
let q: Vec<f32> = residual[k].iter().map(|x| x * inv).collect();
for (j, r) in residual.iter_mut().enumerate() {
if taken[j] {
continue;
}
let d: f32 = r.iter().zip(&q).map(|(a, b)| a * b).sum();
for (x, y) in r.iter_mut().zip(&q) {
*x -= d * y;
}
}
}
chosen
}
#[cfg(test)]
mod tests {
use super::*;
fn unit(v: &[f32]) -> Vec<f32> {
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect()
}
#[test]
fn a_person_under_the_cap_keeps_every_eligible_face() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0]), unit(&[1.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), None, Some(16.0)];
assert_eq!(select(&refs, &q, 100), vec![0, 1, 2]);
}
#[test]
fn a_short_vector_never_stands_for_a_person() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), Some(MIN_REFERENCE_QUALITY - 0.01)];
assert_eq!(select(&refs, &q, 100), vec![0]);
}
/// Two hundred frames from one afternoon and one profile shot: the
/// profile is the second choice, not the two-hundred-and-first.
#[test]
fn the_odd_one_out_is_chosen_before_any_duplicate() {
let mut e: Vec<Vec<f32>> = Vec::new();
let mut q = Vec::new();
for i in 0..200 {
// Near-duplicates of one direction, with a little noise.
let t = (i as f32) * 1e-3;
e.push(unit(&[1.0, t, t * 0.5]));
q.push(Some(20.0 + (i % 7) as f32));
}
e.push(unit(&[0.0, 0.0, 1.0]));
q.push(Some(16.0));
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let chosen = select(&refs, &q, 3);
assert_eq!(chosen.len(), 3);
assert_eq!(
chosen[1], 200,
"the profile shot was not second: {chosen:?}"
);
// Seeded on the longest vector.
assert_eq!(q[chosen[0]], Some(26.0));
}
/// Faces inside the span of the chosen ones add no volume and are not
/// taken to fill the cap.
#[test]
fn the_cap_is_not_filled_from_inside_the_span() {
let e = [
unit(&[1.0, 0.0]),
unit(&[0.0, 1.0]),
unit(&[1.0, 1.0]),
unit(&[2.0, -1.0]),
];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0); 4];
assert_eq!(select(&refs, &q, 3).len(), 2);
}
#[test]
fn the_choice_is_deterministic() {
let e: Vec<Vec<f32>> = (0..50)
.map(|i| {
let a = (i as f32) * 0.37;
unit(&[a.cos(), a.sin(), (a * 3.0).sin(), 0.2])
})
.collect();
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = vec![Some(18.0); 50];
assert_eq!(select(&refs, &q, 5), select(&refs, &q, 5));
}
}
+2 -2
View File
@@ -1,6 +1,6 @@
//! What a frame actually costs — the measurement FR-DSP-2 is waiting on.
//!
//! `docs/display-and-extension.md` §2 argues that tiled computation predates
//! `docs/dev/display-and-extension.md` §2 argues that tiled computation predates
//! the fused-shader design and may not need to exist: the composer folds every
//! active operation into **one dispatch over a viewport-sized target**, so the
//! problem tiles were invented to solve may already be solved. That argument
@@ -28,7 +28,7 @@
//! the per-frame CPU half is dominated by shader-source assembly, which is
//! string formatting and is several times slower unoptimised.
//!
//! The committed numbers live in `docs/frame-budget.md`. Rerun this and diff
//! The committed numbers live in `docs/dev/frame-budget.md`. Rerun this and diff
//! that file; a regression should be a diff rather than somebody's memory.
//!
//! # Why the 99th percentile and not the mean
+1 -1
View File
@@ -1,6 +1,6 @@
//! Segment an image and write the granularity ladder as false-coloured PPMs.
//!
//! The whole point of S15 step 2 (docs/segmentation.md §11): look at the
//! The whole point of S15 step 2 (docs/dev/segmentation.md §11): look at the
//! ladder and decide whether clicking through it would land on the things a
//! person means. No amount of design settles that — the pictures do.
//!
+56
View File
@@ -1898,6 +1898,62 @@ mod tests {
);
}
/// TRACES: FR-DEV-20
#[test]
fn a_keystone_reshapes_the_frame_without_exposing_a_corner() {
// The shader half of perspective correction, end to end. A top-bright
// frame with a full vertical keystone spreads its top across the
// output, so the bright half reaches further down than the middle;
// and since the frame is mapped onto a trapezoid *inside* the source,
// no corner is left without a pixel behind it.
let Some(ctx) = ctx() else { return };
let mut pass = AdjustPass::new(&ctx);
let img = split_image(&ctx, true);
let plain = EditGraph::default_chain().compose();
let tex = pass.render(&img, &plain, 32, 32).expect("render");
let below_middle = read_pixel(&ctx, tex, 16, 19)[0];
assert!(
below_middle < 90,
"unkeyed, row 19 is the dark half: {below_middle}"
);
let mut g = EditGraph::default_chain();
g.set_param(
dr_pipeline::framing::ID,
dr_pipeline::framing::KEYSTONE_V,
100.0,
);
g.set_param(
dr_pipeline::framing::ID,
dr_pipeline::framing::KEYSTONE_H,
100.0,
);
let shader = g.compose();
let tex = pass.render(&img, &shader, 32, 32).expect("render");
for (x, y) in [(0, 0), (31, 0), (0, 31), (31, 31)] {
assert_ne!(
read_pixel(&ctx, tex, x, y),
[0, 0, 0, 255],
"corner ({x},{y}) has no source pixel behind it"
);
}
let mut g = EditGraph::default_chain();
g.set_param(
dr_pipeline::framing::ID,
dr_pipeline::framing::KEYSTONE_V,
100.0,
);
let tex = pass.render(&img, &g.compose(), 32, 32).expect("render");
let keyed = read_pixel(&ctx, tex, 16, 19)[0];
assert!(
keyed > 128,
"the spread top half must reach row 19: {keyed}"
);
}
#[test]
fn dragging_the_crop_does_not_recompile() {
// The cache contract for framing, which is what makes an interactive
+47
View File
@@ -1347,6 +1347,53 @@ mod tests {
}
}
#[test]
fn a_grey_step_edge_stays_grey() {
// A flat patch cannot tell the Malvar kernels from any other set of
// weights that sum to zero. An edge can. A grey vertical step, so
// every photosite records the same profile, must come back with the
// three channels close together on both sides; any spread is false
// colour from interpolating across the edge.
//
// The bound is set by the paper's kernels, which peak at 0.19 here.
// With the ±2 terms of the green-site kernels transposed — the bug
// this test was written against — the peak is 0.375.
let Some(ctx) = ctx() else { return };
let d = Demosaicer::new(&ctx).expect("demosaicer");
let size = 32u32;
let white = 16383u16;
let mut raw = flat_cfa(CfaPattern::Rggb, size, [0, 0, 0], 0, white);
for y in 0..size {
for x in size / 2..size {
raw.data[(y * size + x) as usize] = white;
}
}
let img = d.run(&raw).expect("demosaic");
let px = read_rgba(&ctx, &img);
let (w, _) = img.size();
let mut worst = (0.0f32, 0u32, 0u32);
for y in 2..size - 2 {
for x in 2..size - 2 {
let p = px[(y * w + x) as usize];
let spread = (p[0] - p[1]).abs().max((p[2] - p[1]).abs());
if spread > worst.0 {
worst = (spread, x, y);
}
}
}
assert!(
worst.0 < 0.25,
"false colour of {} at ({}, {}) on a grey edge — the green-site \
kernels are interpolating across the edge",
worst.0,
worst.1,
worst.2
);
}
#[test]
fn output_is_free_of_nan_and_negatives() {
// f16 NaN propagates silently through every later stage; a negative
+6 -13
View File
@@ -470,7 +470,7 @@ impl FocusPeakPass {
// TEXTURE_BINDING to be sampled by the compositor.
// RENDER_ATTACHMENT is not used by anything here and is required
// anyway: Slint rejects an imported texture without it. COPY_SRC
// is for `read_overlay` and its two callers.
// is for `read_overlay` and the tests that call it.
usage: wgpu::TextureUsages::STORAGE_BINDING
| wgpu::TextureUsages::TEXTURE_BINDING
| wgpu::TextureUsages::RENDER_ATTACHMENT
@@ -490,18 +490,11 @@ impl FocusPeakPass {
/// TRACES: AC-8
/// Copy the overlay to the CPU, as RGBA8 rows with no padding.
///
/// **Two callers, and neither is the desktop display path.** The tests
/// below are one: an overlay is a claim about which pixels are sharp, and
/// there is no way to check that claim without looking at the pixels. The
/// other is the Android develop view, which reads the *frame* back for the
/// reasons `technical-debt.md` TD-1 records — wgpu's Android swapchain
/// tears a portrait window, so Slint is not drawing with wgpu there and no
/// texture can be handed over. An overlay that stayed on the device on a
/// platform where the picture underneath it does not would simply never be
/// seen.
///
/// On desktop nothing calls this, and ARCH §6.1 holds on the path that
/// matters: the overlay reaches the compositor as a texture.
/// **The tests below are the only caller, and never the display path.** An
/// overlay is a claim about which pixels are sharp, and there is no way to
/// check that claim without looking at the pixels. On screen, on desktop
/// and Android alike, the overlay reaches the compositor as a texture and
/// ARCH §6.1 holds. (Android read it back here until TD-1 was paid off.)
pub fn read_overlay(&self) -> Result<(Vec<u8>, u32, u32), GpuError> {
let Some(layer) = self.layers[self.current].as_ref() else {
return Err(GpuError::Readback("no overlay has been rendered".into()));
+1 -1
View File
@@ -6,7 +6,7 @@
//! module doc said for eight releases that it held no pipeline and no masks.
//! It holds both now, plus demosaic, detail, segmentation masks, two
//! histograms and focus peaking. The zero-copy claim is still the one that
//! matters, and TD-1 records the one platform where it does not hold.
//! matters, and since TD-1 was paid off it holds on Android too.
//!
//! Deliberately free of UI dependencies (ARCH §6.5a). The texture is handed
//! out as a `wgpu::Texture`; who composites it is not this crate's concern.
+13
View File
@@ -395,6 +395,7 @@ pub struct MaskPass {
combine_layout: wgpu::BindGroupLayout,
combine_union: wgpu::RenderPipeline,
combine_subtract: wgpu::RenderPipeline,
combine_intersect: wgpu::RenderPipeline,
/// Where a part is drawn before it is joined.
///
/// One texture for the whole stack rather than one per layer, because
@@ -651,6 +652,16 @@ impl MaskPass {
"mask-combine-subtract",
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::OneMinusSrc),
);
// TRACES: FR-DEV-19a
// `dst * src`: what the mask had, kept only in proportion to how much
// of it this part also covers. The same three vertices and the same
// scratch, so a third set operation is a third blend state and
// nothing more — which is what `Join::apply` states on the CPU and
// `the_joins_match_their_definition` holds this to.
let combine_intersect = combine(
"mask-combine-intersect",
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::Src),
);
// The same, with the deposit thrown away: coverage is only ever taken
// off what earlier strokes on this layer put down. There is no negative
@@ -681,6 +692,7 @@ impl MaskPass {
combine_layout,
combine_union,
combine_subtract,
combine_intersect,
scratch: None,
array: None,
allocations: 0,
@@ -1376,6 +1388,7 @@ impl MaskPass {
pass.set_pipeline(match join {
Join::Union => &self.combine_union,
Join::Subtract => &self.combine_subtract,
Join::Intersect => &self.combine_intersect,
});
pass.set_bind_group(0, &bind_group, &[]);
pass.draw(0..3, 0..1);
+7 -7
View File
@@ -1,4 +1,4 @@
//! Watershed segmentation — arm A's GPU half (S15, docs/segmentation.md).
//! Watershed segmentation — arm A's GPU half (S15, docs/dev/segmentation.md).
//!
//! Runs the five passes in `shaders/watershed.wgsl` over a demosaiced image
//! and leaves a basin label per pixel on the GPU. The hierarchy built from
@@ -20,7 +20,7 @@
//! one AC-8 forbids is per frame in the render loop, and sharing a switch
//! would force a build wanting local masking to unlock the other.
//!
//! It is still a real cost and still unfinished. F3 in docs/segmentation.md
//! It is still a real cost and still unfinished. F3 in docs/dev/segmentation.md
//! §12 stands: the adjacency accumulation belongs GPU-side with atomics, and
//! until it moves there every segmentation pays a full-resolution transfer.
//! Read the feature name as a description of a known gap rather than as
@@ -36,7 +36,7 @@ pub struct SegmentOptions {
/// Longest proxy edge. The segmentation runs here, not at sensor
/// resolution: a 24 MP watershed costs 12× the memory to place boundaries
/// a person cannot see, and the boundary refinement that matters at 1:1
/// is a separate stage (docs/segmentation.md §4).
/// is a separate stage (docs/dev/segmentation.md §4).
pub max_edge: u32,
/// Pre-smoothing radius in proxy pixels. The caller's to raise with ISO —
/// this is the single knob that decides whether a noisy file segments
@@ -69,7 +69,7 @@ impl Default for SegmentOptions {
w_chroma: 0.5,
// **Zero: the pass is off.** It is implemented, dispatched
// correctly and measurably changes nothing — see the ignored test
// below and §12 of docs/segmentation.md. Until that is understood,
// below and §12 of docs/dev/segmentation.md. Until that is understood,
// running it would buy 64 dispatches per segmentation and no
// improvement, so the default declines to pay.
plateau_iterations: 0,
@@ -486,7 +486,7 @@ impl Segmentation {
/// a region graph of a few thousand nodes that every later interaction
/// reads from the CPU anyway.
///
/// What it is *not* is finished. F3 in docs/segmentation.md §12 stands:
/// What it is *not* is finished. F3 in docs/dev/segmentation.md §12 stands:
/// the adjacency accumulation belongs on the GPU with atomics, and until
/// it moves there a segmentation costs one full-resolution transfer of the
/// label and gradient buffers. That is a real cost on a phone and the
@@ -724,7 +724,7 @@ mod tests {
px
}
#[test]
#[ignore = "the plateau pass is a measured no-op; see docs/segmentation.md §12"]
#[ignore = "the plateau pass is a measured no-op; see docs/dev/segmentation.md §12"]
fn lower_completion_drains_a_plateau_instead_of_shattering_it() {
// F1, asserted rather than eyeballed, and asserted at the level where
// it matters.
@@ -741,7 +741,7 @@ mod tests {
// with no exit anywhere — cannot be drained by a distance that has
// nowhere to descend to, and collapsing it fully would need connected
// component labelling rather than a local rule. It is not worth it:
// see docs/segmentation.md §12.
// see docs/dev/segmentation.md §12.
let Some(ctx) = ctx() else { return };
let (w, h) = (96u32, 96u32);
let src = DemosaicedImage::from_rgba8(&ctx, &ramp(w, h), w, h).expect("source");
+10 -4
View File
@@ -160,12 +160,18 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
// Green is measured. Red and blue are interpolated from their own
// axis, with a correction from the green Laplacian.
//
// Malvar "G at R/B locations" kernels, transposed per axis:
// chroma along the row: (5c + 4(w1+e1) - (nw+ne+sw+se) - (n2+s2) + 0.5(w2+e2)) / 8
// Malvar "R at green in R row" kernel, and its transpose:
// chroma along the row: (5c + 4(w1+e1) - (nw+ne+sw+se) - (w2+e2) + 0.5(n2+s2)) / 8
//
// The -1 goes on the two greens *along* the chroma axis and the +0.5
// on the pair across it. Transposed, both kernels still sum to zero
// and reconstruct a flat patch exactly, but on an edge the correction
// at green sites is half strength and the false colour doubles: a
// blue/yellow zipper around every clipped highlight.
let along_row =
(5.0 * c + 4.0 * (w1 + e1) - diag1 - vert2 + 0.5 * horiz2) * 0.125;
(5.0 * c + 4.0 * (w1 + e1) - diag1 - horiz2 + 0.5 * vert2) * 0.125;
let along_col =
(5.0 * c + 4.0 * (n1 + s1) - diag1 - horiz2 + 0.5 * vert2) * 0.125;
(5.0 * c + 4.0 * (n1 + s1) - diag1 - vert2 + 0.5 * horiz2) * 0.125;
let red_horizontal = red_is_horizontal(gid.x, gid.y);
let r = select(along_col, along_row, red_horizontal);
+2 -2
View File
@@ -1,4 +1,4 @@
// Watershed segmentation — the passes behind arm A of S15 (docs/segmentation.md).
// Watershed segmentation — the passes behind arm A of S15 (docs/dev/segmentation.md).
//
// Seven entry points forming one chain:
//
@@ -197,7 +197,7 @@ fn gradient(@builtin(global_invocation_id) gid: vec3<u32>) {
// lowest-indexed neighbour, which is up and to the left. Each pixel therefore
// walks diagonally until it falls off the plateau, and one flat region becomes
// a fan of diagonal chains rather than one basin — visible as hatching across
// what should be a single area (docs/segmentation.md §12, F1).
// what should be a single area (docs/dev/segmentation.md §12, F1).
//
// The fix is the standard lower-completion: give each plateau pixel its
// geodesic distance to the nearest pixel that *does* have a lower neighbour,
+5 -5
View File
@@ -2,11 +2,11 @@
//!
//! FR-DSP-3 says a slider updates the visible region within one frame budget at
//! proxy resolution. Until this file existed nothing checked it, which made it
//! a wish — `docs/display-and-extension.md` §3 is blunt about that, and §7 is
//! a wish — `docs/dev/display-and-extension.md` §3 is blunt about that, and §7 is
//! blunt about what tagging an unchecked requirement does to the coverage
//! figure.
//!
//! The measurements this guards are in [`docs/frame-budget.md`], produced by
//! The measurements this guards are in [`docs/dev/frame-budget.md`], produced by
//! `examples/frame_budget.rs`. This file is the part of them that has to keep
//! being true: it renders the **whole point-operation chain** through the real
//! `render_detailed` for a hundred frames, moving a slider between each, and
@@ -17,7 +17,7 @@
//! **The neighbourhood stage is deliberately not in the asserted chain.** It is
//! over the budget today — clarity alone is 34 ms at 4K, because its kernel is
//! a fraction of the frame and reaches a 52-pixel radius there — and
//! `docs/frame-budget.md` records that, names the fix (a base computed at
//! `docs/dev/frame-budget.md` records that, names the fix (a base computed at
//! reduced resolution) and does not pretend otherwise. Asserting a budget the
//! code does not meet would produce a red suite that everyone learns to ignore;
//! asserting it on a chain that quietly excluded the expensive stage *without
@@ -81,7 +81,7 @@ const SOURCE: (u32, u32) = (6000, 4000);
/// The viewport the budget is asserted at: a 16:10 desktop display.
///
/// Not 4K, and the reason is worth stating. At 4K the fused chain still passes
/// with room to spare (4.5 ms of GPU; see `docs/frame-budget.md`), but a test
/// with room to spare (4.5 ms of GPU; see `docs/dev/frame-budget.md`), but a test
/// that renders 8.3 M pixels a hundred times twice over is four seconds of
/// suite time to re-establish a conclusion 4.1 M pixels already establishes.
const VIEWPORT: (u32, u32) = (2560, 1600);
@@ -166,7 +166,7 @@ impl Run {
judged <= BUDGET_MS,
"{case} at {}x{}: p99 of {FRAMES} frames was {judged:.2} ms, over the \
{BUDGET_MS:.0} ms budget (cpu {:.2} ms, gpu {:.2} ms, total {:.2} ms). \
FR-DSP-3 is what this violates; docs/frame-budget.md holds the \
FR-DSP-3 is what this violates; docs/dev/frame-budget.md holds the \
numbers it used to be.",
viewport.0,
viewport.1,
+113
View File
@@ -790,6 +790,119 @@ fn the_order_parts_are_joined_in_is_the_mask() {
);
}
/// TRACES: FR-DEV-19a
/// The truth table `docs/dev/mask-editing.md` §13 asks for: one base, one
/// part that half-covers it, joined each of the three ways. The base is the
/// left half of the frame and the part a dab in the middle, so the four
/// quarters of the table are four pixels.
#[test]
fn a_part_unioned_subtracted_and_intersected_gives_the_three_fields() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let field = split_field(&ctx);
let joined = |join: Join| {
let mut layer = brighten(MaskSource::Regions {
signature: 1,
level: 2,
ids: vec![0],
});
assert!(layer.push_part(MaskPart::painted("p2", join)));
paint(&mut layer, 1, false, &[(0.5, 0.5)]);
let mut stack = MaskStack::new();
stack.push(layer);
render(&ctx, &stack, Some(&field))
};
// (base, part): left outside the dab, left inside, right inside, right
// outside.
let cells = [(4, 16), (14, 16), (18, 16), (27, 16)];
let lit = |pixels: &[u8]| cells.map(|(x, y)| luma_at(pixels, x, y) > 200);
assert_eq!(
lit(&joined(Join::Union)),
[true, true, true, false],
"union: either"
);
assert_eq!(
lit(&joined(Join::Subtract)),
[true, false, false, false],
"subtract: the base without the dab"
);
assert_eq!(
lit(&joined(Join::Intersect)),
[false, true, false, false],
"intersect: only where both are"
);
}
/// TRACES: FR-DEV-19a
/// Intersection on soft coverage is the product `Join::apply` defines, on
/// either side of the join: a gradient intersected with a region it fills is
/// the gradient there and nothing elsewhere, and a region intersected with a
/// gradient is the gradient wherever the region is.
#[test]
fn the_joins_match_their_definition() {
let Some(ctx) = ctx() else {
eprintln!("no adapter; skipping");
return;
};
let field = split_field(&ctx);
let ramp = || MaskSource::Linear {
centre: (0.5, 0.5),
angle: 0.0,
width: 1.0,
};
let right_half = || MaskSource::Regions {
signature: 1,
level: 2,
ids: vec![1],
};
let draw = |layer: MaskLayer| {
let mut stack = MaskStack::new();
stack.push(layer);
render(&ctx, &stack, Some(&field))
};
let alone = draw(brighten(ramp()));
let mut ramp_then_region = brighten(ramp());
assert!(ramp_then_region.push_part(MaskPart::new("p2", Join::Intersect, right_half())));
let ramp_then_region = draw(ramp_then_region);
let mut region_then_ramp = brighten(whole_frame());
assert!(region_then_ramp.push_part(MaskPart::new("p2", Join::Intersect, ramp())));
let region_then_ramp = draw(region_then_ramp);
for x in 0..SIZE {
let y = SIZE / 2;
let want = luma_at(&alone, x, y);
// dst · 1 = dst on the right; dst · 0 = 0 on the left. Pixels
// within two of the seam are left out: the region's own edge
// is soft there, so neither side of the table is 0 or 1.
let got = luma_at(&ramp_then_region, x, y);
if x.abs_diff(SIZE / 2) <= 2 {
// The seam.
} else if x > SIZE / 2 {
assert!(
got.abs_diff(want) <= 1,
"x={x}: the ramp survives where the region is ({got} vs {want})"
);
} else {
assert_eq!(got, 128, "x={x}: and nothing survives where it is not");
}
// 1 · src = src everywhere.
let got = luma_at(&region_then_ramp, x, y);
assert!(
got.abs_diff(want) <= 1,
"x={x}: a full base intersected with the ramp is the ramp ({got} vs {want})"
);
}
}
// --- seeing the mask (FR-DEV-19c) ------------------------------------------
/// A radial that covers the middle of the frame and nothing near the corners.
+1 -1
View File
@@ -326,7 +326,7 @@ fn a_proxy_and_an_export_agree_about_the_effect() {
#[test]
fn crossing_the_reduction_threshold_does_not_change_the_picture() {
// TRACES: FR-DSP-3 — `docs/technical-debt.md` TD-4, held in pixels.
// TRACES: FR-DSP-3 — `docs/dev/technical-debt.md` TD-4, held in pixels.
//
// Clarity's base is computed on a reduced grid, and how reduced depends on
// the viewport: `LocalContrast::reduction` steps 4 -> 2 -> 1 as sigma
+1 -1
View File
@@ -10,7 +10,7 @@
//! code path — the zoom is the full-resolution path — which is why the
//! requirement has been satisfied for some time without anyone tagging it.
//!
//! `docs/display-and-extension.md` §7 is the reason this file exists rather
//! `docs/dev/display-and-extension.md` §7 is the reason this file exists rather
//! than a tag on `framing.rs`: a requirement counts as covered when a `TRACES`
//! comment names it, and nothing checks that the code under the tag does the
//! thing. `FR-DEV-8` is tagged against plumbing a future operation would use.
+10 -2
View File
@@ -6,7 +6,7 @@ rust-version.workspace = true
license.workspace = true
# The one crate that names a runtime, a provider, a vendor library or a
# device (docs/inference.md §8). `dr-face` and `dr-segment` ask it for a
# device (docs/dev/inference.md §8). `dr-face` and `dr-segment` ask it for a
# session by role and never see which of these answered.
[dependencies]
@@ -28,7 +28,10 @@ ort-sys = { version = "2.0.0-rc.13", default-features = false, features = ["disa
# The NVIDIA rungs exist on the desktop only. These features add `ort`'s
# option builders and nothing else — no linking under `alternative-backend` —
# but an Android binary has no business carrying even the option names, and
# the packaging must never be tempted to (§2, §3.1).
# the packaging must never be tempted to (§2, §3.1). The AMD rung needs no
# feature: MIGraphX is registered through the runtime's generic key/value
# entry point (`session::migraphx`), because `ort`'s own builder cannot
# name the compiled-program cache.
[target.'cfg(not(target_os = "android"))'.dependencies]
ort = { workspace = true, features = ["cuda", "tensorrt"] }
@@ -42,3 +45,8 @@ default = ["tract"]
tract = ["dep:ort-tract"]
# Look for `libonnxruntime` on disk and hand its table to `ort`.
native = ["dep:libloading", "dep:ort-sys"]
[dev-dependencies]
# The `ep_probe` example prints the provider's own diagnostics, which is most
# of what a failed rung tells you.
env_logger.workspace = true
@@ -0,0 +1,207 @@
//! Time each execution provider a runtime offers, on the models this
//! repository ships — the measurement docs/inference.md §1 requires before a
//! rung is added to §2's ladder.
//!
//! DARKROOM_ORT_DIR=/usr/lib \
//! cargo run --release -p dr-inference-engine --features native,tract \
//! --example ep_probe -- models/face/scrfd_500m_640.onnx ...
//!
//! Prints one row per (model, provider): the median of timed runs after
//! warm-ups, and the build time, which for a compiling provider is the
//! number that decides whether it needs an engine cache. MIGraphX is built
//! twice per precision — cold, then again from the cache it just wrote —
//! so both numbers are on the page.
//!
//! The ROCm provider is not in the list: ONNX Runtime removed it in 1.23,
//! and 1.29's `onnxruntime-rocm` ships `libonnxruntime_providers_migraphx.so`
//! and nothing else for AMD.
use std::path::{Path, PathBuf};
use std::time::Instant;
#[derive(Clone, Copy, PartialEq)]
enum Ep {
Cpu,
MiGraphX,
MiGraphXFp16,
}
impl Ep {
fn label(self) -> &'static str {
match self {
Ep::Cpu => "CPU",
Ep::MiGraphX => "MIGraphX f32",
Ep::MiGraphXFp16 => "MIGraphX fp16",
}
}
}
fn build(ep: Ep, bytes: &[u8], threads: usize, cache: &Path) -> ort::Result<ort::session::Session> {
let mut b = ort::session::Session::builder()?.with_intra_threads(threads)?;
match ep {
Ep::Cpu => {}
Ep::MiGraphX => migraphx(&mut b, false, &cache.join("f32"))?,
Ep::MiGraphXFp16 => migraphx(&mut b, true, &cache.join("fp16"))?,
}
b.commit_from_memory(bytes)
}
/// Register MIGraphX through the generic key/value API. `ort`'s own
/// builder fills the legacy `OrtMIGraphXProviderOptions`, which 1.29 reads
/// for its precision flags and nothing else: the model cache directory —
/// the difference between a 40 s load and a 0.3 s one — only travels this
/// way. The cache key is the graph, the GPU and the MIGraphX version, not
/// the precision, so each precision gets its own directory.
fn migraphx(
b: &mut ort::session::builder::SessionBuilder,
fp16: bool,
cache: &Path,
) -> ort::Result<()> {
use ort::AsPointer;
use std::ffi::CString;
std::fs::create_dir_all(cache).map_err(|e| ort::Error::new(e.to_string()))?;
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
let values = [
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
CString::new(cache.to_string_lossy().as_bytes()).unwrap(),
];
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
// SAFETY: the documented C call, over arrays that outlive it; the
// runtime copies the strings into its own options map.
unsafe {
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
b.ptr_mut(),
c"MIGraphX".as_ptr(),
key_ptrs.as_ptr(),
value_ptrs.as_ptr(),
keys.len(),
);
ort::Error::result_from_status(status)
}
}
/// Median of `runs` timed runs over zeros, in milliseconds, after warm-ups.
fn time(session: &mut ort::session::Session, warmups: usize, runs: usize) -> Result<f64, String> {
let shape: Vec<usize> = session.inputs()[0]
.dtype()
.tensor_shape()
.ok_or("input is not a tensor")?
.iter()
.map(|&d| if d > 0 { d as usize } else { 1 })
.collect();
let zeros = vec![0f32; shape.iter().product()];
let once = |s: &mut ort::session::Session| -> Result<f64, String> {
let input = ort::value::Tensor::from_array((shape.clone(), zeros.clone()))
.map_err(|e| e.to_string())?;
let t = Instant::now();
let out = s.run(ort::inputs![input]).map_err(|e| e.to_string())?;
let _ = out[0]
.try_extract_tensor::<f32>()
.map_err(|e| e.to_string())?;
Ok(t.elapsed().as_secs_f64() * 1e3)
};
for _ in 0..warmups {
once(session)?;
}
let mut times = Vec::with_capacity(runs);
for _ in 0..runs {
times.push(once(session)?);
}
times.sort_by(|a, b| a.partial_cmp(b).unwrap());
Ok(times[times.len() / 2])
}
fn first_line(s: &str) -> String {
s.lines().next().unwrap_or("").chars().take(120).collect()
}
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let models: Vec<PathBuf> = std::env::args_os().skip(1).map(PathBuf::from).collect();
if models.is_empty() {
eprintln!("usage: ep_probe MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
}
dr_inference_engine::ensure_runtime();
let runtime = dr_inference_engine::status().runtime;
println!("runtime: {}", runtime.label());
if !runtime.is_native() {
println!("(tract: no provider to compare; set DARKROOM_ORT_DIR)");
}
let threads = std::thread::available_parallelism()
.map(|n| n.get().saturating_sub(2).max(1))
.unwrap_or(1);
println!("intra-op threads: {threads}");
let cache = std::env::temp_dir().join("darkroom-ep-probe");
let _ = std::fs::remove_dir_all(&cache);
println!("compiled-program cache: {}\n", cache.display());
println!(
"{:<28} {:<15} {:>10} {:>10}",
"model", "provider", "build s", "median ms"
);
for model in &models {
let bytes = match std::fs::read(model) {
Ok(b) => b,
Err(e) => {
println!("{:<28} read failed: {e}", name(model));
continue;
}
};
// A compiling provider is built twice: the second build reads the
// program the first wrote, and its time is what a launch after the
// first costs.
let plan = [
(Ep::Cpu, false),
(Ep::MiGraphX, false),
(Ep::MiGraphX, true),
(Ep::MiGraphXFp16, false),
(Ep::MiGraphXFp16, true),
];
for (ep, cached) in plan {
let started = Instant::now();
match build(ep, &bytes, threads, &cache) {
Ok(mut session) => {
let built = started.elapsed().as_secs_f64();
match time(&mut session, 3, 15) {
Ok(ms) => println!(
"{:<28} {:<15} {:>10.1} {:>10.1}{}",
name(model),
ep.label(),
built,
ms,
if cached { " (from cache)" } else { "" }
),
Err(e) => println!(
"{:<28} {:<15} {:>10.1} {:>10} {}",
name(model),
ep.label(),
built,
"ran ✗",
first_line(&e)
),
}
}
Err(e) => println!(
"{:<28} {:<15} {:>21} {}",
name(model),
ep.label(),
"build ✗",
first_line(&e.to_string())
),
}
}
println!();
}
}
fn name(p: &Path) -> String {
p.file_name()
.unwrap_or(p.as_os_str())
.to_string_lossy()
.into_owned()
}
+100
View File
@@ -0,0 +1,100 @@
//! Walk the ladder as the app does — probe, engines, then a session — and
//! say what each step chose. The M5 check of docs/inference.md §6 without
//! the app around it.
//!
//! DARKROOM_ORT_DIR=/usr/lib \
//! cargo run --release -p dr-inference-engine --features native,tract \
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [MODEL.onnx ...]
//!
//! Every model named is a `Detector` for the config's purposes, which is
//! enough to see the rung taken, the engines compiled and a session land
//! on it. Delete `CACHE_DIR` to see the first run again; keep it to see the
//! second.
use std::path::PathBuf;
use std::time::{Duration, Instant};
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let mut args = std::env::args_os().skip(1).map(PathBuf::from);
let (Some(cache_dir), models) = (args.next(), args.collect::<Vec<_>>()) else {
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
};
if models.is_empty() {
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
}
let runtime_dirs: Vec<PathBuf> = std::env::var_os("DARKROOM_ORT_DIR")
.map(PathBuf::from)
.into_iter()
.collect();
let started = Instant::now();
dr_inference_engine::init(dr_inference_engine::Config {
runtime_dirs,
cache_dir: cache_dir.clone(),
models: models
.iter()
.map(|p| (dr_inference_engine::Role::Detector, p.clone()))
.collect(),
embedded: Vec::new(),
ceiling: None,
threads: 0,
decay: Duration::ZERO,
});
let mut last = String::new();
loop {
let s = dr_inference_engine::status();
let line = format!(
"{} · {} · engines {}/{}{}",
s.line(),
if s.probing {
"probing"
} else {
s.reason.as_str()
},
s.engines.0,
s.engines.1,
if s.failed.is_empty() {
String::new()
} else {
format!(
" · tried {}",
s.failed
.iter()
.map(|(r, why)| format!("{}: {why}", r.label()))
.collect::<Vec<_>>()
.join(" · ")
)
}
);
if line != last {
println!("{:>6.1} s {line}", started.elapsed().as_secs_f64());
last = line;
}
if !s.probing && s.engines.0 >= s.engines.1 {
break;
}
std::thread::sleep(Duration::from_millis(500));
}
for path in &models {
let bytes = std::fs::read(path).expect("read model");
let t = Instant::now();
let model = dr_inference_engine::open(
dr_inference_engine::Role::Detector,
dr_inference_engine::Form::F32,
&bytes,
)
.expect("open model");
let acquired = model.acquire().expect("acquire session");
println!(
"{} on {} in {:.2} s",
path.file_name().unwrap().to_string_lossy(),
acquired.rung().label(),
t.elapsed().as_secs_f64()
);
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! The API table `ort` runs on, chosen once (docs/inference.md §3).
//! The API table `ort` runs on, chosen once (docs/dev/inference.md §3).
//!
//! `ort` with `alternative-backend` links no runtime and asks, on first use,
//! for an `OrtApi` — a struct of function pointers. Two things can fill it:
+1 -1
View File
@@ -1,5 +1,5 @@
//! Compiled engines: what a rung builds once per device, and the thread that
//! builds them before anyone asks (docs/inference.md §5, §6).
//! builds them before anyone asks (docs/dev/inference.md §5, §6).
//!
//! TensorRT keeps its own engine cache keyed by graph hash; QNN writes a
//! context model. Both are opaque to this crate, which tracks only *that* a
+58 -13
View File
@@ -1,10 +1,10 @@
//! Which runtime, which provider and which model form — decided once per
//! device, and the only crate that knows the answer (docs/inference.md).
//! device, and the only crate that knows the answer (docs/dev/inference.md).
//!
//! Consumers ask for a session by [`Role`] and get `ort`'s `Session` back;
//! what built it — tract on one core, ONNX Runtime's CPU pool, a TensorRT
//! engine, the Hexagon — is this crate's business and shows up in
//! [`status`] for the settings row and nowhere else.
//! engine, a MIGraphX program, the Hexagon — is this crate's business and
//! shows up in [`status`] for the settings row and nowhere else.
//!
//! The shape follows §3 of the spec: `ort` links nothing (`alternative-backend`),
//! and the first call hands it an API table from either a `libonnxruntime`
@@ -35,13 +35,13 @@ pub enum Role {
Embedder,
Segmenter,
Scene,
/// The dense landmark model behind the eye reading (docs/faces.md §7c).
/// The dense landmark model behind the eye reading (docs/dev/faces.md §7c).
Landmarks,
/// The eye-state and sunglasses classifiers, a few hundred kilobytes.
EyeClassifier,
/// XFeat, the panorama keypoint detector (docs/panorama.md).
/// XFeat, the panorama keypoint detector (docs/dev/panorama.md).
Keypoints,
/// MI-GAN, the panorama border filler (docs/panorama.md §12). Plain
/// MI-GAN, the panorama border filler (docs/dev/panorama.md §12). Plain
/// convolutions, so any rung serves it; fp16 on TensorRT and int8 on
/// the Hexagon are the point of it.
Inpainter,
@@ -60,7 +60,9 @@ pub enum Form {
/// A rung of the ladder (§2). Ordered: a user override names the highest rung
/// the probe may take, and a compiling rung falls back to the one below it
/// until its engine exists.
/// until its engine exists. The order is within a vendor's ladder — a
/// machine has NVIDIA rungs or an AMD rung, never both — so a ceiling is
/// read as "no higher than this on whichever ladder the device has".
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
pub enum Rung {
/// ONNX Runtime's CPU provider, or tract when no runtime file was found.
@@ -69,6 +71,11 @@ pub enum Rung {
Cuda,
/// NVIDIA, through a TensorRT engine compiled on this device. Desktop only.
TensorRt,
/// AMD, through a MIGraphX program compiled on this device. Desktop
/// only. ONNX Runtime's ROCm provider, the CUDA provider's twin, was
/// removed in ONNX Runtime 1.23, so there is no non-compiling AMD rung
/// to fall back to: this one falls back to the CPU.
MiGraphX,
/// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only.
Hexagon,
}
@@ -79,6 +86,7 @@ impl Rung {
Rung::Cpu => "CPU",
Rung::Cuda => "CUDA",
Rung::TensorRt => "TensorRT",
Rung::MiGraphX => "MIGraphX",
Rung::Hexagon => "Hexagon NPU",
}
}
@@ -88,13 +96,13 @@ impl Rung {
fn fallback(self) -> Rung {
match self {
Rung::TensorRt => Rung::Cuda,
Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
Rung::MiGraphX | Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
}
}
/// Whether a session on this rung needs an engine built first.
fn compiles(self) -> bool {
matches!(self, Rung::TensorRt | Rung::Hexagon)
matches!(self, Rung::TensorRt | Rung::MiGraphX | Rung::Hexagon)
}
/// The model form this rung wants for a role.
@@ -155,6 +163,8 @@ pub struct Status {
/// Engines compiled and engines wanted, for a compiling rung; `(0, 0)`
/// otherwise.
pub engines: (usize, usize),
/// Every rung above the selected one that was tried, and why it lost.
pub failed: Vec<(Rung, String)>,
}
impl Status {
@@ -162,7 +172,7 @@ impl Status {
pub fn line(&self) -> String {
let form = match self.rung {
Rung::Hexagon => " · int8",
Rung::TensorRt => " · fp16",
Rung::TensorRt | Rung::MiGraphX => " · fp16",
_ => "",
};
format!("{}{} · {}", self.rung.label(), form, self.runtime.label())
@@ -267,8 +277,8 @@ fn acquire(role: Role, form: Form, bytes: &Arc<[u8]>, hash: u64) -> Result<Acqui
return Ok(Acquired { entry });
}
// Built outside the registry lock: a TensorRT engine load is long enough
// that another role's acquire should not wait on it.
// Built outside the registry lock: a TensorRT or MIGraphX engine load
// is long enough that another role's acquire should not wait on it.
let session = session::build(rung, role, bytes, &cfg)?;
log::debug!("inference: {role:?} loaded on {}", rung.label());
let entry = Arc::new(Loaded {
@@ -399,6 +409,16 @@ pub fn status() -> Status {
runtime: api::runtime(),
rung,
reason: s.cache.reason.clone(),
// Only what explains the selection: on an AMD machine the NVIDIA
// rungs "not enabled in this build" say nothing about why MIGraphX
// was taken. With the floor selected, everything tried is above it.
failed: s
.cache
.failed
.iter()
.filter(|(r, _)| *r > rung)
.cloned()
.collect(),
probing: s.probing,
engines: if rung.compiles() {
(s.cache.compiled.len(), s.wanted)
@@ -496,7 +516,7 @@ mod tests {
/// The smallest shipped graph, if this checkout has the weights; a test
/// suite that needs a research-licensed download is one that does not
/// run in CI (docs/faces.md §3), so absence is a skip.
/// run in CI (docs/dev/faces.md §3), so absence is a skip.
fn probe_bytes() -> Option<Vec<u8>> {
let path = concat!(
env!("CARGO_MANIFEST_DIR"),
@@ -611,8 +631,33 @@ mod tests {
);
}
#[test]
fn the_status_reports_only_the_rungs_above_the_selection() {
let _serial = serial();
let failed = vec![
(Rung::TensorRt, "not enabled".to_string()),
(Rung::Cuda, "not enabled".to_string()),
];
let before = state().lock().unwrap().cache.clone();
state().lock().unwrap().cache = Cache {
rung: Some(Rung::MiGraphX),
failed: failed.clone(),
..Cache::default()
};
// An AMD desktop: the NVIDIA rungs below MIGraphX are not the story.
assert!(status().failed.is_empty());
// An NVIDIA desktop on the CUDA provider: TensorRT's failure is.
state().lock().unwrap().cache.rung = Some(Rung::Cuda);
assert_eq!(status().failed, vec![failed[0].clone()]);
// The floor: everything tried explains it.
state().lock().unwrap().cache.rung = Some(Rung::Cpu);
assert_eq!(status().failed.len(), 2);
state().lock().unwrap().cache = before;
}
#[test]
fn the_status_line_reads_as_the_floor_before_init() {
let _serial = serial();
let s = status();
assert_eq!(s.rung, Rung::Cpu);
assert!(s.line().starts_with("CPU"), "{}", s.line());
+46 -8
View File
@@ -1,4 +1,4 @@
//! Walk the ladder, once, by building real sessions (docs/inference.md §4).
//! Walk the ladder, once, by building real sessions (docs/dev/inference.md §4).
//!
//! A rung is taken when a session builds on it, runs, and is faster than
//! the floor. Both halves matter: a provider can register and then fail at
@@ -16,8 +16,11 @@ use crate::{api::Runtime, state, Cache, Config, Form, Role, Rung};
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
#[cfg(target_os = "android")]
let all = [Rung::Hexagon];
// A desktop has one vendor's GPU; the other vendor's providers are
// "not enabled in this build" or a library that fails to load, and
// either answer arrives in milliseconds.
#[cfg(not(target_os = "android"))]
let all = [Rung::TensorRt, Rung::Cuda];
let all = [Rung::TensorRt, Rung::Cuda, Rung::MiGraphX];
all.into_iter()
.filter(|r| ceiling.is_none_or(|c| *r <= c))
.collect()
@@ -206,15 +209,22 @@ fn first_line(s: &str) -> String {
line[start..].chars().take(200).collect()
}
/// Everything a change of which should re-probe: the runtime and where it
/// came from, this crate, the platform, the driver or SoC, and the models.
/// Everything a change of which should re-probe: the runtime, where it
/// came from and which providers sit beside it, this crate, the platform,
/// the driver or SoC, and the models.
fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
let mut parts = vec![
format!("engine {}", env!("CARGO_PKG_VERSION")),
format!("{} {}", std::env::consts::OS, std::env::consts::ARCH),
match runtime {
Runtime::Tract => "tract".to_string(),
Runtime::OnnxRuntime { path, version } => format!("ort {version} {}", path.display()),
Runtime::OnnxRuntime { path, version } => {
format!(
"ort {version} {} [{}]",
path.display(),
providers_beside(path)
)
}
},
device_identity(),
];
@@ -237,13 +247,41 @@ fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
parts.join("\n")
}
/// The `libonnxruntime_providers_*.so` files in the runtime's directory.
/// A distribution's CPU-only and ROCm builds are the same version at the
/// same path; the provider libraries beside them are what differs.
fn providers_beside(runtime: &Path) -> String {
let Some(dir) = runtime.parent() else {
return String::new();
};
let mut names: Vec<String> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.filter_map(|e| e.ok())
.filter_map(|e| e.file_name().into_string().ok())
.filter(|n| {
n.starts_with("libonnxruntime_providers_") || n.starts_with("onnxruntime_providers_")
})
.collect();
names.sort();
names.join(" ")
}
#[cfg(target_os = "linux")]
fn device_identity() -> String {
// The NVIDIA driver's version line; absent means no NVIDIA driver.
std::fs::read_to_string("/proc/driver/nvidia/version")
// The NVIDIA driver's version line, or the ROCm release the AMD stack
// came from (`rocm-core` writes it; the kernel driver has no version
// of its own). Absent means neither.
if let Some(line) = std::fs::read_to_string("/proc/driver/nvidia/version")
.ok()
.and_then(|s| s.lines().next().map(str::to_string))
.unwrap_or_else(|| "no nvidia driver".into())
{
return line;
}
if let Ok(rocm) = std::fs::read_to_string("/opt/rocm/.info/version") {
return format!("rocm {}", rocm.trim());
}
"no nvidia driver, no rocm".into()
}
#[cfg(target_os = "android")]
+59 -2
View File
@@ -1,4 +1,4 @@
//! One session builder per rung (docs/inference.md §2, §7, §9).
//! One session builder per rung (docs/dev/inference.md §2, §7, §9).
use ort::session::Session;
@@ -83,10 +83,65 @@ fn providers(
ep::CUDA::default().build(),
])?)
}
Rung::MiGraphX => {
// fp16 on the same terms as TensorRT (§7). MIGraphX compiles a
// program per graph — 20–60 s here — and keeps it in the cache
// directory, keyed on the graph, the GPU and its own version
// but not the precision: hence one directory per precision.
// The CPU takes any node it declines.
let fp16 = role != Role::Embedder;
let cache = cfg
.cache_dir
.join("migraphx")
.join(if fp16 { "fp16" } else { "f32" });
let _ = std::fs::create_dir_all(&cache);
let mut b = b;
migraphx(&mut b, fp16, &cache)?;
Ok(b)
}
Rung::Hexagon => unreachable!("the Hexagon rung is not on a desktop ladder"),
}
}
/// Register MIGraphX through ONNX Runtime's generic key/value entry point.
///
/// `ort`'s own builder (`ep::MIGraphX`) fills the legacy
/// `OrtMIGraphXProviderOptions`, and 1.29 reads that struct for its
/// precision flags and nothing else — the compiled-program cache directory
/// is only a key in the generic map (`migraphx_model_cache_dir`), and
/// without it every session is a full compile. Registration through the
/// generic entry point needs no `ort` feature: it is one call on the API
/// table, which is why the crate's `ort` dependency names no AMD feature.
#[cfg(not(target_os = "android"))]
fn migraphx(
b: &mut ort::session::builder::SessionBuilder,
fp16: bool,
cache: &std::path::Path,
) -> ort::Result<()> {
use ort::AsPointer;
use std::ffi::CString;
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
let values = [
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
CString::new(cache.to_string_lossy().as_bytes())
.map_err(|e| ort::Error::new(e.to_string()))?,
];
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
// SAFETY: the documented C call over arrays that outlive it; the
// runtime copies the strings into its own options map before returning.
unsafe {
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
b.ptr_mut(),
c"MIGraphX".as_ptr(),
key_ptrs.as_ptr(),
value_ptrs.as_ptr(),
keys.len(),
);
ort::Error::result_from_status(status)
}
}
#[cfg(target_os = "android")]
fn providers(
b: ort::session::builder::SessionBuilder,
@@ -120,6 +175,8 @@ fn providers(
.build()
.error_on_failure()])?)
}
Rung::Cuda | Rung::TensorRt => unreachable!("no NVIDIA rung on Android"),
Rung::Cuda | Rung::TensorRt | Rung::MiGraphX => {
unreachable!("no desktop GPU rung on Android")
}
}
}
+1 -1
View File
@@ -13,7 +13,7 @@ log.workspace = true
# Inference for the learned keypoint detector, on the same footing as
# `dr-segment`: `ort` is the API, `dr-inference-engine` decides what runs
# it (docs/inference.md), and both are optional so that the geometry —
# it (docs/dev/inference.md), and both are optional so that the geometry —
# matching, the rotation solve, the projections — is a dependency-free crate
# that tests without a model.
ort = { workspace = true, optional = true }
+142 -18
View File
@@ -89,12 +89,19 @@ pub struct Params {
pub coarse: usize,
/// The fine passes' band width.
pub band: usize,
/// How deep into the picture the mirrored context reaches. A plain
/// reflection of a deep hole pulls in whatever is that far from the
/// edge — a ridge, a peak — and the model, told that is what lies
/// beyond, paints it upside down. Folding the reflection within this
/// band keeps the ring looking like the edge it continues (sky beside
/// sky, grass beside grass) and nothing further away.
/// How deep into the picture the mirrored context reaches, or **zero
/// for no mirrored context at all**: the void is then shown to the
/// model as it is — reaching the picture's edge with nothing beyond,
/// and, beyond the band being filled, still unknown. That is what the
/// shipped model was trained on (a fine-tune of MI-GAN on voids cut
/// from photographs the way a cylindrical merge cuts them, see
/// `docs/dev/panorama.md` §14); a ring would give it a fold to continue.
///
/// Non-zero is the stock model's crutch: a plain reflection of a deep
/// hole pulls in whatever is that far from the edge — a ridge, a peak —
/// and the model, told that is what lies beyond, paints it upside down.
/// Folding the reflection within this band keeps the ring looking like
/// the edge it continues and nothing further away.
pub mirror_depth: usize,
/// How far inside the real edge the fill also regenerates, the two
/// blended by distance. A hard cut between real pixels and invented
@@ -108,9 +115,9 @@ pub struct Params {
impl Default for Params {
fn default() -> Self {
Params {
coarse: 4,
band: 96,
mirror_depth: 48,
coarse: 1,
band: 192,
mirror_depth: 0,
feather: 24,
stride: 384,
}
@@ -141,7 +148,10 @@ pub fn fill_border(
} = params;
let q = q.max(1);
let band = band.max(8);
let mirror_depth = mirror_depth.max(1);
// No ring: the void beyond the band stays unknown, as in the model's
// training; with a ring the far side is the coarse fill, presented as
// known, which the stock model needed to see something there.
let open = mirror_depth == 0;
if width == 0 || height == 0 || rgb.len() != width * height * 3 || known.len() != width * height
{
return Err(PanoError::Input("fill: buffer sizes disagree".into()));
@@ -227,7 +237,11 @@ pub fn fill_border(
let mut any = false;
for i in 0..width * height {
let in_band = !known[i] && dist[i] > lo && dist[i] <= hi;
band_known[i] = !in_band;
band_known[i] = if open {
known[i] || dist[i] <= lo
} else {
!in_band
};
any |= in_band;
}
if !any {
@@ -289,7 +303,7 @@ fn fill_once(
}
// The padded canvas with mirrored context, and the hole within it.
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth);
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth, t);
let (pw, ph) = (ctx.width, ctx.height);
// Tiles that touch the hole, on a grid that reaches both far edges.
@@ -369,7 +383,7 @@ fn fill_once(
if known[i] {
continue;
}
let p = (yy + RING) * pw + (xx + RING);
let p = (yy + ctx.ring) * pw + (xx + ctx.ring);
if wsum[p] > 0.0 {
for ch in 0..3 {
rgb[i * 3 + ch] = (acc[p * 3 + ch] / wsum[p]).clamp(0.0, 1.0);
@@ -478,12 +492,45 @@ fn fold(d: usize, depth: usize) -> usize {
struct MirroredContext {
width: usize,
height: usize,
/// The padding on every side: `RING` with mirrored context, 0 without.
ring: usize,
rgb: Vec<f32>,
hole: Vec<bool>,
}
impl MirroredContext {
fn build(rgb: &[f32], width: usize, height: usize, known: &[bool], depth: usize) -> Self {
fn build(
rgb: &[f32],
width: usize,
height: usize,
known: &[bool],
depth: usize,
tile: usize,
) -> Self {
if depth == 0 {
// Open: the picture as it is, the hole as it is. What the hole
// holds does not matter — the model masks it out. A picture
// smaller than a tile (the merge page's preview) sits at the
// origin of a tile-sized canvas whose rest is hole: still the
// void as it is, and the only way a tile fits at all.
let (pw, ph) = (width.max(tile), height.max(tile));
let mut canvas = vec![0.0f32; pw * ph * 3];
let mut hole = vec![true; pw * ph];
for y in 0..height {
canvas[y * pw * 3..(y * pw + width) * 3]
.copy_from_slice(&rgb[y * width * 3..(y + 1) * width * 3]);
for x in 0..width {
hole[y * pw + x] = !known[y * width + x];
}
}
return MirroredContext {
width: pw,
height: ph,
ring: 0,
rgb: canvas,
hole,
};
}
let fold = |d: usize| fold(d, depth);
let (pw, ph) = (width + 2 * RING, height + 2 * RING);
let mut canvas = vec![0.0f32; pw * ph * 3];
@@ -534,6 +581,7 @@ impl MirroredContext {
MirroredContext {
width: pw,
height: ph,
ring: RING,
rgb: canvas,
hole,
}
@@ -634,7 +682,10 @@ mod tests {
200,
&known,
&mut model,
test_params(0),
Params {
mirror_depth: 48,
..test_params(0)
},
&mut |_, _| {},
)
.unwrap();
@@ -648,6 +699,74 @@ mod tests {
}
}
#[test]
fn an_open_void_reaches_the_tile_edge_and_stays_unknown_beyond_the_band() {
// A 150-tall hole above and below; bands of 96. With no ring the
// first band's tiles sit at the picture's edge, so a tile's top
// row is unknown, and the rows deeper than the band are unknown
// too — not "known" coarse fill — exactly as the model was trained.
let (mut rgb, known) = picture(200, 500, 150);
let mut model = Flat {
tile: 64,
seen: Vec::new(),
};
fill_border(
&mut rgb,
200,
500,
&known,
&mut model,
Params {
band: 96,
..test_params(0)
},
&mut |_, _| {},
)
.unwrap();
// The first pass's tile at the picture's top edge is unknown
// through and through: the hole is 150 deep, the tile 64, and
// nothing beyond the band was presented as known. With a ring, or
// with the far side shown as coarse fill, no tile is ever all hole.
assert!(model.seen.iter().any(|(_, k)| k.iter().all(|&v| !v)));
for i in 0..200 * 500 {
if !known[i] {
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
}
}
}
#[test]
fn a_picture_smaller_than_the_tile_is_still_filled_when_the_void_is_open() {
// The merge page's preview is 1600 wide and a few hundred tall —
// shorter than a 512 tile. With no ring the canvas is padded to a
// tile, the padding hole, and the border is still filled.
let (mut rgb, known) = picture(300, 40, 8);
let mut model = Flat {
tile: 64,
seen: Vec::new(),
};
let tiles = fill_border(
&mut rgb,
300,
40,
&known,
&mut model,
test_params(0),
&mut |_, _| {},
)
.unwrap();
assert!(tiles > 0, "no tile fitted a picture shorter than the tile");
for i in 0..300 * 40 {
if !known[i] {
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
}
}
// And the model saw the padding as hole, never as black content.
for (_, k) in &model.seen {
assert_eq!(k.len(), 64 * 64);
}
}
#[test]
fn the_fine_passes_run_in_bands_after_the_coarse_one() {
// A 150-tall hole above and below a picture: the coarse pass sees
@@ -663,7 +782,12 @@ mod tests {
500,
&known,
&mut model,
test_params(0),
Params {
coarse: 4,
band: 96,
mirror_depth: 48,
..test_params(0)
},
&mut |_, _| {},
)
.unwrap();
@@ -752,7 +876,7 @@ mod tests {
#[test]
fn the_context_mirrors_the_top_rows_upward() {
let (rgb, known) = picture(40, 30, 5);
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48);
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48, 64);
let x = RING + 10;
let first = RING + 5;
for k in 1..=4 {
@@ -774,7 +898,7 @@ mod tests {
for x in 0..40 {
rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]);
}
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48);
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48, 64);
let x = RING + 10;
for y in 0..RING + 5 {
let p = (y * ctx.width + x) * 3;
+1 -1
View File
@@ -9,7 +9,7 @@
//! on every rung, and what they cost is the whole story of whether a fill
//! is interactive: 7.4 s a tile under tract, 0.4 s under ONNX Runtime's
//! CPU pool, 23 ms in fp16 and 13 ms in int8 on a laptop's TensorRT
//! (2026-09-19, docs/panorama.md §12).
//! (2026-09-19, docs/dev/panorama.md §12).
//!
//! The model's contract, from the reference `export_inference_model.py`:
//! input `1×4×512×512` float — channel 0 is `mask − 0.5` with 1 where the
+2 -2
View File
@@ -4,7 +4,7 @@
//! Apache-2.0 weights (`models/LICENCE.md`), exported at a fixed shape by
//! `tools/export-xfeat.sh` and loaded through the same `dr-inference-engine`
//! `dr-segment` and `dr-face` use, so this adds no runtime and no C to the
//! tree; what runs it is the device's business (docs/inference.md). ~300 ms
//! tree; what runs it is the device's business (docs/dev/inference.md). ~300 ms
//! per frame on tract on the reference desktop, ~400 ms on the tablet
//! (S15.2, S15.4).
@@ -37,7 +37,7 @@ pub struct XFeat {
}
/// The bytes of both exports compiled into the binary, for whoever compiles
/// engines ahead of the first request (docs/inference.md §6).
/// engines ahead of the first request (docs/dev/inference.md §6).
#[cfg(feature = "embedded-model")]
pub fn embedded_model_bytes() -> [&'static [u8]; 2] {
[EMBEDDED_LANDSCAPE, EMBEDDED_PORTRAIT]
+2 -2
View File
@@ -315,7 +315,7 @@ pub struct DetailPass {
/// 52 render pixels at 4K — holds no spatial frequency a quarter-scale
/// grid cannot represent. Computing it at the render size therefore buys
/// nothing and costs everything: 105 taps over 8.3 M pixels, twice, which
/// measured at 34 ms and is where `docs/technical-debt.md` TD-4 came from.
/// measured at 34 ms and is where `docs/dev/technical-debt.md` TD-4 came from.
/// At a quarter it is a sixteenth of the pixels at a quarter of the
/// radius, and the result is not an approximation of the full-resolution
/// base — it is the same band-limited function, sampled where it is still
@@ -568,7 +568,7 @@ pub fn compose_detail(
/// photograph the photographer thinks they are sharpening.
///
/// It also means ARCH §5.2's stage list, which draws spot removal after
/// texture and clarity, is not what this does — see `docs/spot-removal.md`
/// texture and clarity, is not what this does — see `docs/dev/spot-removal.md`
/// §5.1, which is where the disagreement is written down.
pub fn compose_detail_with(
ops: &[Box<dyn Operation>],
+681 -9
View File
@@ -27,6 +27,33 @@
//! chain exactly the space it documents: normalised, centred, `r == 1` at the
//! corner. Neither stage needs to know the other exists.
//!
//! # Perspective sits inside framing (FR-DEV-20)
//!
//! A keystone correction is composition too — straightening converging
//! verticals reframes the photograph — so it is a step *of* framing rather
//! than a stage beside it, and inherits framing's `Compose` attribute, its
//! place in the sidecar and its exclusion from a default paste. Expanded, the
//! chain reads:
//!
//! ```text
//! output pixel → crop → straighten → perspective → orientation → warp (lens) → sample
//! ```
//!
//! After the straightening, because the angle is a nudge applied to the
//! corrected picture: the verticals are made parallel and *then* the whole is
//! levelled. Before the stored orientation, because "vertical" means vertical
//! in the photograph as it is shown — a portrait frame the camera stored on
//! its side must converge along its displayed height, not along the sensor's
//! rows. And before the lens warp, which still sees the whole frame it
//! corrects, for the reason given above.
//!
//! The correction maps the output frame onto a trapezoid **inside** the
//! source rather than pulling the source edges in. So a keystone on its own
//! never exposes an empty corner, and the crop the user drew is still valid
//! after it; only in combination with a straightening angle does the
//! inscribed crop have anything to account for — see
//! [`Framing::max_inscribed_crop`].
//!
//! # Why sampling changes with the angle
//!
//! At 90° steps and flips, output pixels land exactly on source pixels, so
@@ -58,6 +85,13 @@ pub const CROP_X: ParamId = ParamId("crop_x");
pub const CROP_Y: ParamId = ParamId("crop_y");
pub const CROP_W: ParamId = ParamId("crop_w");
pub const CROP_H: ParamId = ParamId("crop_h");
/// TRACES: FR-DEV-20
/// Vertical keystone. Positive spreads the top of the frame — the correction
/// for a building photographed looking up, whose verticals lean together.
pub const KEYSTONE_V: ParamId = ParamId("keystone_v");
/// TRACES: FR-DEV-20
/// Horizontal keystone. Positive spreads the right-hand side of the frame.
pub const KEYSTONE_H: ParamId = ParamId("keystone_h");
/// Widest straightening the control offers, in degrees either way.
///
@@ -66,12 +100,28 @@ pub const CROP_H: ParamId = ParamId("crop_h");
/// the edits actually are.
pub const MAX_STRAIGHTEN: f32 = 45.0;
/// The keystone sliders' travel either way.
///
/// A plain amount rather than degrees of tilt: the angle a camera was tilted
/// by depends on a focal length the correction does not know, and a number
/// that claimed to be one would be wrong for every lens but one.
pub const MAX_KEYSTONE: f32 = 100.0;
/// How far a full keystone narrows the far edge of the frame, as a fraction
/// of its width: at `MAX_KEYSTONE` the source trapezoid's short side is half
/// its long one.
///
/// Enough for a tall building from its own pavement, and short of the point
/// where the stretched edge is so magnified that the correction reads as a
/// fault of its own.
const KEYSTONE_REACH: f64 = 0.5;
/// The parameters the framing widget owns — every one of them.
///
/// In the order the widget expects: the rect first, then the angle it is
/// straightened by, then the exact reorientations.
static FRAMING_PARAMS: [ParamId; 8] = [
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, ROTATION, FLIP_H, FLIP_V,
static FRAMING_PARAMS: [ParamId; 10] = [
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, KEYSTONE_V, KEYSTONE_H, ROTATION, FLIP_H, FLIP_V,
];
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
@@ -117,6 +167,30 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
ParamDescriptor::fraction("crop_y", "param.crop_y", 0.0),
ParamDescriptor::fraction("crop_w", "param.crop_w", 1.0),
ParamDescriptor::fraction("crop_h", "param.crop_h", 1.0),
// TRACES: FR-DEV-20
// Perspective. Last so every sidecar written before these existed
// reads exactly as it did: a missing parameter is its default, and
// the default is no correction.
ParamDescriptor::scalar(
"keystone_v",
"param.keystone_v",
-MAX_KEYSTONE,
MAX_KEYSTONE,
0.0,
Unit::None,
Scale::Linear,
0,
),
ParamDescriptor::scalar(
"keystone_h",
"param.keystone_h",
-MAX_KEYSTONE,
MAX_KEYSTONE,
0.0,
Unit::None,
Scale::Linear,
0,
),
],
})
});
@@ -311,6 +385,86 @@ fn finite(v: f32, fallback: f32) -> f32 {
}
}
/// TRACES: FR-DEV-20
/// A plane projective map, row-major, acting on `(x, y, 1)`.
///
/// Held in `f64` because it is built by solving for four corners and then
/// inverted for [`Framing::output_at`]; the shader gets `f32` copies of the
/// forward map only.
#[derive(Debug, Clone, Copy, PartialEq)]
struct Homography([[f64; 3]; 3]);
impl Homography {
/// The map taking the square `[-0.5, 0.5]²` onto the quadrilateral whose
/// corners are `q`, listed top-left, top-right, bottom-right, bottom-left.
///
/// Heckbert's closed form for the unit square, composed with the shift
/// from the centred square onto it.
fn square_to_quad(q: [(f64, f64); 4]) -> Self {
let [(x0, y0), (x1, y1), (x2, y2), (x3, y3)] = q;
let (dx1, dx2, dx3) = (x1 - x2, x3 - x2, x0 - x1 + x2 - x3);
let (dy1, dy2, dy3) = (y1 - y2, y3 - y2, y0 - y1 + y2 - y3);
let den = dx1 * dy2 - dx2 * dy1;
let (g, h) = if den.abs() < 1e-12 {
(0.0, 0.0)
} else {
((dx3 * dy2 - dx2 * dy3) / den, (dx1 * dy3 - dx3 * dy1) / den)
};
// Unit square (u, v) -> quad.
let unit = [
[x1 - x0 + g * x1, x3 - x0 + h * x3, x0],
[y1 - y0 + g * y1, y3 - y0 + h * y3, y0],
[g, h, 1.0],
];
// Centred square -> unit square is `u = x + 0.5`, so fold the shift
// into the constant column.
let mut m = unit;
for row in &mut m {
row[2] += 0.5 * (row[0] + row[1]);
}
Self(m)
}
/// Where `(x, y)` lands, or `None` past the line the map sends to
/// infinity — a point with no image, which the caller treats as outside
/// the source.
fn apply(&self, (x, y): (f64, f64)) -> Option<(f64, f64)> {
let m = &self.0;
let w = m[2][0] * x + m[2][1] * y + m[2][2];
if w <= 1e-9 {
return None;
}
Some((
(m[0][0] * x + m[0][1] * y + m[0][2]) / w,
(m[1][0] * x + m[1][1] * y + m[1][2]) / w,
))
}
/// The inverse map, by the adjugate. Scale is irrelevant to a projective
/// map, so the determinant is only divided out to keep `w` positive and
/// near one — which is what [`Self::apply`]'s horizon test relies on.
fn inverse(&self) -> Self {
let m = &self.0;
let c = |r0: usize, c0: usize, r1: usize, c1: usize| {
m[r0][c0] * m[r1][c1] - m[r0][c1] * m[r1][c0]
};
let adj = [
[c(1, 1, 2, 2), -c(0, 1, 2, 2), c(0, 1, 1, 2)],
[-c(1, 0, 2, 2), c(0, 0, 2, 2), -c(0, 0, 1, 2)],
[c(1, 0, 2, 1), -c(0, 0, 2, 1), c(0, 0, 1, 1)],
];
let det = m[0][0] * adj[0][0] + m[0][1] * adj[1][0] + m[0][2] * adj[2][0];
let det = if det.abs() < 1e-12 { 1.0 } else { det };
let mut inv = adj;
for row in &mut inv {
for v in row.iter_mut() {
*v /= det;
}
}
Self(inv)
}
}
/// TRACES: FR-DEV-3 | FR-DEV-3d
/// Crop, straighten, rotation and flips for one image.
///
@@ -325,6 +479,10 @@ pub struct Framing {
quarter_turns: u8,
flip_h: bool,
flip_v: bool,
/// TRACES: FR-DEV-20
/// Vertical and horizontal keystone, each `-MAX_KEYSTONE..=MAX_KEYSTONE`.
keystone_v: f32,
keystone_h: f32,
/// TRACES: FR-DEV-3h
/// How the file's pixels were stored, from its EXIF orientation.
///
@@ -376,6 +534,8 @@ impl Default for Framing {
quarter_turns: 0,
flip_h: false,
flip_v: false,
keystone_v: 0.0,
keystone_h: 0.0,
baseline: dr_types::Orientation::NORMAL,
crop: CropRect::default(),
view: CropRect::default(),
@@ -396,7 +556,7 @@ impl Framing {
/// How framing would like to be presented.
///
/// **This is what stops a frontend having to name this stage.** Rendered
/// generically these eight parameters are eight bad controls: four crop
/// generically these ten parameters are ten bad controls: four crop
/// edges the photographer would have to type coordinates into, a "rotate"
/// slider running 0..3, and two switches. Every one of them is a worse
/// control than the gesture it stands for — a crop is dragged on the
@@ -407,10 +567,10 @@ impl Framing {
/// a second frontend would have had to learn the same special case, and
/// nothing in the capability output said why. Now the preference is
/// declared, the demand says what the widget needs, and a frontend that
/// cannot meet it falls back to the eight sliders — tedious, but complete,
/// cannot meet it falls back to the ten sliders — tedious, but complete,
/// which is the guarantee the whole hint mechanism rests on.
///
/// The widget owns **all eight** parameters rather than only the rect: a
/// The widget owns **all ten** parameters rather than only the rect: a
/// frontend that takes this on is taking on the whole framing control
/// surface, and leaving rotation and the flips behind would scatter them
/// into the generated panel underneath a crop control that already exists.
@@ -476,6 +636,52 @@ impl Framing {
(self.flip_h, self.flip_v)
}
/// TRACES: FR-DEV-20
/// The vertical and horizontal keystone, as the sliders show them.
pub fn keystone(&self) -> (f32, f32) {
(self.keystone_v, self.keystone_h)
}
/// Whether a perspective correction is applied at all.
pub fn has_keystone(&self) -> bool {
self.keystone_v != 0.0 || self.keystone_h != 0.0
}
/// TRACES: FR-DEV-20
/// The perspective map, from the straightened output frame to the upright
/// source frame, both measured as the centred square `[-0.5, 0.5]²`.
///
/// **Measured in fractions of the frame, not in the aspect-scaled space
/// the rest of the prologue works in**, so the map is the same for every
/// frame shape and the uniforms need no image size. The prologue divides
/// `p.x` by the frame's aspect on the way in and multiplies it back on
/// the way out.
///
/// The output frame's corners go to a trapezoid inside the source: a
/// positive vertical keystone brings the top corners in, so the top of
/// the source is spread across the full width of the output and lines
/// that converged upward come out parallel. Nothing is ever mapped from
/// outside the source, which is why a keystone alone needs no crop.
fn keystone_map(&self) -> Option<Homography> {
if !self.has_keystone() {
return None;
}
let amount = |v: f32| f64::from(v / MAX_KEYSTONE).clamp(-1.0, 1.0) * KEYSTONE_REACH;
let (tv, th) = (amount(self.keystone_v), amount(self.keystone_h));
// How much of each edge survives: the top and bottom rows' widths,
// the left and right columns' heights.
let top = 1.0 - tv.max(0.0);
let bottom = 1.0 + tv.min(0.0);
let right = 1.0 - th.max(0.0);
let left = 1.0 + th.min(0.0);
Some(Homography::square_to_quad([
(-0.5 * top, -0.5 * left),
(0.5 * top, -0.5 * right),
(0.5 * bottom, 0.5 * right),
(-0.5 * bottom, 0.5 * left),
]))
}
/// Add quarter turns, wrapping. The rotate-left/right buttons.
pub fn rotate_quarters(&mut self, turns: i32) {
self.quarter_turns = (i32::from(self.quarter_turns) + turns).rem_euclid(4) as u8;
@@ -544,6 +750,7 @@ impl Framing {
pub fn is_active(&self) -> bool {
let (turns, flip_h, flip_v) = self.effective();
self.angle != 0.0
|| self.has_keystone()
// Effective, not the user's: a file stored sideways needs the
// prologue emitted even on an untouched image, or it renders
// through the identity map and lies on its side.
@@ -572,6 +779,7 @@ impl Framing {
/// edited, the file was merely read correctly.
pub fn edits_image(&self) -> bool {
self.angle != 0.0
|| self.has_keystone()
|| self.quarter_turns != 0
|| self.flip_h
|| self.flip_v
@@ -591,7 +799,9 @@ impl Framing {
/// warp being active forces interpolation regardless, which is the
/// composer's call to make rather than this stage's.
pub fn needs_interpolation(&self) -> bool {
self.angle != 0.0
// A keystone stretches the frame by a different amount at every row,
// so it lands between pixels everywhere but on its centre line.
self.angle != 0.0 || self.has_keystone()
}
pub fn set_param(&mut self, id: ParamId, value: f32) {
@@ -629,6 +839,8 @@ impl Framing {
}
.normalised()
}
KEYSTONE_V => self.keystone_v = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
KEYSTONE_H => self.keystone_h = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
_ => log::warn!("framing: unknown parameter {id}"),
}
}
@@ -643,6 +855,8 @@ impl Framing {
CROP_Y => self.crop.y,
CROP_W => self.crop.width,
CROP_H => self.crop.height,
KEYSTONE_V => self.keystone_v,
KEYSTONE_H => self.keystone_h,
_ => 0.0,
}
}
@@ -707,8 +921,22 @@ impl Framing {
///
/// The standard largest-inscribed-rectangle result for a rotated
/// rectangle of the same aspect ratio.
///
/// TRACES: FR-DEV-20
/// **With a keystone the closed form no longer applies**: the area with a
/// source pixel behind it is the source rectangle pulled back through the
/// perspective map and then turned, a quadrilateral no textbook result
/// describes. That case is searched instead — see
/// [`Self::inscribed_by_search`]. A keystone alone never needs a crop, so
/// the search returns the whole frame for it, exactly.
pub fn max_inscribed_crop(&self, width: u32, height: u32) -> CropRect {
if self.angle == 0.0 || width == 0 || height == 0 {
if width == 0 || height == 0 {
return CropRect::default();
}
if self.has_keystone() {
return self.inscribed_by_search(width, height);
}
if self.angle == 0.0 {
return CropRect::default();
}
@@ -750,6 +978,123 @@ impl Framing {
.normalised()
}
/// TRACES: FR-DEV-20
/// The largest centred crop with a source pixel behind every point, found
/// by search rather than by formula.
///
/// The area that has a source pixel behind it is convex — the source
/// rectangle pulled back through a projective map whose horizon lies
/// outside it, then turned — and a rectangle lies inside a convex region
/// exactly when its four corners do. For a given width the tallest
/// rectangle that fits is therefore found by bisection, and the area
/// `width × tallest(width)` is unimodal in the width (a positive concave
/// function times a line), so a golden-section search finds the best
/// width. Every rectangle returned has been tested corner by corner, so
/// the answer errs inside, never outside.
///
/// A few hundred corner tests, on a gesture's release, is nothing next to
/// the render that follows it.
fn inscribed_by_search(&self, width: u32, height: u32) -> CropRect {
let (w, h) = if self.swaps_axes() {
(f64::from(height), f64::from(width))
} else {
(f64::from(width), f64::from(height))
};
let fa = w / h;
let rad = f64::from(self.angle).to_radians();
let (sn, cs) = (rad.sin(), rad.cos());
let map = self.keystone_map();
// Whether the output point `(fx, fy)`, in fractions of the frame from
// its centre, has a source pixel behind it. The prologue's steps, in
// its order, stopping short of the turns: those are a permutation of
// the frame and cannot move a point across its edge.
let defined = |fx: f64, fy: f64| {
let p = (fx * fa, fy);
let q = (p.0 * cs - p.1 * sn, p.0 * sn + p.1 * cs);
let n = (q.0 / fa, q.1);
let src = match &map {
Some(m) => m.apply(n),
None => Some(n),
};
const EDGE: f64 = 0.5 + 1e-9;
src.is_some_and(|(x, y)| x.abs() <= EDGE && y.abs() <= EDGE)
};
let fits = |hw: f64, hh: f64| {
[(-1.0, -1.0), (1.0, -1.0), (1.0, 1.0), (-1.0, 1.0)]
.iter()
.all(|(sx, sy)| defined(sx * hw, sy * hh))
};
if fits(0.5, 0.5) {
return CropRect::default();
}
// Half the tallest height that fits at half-width `hw`.
let tallest = |hw: f64| {
if fits(hw, 0.5) {
return 0.5;
}
if !fits(hw, 0.0) {
return 0.0;
}
let (mut lo, mut hi) = (0.0, 0.5);
for _ in 0..40 {
let mid = 0.5 * (lo + hi);
if fits(hw, mid) {
lo = mid;
} else {
hi = mid;
}
}
lo
};
let area = |hw: f64| hw * tallest(hw);
let ratio = (5.0_f64.sqrt() - 1.0) * 0.5;
let (mut a, mut b) = (0.0, 0.5);
let mut c = b - ratio * (b - a);
let mut d = a + ratio * (b - a);
let (mut fc, mut fd) = (area(c), area(d));
for _ in 0..48 {
if fc < fd {
a = c;
c = d;
fc = fd;
d = a + ratio * (b - a);
fd = area(d);
} else {
b = d;
d = c;
fd = fc;
c = b - ratio * (b - a);
fc = area(c);
}
}
let hw = 0.5 * (a + b);
let hh = tallest(hw);
// Tested at `hw` as returned, so a width that the search's last step
// nudged past the boundary cannot come back with a height that no
// longer fits it.
let (hw, hh) = if hh > 0.0 && fits(hw, hh) {
(hw, hh)
} else {
(c.min(d), tallest(c.min(d)))
};
let fw = (2.0 * hw) as f32;
let fh = (2.0 * hh) as f32;
let fw = fw.clamp(CropRect::MIN_EXTENT, 1.0);
let fh = fh.clamp(CropRect::MIN_EXTENT, 1.0);
CropRect {
x: (1.0 - fw) * 0.5,
y: (1.0 - fh) * 0.5,
width: fw,
height: fh,
}
.normalised()
}
/// TRACES: FR-DEV-3
/// Where an output point comes from in the source, both in normalised
/// `0..1` coordinates.
@@ -782,6 +1127,15 @@ impl Framing {
p = (p.0 * c - p.1 * s, p.0 * s + p.1 * c);
}
if let Some(m) = self.keystone_map() {
p = match m.apply((f64::from(p.0 / fx), f64::from(p.1))) {
Some((x, y)) => (x as f32 * fx, y as f32),
// Beyond the map's horizon: no source point at all, reported
// as one far outside the frame rather than as a NaN.
None => (1e6, 1e6),
};
}
let (turns, flip_h, flip_v) = self.effective();
p = match turns {
1 => (p.1 * ax, -p.0 / fx),
@@ -824,6 +1178,13 @@ impl Framing {
_ => p,
};
if let Some(m) = self.keystone_map() {
p = match m.inverse().apply((f64::from(p.0 / fx), f64::from(p.1))) {
Some((x, y)) => (x as f32 * fx, y as f32),
None => (1e6, 1e6),
};
}
if self.angle != 0.0 {
let rad = -self.angle * PI / 180.0;
let (s, c) = (rad.sin(), rad.cos());
@@ -865,6 +1226,19 @@ impl Framing {
// identical whether or not the user is zoomed in, and costs no extra
// uniform slot.
let rect = self.visible_rect();
// The perspective map by columns, so the prologue can apply it as
// three multiply-adds. The identity when there is none: the slots
// exist either way and the prologue does not read them.
let m = self
.keystone_map()
.unwrap_or(Homography([
[1.0, 0.0, 0.0],
[0.0, 1.0, 0.0],
[0.0, 0.0, 1.0],
]))
.0;
let col = |c: usize| [m[0][c] as f32, m[1][c] as f32, m[2][c] as f32, 0.0];
let [c0, c1, c2] = [col(0), col(1), col(2)];
[
rect.x,
rect.y,
@@ -874,6 +1248,18 @@ impl Framing {
rad.cos(),
0.0,
0.0,
c0[0],
c0[1],
c0[2],
c0[3],
c1[0],
c1[1],
c1[2],
c1[3],
c2[0],
c2[1],
c2[2],
c2[3],
]
}
@@ -972,6 +1358,28 @@ impl Framing {
);
}
if self.has_keystone() {
// TRACES: FR-DEV-20
// After the straightening and before the turns, so the keystone
// acts on the photograph as it is shown. The map is measured in
// fractions of the frame (see `Framing::keystone_map`), hence the
// aspect divided out and put back. A point past the map's horizon
// has no source at all and is sent far outside it, where the
// sampler's bounds test renders it void.
s.push_str(
"
// Perspective: the straightened frame onto a trapezoid of the source.
let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);
let key_h = u.keystone_c0.xyz * key_n.x + u.keystone_c1.xyz * key_n.y + u.keystone_c2.xyz;
p = select(
vec2<f32>(1.0e6),
vec2<f32>(key_h.x / key_h.z * frame_aspect.x, key_h.y / key_h.z),
key_h.z > 1.0e-6,
);
",
);
}
// The user's turns and mirrors composed with the file's stored
// orientation. One permutation covers both, so honouring the EXIF tag
// adds no per-pixel work over an untagged file.
@@ -1040,13 +1448,15 @@ impl Framing {
| u64::from(flip_v) << 3
| u64::from(turns) << 4
| u64::from(self.is_active()) << 6
| u64::from(self.has_keystone()) << 7
}
}
/// Floats the framing block occupies in the generated uniform struct.
///
/// Two `vec4`s: the crop rect, and the angle's sin/cos with padding.
pub const FRAMING_UNIFORM_FIELDS: usize = 8;
/// Five `vec4`s: the crop rect, the angle's sin/cos with padding, and the
/// perspective map's three columns, each padded.
pub const FRAMING_UNIFORM_FIELDS: usize = 20;
#[cfg(test)]
mod tests {
@@ -2025,6 +2435,8 @@ mod tests {
(CROP_Y, 0.2),
(CROP_W, 0.5),
(CROP_H, 0.4),
(KEYSTONE_V, 35.0),
(KEYSTONE_H, -20.0),
] {
f.set_param(id, v);
assert_eq!(f.param(id), v, "{id} did not round-trip");
@@ -2241,6 +2653,8 @@ mod tests {
f.rotate_quarters(turns);
f.set_param(FLIP_H, 1.0);
f.set_param(FLIP_V, 1.0);
f.set_param(KEYSTONE_V, 60.0);
f.set_param(KEYSTONE_H, -25.0);
for out in [(0.0, 0.0), (0.5, 0.5), (0.2, 0.9), (0.95, 0.05)] {
let src = f.source_at(out, SRC.0, SRC.1);
@@ -2316,6 +2730,27 @@ mod tests {
"the three-turn permutation moved; `source_at` must move with it"
);
// The perspective step: the map applied in fractions of the frame,
// which is what `source_at` divides the aspect out for.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 40.0);
let prologue = f.wgsl_prologue();
assert!(
prologue.contains("let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);")
&& prologue.contains("key_h.x / key_h.z * frame_aspect.x"),
"the perspective step moved; `source_at` must move with it"
);
// After the straightening and before the turns, as `source_at` has it.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 40.0);
f.set_param(ANGLE, 3.0);
f.rotate_quarters(1);
let prologue = f.wgsl_prologue();
let straighten = prologue.find("// Straighten").unwrap();
let keystone = prologue.find("// Perspective").unwrap();
let turn = prologue.find("90° clockwise").unwrap();
assert!(straighten < keystone && keystone < turn, "{prologue}");
// And the sampler's last step, which lives in `operation.rs` and is
// the half of the map this file does not emit.
assert!(
@@ -2323,4 +2758,241 @@ mod tests {
"the sampler's return to texture coordinates moved"
);
}
// ---- perspective (FR-DEV-20) -----------------------------------------
/// A grid over the whole output frame, edges included.
fn grid() -> impl Iterator<Item = (f32, f32)> {
(0..=10).flat_map(|j| (0..=10).map(move |i| (i as f32 / 10.0, j as f32 / 10.0)))
}
fn inside(p: (f32, f32)) -> bool {
(-1e-4..=1.0 + 1e-4).contains(&p.0) && (-1e-4..=1.0 + 1e-4).contains(&p.1)
}
#[test]
fn a_keystone_is_an_edit_and_a_resample() {
let mut f = Framing::new();
let neutral = f.structure_key();
f.set_param(KEYSTONE_V, 30.0);
assert!(f.is_active());
assert!(f.edits_image(), "a keystone must light the modified dot");
assert!(f.needs_interpolation());
assert_ne!(f.structure_key(), neutral);
assert!(f.wgsl_prologue().contains("u.keystone_c0"));
// Neither the output size nor the crop moves: the frame is reshaped
// inside itself, so what the user cropped stays cropped.
assert_eq!(f.output_size(6000, 4000), (6000, 4000));
assert!(f.crop().is_full());
}
#[test]
fn the_keystone_magnitude_does_not_reach_the_structure_key() {
// Dragging the slider is a uniform upload, never a shader build.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 10.0);
let key = f.structure_key();
for (v, h) in [(80.0, 0.0), (-45.0, 30.0), (1.0, -100.0)] {
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
assert_eq!(f.structure_key(), key, "{v}/{h} forced a recompile");
}
}
#[test]
fn the_keystone_is_clamped_to_its_travel() {
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 1e9);
f.set_param(KEYSTONE_H, f32::NAN);
assert_eq!(f.keystone(), (MAX_KEYSTONE, 0.0));
assert!(f.uniforms().iter().all(|v| v.is_finite()));
}
#[test]
fn a_keystone_alone_never_reaches_outside_the_source() {
// The design decision the crop relies on: the output frame is mapped
// onto a trapezoid *inside* the source, so no corner goes empty and
// a crop drawn before the keystone is still a crop of the picture.
for (v, h) in [
(100.0, 0.0),
(-100.0, 0.0),
(0.0, 100.0),
(0.0, -100.0),
(100.0, 100.0),
(-100.0, 100.0),
(37.0, -64.0),
] {
for turns in 0..4 {
let mut f = Framing::new();
f.rotate_quarters(turns);
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
for out in grid() {
let src = f.source_at(out, SRC.0, SRC.1);
assert!(inside(src), "{v}/{h}, {turns} turn(s): {out:?} -> {src:?}");
}
assert!(f.max_inscribed_crop(SRC.0, SRC.1).is_full());
}
}
}
#[test]
fn a_vertical_keystone_makes_upward_converging_lines_parallel() {
// What the control is for. Output columns are straight verticals;
// with a positive keystone each must come from a straight source line
// that leans in toward the centre as it rises — the shape a building
// has when photographed looking up.
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 60.0);
for x in [0.1f32, 0.3, 0.7, 0.9] {
let bottom = f.source_at((x, 1.0), SRC.0, SRC.1);
let middle = f.source_at((x, 0.5), SRC.0, SRC.1);
let top = f.source_at((x, 0.0), SRC.0, SRC.1);
// Straight: the middle sits on the line through the two ends.
let cross = (top.0 - bottom.0) * (middle.1 - bottom.1)
- (top.1 - bottom.1) * (middle.0 - bottom.0);
assert!(cross.abs() < 1e-4, "column {x} is not a straight line");
// Leaning in: the top is nearer the centre than the bottom.
assert!(
(top.0 - 0.5).abs() < (bottom.0 - 0.5).abs(),
"column {x}: top {top:?} is not inside bottom {bottom:?}"
);
}
// The bottom row is left where it was; the top row is the one spread.
close(
f.source_at((0.0, 1.0), SRC.0, SRC.1),
(0.0, 1.0),
"bottom-left",
);
close(
f.source_at((0.0, 0.0), SRC.0, SRC.1),
(0.15, 0.0),
"top-left",
);
}
#[test]
fn a_horizontal_keystone_spreads_the_right_hand_side() {
let mut f = Framing::new();
f.set_param(KEYSTONE_H, 100.0);
// The right-hand column comes from half the source's height.
close(
f.source_at((1.0, 0.0), SRC.0, SRC.1),
(1.0, 0.25),
"top-right",
);
close(
f.source_at((1.0, 1.0), SRC.0, SRC.1),
(1.0, 0.75),
"bottom-right",
);
close(
f.source_at((0.0, 0.0), SRC.0, SRC.1),
(0.0, 0.0),
"top-left",
);
}
#[test]
fn the_keystone_acts_on_the_frame_as_shown() {
// A portrait frame the camera stored on its side: "vertical" is the
// frame's displayed height, so the same keystone must move the same
// *displayed* points whatever the file's stored orientation.
let mut upright = Framing::new();
upright.set_param(KEYSTONE_V, 50.0);
let mut sideways = Framing::new();
sideways.set_baseline(dr_types::Orientation::from_exif(6));
sideways.set_param(KEYSTONE_V, 50.0);
// Compare in the displayed frame: map the sideways result back
// through the orientation alone.
let mut turn_only = Framing::new();
turn_only.set_baseline(dr_types::Orientation::from_exif(6));
for out in grid() {
let a = upright.source_at(out, SRC.1, SRC.0);
let b = turn_only.output_at(sideways.source_at(out, SRC.0, SRC.1), SRC.0, SRC.1);
close(a, b, "the keystone turned with the file");
}
}
#[test]
fn the_inscribed_crop_accounts_for_the_keystone() {
// Straightening a keystoned frame: the empty area is no longer the
// rotated rectangle's, and the crop must avoid the area that is.
for (angle, v, h) in [
(5.0f32, 50.0f32, 0.0f32),
(-8.0, -70.0, 20.0),
(12.0, 100.0, 100.0),
(2.0, 0.0, -40.0),
] {
for turns in [0, 1] {
let mut f = Framing::new();
f.rotate_quarters(turns);
f.set_param(ANGLE, angle);
f.set_param(KEYSTONE_V, v);
f.set_param(KEYSTONE_H, h);
let c = f.max_inscribed_crop(SRC.0, SRC.1);
assert!(
c.width > 0.3 && c.height > 0.3 && !c.is_full(),
"{angle}°/{v}/{h}: {c:?}"
);
assert!(
((c.x + c.width * 0.5) - 0.5).abs() < 1e-4
&& ((c.y + c.height * 0.5) - 0.5).abs() < 1e-4,
"{c:?} is not centred"
);
// Every point of it, edges included, has a source pixel.
f.set_crop(c);
for out in grid() {
let src = f.source_at(out, SRC.0, SRC.1);
assert!(inside(src), "{angle}°/{v}/{h}: {out:?} -> {src:?}");
}
}
}
}
#[test]
fn the_inscribed_crop_with_a_keystone_is_not_needlessly_small() {
// The search must find the best rectangle, not merely a safe one. A
// tenth larger in either direction has to reach outside the source.
let mut f = Framing::new();
f.set_param(ANGLE, 6.0);
f.set_param(KEYSTONE_V, 60.0);
let c = f.max_inscribed_crop(SRC.0, SRC.1);
for (gw, gh) in [(1.1, 1.0), (1.0, 1.1)] {
let mut g = f;
let (w, h) = (c.width * gw, c.height * gh);
g.set_crop(CropRect {
x: 0.5 - w * 0.5,
y: 0.5 - h * 0.5,
width: w,
height: h,
});
let spills = [(0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)]
.into_iter()
.any(|out| !inside(g.source_at(out, SRC.0, SRC.1)));
// Either the grown rect spills, or it could not grow at all
// because the crop was already at the frame's edge on that axis.
assert!(
spills
|| (gw > 1.0 && c.width >= 1.0 - 1e-4)
|| (gh > 1.0 && c.height >= 1.0 - 1e-4),
"{c:?} grown by {gw}x{gh} still fits"
);
}
}
#[test]
fn reset_clears_the_keystone() {
let mut f = Framing::new();
f.set_param(KEYSTONE_V, 30.0);
f.set_param(KEYSTONE_H, -30.0);
f.reset();
assert_eq!(f.keystone(), (0.0, 0.0));
assert!(!f.is_active());
}
}
+35 -2
View File
@@ -113,7 +113,7 @@ pub struct EditGraph {
/// a sidecar comes to name one stock while the shader draws another.
film: Option<Film>,
/// TRACES: FR-DEV-8
/// The repairs (`docs/spot-removal.md`).
/// The repairs (`docs/dev/spot-removal.md`).
///
/// Apart from `ops` for the third time and the same reason: a spot is not
/// a scalar, and a list of them is not a slider. It sits beside the masks
@@ -122,7 +122,7 @@ pub struct EditGraph {
/// photograph comes from.
spots: SpotSet,
/// The lens corrections that rewrite coordinates: distortion and lateral
/// chromatic aberration (`docs/architecture.md` §5.2).
/// chromatic aberration (`docs/dev/architecture.md` §5.2).
///
/// Apart from `ops` for the fourth time, and this one is not about shape
/// but about direction. Every [`Operation`] is a function from colour to
@@ -857,6 +857,39 @@ impl EditGraph {
crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view)
}
/// TRACES: FR-DEV-3
/// The camera-space tap over one patch of what is on the canvas.
///
/// `patch` is in fractions of the visible region — the coordinates a
/// click on the canvas arrives in — and is laid over this edit's own
/// framing: crop, view, rotation and all, so a fraction of the canvas is
/// a fraction of the probe. Nothing else of the edit: no operation, no
/// mask, no repair. Rendering only the patch is what lets a small target
/// cover every sensor pixel under it rather than sampling one in fifty;
/// see [`crate::operation::compose_camera_probe`] for why the white
/// balance picker reads from here and not from the display.
///
/// The patch is centred where asked and held to the view's own minimum
/// extent: at a deep zoom a patch a fraction of the view would be
/// smaller than a view may be, and letting `set_view` widen it from one
/// corner would move the sample off the point that was clicked.
pub fn compose_camera_probe(&self, patch: crate::framing::CropRect) -> ComposedShader {
use crate::framing::CropRect;
let mut framing = self.framing;
let view = framing.view();
let width = (patch.width * view.width).max(CropRect::MIN_EXTENT);
let height = (patch.height * view.height).max(CropRect::MIN_EXTENT);
let cx = view.x + (patch.x + patch.width * 0.5) * view.width;
let cy = view.y + (patch.y + patch.height * 0.5) * view.height;
framing.set_view(CropRect {
x: (cx - width * 0.5).clamp(0.0, 1.0 - width),
y: (cy - height * 0.5).clamp(0.0, 1.0 - height),
width,
height,
});
crate::operation::compose_camera_probe(&self.warps, &framing)
}
/// TRACES: FR-DEV-19c
/// [`Self::compose_for`], with one layer's mask drawn over the picture.
///
+3 -1
View File
@@ -44,6 +44,7 @@ pub mod mask;
pub mod neutral;
pub mod operation;
pub mod ops;
pub mod orphan;
pub mod preset;
pub mod sidecar;
pub mod spot;
@@ -65,7 +66,8 @@ pub use history::{Edit, Entry as HistoryEntry, History, Step};
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
RESERVED_UNIFORM_FIELDS,
};
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
pub use sidecar::{Sidecar, Version};
+116 -8
View File
@@ -27,7 +27,7 @@
//! [`MaskSource::Regions`] stores integers naming regions in the segmentation
//! hierarchy (`dr-segment`). That choice is what makes a mask diffable, cheap
//! in a sidecar, and mergeable per-field under FR-NC-9 — three properties a
//! stored raster has none of (docs/segmentation.md §1). Two devices that
//! stored raster has none of (docs/dev/segmentation.md §1). Two devices that
//! select the same subject produce the same small sorted list, and a sync
//! conflict between them is resolvable rather than a binary blob fight.
//!
@@ -564,7 +564,7 @@ pub enum MaskSource {
/// This is what the watershed and the semantic model exist to produce.
/// Selecting a subject means "the regions the model's instance covers",
/// and the resulting edge is the watershed's, which is to say the image's
/// own (docs/segmentation.md §5).
/// own (docs/dev/segmentation.md §5).
Regions {
/// Which segmentation these ids index into.
///
@@ -590,7 +590,7 @@ pub enum MaskSource {
/// **The primary way a local adjustment is made.** The watershed hierarchy
/// this crate was first built around does not survive a photograph: its
/// saddles are near zero almost everywhere, so a global cut collapses the
/// frame into one region plus noise (docs/segmentation.md §15). A model
/// frame into one region plus noise (docs/dev/segmentation.md §15). A model
/// instance is a whole object, found as one thing, and needs no ladder.
///
/// The trade is that the boundary is the model's — a quarter-resolution
@@ -625,7 +625,7 @@ pub enum MaskSource {
/// reason both exist. A subject is *one* instance — this dog, not that one
/// — found by a COCO-trained instance model. A category is *all* the sky,
/// or all the foliage, from an ADE20K-trained semantic model that has no
/// notion of instances at all (docs/segmentation.md §16).
/// notion of instances at all (docs/dev/segmentation.md §16).
///
/// So this is what a global grade attaches to: lift the sky, desaturate
/// the vegetation, warm the architecture. Asking it for "that person
@@ -926,6 +926,21 @@ pub enum Join {
/// erase stroke is a hole in the part it was painted into and reads as
/// nothing at all.
Subtract,
/// TRACES: FR-DEV-19a
/// Only where both agree. "Keep the part of this mask that is also that."
///
/// The join that makes cheap criteria precise: a sky is a category *and*
/// a luminance band, skin is a subject *and* a hue. Neither alone is the
/// selection, and no feather on either makes it one.
///
/// The product of the two coverages rather than their minimum, because
/// that is what one fixed-function blend gives on the device
/// (`dst · src`, `docs/dev/mask-editing.md` §5.2) and it agrees with the
/// minimum wherever either side is fully in or fully out. Between two soft
/// edges it is the softer of the two readings, which is the right way to
/// be wrong: an overlap of two partial selections is less certainly
/// selected than either.
Intersect,
}
impl Join {
@@ -933,6 +948,7 @@ impl Join {
match self {
Self::Union => "union",
Self::Subtract => "subtract",
Self::Intersect => "intersect",
}
}
@@ -940,12 +956,30 @@ impl Join {
Some(match name {
"union" => Self::Union,
"subtract" => Self::Subtract,
"intersect" => Self::Intersect,
_ => return None,
})
}
/// Every variant, for a UI building a choice control.
pub const ALL: [Join; 2] = [Join::Union, Join::Subtract];
/// TRACES: FR-DEV-19a
/// The coverage this join leaves at one point, given what the mask had
/// there (`dst`) and what the part covers (`src`), both in `0..=1`.
///
/// The definition the device's blend states implement, spelled out once
/// on the CPU so a test can hold the GPU to it and a reader can see the
/// three set operations side by side without reading `wgpu` enums.
pub fn apply(self, dst: f32, src: f32) -> f32 {
match self {
Self::Union => dst.max(src),
Self::Subtract => dst * (1.0 - src),
Self::Intersect => dst * src,
}
}
/// Every variant, for a UI building a choice control. The order is the
/// panel's: a chip cycles through it and a stored index names a place in
/// it, so a new join goes on the end.
pub const ALL: [Join; 3] = [Join::Union, Join::Subtract, Join::Intersect];
}
/// One selection inside a layer's mask.
@@ -1580,9 +1614,20 @@ impl MaskLayer {
// A hidden part is not in the build, whichever way it joins — and a
// hidden base hands its role to the first part that is shown, which
// is why "adds" is asked of the shown parts rather than of index 0.
//
// Folded rather than asked with `any`, because an intersection can
// take away everything the parts before it added: a subject
// intersected with an unpainted brush covers nothing. An inverted
// part is taken to cover, whatever its source — an inverted empty
// brush is the whole frame, and saying "covers nothing" of a layer
// that does would hide an adjustment the photographer made.
self.shown_parts()
.enumerate()
.any(|(i, p)| (i == 0 || p.join == Join::Union) && p.covers())
.fold(false, |acc, (i, p)| match (i, p.join) {
(0, _) | (_, Join::Union) => acc || p.covers(),
(_, Join::Subtract) => acc,
(_, Join::Intersect) => acc && (p.covers() || p.invert),
})
}
/// TRACES: FR-DEV-19a
@@ -2231,7 +2276,7 @@ fn reveal_block(slot: usize, layer: &MaskLayer, style: RevealStyle, colour: [f32
/// **not** a hash of the label field: that would be a readback on a path that
/// must not have one (ARCH §6.1), and would also make the signature depend on
/// float arithmetic whose cross-vendor determinism is exactly the open
/// question (docs/segmentation.md §6, M5).
/// question (docs/dev/segmentation.md §6, M5).
pub fn segmentation_signature(width: u32, height: u32, regions: u32, tuning: u64) -> u64 {
// FNV-1a over the four fields. Small, dependency-free, and adequate: this
// guards against accidental mismatch, not against a forged sidecar.
@@ -3037,6 +3082,69 @@ mod tests {
);
}
/// TRACES: FR-DEV-19a
/// The three joins, pointwise, on every pair of coverages a part and a
/// mask can meet at: fully in, fully out, and each soft edge. Intersection
/// is the product, which agrees with the minimum wherever either side is
/// decided and is the softer reading where both are not.
#[test]
fn the_joins_are_max_cut_and_product() {
let levels = [0.0f32, 0.25, 0.5, 0.75, 1.0];
for &dst in &levels {
for &src in &levels {
assert_eq!(Join::Union.apply(dst, src), dst.max(src));
assert_eq!(Join::Subtract.apply(dst, src), dst * (1.0 - src));
let meet = Join::Intersect.apply(dst, src);
assert_eq!(meet, dst * src);
assert!(meet <= dst.min(src), "never more than either side");
if dst == 0.0 || dst == 1.0 || src == 0.0 || src == 1.0 {
assert_eq!(meet, dst.min(src), "the minimum where either is decided");
}
}
}
}
/// A stored index names a place in [`Join::ALL`], and a sidecar names a
/// join by word: both have to survive a third join arriving, which means
/// the first two keep their places and every name reads back as itself.
#[test]
fn every_join_reads_back_by_name_and_keeps_its_place() {
assert_eq!(Join::ALL[0], Join::Union);
assert_eq!(Join::ALL[1], Join::Subtract);
for join in Join::ALL {
assert_eq!(Join::from_name(join.name()), Some(join));
}
assert_eq!(Join::from_name("intersect"), Some(Join::Intersect));
}
/// TRACES: FR-DEV-19a
/// An intersection can empty a mask the parts before it filled: a range
/// meeting an unpainted brush selects nothing, and rasterising it would
/// spend a slice to draw an empty field. Painting the brush, or inverting
/// it, gives the intersection something to keep.
#[test]
fn an_intersection_with_nothing_covers_nothing() {
let mut layer = MaskLayer::new("m1", MaskSource::highlights());
layer.set_param("exposure", ParamId("exposure"), 1.0);
layer.push_part(MaskPart::painted("p2", Join::Intersect));
assert!(
!layer.is_active(),
"a range intersected with an unpainted brush selects nothing"
);
layer.part_mut(1).expect("p2").invert = true;
assert!(layer.is_active(), "inverted, the empty brush is everywhere");
layer.part_mut(1).expect("p2").invert = false;
layer.begin_stroke(1, false, 0.1, 0.5, 1.0);
layer.extend_stroke(1, 0.5, 0.5);
layer.end_stroke(1);
assert!(layer.is_active(), "and painted, it keeps what it covers");
layer.part_mut(1).expect("p2").hidden = true;
assert!(layer.is_active(), "hidden, it is out of the build");
}
/// One stale part is a stale layer: the mask is the fold over all of them,
/// so a part that would draw a confidently wrong shape makes the result
/// wrong whichever way it joins.
+15 -9
View File
@@ -69,20 +69,26 @@ const ROUNDS: u32 = 3;
/// Below this a channel carries no ratio worth balancing.
///
/// A sample in the deep shadows, or one taken on a blown highlight where a
/// channel has already clipped to nothing, has no white balance in it: the
/// logarithms below would run away and the picker would slam a slider to its
/// stop. Refusing is the honest answer, and the caller reports that the point
/// was not usable rather than moving the photograph.
/// A sample in the deep shadows has no white balance in it: the logarithms
/// below would run away and the picker would slam a slider to its stop.
/// Refusing is the honest answer, and the caller reports that the point was
/// not usable rather than moving the photograph. The other end — a blown
/// highlight, where every channel has stopped counting — is refused by the
/// caller before the sample is taken, because only the caller can see the
/// sensor value; see [`crate::operation::CLIP_ONSET`].
const FLOOR: f32 = 1e-4;
/// TRACES: FR-DEV-3
/// Move the graph so that `sample` renders neutral.
///
/// `sample` is linear RGB, as the operation's own gains multiply it — that is,
/// measured with the sampling operation at its defaults. Returns whether the
/// graph was moved: `false` where the chain offers no white point widget, or
/// where the colour has no balance in it to correct.
/// `sample` is the linear triple the operation's own gains multiply — camera
/// RGB with the camera's as-shot balance on, *before* the body's base curve
/// and matrix, and with the sampling operation at its defaults. Not the
/// pixel on the screen: the matrix mixes the channels on the way there, so
/// a colour read after it does not answer to these gains, and a solve over
/// one lands somewhere no sample asked for. Returns whether the graph was
/// moved: `false` where the chain offers no white point widget, or where the
/// colour has no balance in it to correct.
///
/// **Absolute, not relative.** The values written depend on the colour and not
/// on where the sliders happened to be, so sampling the same wall twice lands
+73 -10
View File
@@ -54,7 +54,7 @@ pub enum Affects {
/// A pixel's *neighbourhood* — sharpening, noise reduction, clarity,
/// texture, dehaze, spot removal.
///
/// The seam `docs/requirements.md` §3.3 designed and nothing cut until
/// The seam `docs/dev/requirements.md` §3.3 designed and nothing cut until
/// [`crate::detail`] existed. It is a separate variant rather than a flavour
/// of `Colour` because it is a separate *dispatch*: a fragment in the fused
/// pass is handed a colour and has no way back to a coordinate, so a
@@ -431,9 +431,11 @@ pub enum OutputMode {
/// no operations, and the caller fills the reserved uniforms neutral —
/// unit white balance, identity matrix, base curve off — so what is
/// stored is the sensor's own numbers, demosaiced and undistorted. Only
/// [`compose_camera_linear`] produces it, and only
/// `AdjustPass::render_camera_linear` accepts it, so the neutral
/// uniforms cannot be forgotten by a caller that composed it by mistake.
/// [`compose_camera_probe`] produces it — for a merge through
/// [`compose_camera_linear`], and for the white balance picker under the
/// edit's own framing — and only `AdjustPass::render_camera_linear`
/// accepts it, so the neutral uniforms cannot be forgotten by a caller
/// that composed it by mistake.
///
/// Thirty-two bits rather than sixteen because the composite is written
/// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has
@@ -490,6 +492,19 @@ pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
/// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5;
/// Where the highlight desaturation begins: the fraction of the white level
/// above which a photosite is treated as clipped.
///
/// A photosite this close to saturation has stopped counting, so its ratio
/// to its neighbours is not a colour. The generated prologue fades a pixel
/// above this toward a neutral of the same brightness before any operation
/// runs, and the white balance probe refuses to sample one: a blown sky is
/// sensor white, which the as-shot multipliers make magenta, and a solve
/// over that slams tint to its stop. One number, so the two cannot drift
/// apart — a probe that accepted what the shader had already desaturated
/// would be balancing against a pixel the photographer cannot see.
pub const CLIP_ONSET: f32 = 0.985;
/// Where an operation's own uniforms begin in the generated block.
///
/// The base fields, then framing's. Exported because `dr-gpu` writes the
@@ -589,7 +604,9 @@ pub fn compose_full_revealing(
warps: &[Box<dyn crate::lens::Warp>],
reveal: Option<&crate::mask::Reveal>,
) -> ComposedShader {
compose_inner(ops, framing, output, masks, spots, warps, reveal, None)
compose_inner(
ops, framing, output, masks, spots, warps, reveal, None, false,
)
}
/// TRACES: FR-MRG-2
@@ -614,15 +631,50 @@ pub fn compose_camera_linear(
// upright too, and `view` is a fraction of the upright frame.
framing.set_baseline(baseline);
framing.set_view(view);
compose_camera_tap(warps, &framing, false)
}
/// TRACES: FR-DEV-3
/// The same tap under a framing the caller chose: the white balance probe.
///
/// A neutral picked off the canvas has to be measured in the space the
/// white balance gains multiply, and that is camera RGB — the operation
/// runs before the body's matrix, and a probe read after the matrix would
/// be solving the wrong equation on any body whose matrix mixes the
/// channels, which is every body. It also has to be measured at the pixel
/// the canvas is showing, which is why this takes the edit's own framing
/// where a merge passes the file's orientation and a tile.
///
/// **Interpolated whatever the framing says.** The point of rendering a
/// patch is to average what is under it, and the nearest sampling an
/// unrotated frame otherwise gets is a comb: at two source pixels per
/// probe pixel it lands on the same column of any pattern every time, and
/// the average of a thousand samples is then the average of nothing.
pub fn compose_camera_probe(
warps: &[Box<dyn crate::lens::Warp>],
framing: &Framing,
) -> ComposedShader {
compose_camera_tap(warps, framing, true)
}
/// The camera-space tap proper: no operations, `rgba32float`, and the
/// profile uniforms left for the GPU side to fill neutral. `smooth` forces
/// the interpolating sampler; see the two callers for who wants it and why.
fn compose_camera_tap(
warps: &[Box<dyn crate::lens::Warp>],
framing: &Framing,
smooth: bool,
) -> ComposedShader {
compose_inner(
&[],
&framing,
framing,
ColourSpace::Srgb,
&MaskStack::new(),
&crate::spot::SpotSet::new(),
warps,
None,
Some(OutputMode::CameraLinear),
smooth,
)
}
@@ -636,6 +688,7 @@ fn compose_inner(
warps: &[Box<dyn crate::lens::Warp>],
reveal: Option<&crate::mask::Reveal>,
forced: Option<OutputMode>,
smooth: bool,
) -> ComposedShader {
// The lens corrections, composed into one coordinate transform. Beside
// `framing` because they are the other half of the same stage: framing
@@ -668,7 +721,7 @@ fn compose_inner(
// twice and be bound to a texture of the wrong format.
//
// `forced` is the one exception, and it is not a caller flag in the
// sense above: `compose_camera_linear` is the only function that passes
// sense above: `compose_camera_probe` is the only function that passes
// it, with an empty operation list, and the mode it forces has its own
// storage format and its own render entry on the GPU side.
let output_mode = forced.unwrap_or(
@@ -748,7 +801,11 @@ fn compose_inner(
\x20 // angle as sin/cos — a trig call per pixel would recompute a\n\
\x20 // value that is constant across the dispatch.\n\
\x20 crop_rect: vec4<f32>,\n\
\x20 framing_angle: vec4<f32>,\n",
\x20 framing_angle: vec4<f32>,\n\
\x20 // The perspective map (FR-DEV-20) by columns, `.w` unused.\n\
\x20 keystone_c0: vec4<f32>,\n\
\x20 keystone_c1: vec4<f32>,\n\
\x20 keystone_c2: vec4<f32>,\n",
);
uniform_values.extend_from_slice(&framing.uniforms());
@@ -842,7 +899,10 @@ fn compose_inner(
// framing alone — which is what this did before the warps existed — would
// have nearest-neighboured a distortion correction on an unstraightened
// frame, and the aliasing would have looked like a bad profile.
let interpolate = framing.needs_interpolation() || warp.is_active();
// `smooth` is the third reason, and the only one a caller states: the
// white balance probe averages a patch and cannot do that through a
// nearest-neighbour comb (see `compose_camera_probe`).
let interpolate = smooth || framing.needs_interpolation() || warp.is_active();
// Declared ahead of the warp block, which assigns to them. They enter
// equal to `p` so that a chain mixing a splitting warp with a
@@ -997,6 +1057,9 @@ fn compose_inner(
.to_string()
};
// Formatted with Rust's `Display` so the shader reads the same threshold
// the probe checks against; see `CLIP_ONSET`.
let clip_onset = CLIP_ONSET;
let source = format!(
"// GENERATED — do not edit.
//
@@ -1067,7 +1130,7 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
// A photosite at its white level carries no colour information — every
// channel simply stopped counting — so the balance below must not be
// allowed to tint it.
let clipped = smoothstep(0.985, 1.0, max(c.r, max(c.g, c.b)));
let clipped = smoothstep({clip_onset}, 1.0, max(c.r, max(c.g, c.b)));
c = c * u.as_shot_wb.rgb;
+1 -1
View File
@@ -99,7 +99,7 @@
//! A minimum over a patch is separable, as a Gaussian is: minimum along x,
//! then along y. That alone is not enough. The patch is 1% of the shorter edge
//! — 61 taps across at 4K — and two passes of 61 taps is the arithmetic that
//! measured 34 ms for clarity and became `docs/technical-debt.md` TD-4.
//! measured 34 ms for clarity and became `docs/dev/technical-debt.md` TD-4.
//!
//! A minimum has a property a Gaussian does not: **erosions compose by adding
//! their structuring elements**. The minimum over a contiguous run of `d`
+2 -2
View File
@@ -150,7 +150,7 @@
//! the artefact this control must not have.
//!
//! Run at the render size, that measured **34 ms at 4K** — seven times the
//! entire fused point chain, for one slider — which is `docs/technical-debt.md`
//! entire fused point chain, for one slider — which is `docs/dev/technical-debt.md`
//! TD-4 and is what [`Recipe::base_scale`] now answers. The base is computed on
//! a grid a quarter the size on each axis: a sixteenth of the pixels at a
//! quarter of the radius.
@@ -271,7 +271,7 @@ impl Band for Coarse {
threshold: 0.35,
gain: 1.0,
midtone_taper: true,
// A quarter, which is what `docs/technical-debt.md` TD-4 bought back.
// A quarter, which is what `docs/dev/technical-debt.md` TD-4 bought back.
//
// σ is 1.2% of the shorter edge — 26 px at 4K — so the base holds no
// spatial frequency anywhere near the quarter-scale Nyquist of one
+645
View File
@@ -0,0 +1,645 @@
//! TRACES: FR-DEV-17
//! Whether a crop leaves a mask layer's work outside the frame.
//!
//! # Why this is a question worth asking
//!
//! Mask geometry is stored in normalised *source* coordinates (see
//! [`MaskSource::Linear`] and [`Stroke::points`]), so a tighter crop never
//! destroys a layer. It makes it invisible — which is worse, because nothing
//! announces it. The layer is still in the panel, still in the sidecar, still
//! costing a rasterisation, and its adjustment lands on pixels nobody will
//! ever see. The crop that did that is exactly the kind of edit made early
//! and quickly, and the loss is found, if at all, much later.
//!
//! This module answers one question on the CPU, cheaply enough to ask once
//! per committed crop: *which layers did this change of crop take out of the
//! picture?* The interface turns the answer into a notice. It never refuses
//! the crop — the photographer may well mean it.
//!
//! # How it is measured
//!
//! Each layer's mask is sampled on a [`GRID`]×[`GRID`] lattice over the
//! source, with the same geometry the mask shader uses (`mask.wgsl`), folded
//! part by part with the layer's joins and inversions. The sample points are
//! then mapped through the framing — crop, straighten, turns and flips — into
//! the frame, and the layer's *share inside* is the coverage that lands in
//! the crop over the coverage there is. A layer is hidden by a crop when that
//! share falls below [`HIDDEN_SHARE`] and was not already below it.
//!
//! The lattice is coarse on purpose. The question is "is this layer mostly
//! gone", not "which pixels are": the edge treatment — feather, morphology —
//! moves a boundary by a few hundredths of the frame and cannot turn a layer
//! that is mostly inside into one that is mostly outside, so it is left out.
//!
//! # What cannot be orphaned
//!
//! A range ([`MaskSource::Luminance`], [`MaskSource::Colour`]) selects by a
//! property of the picture, so wherever the crop falls it selects whatever
//! part of the picture remains — there is nothing for a crop to strand. A
//! region selection needs the segmentation's label map, which this crate does
//! not hold, and a model's selection with no raster to hand has nothing to
//! measure. A layer that *adds* any of these is therefore never reported: a
//! false alarm on the common path would teach the photographer to dismiss the
//! notice unread, which is the failure it exists to prevent. Subtracting one
//! is ignored, which can only make the layer look larger — the safe side.
use std::borrow::Cow;
use crate::framing::{CropRect, Framing};
use crate::mask::{Join, MaskLayer, MaskPart, MaskSource, MaskStack, Stroke};
/// Samples per axis over the source.
///
/// 4096 points: enough that a brush dab a twentieth of the frame across is
/// several samples wide, and few enough that the whole stack is measured in
/// well under a frame when a crop is let go.
pub const GRID: usize = 64;
/// The share of a layer's coverage below which it counts as cropped away.
///
/// "Mostly outside" rather than "entirely": a gradient reduced to a sliver
/// along one edge, or a subject of which one elbow survives, has lost the
/// work as surely as one that is wholly gone. Low enough that an ordinary
/// recomposition which trims part of a subject is not reported.
pub const HIDDEN_SHARE: f32 = 0.1;
/// A model's selection, as bytes over the source at some proxy size.
///
/// Supplied by the caller for a part whose own [`MaskPart::coverage`] is not
/// set — the live session holds its model output outside the graph and folds
/// it into the parts only when saving.
pub struct Raster<'a> {
pub values: Cow<'a, [u8]>,
pub width: usize,
pub height: usize,
}
/// One layer's coverage, sampled over the source.
#[derive(Debug, Clone, PartialEq)]
pub struct Footprint {
/// Row-major over the lattice, `0.0..=1.0`, one per cell centre.
weights: Vec<f32>,
}
impl Footprint {
/// The layer's coverage on the lattice, or `None` when it has none to
/// strand — see the module header for which layers those are.
///
/// `model` is asked for the raster behind a subject or category part that
/// carries none of its own.
pub fn of<'r>(
layer: &MaskLayer,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Option<Self> {
let mut acc: Option<Vec<f32>> = None;
for (i, part) in layer.shown_parts().enumerate() {
let adds = i == 0 || part.join == Join::Union;
let Some(values) = part_values(part, source, model) else {
if adds {
// It follows the picture, or cannot be measured: either
// way it is not a shape a crop can leave behind.
return None;
}
continue;
};
acc = Some(match acc {
None => values,
Some(mut a) => {
for (d, s) in a.iter_mut().zip(values) {
*d = part.join.apply(*d, s);
}
a
}
});
}
let mut weights = acc?;
if layer.invert {
weights.iter_mut().for_each(|w| *w = 1.0 - *w);
}
Some(Self { weights })
}
/// Of this layer's coverage, the share `inside` marks as in frame.
/// `None` when there is no coverage at all.
fn share(&self, inside: &[bool]) -> Option<f32> {
let (mut total, mut kept) = (0.0f32, 0.0f32);
for (w, &i) in self.weights.iter().zip(inside) {
total += w;
if i {
kept += w;
}
}
(total > 1e-3).then(|| kept / total)
}
/// Of this layer's coverage, the share `framing`'s crop keeps.
pub fn share_inside(&self, framing: &Framing, source: (u32, u32)) -> Option<f32> {
self.share(&in_frame(framing, source))
}
}
/// TRACES: FR-DEV-17
/// The layers that changing the framing from `before` to `after` took out of
/// the picture, in stack order.
///
/// Only those it *newly* hid: a layer already cropped away by `before` is not
/// reported again, or every later adjustment of the crop would repeat a notice
/// the photographer has already answered.
///
/// The view (zoom and pan) of either framing is ignored. It is a way of
/// looking, not the frame.
pub fn hidden_by_crop<'m, 'r>(
masks: &'m MaskStack,
before: &Framing,
after: &Framing,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Vec<&'m MaskLayer> {
if masks.is_empty() || framed_alike(before, after) {
return Vec::new();
}
let was = in_frame(before, source);
let now = in_frame(after, source);
masks
.layers()
.iter()
.filter(|layer| {
let Some(print) = Footprint::of(layer, source, model) else {
return false;
};
let (Some(was), Some(now)) = (print.share(&was), print.share(&now)) else {
return false;
};
was >= HIDDEN_SHARE && now < HIDDEN_SHARE
})
.collect()
}
/// Whether the two frame the same part of the source, zoom aside.
fn framed_alike(a: &Framing, b: &Framing) -> bool {
let mut a = *a;
let mut b = *b;
a.set_view(CropRect::default());
b.set_view(CropRect::default());
a == b
}
/// Which lattice cells the framing's crop keeps.
fn in_frame(framing: &Framing, source: (u32, u32)) -> Vec<bool> {
let mut f = *framing;
f.set_view(CropRect::default());
lattice()
.map(|uv| {
let (x, y) = f.output_at(uv, source.0, source.1);
(0.0..=1.0).contains(&x) && (0.0..=1.0).contains(&y)
})
.collect()
}
/// Cell centres, row-major, in normalised source coordinates.
fn lattice() -> impl Iterator<Item = (f32, f32)> {
let step = 1.0 / GRID as f32;
(0..GRID).flat_map(move |y| {
(0..GRID).map(move |x| ((x as f32 + 0.5) * step, (y as f32 + 0.5) * step))
})
}
/// One part's coverage on the lattice, its own inversion applied, or `None`
/// where it cannot be measured as a shape.
fn part_values<'r>(
part: &MaskPart,
source: (u32, u32),
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
) -> Option<Vec<f32>> {
let aspect = source.0.max(1) as f32 / source.1.max(1) as f32;
let mut values: Vec<f32> = match &part.source {
MaskSource::Linear {
centre,
angle,
width,
} => {
let axis = (angle.cos(), angle.sin());
lattice()
.map(|(u, v)| {
let d = (u - centre.0) * aspect * axis.0 + (v - centre.1) * axis.1;
if *width <= 0.0 {
if d >= 0.0 {
1.0
} else {
0.0
}
} else {
smoothstep(-width * 0.5, width * 0.5, d)
}
})
.collect()
}
MaskSource::Radial {
centre,
radii,
angle,
feather,
} => {
let (sa, ca) = (-angle).sin_cos();
let radii = (radii.0.max(1e-6), radii.1.max(1e-6));
let edge = feather.clamp(0.0, 1.0);
lattice()
.map(|(u, v)| {
let d = ((u - centre.0) * aspect, v - centre.1);
let local = (d.0 * ca - d.1 * sa, d.0 * sa + d.1 * ca);
let r = (local.0 / radii.0).hypot(local.1 / radii.1);
if edge <= 0.0 {
if r <= 1.0 {
1.0
} else {
0.0
}
} else {
1.0 - smoothstep(1.0 - edge, 1.0, r)
}
})
.collect()
}
MaskSource::Brush { strokes } => brush_values(strokes, source),
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
if let Some(coverage) = &part.coverage {
// The lattice is a regular grid over the source, which is
// exactly the resample `decode_at` does.
coverage
.decode_at(GRID, GRID)
.into_iter()
.map(|b| f32::from(b) / 255.0)
.collect()
} else {
let raster = model(part)?;
if raster.width == 0
|| raster.height == 0
|| raster.values.len() < raster.width * raster.height
{
return None;
}
lattice()
.map(|(u, v)| {
let x = ((u * raster.width as f32) as usize).min(raster.width - 1);
let y = ((v * raster.height as f32) as usize).min(raster.height - 1);
f32::from(raster.values[y * raster.width + x]) / 255.0
})
.collect()
}
}
MaskSource::Regions { .. } | MaskSource::Luminance { .. } | MaskSource::Colour { .. } => {
return None
}
};
if part.invert {
values.iter_mut().for_each(|w| *w = 1.0 - *w);
}
Some(values)
}
/// Strokes composited in order, the way `fs_brush` and its blend states do.
fn brush_values(strokes: &[Stroke], source: (u32, u32)) -> Vec<f32> {
// Into units of the shorter edge, as `to_square` does, so a dab is round.
let short = source.0.min(source.1).max(1) as f32;
let scale = (
source.0.max(1) as f32 / short,
source.1.max(1) as f32 / short,
);
let square = |p: (f32, f32)| (p.0 * scale.0, p.1 * scale.1);
let cells: Vec<(f32, f32)> = lattice().map(square).collect();
let mut out = vec![0.0f32; cells.len()];
for stroke in strokes.iter().filter(|s| !s.is_empty()) {
let points: Vec<(f32, f32)> = stroke.points.iter().copied().map(square).collect();
let r = stroke.radius;
let inner = r * stroke.hardness.clamp(0.0, 1.0);
// Only cells inside the stroke's bounding box can be reached.
let (lo, hi) = points.iter().fold(
((f32::MAX, f32::MAX), (f32::MIN, f32::MIN)),
|(lo, hi), p| {
(
(lo.0.min(p.0), lo.1.min(p.1)),
(hi.0.max(p.0), hi.1.max(p.1)),
)
},
);
for (cell, dst) in cells.iter().zip(out.iter_mut()) {
if cell.0 < lo.0 - r || cell.0 > hi.0 + r || cell.1 < lo.1 - r || cell.1 > hi.1 + r {
continue;
}
let d = if points.len() == 1 {
dist(*cell, points[0])
} else {
points
.windows(2)
.map(|s| segment_distance(*cell, s[0], s[1]))
.fold(f32::MAX, f32::min)
};
let c = ((1.0 - smoothstep(inner, r, d)) * stroke.flow).clamp(0.0, 1.0);
*dst = if stroke.erase {
*dst * (1.0 - c)
} else {
*dst + c - *dst * c
};
}
}
out
}
fn dist(a: (f32, f32), b: (f32, f32)) -> f32 {
(a.0 - b.0).hypot(a.1 - b.1)
}
fn segment_distance(q: (f32, f32), a: (f32, f32), b: (f32, f32)) -> f32 {
let ab = (b.0 - a.0, b.1 - a.1);
let len2 = ab.0 * ab.0 + ab.1 * ab.1;
if len2 <= 1e-12 {
return dist(q, a);
}
let t = (((q.0 - a.0) * ab.0 + (q.1 - a.1) * ab.1) / len2).clamp(0.0, 1.0);
dist(q, (a.0 + ab.0 * t, a.1 + ab.1 * t))
}
/// WGSL's `smoothstep`, including its behaviour when the edges meet.
fn smoothstep(e0: f32, e1: f32, x: f32) -> f32 {
if e1 <= e0 {
return if x < e0 { 0.0 } else { 1.0 };
}
let t = ((x - e0) / (e1 - e0)).clamp(0.0, 1.0);
t * t * (3.0 - 2.0 * t)
}
#[cfg(test)]
mod tests {
use std::sync::Arc;
use super::*;
use crate::coverage::Coverage;
const SOURCE: (u32, u32) = (3000, 2000);
fn no_model(_: &MaskPart) -> Option<Raster<'static>> {
None
}
fn cropped(x: f32, y: f32, w: f32, h: f32) -> Framing {
let mut f = Framing::new();
f.set_crop(CropRect {
x,
y,
width: w,
height: h,
});
f
}
/// A small circle near the source's top-left corner.
fn top_left_circle() -> MaskLayer {
MaskLayer::new(
"m1",
MaskSource::Radial {
centre: (0.15, 0.15),
radii: (0.08, 0.08),
angle: 0.0,
feather: 0.2,
},
)
}
fn stack(layers: Vec<MaskLayer>) -> MaskStack {
let mut s = MaskStack::new();
for l in layers {
assert!(s.push(l));
}
s
}
fn hidden(masks: &MaskStack, before: &Framing, after: &Framing) -> Vec<String> {
hidden_by_crop(masks, before, after, SOURCE, &no_model)
.into_iter()
.map(|l| l.id.clone())
.collect()
}
#[test]
fn a_crop_away_from_a_shape_hides_it() {
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.5, 0.5, 0.5, 0.5);
assert_eq!(hidden(&masks, &Framing::new(), &after), vec!["m1"]);
}
#[test]
fn a_crop_that_keeps_the_shape_is_silent() {
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.0, 0.0, 0.6, 0.6);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_crop_that_trims_part_of_a_shape_is_silent() {
// Half the circle survives, which is a recomposition, not a loss.
let masks = stack(vec![top_left_circle()]);
let after = cropped(0.15, 0.0, 0.85, 1.0);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_layer_already_cropped_away_is_not_reported_again() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
let after = cropped(0.6, 0.6, 0.4, 0.4);
assert!(hidden(&masks, &before, &after).is_empty());
}
#[test]
fn uncropping_brings_a_layer_back_and_says_nothing() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
assert!(hidden(&masks, &before, &Framing::new()).is_empty());
}
#[test]
fn an_unchanged_frame_is_silent_whatever_the_zoom() {
let masks = stack(vec![top_left_circle()]);
let before = cropped(0.5, 0.5, 0.5, 0.5);
let mut after = before;
after.set_view(CropRect {
x: 0.5,
y: 0.5,
width: 0.5,
height: 0.5,
});
assert!(hidden(&masks, &Framing::new(), &after).len() == 1);
assert!(hidden(&masks, &before, &after).is_empty());
}
#[test]
fn a_range_follows_the_picture_and_is_never_stranded() {
let masks = stack(vec![
MaskLayer::new("lum", MaskSource::highlights()),
MaskLayer::new("skin", MaskSource::skin_tones()),
]);
let after = cropped(0.9, 0.9, 0.1, 0.1);
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
}
#[test]
fn a_linear_gradient_over_the_bottom_is_hidden_by_keeping_the_top() {
let masks = stack(vec![MaskLayer::new(
"grad",
MaskSource::Linear {
centre: (0.5, 0.8),
angle: std::f32::consts::FRAC_PI_2,
width: 0.05,
},
)]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 1.0, 0.5)),
vec!["grad"]
);
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.5, 1.0, 0.5)).is_empty());
}
#[test]
fn a_painted_stroke_is_measured_where_it_was_painted() {
let mut layer = MaskLayer::new("paint", MaskSource::brush());
layer.begin_stroke(0, false, 0.05, 0.8, 1.0);
for i in 0..10 {
layer.extend_stroke(0, 0.8 + i as f32 * 0.01, 0.8);
}
layer.end_stroke(0);
let masks = stack(vec![layer]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.5, 0.5)),
vec!["paint"]
);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn an_unpainted_brush_has_nothing_to_lose() {
let masks = stack(vec![MaskLayer::new("empty", MaskSource::brush())]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.3, 0.3)).is_empty());
}
#[test]
fn an_inverted_layer_is_measured_as_what_it_selects() {
// Everything *but* a corner circle: a crop into the other corner
// keeps most of it.
let mut layer = top_left_circle();
layer.invert = true;
let masks = stack(vec![layer]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn a_subject_is_measured_from_its_stored_coverage_or_the_model() {
// A subject occupying the right-hand quarter of a 40x20 proxy.
let (w, h) = (40usize, 20usize);
let bytes: Vec<u8> = (0..w * h)
.map(|i| if i % w >= 30 { 255 } else { 0 })
.collect();
let subject = MaskSource::Subject {
signature: 1,
index: 0,
class: "dog".into(),
score: 0.9,
};
let left = cropped(0.0, 0.0, 0.5, 1.0);
let mut stored = MaskLayer::new("stored", subject.clone());
stored.base_mut().coverage = Coverage::encode(&bytes, w, h, 2).map(Arc::new);
let live = MaskLayer::new("live", subject);
let masks = stack(vec![stored, live]);
// No model to hand: only the layer carrying its raster is measured.
assert_eq!(hidden(&masks, &Framing::new(), &left), vec!["stored"]);
let model = |_: &MaskPart| {
Some(Raster {
values: Cow::Borrowed(bytes.as_slice()),
width: w,
height: h,
})
};
let ids: Vec<_> = hidden_by_crop(&masks, &Framing::new(), &left, SOURCE, &model)
.into_iter()
.map(|l| l.id.as_str())
.collect();
assert_eq!(ids, vec!["stored", "live"]);
}
#[test]
fn a_crop_is_read_in_the_turned_frame() {
// Turned a quarter clockwise, the source's top-left lands at the
// frame's top-right, so keeping the right half of the frame keeps it.
let masks = stack(vec![top_left_circle()]);
let mut before = Framing::new();
before.rotate_quarters(1);
let mut right = before;
right.set_crop(CropRect {
x: 0.5,
y: 0.0,
width: 0.5,
height: 1.0,
});
let mut left = before;
left.set_crop(CropRect {
x: 0.0,
y: 0.0,
width: 0.5,
height: 1.0,
});
assert!(hidden(&masks, &before, &right).is_empty());
assert_eq!(hidden(&masks, &before, &left), vec!["m1"]);
}
#[test]
fn a_subtracted_range_does_not_hide_the_shape_it_cuts() {
let mut layer = top_left_circle();
assert!(layer.push_part(MaskPart::new(
"p2",
Join::Subtract,
MaskSource::highlights()
)));
let masks = stack(vec![layer.clone()]);
assert_eq!(
hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)),
vec!["m1"]
);
// Added instead, the range reaches everywhere and nothing is lost.
let mut layer = top_left_circle();
assert!(layer.push_part(MaskPart::new("p2", Join::Union, MaskSource::highlights())));
let masks = stack(vec![layer]);
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
}
#[test]
fn an_intersection_keeps_only_what_both_parts_cover() {
let circle = |id: &str, join, centre, r| {
MaskPart::new(
id,
join,
MaskSource::Radial {
centre,
radii: (r, r),
angle: 0.0,
feather: 0.2,
},
)
};
// Two circles in opposite corners: keeping the bottom-right quarter
// keeps one of them, and nothing is lost.
let mut layer = top_left_circle();
assert!(layer.push_part(circle("p2", Join::Union, (0.85, 0.85), 0.08)));
let bottom_right = cropped(0.5, 0.5, 0.5, 0.5);
let masks = stack(vec![layer.clone()]);
assert!(hidden(&masks, &Framing::new(), &bottom_right).is_empty());
// Intersected with a disc around the top-left, only that corner's
// circle survives, and the same crop takes it out of the frame.
assert!(layer.push_part(circle("p3", Join::Intersect, (0.15, 0.15), 0.3)));
let masks = stack(vec![layer]);
assert_eq!(hidden(&masks, &Framing::new(), &bottom_right), vec!["m1"]);
}
}
+10
View File
@@ -725,6 +725,7 @@ mod tests {
height: 0.5,
});
g.set_param(framing::ID, framing::ANGLE, -2.0);
g.set_param(framing::ID, framing::KEYSTONE_V, 40.0);
g
}
@@ -820,6 +821,14 @@ mod tests {
Some(0.0),
"the source's straightening must not travel on this scope"
);
// TRACES: FR-DEV-20
// Perspective is composition on the same terms as the crop: it is
// withheld by the default scope, not pasted over the target's frame.
assert_eq!(
target.param(framing::ID, framing::KEYSTONE_V),
Some(0.0),
"the source's keystone must not travel on this scope"
);
}
#[test]
@@ -829,6 +838,7 @@ mod tests {
preset.apply(&mut target, Scope::everything());
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(-2.0));
assert_eq!(target.param(framing::ID, framing::KEYSTONE_V), Some(40.0));
assert!(
(target.crop().width - 0.5).abs() < 1e-5,
"{:?}",
+74 -1
View File
@@ -93,6 +93,9 @@ pub const MAX_RATING: u8 = 5;
/// Highest flag code: 0 unflagged, 1 pick, 2 reject.
pub const MAX_FLAG: u8 = 2;
/// Highest colour-label code: purple. Mirrors `dr_catalog::rating::label_code`.
pub const MAX_LABEL: u8 = 5;
/// TRACES: FR-CAT-8 | FR-NC-8
/// One image's sidecar: a keyed set of versions.
///
@@ -187,6 +190,17 @@ pub struct Version {
/// so a value moving between the two stores needs no translation table
/// that could drift.
pub flag: u8,
/// TRACES: FR-CAT-5
/// The colour label, `0` for none and `1..=5` red, yellow, green, blue,
/// purple — the catalog's `versions.label` codes, for the reason
/// [`Self::flag`] shares its encoding.
///
/// Here for the reason the rating is: a label that lived only in the
/// catalog would go with the catalog, and would never reach the
/// photographer's other devices, which learn judgements from this file.
/// A build that predates the key keeps it as an unknown line and writes
/// it back, so an older device passes it on rather than erasing it.
pub label: u8,
/// The edit itself: `(op, param) -> value`, non-default values only.
pub params: BTreeMap<(String, String), f32>,
/// TRACES: FR-DEV-3 | FR-NC-9
@@ -217,7 +231,7 @@ pub struct Version {
/// graph's film cleared and the caller re-bakes — see `EditGraph::set_film`.
pub film: Option<FilmRef>,
/// TRACES: FR-DEV-8 | FR-NC-9
/// The repairs (`docs/spot-removal.md`).
/// The repairs (`docs/dev/spot-removal.md`).
///
/// A line per spot, keyed `spot.<id>`, rather than a block per spot as a
/// mask gets: a spot is eight numbers, and sixty-four blocks would bury the
@@ -264,6 +278,7 @@ impl Version {
// it in the "not yet looked at" state a cull resumes from.
rating: 0,
flag: 0,
label: 0,
params,
masks,
film,
@@ -572,6 +587,10 @@ impl Version {
// that never had it.
self.rating = merge_judgement(self.rating, remote.rating, remote_wins);
self.flag = merge_judgement(self.flag, remote.flag, remote_wins);
// TRACES: FR-CAT-5
// A label under the same rule: 0 is "none given", so a device that
// never labelled a frame cannot clear another's label.
self.label = merge_judgement(self.label, remote.label, remote_wins);
// TRACES: FR-DEV-3f
// The film resolves wholesale to the higher revision, like a mask
@@ -873,6 +892,9 @@ impl Sidecar {
if v.flag > 0 {
let _ = writeln!(out, "flag = {}", v.flag);
}
if v.label > 0 {
let _ = writeln!(out, "label = {}", v.label);
}
// TRACES: FR-DEV-3f
// Before the parameters, because it decides what they mean: the
// film's exposure slider is a slider on *that stock's* curve.
@@ -1068,6 +1090,17 @@ impl Sidecar {
Some(value.to_string());
}
"flag" => version.flag = value.parse::<u8>().unwrap_or(0).min(MAX_FLAG),
// TRACES: FR-CAT-5
// A code this build does not know reads as none rather than
// being clamped onto purple: a wrong colour is a claim, and
// no colour is only a gap.
"label" => {
version.label = value
.parse::<u8>()
.ok()
.filter(|l| *l <= MAX_LABEL)
.unwrap_or(0)
}
// TRACES: FR-DEV-8
// Ahead of the `op.param` arm below, which would otherwise try
// to read eight numbers as one float and drop the repair with a
@@ -2141,6 +2174,8 @@ mod tests {
height: 0.6,
});
g.set_param(framing::ID, framing::ANGLE, -1.5);
g.set_param(framing::ID, framing::KEYSTONE_V, 42.0);
g.set_param(framing::ID, framing::KEYSTONE_H, -17.0);
g.rotate_quarters(1);
let mut sidecar = Sidecar::new();
@@ -2157,6 +2192,12 @@ mod tests {
assert_eq!(restored.crop(), g.crop());
assert_eq!(restored.param(framing::ID, framing::ANGLE), Some(-1.5));
assert_eq!(restored.param(framing::ID, framing::ROTATION), Some(1.0));
// TRACES: FR-DEV-20
assert_eq!(restored.param(framing::ID, framing::KEYSTONE_V), Some(42.0));
assert_eq!(
restored.param(framing::ID, framing::KEYSTONE_H),
Some(-17.0)
);
}
#[test]
@@ -2828,6 +2869,37 @@ mod tests {
assert_eq!(back.flag, 1);
}
#[test]
fn a_label_survives_the_round_trip_and_an_unknown_one_reads_as_none() {
// TRACES: FR-CAT-5
let mut v = version_of(&EditGraph::default_chain());
v.label = 3;
let mut sidecar = Sidecar::new();
sidecar.put(v);
let text = sidecar.to_text();
assert!(text.contains("label = 3"), "{text}");
let parsed = Sidecar::parse(&text).expect("valid");
assert_eq!(parsed.default_version().expect("a version").label, 3);
let odd = "drsc 1\n\n[version u1]\nname = Default\nrevision = 1\nmodified = 0\n\
label = 9\n";
let parsed = Sidecar::parse(odd).expect("valid");
assert_eq!(parsed.default_version().expect("a version").label, 0);
}
#[test]
fn an_unlabelled_device_cannot_clear_another_devices_label() {
// TRACES: FR-CAT-5
let mut local = version_of(&EditGraph::default_chain());
local.label = 0;
local.revision = 9;
let mut remote = version_of(&EditGraph::default_chain());
remote.label = 1;
remote.revision = 1;
local.merge(&remote, None);
assert_eq!(local.label, 1);
}
#[test]
fn an_unrated_image_writes_no_judgement_lines() {
// The non-default rule applied to judgement: a library that has never
@@ -2838,6 +2910,7 @@ mod tests {
let text = sidecar.to_text();
assert!(!text.contains("rating"), "{text}");
assert!(!text.contains("flag"), "{text}");
assert!(!text.contains("label"), "{text}");
}
#[test]
+3 -3
View File
@@ -6,7 +6,7 @@
//! are blended. No pixels are stored, here or anywhere: the shader draws the
//! repair from these numbers every time the photograph is rendered, which is
//! what makes it non-destructive, cheap to sync, and undoable
//! (`docs/spot-removal.md`).
//! (`docs/dev/spot-removal.md`).
//!
//! # Why this is not an operation
//!
@@ -61,7 +61,7 @@ pub const MAX_SPOTS: usize = 64;
/// bounds something that is otherwise unbounded: a detail pass declares how far
/// it reads from the pixel it writes, and for a spot that is the offset plus
/// the radius. An unbounded offset is an unbounded halo, which is a pass the
/// tile scheduler cannot plan (ARCH §5.3, `docs/spot-removal.md` §5.3).
/// tile scheduler cannot plan (ARCH §5.3, `docs/dev/spot-removal.md` §5.3).
pub const MAX_SOURCE_DISTANCE: f32 = 0.5;
/// The radius a new spot starts at, in frame units.
@@ -210,7 +210,7 @@ impl Spot {
/// **FR-DEV-8 asks for automatic source placement, and this is the cheap
/// half of it.** The good half searches the photograph for a patch whose
/// surroundings match — a compute dispatch scoring candidate offsets, and
/// one small readback when the spot is created (`docs/spot-removal.md`
/// one small readback when the spot is created (`docs/dev/spot-removal.md`
/// §8). This is what stands in for it, and it is worth having on its own
/// terms rather than as a placeholder: dust sits on skies, skies are
/// smooth, and a patch two and a half radii away is nearly always the same
+22
View File
@@ -1344,6 +1344,28 @@ fn a_correction_painted_onto_a_subject_survives_a_round_trip() {
);
}
/// TRACES: FR-DEV-19a
/// An intersecting part is written under its own word and reads back as an
/// intersection. A build from before intersection read that word as a union
/// (see `an_unknown_join_adds_the_part`), which keeps the part visible and
/// fixable rather than silently cutting the mask down.
#[test]
fn an_intersecting_part_survives_a_round_trip() {
let mut graph = EditGraph::default_chain();
graph.masks_mut().push(corrected("m1", Join::Intersect));
let mut sidecar = Sidecar::new();
sidecar.put(Version::from_graph("default", "Default", &graph));
let text = sidecar.to_text();
assert!(text.contains("join = intersect"), "{text}");
let restored = round_trip(&graph);
let layer = &restored.masks().layers()[0];
assert_eq!(layer.parts().len(), 2);
assert_eq!(layer.parts()[1].join, Join::Intersect);
assert_eq!(layer.parts()[0].join, Join::Union, "the base is untouched");
}
/// TRACES: FR-DEV-19a
/// A part left out of the build comes back left out, and the file says so
/// under a word that cannot be confused with the layer's own switch.
+1 -1
View File
@@ -13,7 +13,7 @@ log.workspace = true
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
# business** — tract, or an ONNX Runtime the app found on disk, on whichever
# provider the device has (docs/inference.md). This crate never names either.
# provider the device has (docs/dev/inference.md). This crate never names either.
ort = { workspace = true, optional = true }
dr-inference-engine = { workspace = true, optional = true }
ndarray = { workspace = true, optional = true }
+1 -1
View File
@@ -34,7 +34,7 @@
//! And doing it here buys two things a shader could not. It is **exactly
//! deterministic**, which matters because masks reach the sidecar as indices
//! and a field that varied by vendor would mean a mask meaning one thing on
//! the desktop and another on the phone (docs/segmentation.md §6, M5). And it
//! the desktop and another on the phone (docs/dev/segmentation.md §6, M5). And it
//! is testable against hand-computed distances with no adapter present.
//!
//! # The transform
+1 -1
View File
@@ -40,7 +40,7 @@ pub struct Edge {
/// A partition of the image into labelled regions, plus how they adjoin.
///
/// The shared interface from docs/segmentation.md §2: arm A produces this
/// The shared interface from docs/dev/segmentation.md §2: arm A produces this
/// from a watershed, arm B would produce it from a class map, and the
/// consumers above cannot tell which.
#[derive(Debug, Clone, PartialEq)]
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-DEV-3i
//! Region segmentation for local masking (S15, docs/segmentation.md).
//! Region segmentation for local masking (S15, docs/dev/segmentation.md).
//!
//! Local adjustments need to know where the image's regions are before they
//! can snap a mask to one. This crate is that map, and it is deliberately
+2 -2
View File
@@ -1,6 +1,6 @@
//! Arm C — semantic instances as a prior over the watershed merge order.
//!
//! docs/segmentation.md §5. The spec calls this the expected winner and it is
//! docs/dev/segmentation.md §5. The spec calls this the expected winner and it is
//! what ships, for a reason that survives the model turning out to be narrower
//! than §4 assumed: the two arms fail in *opposite* directions, so each one
//! covers the other's failure.
@@ -233,7 +233,7 @@ pub fn apply_semantic_prior(
/// This is the interaction the whole spike exists to enable, and the reason it
/// returns *region ids* rather than a raster: a mask that is a set of integers
/// is diffable, mergeable at node level under FR-NC-9, and cheap in a sidecar
/// (docs/segmentation.md §1). A raster is none of those.
/// (docs/dev/segmentation.md §1). A raster is none of those.
///
/// The returned ids are sorted, so the same click always produces the same
/// mask — which is what lets it be a cache key.
+5 -5
View File
@@ -60,7 +60,7 @@
//! So the last step is a **marker-based watershed**. The mask is eroded to
//! give two markers — confidently inside, confidently outside — and the flood
//! runs in the ribbon left between them, meeting along the most expensive line
//! it can find. The cost is a sum of terms, as docs/segmentation.md §2 says it
//! it can find. The cost is a sum of terms, as docs/dev/segmentation.md §2 says it
//! should be: the photograph's own edges, and the colour model's disagreement.
//!
//! Markers are what make this the right shape rather than the watershed §15
@@ -244,7 +244,7 @@ pub struct RefineOptions {
/// How much the photograph's own edges count against the colour model in
/// the flood's cost, `0.0..=1.0`.
///
/// docs/segmentation.md §2 specifies the cost as *a sum of terms* — image
/// docs/dev/segmentation.md §2 specifies the cost as *a sum of terms* — image
/// gradient always available, semantic evidence added when a model is
/// present — and this is the mix. At one the boundary lands purely on the
/// strongest edge in the band; at zero purely where the colour verdict
@@ -378,7 +378,7 @@ const MAX_SAMPLES: usize = 20_000;
/// floating-point comparison is a stopping rule that can differ between
/// machines, and a mask that differs between machines reaches the sidecar as
/// indices meaning one thing on the desktop and another on the phone
/// (docs/segmentation.md §6).
/// (docs/dev/segmentation.md §6).
const ITERATIONS: usize = 12;
/// Half-width of the verdict scale, in nats.
@@ -676,7 +676,7 @@ impl Refinement {
/// fronts meet along the most expensive line in the ribbon — which is the
/// watershed, and which is where the boundary belongs.
///
/// The cost is a sum of terms, as docs/segmentation.md §2 says it should
/// The cost is a sum of terms, as docs/dev/segmentation.md §2 says it should
/// be: the photograph's own edges, and the colour model's disagreement.
/// Neither alone is right. An edge with no colour meaning is a texture,
/// and a colour change with no edge is a gradient.
@@ -889,7 +889,7 @@ fn neighbours(p: usize, w: usize, h: usize) -> impl Iterator<Item = usize> {
/// Edge strength over the opponent features, as one byte per pixel.
///
/// Sobel over the same three numbers the colour model is fitted on, rather
/// than over plain luma — docs/segmentation.md §3 is explicit that a
/// than over plain luma — docs/dev/segmentation.md §3 is explicit that a
/// channel-weighted RGB gradient reads a saturated red edge as weaker than it
/// looks, and a flag against sky is exactly that edge.
///
+3 -3
View File
@@ -1,4 +1,4 @@
//! Semantic segmentation — arm B (S15, docs/segmentation.md §4).
//! Semantic segmentation — arm B (S15, docs/dev/segmentation.md §4).
//!
//! Runs a YOLO instance-segmentation graph over a proxy-resolution image and
//! returns the instances it found: a class, a score, a box, and a soft mask
@@ -209,7 +209,7 @@ const EMBEDDED_MODEL: &[u8] = include_bytes!("../../../models/segment/yolo26n-se
const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json");
/// The bytes of the model that ships with this crate, for whoever compiles
/// engines ahead of the first request (docs/inference.md §6).
/// engines ahead of the first request (docs/dev/inference.md §6).
#[cfg(feature = "embedded-model")]
pub fn embedded_model_bytes() -> &'static [u8] {
EMBEDDED_MODEL
@@ -237,7 +237,7 @@ impl SemanticModel {
pub fn from_bytes(bytes: &[u8], classes: Vec<Arc<str>>) -> Result<Self, SegmentError> {
// The f32 graph on whatever the device's backend is. An int8 form
// for the Hexagon waits on docs/inference.md §10 M7 — the mask
// for the Hexagon waits on docs/dev/inference.md §10 M7 — the mask
// boundary has to be measured before it moves.
let session = dr_inference_engine::open(
dr_inference_engine::Role::Segmenter,
+121 -4
View File
@@ -22,6 +22,11 @@ pub struct LoginFlow {
pub login_url: String,
#[serde(rename = "poll")]
pub poll: PollInfo,
/// The server the flow was started against — the address the user typed,
/// already normalised. Not part of the response: [`begin`] fills it in so
/// [`poll`] can put it in the credentials instead of the server's answer.
#[serde(skip)]
pub server: String,
}
#[derive(Debug, Clone, Deserialize)]
@@ -66,9 +71,57 @@ pub async fn begin(
});
}
resp.json::<LoginFlow>()
let mut flow = resp
.json::<LoginFlow>()
.await
.map_err(|e| RemoteError::Protocol(e.to_string()))
.map_err(|e| RemoteError::Protocol(e.to_string()))?;
// Both URLs are the server's to choose, and neither may be trusted as
// sent. The login URL is handed to the operating system to open, where a
// `file:` or UNC path is a program launch rather than a web page; the poll
// endpoint is where the app password comes back from.
flow.login_url = upgraded("login URL", &flow.login_url)?;
flow.poll.endpoint = upgraded("poll endpoint", &flow.poll.endpoint)?;
flow.server = server.trim_end_matches('/').to_string();
Ok(flow)
}
/// TRACES: NFR-SEC-3
/// A URL the server sent, upgraded to HTTPS, or refused.
///
/// `http` is upgraded rather than refused: a Nextcloud behind a TLS-terminating
/// proxy without `overwriteprotocol` builds every absolute URL it returns with
/// `http`, and the same path over `https` is the one that works. Any other
/// scheme is refused, because the only thing it could be for is reaching
/// something that is not this server.
///
/// The host is not checked. A server reached by its LAN address may answer with
/// its public name, and nothing here is safer for refusing that: the account
/// is stored under the address the user typed (see [`poll`]), not under
/// anything the server said.
pub(crate) fn upgraded(what: &str, url: &str) -> Result<String, RemoteError> {
let mut parsed = url::Url::parse(url).map_err(|e| {
RemoteError::Protocol(format!(
"the server sent a {what} that is not a URL ({e}): {url}"
))
})?;
match parsed.scheme() {
"https" => {}
"http" => parsed.set_scheme("https").map_err(|()| {
RemoteError::Protocol(format!("{what} cannot be upgraded to https: {url}"))
})?,
other => {
return Err(RemoteError::Protocol(format!(
"the server sent a {what} using {other}:, and only https is accepted: {url}"
)))
}
}
if parsed.host_str().is_none_or(str::is_empty) {
return Err(RemoteError::Protocol(format!(
"the server sent a {what} with no host: {url}"
)));
}
Ok(parsed.into())
}
/// Poll until the user finishes authenticating in the browser.
@@ -98,10 +151,11 @@ pub async fn poll(
match resp.status().as_u16() {
200 => {
return resp
let creds = resp
.json::<AppCredentials>()
.await
.map_err(|e| RemoteError::Protocol(e.to_string()))
.map_err(|e| RemoteError::Protocol(e.to_string()))?;
return Ok(under_typed_server(creds, &flow.server));
}
// Still waiting for the user.
404 => tokio::time::sleep(POLL_INTERVAL).await,
@@ -117,6 +171,26 @@ pub async fn poll(
Err(RemoteError::AuthFailed)
}
/// TRACES: NFR-SEC-3
/// Credentials filed under the address the user typed, not the one the server
/// reports.
///
/// That report is the server's idea of its own URL, and behind a proxy
/// without `overwriteprotocol` it says `http://` — which, stored, would send
/// the app password in the clear on every request from then on. The typed
/// address has just carried the whole flow, so it is known to reach the
/// server.
fn under_typed_server(mut creds: AppCredentials, server: &str) -> AppCredentials {
if creds.server.trim_end_matches('/') != server {
log::info!(
"server reports itself as {}; keeping {server}",
creds.server
);
}
creds.server = server.to_string();
creds
}
/// Percent-encode a form value.
fn urlencode(s: &str) -> String {
s.bytes()
@@ -167,6 +241,49 @@ mod tests {
assert!(flow.login_url.contains("/login/v2/flow/"));
}
/// TRACES: NFR-SEC-3
#[test]
fn server_urls_are_upgraded_to_https_or_refused() {
assert_eq!(
upgraded("login URL", "http://cloud.example/login/v2/flow/xyz").unwrap(),
"https://cloud.example/login/v2/flow/xyz"
);
// A non-default port stays, and only the scheme changes.
assert_eq!(
upgraded("poll endpoint", "http://cloud.example:8443/login/v2/poll").unwrap(),
"https://cloud.example:8443/login/v2/poll"
);
assert_eq!(
upgraded("login URL", "https://cloud.example/x").unwrap(),
"https://cloud.example/x"
);
// What `rundll32 url.dll,FileProtocolHandler` would run, and what
// `xdg-open` would hand to whatever claims it.
for hostile in [
"file:///C:/Windows/System32/calc.exe",
"\\\\evil\\share\\x.exe",
"C:\\x.exe",
"javascript:alert(1)",
"-v",
"",
] {
assert!(upgraded("login URL", hostile).is_err(), "{hostile:?}");
}
}
/// TRACES: NFR-SEC-3
#[test]
fn credentials_keep_the_typed_server_not_the_reported_one() {
let reported = AppCredentials {
server: "http://cloud.example".into(),
login_name: "duncan".into(),
app_password: "secret-token".into(),
};
let c = under_typed_server(reported, "https://cloud.example");
assert_eq!(c.server, "https://cloud.example");
assert_eq!(c.app_password, "secret-token");
}
#[test]
fn form_values_are_encoded() {
assert_eq!(urlencode("abc123"), "abc123");
+79 -4
View File
@@ -47,6 +47,20 @@ pub struct NextcloudBackend {
/// `/remote.php/dav/files/<user>/` — the prefix stripped from hrefs.
dav_base: String,
caps: Capabilities,
/// Collections this backend has seen exist, so [`create_dir`] asks the
/// server about each one once.
///
/// Every `MOVE` guarantees its destination's parent, and did so with a
/// `MKCOL` for each ancestor down from the account root — for a trash
/// folder three levels deep that was three round trips of `405 Method
/// Not Allowed` before the one request that moved anything, on every
/// image of a batch. A backend lives for one job (`dr_ui::remote::
/// connect` builds one per worker), so a folder deleted by another
/// client mid-job is the one case this can get wrong, and it is reported
/// as the `409` the `MOVE` then earns rather than hidden.
///
/// [`create_dir`]: RemoteBackend::create_dir
known_dirs: std::sync::Mutex<std::collections::HashSet<String>>,
}
impl NextcloudBackend {
@@ -63,6 +77,7 @@ impl NextcloudBackend {
login: creds.login_name.clone(),
password: creds.app_password.clone(),
dav_base,
known_dirs: Default::default(),
caps: Capabilities {
// The property that makes a no-op sync one request (ARCH §8.1).
change_detection: ChangeDetection::PropagatingEtags,
@@ -567,6 +582,16 @@ impl RemoteBackend for NextcloudBackend {
chain.reverse();
for dir in chain {
// Asked once per backend — see `known_dirs`. The lock is held
// across no await: it is taken to look, and again to record.
let known = self
.known_dirs
.lock()
.map(|k| k.contains(dir.as_str()))
.unwrap_or(false);
if known {
continue;
}
let url = self.url_for(&dir);
let resp = self
.client
@@ -581,10 +606,12 @@ impl RemoteBackend for NextcloudBackend {
// 405 is "already a collection here", which is exactly what the
// caller wanted. Anything else is reported.
if resp.status() == reqwest::StatusCode::METHOD_NOT_ALLOWED {
continue;
if resp.status() != reqwest::StatusCode::METHOD_NOT_ALLOWED {
map_status(resp.status(), &url)?;
}
if let Ok(mut k) = self.known_dirs.lock() {
k.insert(dir.as_str().to_string());
}
map_status(resp.status(), &url)?;
}
Ok(())
}
@@ -696,6 +723,15 @@ pub fn http_client(user_agent: &str) -> Result<reqwest::Client, RemoteError> {
// for months. [`EXTRA_ROOTS`] carries those, and is additive — the
// webpki-roots set is still installed alongside.
.tls_certs_only(extra_roots())
// TRACES: NFR-SEC-3
// Refuse `http` here, below every URL this crate builds, rather than
// trusting each place a URL comes from. `normalise_endpoint` upgrades
// what the user types, but the login flow's poll endpoint, the account
// an older build saved and a redirect all arrive from somewhere else,
// and any of them naming `http://` would send the app password in
// Basic auth in the clear. reqwest checks this before connecting and
// again on every redirect, so a refused request never opens a socket.
.https_only(true)
// A request that hangs forever is indistinguishable from a worker that
// died, and cost a long time to tell apart once. These turn that into
// an error the UI can show.
@@ -753,8 +789,16 @@ fn map_send_error(e: reqwest::Error) -> RemoteError {
let mut detail = e.to_string();
let mut src: Option<&dyn std::error::Error> = std::error::Error::source(&e);
let mut tls = false;
// A URL the client refused to send — `http` under `https_only`, on the
// first request or a redirect. Nothing left the process, so this is the
// account's configuration, not the network: reported as `Network` it
// would put the app into offline mode over a connection that is fine.
let mut refused = e.is_builder();
while let Some(s) = src {
let text = s.to_string();
if text.contains("URL scheme is not allowed") {
refused = true;
}
// rustls surfaces every verification failure through this wording:
// UnknownIssuer, Expired, NotValidForName, BadSignature.
if text.contains("invalid peer certificate")
@@ -768,7 +812,9 @@ fn map_send_error(e: reqwest::Error) -> RemoteError {
src = std::error::Error::source(s);
}
if tls {
if refused {
RemoteError::Configuration(detail)
} else if tls {
RemoteError::Tls(detail)
} else {
RemoteError::Network(detail)
@@ -986,6 +1032,35 @@ mod tests {
assert!(c.is_ok(), "client must build without a backend");
}
/// TRACES: NFR-SEC-3
#[tokio::test]
async fn plain_http_is_refused_before_a_connection_opens() {
// A listener that would take the connection if one were made. The
// app password travels in a header, so a request that reached the
// socket has already leaked it; failing on the response is too late.
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
listener.set_nonblocking(true).unwrap();
let url = format!("http://{}/remote.php/dav/", listener.local_addr().unwrap());
let err = http_client("test")
.unwrap()
.get(&url)
.basic_auth("duncan", Some("app-password"))
.send()
.await
.expect_err("http must be refused");
assert!(
matches!(map_send_error(err), RemoteError::Configuration(_)),
"a refused scheme is configuration, not the network"
);
assert_eq!(
listener.accept().err().map(|e| e.kind()),
Some(std::io::ErrorKind::WouldBlock),
"nothing may have connected"
);
}
#[tokio::test]
async fn delta_is_unsupported_and_says_why() {
// Verified absent in the server; the engine must fall back rather
+24
View File
@@ -96,6 +96,18 @@ impl BackendProvider for NextcloudProvider {
}
}
/// TRACES: NFR-SEC-3
/// An `http://` account written before sign-in kept the address the user
/// typed: it was stored as the server reported itself, and behind a proxy
/// without `overwriteprotocol` that is `http`. The client refuses to send
/// to it now, so it is upgraded here rather than left to fail. The
/// namespace ignores the scheme, so the catalog stays where it is.
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
stored
.strip_prefix("http://")
.map(|rest| format!("https://{rest}"))
}
fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
let creds = Self::credentials(conn)?;
Ok(Box::new(NextcloudBackend::new(
@@ -132,6 +144,18 @@ mod tests {
assert!(p.normalise_endpoint(" ").is_err());
}
/// TRACES: NFR-SEC-3
#[test]
fn a_stored_http_endpoint_is_upgraded_and_nothing_else_is_touched() {
let p = NextcloudProvider;
assert_eq!(
p.upgrade_endpoint("http://cloud.example/nextcloud")
.as_deref(),
Some("https://cloud.example/nextcloud")
);
assert_eq!(p.upgrade_endpoint("https://cloud.example"), None);
}
#[test]
fn the_account_keeps_the_dav_user_id_apart_from_the_login() {
// A login can be an email address while the user id is something
+237 -1
View File
@@ -29,7 +29,7 @@ use dr_plat::{SecretError, SecretRef, SecretStore};
use dr_types::{Format, FormatFilter};
use serde::{Deserialize, Serialize};
use crate::RemoteError;
use crate::{BackendRegistry, RemoteError};
/// The connector every account had before there was a choice.
///
@@ -127,6 +127,19 @@ pub struct Account {
#[serde(default)]
pub root: String,
/// Whether [`root`](Self::root) has been chosen at all.
///
/// An empty `root` is two different things: nothing picked yet, and the
/// endpoint itself picked on purpose — a user who keeps everything at
/// the top level, or a folder library, which is its own root. The string
/// cannot tell them apart, and reading empty as "not chosen" meant the
/// top level could be confirmed in the picker and still not open. So the
/// fact is recorded separately. Defaulted, so an account written before
/// it existed loads as it always did: a non-empty root is chosen by
/// virtue of being there, and an empty one asks again.
#[serde(default)]
pub root_chosen: bool,
/// Which formats the scan looks for (the tick-boxes).
#[serde(default)]
pub formats: Vec<String>,
@@ -149,6 +162,7 @@ impl Account {
login: String::new(),
user_id: String::new(),
root: String::new(),
root_chosen: false,
formats: Vec::new(),
last_scan: None,
}
@@ -496,6 +510,95 @@ impl AccountStore {
written
}
/// TRACES: NFR-SEC-3
/// Rewrite the endpoints an older build stored in a form this one would
/// not, asking each account's connector
/// ([`upgrade_endpoint`](crate::BackendProvider::upgrade_endpoint)).
///
/// Per account and best effort: one that cannot be moved — its keyring
/// locked, say — is logged and left as it was, and is tried again on the
/// next launch, rather than stopping the others or the launch. Returns
/// the accounts it rewrote.
pub fn upgrade_endpoints(&self, registry: &BackendRegistry) -> Vec<Account> {
let mut upgraded = Vec::new();
for account in self.list() {
let Ok(provider) = registry.for_account(&account) else {
continue;
};
let Some(endpoint) = provider.upgrade_endpoint(&account.endpoint) else {
continue;
};
if endpoint == account.endpoint {
continue;
}
match self.move_endpoint(&account, &endpoint) {
Ok(moved) => {
log::info!("account {} moved to {endpoint}", account.describe());
upgraded.push(moved);
}
Err(e) => log::warn!(
"account {} could not be moved to {endpoint}: {e}",
account.describe()
),
}
}
upgraded
}
/// Move an account to a new endpoint, taking its credential with it.
///
/// The endpoint is half of two keys, and both have to be dealt with. The
/// credential is filed under it ([`Account::secret_ref`]), so rewriting
/// the record alone would strand the app password under the old key and
/// sign the user out. And it feeds [`Account::namespace`], so a rewrite
/// that changed the namespace would abandon the catalog and everything
/// beside it; that is refused outright rather than left to the caller.
///
/// Ordered so an interruption at any step leaves something that works:
/// the credential is copied before the record names the new key, and the
/// old copy is deleted only once nothing names the old one.
fn move_endpoint(&self, account: &Account, endpoint: &str) -> Result<Account, AccountError> {
let mut moved = account.clone();
moved.endpoint = endpoint.to_string();
if moved.namespace() != account.namespace() {
return Err(AccountError::WouldMoveData {
from: account.namespace(),
to: moved.namespace(),
});
}
let mut config = self.read_config();
// Already there — the user signed in again at the new address. That
// record and its credential are the newer, so the old one just goes.
let duplicate = config.sessions.iter().any(|a| a.is_same_as(&moved));
let old_ref = account.secret_ref();
let secret = match self.secrets.retrieve(&old_ref) {
Ok(s) => Some(s),
Err(SecretError::NotFound) => None,
Err(e) => return Err(e.into()),
};
if let (Some(s), false) = (&secret, duplicate) {
self.secrets.store(&moved.secret_ref(), s)?;
}
if duplicate {
config.sessions.retain(|a| !a.is_same_as(account));
} else {
// In place, not removed and pushed: the last record is the one
// the next launch resumes.
for a in config.sessions.iter_mut().filter(|a| a.is_same_as(account)) {
*a = moved.clone();
}
}
self.write_config(&config)?;
if secret.is_some() {
self.secrets.delete(&old_ref)?;
}
Ok(moved)
}
fn read_config(&self) -> ConfigFile {
std::fs::read_to_string(&self.config_path)
.ok()
@@ -531,6 +634,11 @@ pub enum AccountError {
#[error(transparent)]
Remote(#[from] RemoteError),
/// A change that would give an account a different local data directory,
/// leaving its catalog and caches behind under the old one.
#[error("moving the account would leave its local data behind ({from} → {to})")]
WouldMoveData { from: String, to: String },
}
#[cfg(test)]
@@ -560,6 +668,134 @@ mod tests {
d
}
/// A connector that upgrades `http://` endpoints the way Nextcloud's does,
/// and every other one not at all.
struct Upgrading(&'static str);
impl crate::BackendProvider for Upgrading {
fn id(&self) -> &'static str {
self.0
}
fn display_name(&self) -> &'static str {
"U"
}
fn endpoint_label(&self) -> &'static str {
"Server"
}
fn endpoint_placeholder(&self) -> &'static str {
""
}
fn sign_in(&self) -> crate::SignIn {
crate::SignIn::Browser
}
fn normalise_endpoint(&self, i: &str) -> Result<String, String> {
Ok(i.into())
}
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
stored
.strip_prefix("http://")
.map(|rest| format!("https://{rest}"))
}
fn connect(&self, _: &Connection) -> Result<Box<dyn crate::RemoteBackend>, RemoteError> {
Err(RemoteError::Unsupported("stub"))
}
}
fn upgrading(id: &'static str) -> BackendRegistry {
let mut r = BackendRegistry::new();
r.register(std::sync::Arc::new(Upgrading(id)));
r
}
/// TRACES: NFR-SEC-3
#[test]
fn an_http_account_is_upgraded_and_keeps_its_credential_and_data() {
let dir = tmpdir("upgrade");
let store = store_in(&dir);
let old =
Account::new(LEGACY_BACKEND, "http://cloud.example").with_login("duncan", "duncan");
store.save(&folder(), None).unwrap();
store
.save(&old, Some(&Secret::new("secret-token")))
.unwrap();
let moved = store.upgrade_endpoints(&upgrading(LEGACY_BACKEND));
assert_eq!(moved.len(), 1);
let now = store.current().expect("still the account resumed");
assert_eq!(now.endpoint, "https://cloud.example");
assert_eq!(
now.namespace(),
old.namespace(),
"the catalog directory must not move"
);
assert_eq!(
store
.connection(&now, true)
.unwrap()
.require_secret()
.unwrap()
.expose(),
"secret-token",
"the credential moves with the account"
);
assert!(
store.connection(&old, true).is_err(),
"nothing is left under the old key"
);
assert_eq!(store.list().len(), 2, "the folder library is untouched");
// And a second launch has nothing to do.
assert!(store
.upgrade_endpoints(&upgrading(LEGACY_BACKEND))
.is_empty());
}
/// TRACES: NFR-SEC-3
#[test]
fn a_move_that_would_change_the_data_directory_is_refused() {
// A scheme change keeps the namespace, which is what makes the upgrade
// safe; a host change does not, and would give the library a new,
// empty data directory. The account is left exactly as it was.
let dir = tmpdir("upgrade-refused");
let store = store_in(&dir);
let old = nextcloud();
store
.save(&old, Some(&Secret::new("secret-token")))
.unwrap();
assert!(matches!(
store.move_endpoint(&old, "https://elsewhere.example"),
Err(AccountError::WouldMoveData { .. })
));
assert_eq!(store.current().unwrap().endpoint, old.endpoint);
assert!(store.connection(&old, true).is_ok());
}
/// TRACES: NFR-SEC-3
#[test]
fn an_upgrade_onto_an_existing_sign_in_keeps_the_newer_one() {
let dir = tmpdir("upgrade-duplicate");
let store = store_in(&dir);
let old =
Account::new(LEGACY_BACKEND, "http://cloud.example").with_login("duncan", "duncan");
let new =
Account::new(LEGACY_BACKEND, "https://cloud.example").with_login("duncan", "duncan");
store.save(&old, Some(&Secret::new("stale"))).unwrap();
store.save(&new, Some(&Secret::new("fresh"))).unwrap();
store.upgrade_endpoints(&upgrading(LEGACY_BACKEND));
assert_eq!(store.list().len(), 1);
assert_eq!(
store
.connection(&new, true)
.unwrap()
.require_secret()
.unwrap()
.expose(),
"fresh"
);
}
#[test]
fn a_saved_account_survives_reopening() {
let dir = tmpdir("survives");
+14
View File
@@ -83,6 +83,20 @@ pub trait BackendProvider: Send + Sync {
/// wrong rather than naming a type.
fn normalise_endpoint(&self, input: &str) -> Result<String, String>;
/// The form a *stored* endpoint should take now, where an older build
/// wrote one this build would not.
///
/// Not [`normalise_endpoint`](Self::normalise_endpoint) run again: that
/// judges what a person typed, and may touch the world to do it — a folder
/// is canonicalised and must exist — so rerunning it on every launch would
/// fail a library whose disk is unplugged, or rename one whose path now
/// resolves differently. This is a pure rewrite of the string, and `None`
/// means leave it alone, which is the answer for almost every connector.
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
let _ = stored;
None
}
/// Build an account from a normalised endpoint alone.
///
/// Only meaningful for [`SignIn::EndpointOnly`]; a browser flow produces
+80
View File
@@ -240,6 +240,23 @@ impl ThumbStore {
.collect()
}
/// Every file id stored at one size, read from the index in one query.
///
/// For a pass that asks about thousands of images at once — the face
/// audit, the repair lists. Each [`contains`](Self::contains) is a
/// prepared statement and a b-tree probe; asked ten thousand times over a
/// scan it costs more than the scan does, where one walk of the index is
/// a few milliseconds and answers every row.
pub fn held(&self, size: ThumbSize) -> Result<std::collections::HashSet<u64>, ThumbError> {
let mut stmt = self
.index
.prepare("SELECT file_id FROM entries WHERE size = ?1")?;
let ids = stmt
.query_map([size as i64], |r| r.get::<_, i64>(0))?
.collect::<Result<Vec<_>, _>>()?;
Ok(ids.into_iter().map(|id| id as u64).collect())
}
/// Store a thumbnail, opening a new shard if the active one is full.
///
/// Re-storing an existing id overwrites in place rather than migrating it
@@ -481,6 +498,30 @@ impl ThumbStore {
self.dir.join(format!("shard-{shard:04}.sqlite"))
}
/// Write a coherent copy of one shard to `dest`, ready to upload.
///
/// Not a file copy. Every shard is in WAL mode and every `put` opens its
/// own connection, so while thumbnails are being generated on several
/// threads at once — which is exactly when the first sync pass runs —
/// there is nearly always a connection open and the log is never
/// checkpointed. The main file then holds whatever the *last* quiet
/// moment left in it, which for a shard created seconds ago is nothing:
/// zero bytes, the schema still in the log. Reading it uploaded an empty
/// file, and every other device merging it failed with "no such table:
/// thumbs". The backup API serialises against writers and copies the
/// database as it is, log included.
pub fn snapshot_shard(&self, shard: u32, dest: &Path) -> Result<(), ThumbError> {
let source = self.open_shard(shard, false)?;
let _ = std::fs::remove_file(dest);
let mut out = Connection::open(dest)?;
let backup = rusqlite::backup::Backup::new(&source, &mut out)?;
// rusqlite asserts a positive page count where SQLite would take -1
// for "everything"; a shard is capped well under this many pages.
backup.run_to_completion(i32::MAX, std::time::Duration::ZERO, None)?;
drop(backup);
Ok(())
}
pub fn index_path(&self) -> PathBuf {
self.dir.join("index.sqlite")
}
@@ -726,6 +767,45 @@ mod tests {
}
}
#[test]
fn a_snapshot_carries_what_the_shard_file_does_not_yet() {
// A thumbnail worker holding the shard open keeps the log from being
// checkpointed; the file on disk is then not the database. The
// upload used to read that file.
let (mut s, _d) = store();
s.put(1, ThumbSize::Grid, &thumb(1024)).unwrap();
let path = s.shard_path(0);
let worker = Connection::open(&path).unwrap();
worker.pragma_update(None, "journal_mode", "WAL").unwrap();
s.put(2, ThumbSize::Grid, &thumb(2048)).unwrap();
s.put(3, ThumbSize::Grid, &thumb(2048)).unwrap();
// What a byte-for-byte reader sees is at most what the last
// checkpoint left; the log is the part a copy misses.
let copy = _d.join("copied.sqlite");
std::fs::copy(&path, &copy).unwrap();
let file_rows = Connection::open(&copy)
.ok()
.and_then(|c| {
c.query_row("SELECT count(*) FROM thumbs", [], |r| r.get::<_, i64>(0))
.ok()
})
.unwrap_or(0);
let snap = _d.join("upload.sqlite");
s.snapshot_shard(0, &snap).unwrap();
let snapshot = Connection::open(&snap).unwrap();
let rows: i64 = snapshot
.query_row("SELECT count(*) FROM thumbs", [], |r| r.get(0))
.unwrap();
assert_eq!(rows, 3, "the snapshot is the whole shard");
assert!(
file_rows < 3,
"the file alone lagged ({file_rows} rows), which is what the snapshot exists for"
);
drop(worker);
}
#[test]
fn a_forgotten_thumbnail_is_no_longer_served() {
// The point of the whole method: a purged photograph must not keep a
+8
View File
@@ -115,6 +115,10 @@ pub enum PlaceScope {
#[serde(default)]
pub struct StoredFilter {
pub min_rating: u8,
/// TRACES: FR-UI-5
/// The top of a star range. A record from before ranges existed has none,
/// which reads as no ceiling — what it meant when it was written.
pub max_rating: Option<u8>,
pub unjudged: bool,
pub flag: Option<FlagState>,
pub local_only: bool,
@@ -131,6 +135,10 @@ pub struct StoredFilter {
/// Travels: it is a narrowing like `local_only`, and a record without it
/// — from a build before it existed — reads as off.
pub eyes_open: bool,
/// TRACES: FR-CAT-6
/// The colour label the grid was narrowed to. A record without it reads
/// as none, as every term added after the first does.
pub label: Option<crate::ColourLabel>,
}
/// Where the photographer was, at the moment they were there.

Some files were not shown because too many files have changed in this diff Show More