Compare commits

..
110 Commits
Author SHA1 Message Date
dtourolle 031315bdb6 Run cargo fmt over the develop shortcuts and the star-range filter
Benchmarks / CPU and I/O (per commit) (push) Successful in 2m8s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 33s
Build and test / Android (aarch64) (push) Successful in 14m21s
Build and test / android-image (push) Successful in 1s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / Desktop (Linux) (push) Successful in 47m57s
Build and test / windows-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / Layer separation (push) Successful in 51s
Build and test / Windows (x86_64, cross) (push) Successful in 34m3s
Build and test / Publish the release (push) Successful in 1m3s
bddf325 and 00c028c went in unformatted, so the Desktop job's
`cargo fmt --check` step failed on master (run 1693) and the release job
that needs it was skipped. Whitespace only.
2026-09-24 20:05:02 -04:00
dtourolle 00c028c8c8 Rate under the pointer, filter a star range, and name Help as help
Benchmarks / CPU and I/O (per commit) (push) Successful in 1m53s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 2m50s
Build and test / Layer separation (push) Successful in 33s
Traceability / Requirement traces (push) Successful in 40s
🐳 Android image / Build and push (push) Successful in 4s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 4s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 14m19s
Build and test / Windows (x86_64, cross) (push) Successful in 17m50s
Build and test / Publish the release (push) Skipped
Rating keys in the grid follow darktable's rule: with the pointer over a
photograph outside the selection, 0-5, P, X and U judge that photograph
alone; over one inside it, the whole selection, as before; off the grid,
the selection. The hover is cleared when the grid scrolls, so a key after
a wheel turn cannot judge whatever used to be under the pointer.

Holding F and tapping digits filters by stars: one digit for exactly
that many, two for everything between them, F alone to show every
rating again. The filter gains a ceiling to do it (`max_rating`, one
BETWEEN in the query). The place record carries it, and a record from an
older build reads as having none. The star chips light across a capped
range and the bar says "2-3★ only" beside them.

The grid also takes Ctrl+E and Ctrl+Shift+E for the selection, Ctrl+V to
paste onto it and Ctrl+A to select all. The "Gestures" button is now
"Help", its sheet "Controls and shortcuts", and F1 opens it.
2026-09-24 05:11:36 +02:00
dtourolle bddf3250c5 Add Lightroom's export and copy shortcuts to develop
Ctrl+E opens an export sheet: the export defaults on their own, over the
photograph, with an Export button. Ctrl+Shift+E exports straight away on
those defaults. There is no per-export copy of the settings, so what is
chosen in the sheet is saved as it is on the settings page, and the next
Ctrl+Shift+E uses it.

To make that one set of controls in two places, the export options move
out of the settings page into export.slint: an `ExportOptions` global
that Rust writes once, and two panels that read it. The window no longer
forwards forty `settings-*` properties to the page.

Ctrl+Shift+C opens a copy sheet with the edit-kind chips the preset
sheet already uses and a Copy button, which is how a paste leaves each
photograph's crop and rotation alone (Compose off). A and D step along
the roll beside the arrows. While either sheet is up the develop keys
stand down, so A cannot change the photograph behind the form, and
Escape closes it.
2026-09-24 05:11:16 +02:00
dtourolle 41486bd59b Step to the next photograph from the keyboard in a library's develop
The arrow keys and space in develop called `next-image` and
`prev-image`, which walk the files given on the command line. A
photograph opened from a library leaves that list empty, so the keys did
nothing and only a click on the photo roll moved on.

With a library open the step now goes through the roll: it opens the
neighbouring frame exactly as clicking it would, and saves the outgoing
edit the same way.
2026-09-24 05:09:39 +02:00
dtourolle d6d27fb062 Publish a Gitea Release from CI on every v* tag
Benchmarks / CPU and I/O (per commit) (push) Successful in 2m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 45m9s
Build and test / Layer separation (push) Successful in 38s
Traceability / Requirement traces (push) Successful in 44s
🐳 Android image / Build and push (push) Successful in 5s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 5s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 14m25s
Build and test / Windows (x86_64, cross) (push) Successful in 34m30s
Build and test / Publish the release (push) Skipped
Nothing made a release. CI built the APK and the installer on the master
push and kept them as workflow artefacts, the Linux binary was not kept
at all, and most tags went out with no downloads until they were
attached by hand.

build-and-test now also runs on v* tags. On a tag the desktop job keeps
its release binary, and a release job that needs desktop, Android and
Windows collects the three, names them with the version and runs
tools/publish-release.sh. The script titles and describes the release
from the annotated tag's message as the server holds it, writes
SHA256SUMS, and attaches what is not already there, so a re-run after
an interrupted upload finishes the job instead of duplicating it. The
same script is how a release is made or finished by hand.

Tried on v0.14.1, whose release was made by hand with the same files:
it found the release, reported all four files attached, and changed
nothing.
2026-09-24 03:43:42 +02:00
dtourolle 317a2f40bd Release 0.14.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m19s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Successful in 43m6s
Build and test / Layer separation (push) Successful in 37s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 3s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Traceability / Requirement traces (push) Successful in 46s
Build and test / Android (aarch64) (push) Successful in 29m7s
Build and test / Windows (x86_64, cross) (push) Successful in 34m31s
2026-09-23 19:07:22 -04:00
dtourolle 949fe40d5b Pin "Export N" to the right of the library's selection bar
It was the last of a dozen buttons in a row that scrolls sideways once
it outgrows the window, which at a desktop width it does. The button
sat past the right edge and nothing says the row scrolls to a mouse,
so batch export of a selection looked like a feature the library did
not have. The rest of the row still scrolls; Export, which is also the
cancel for a running batch, now sits beside it and is always visible.
2026-09-23 19:05:42 -04:00
dtourolle 2be80d4203 Show "Paste to N" on every selection, disabled until something is copied
It appeared only once settings had been copied this session, so with an
empty clipboard nothing on the selection bar said pasting onto a
selection was possible. It is one of the things a selection can have
done to it, like filing it in a collection, and now sits with them
beside Presets.
2026-09-22 21:37:41 -04:00
dtourolle 9ebaa15099 Move copy, paste and presets into the develop top bar
They sat in the develop column under a "SETTINGS" heading, which read as
application settings, and went away with the panel toggle and in the
mask and spot modes. The strip is where undo already is for the same
reason: these act on the whole edit, not on any one panel.

The paste button still names what it would apply. The TransferPanel
component is gone; the Transfer global and its Rust wiring are
unchanged.
2026-09-22 21:37:31 -04:00
dtourolle aee355fada Make the in-flight claim test wait for the waiter to park
a_second_claim_waits_for_the_first_to_be_released failed on CI: the
second claim came back Some. The waiter thread signalled the main thread
before calling claim, so the main thread could drop the first guard
before the waiter reached the lock. The path was free by then, and the
waiter claimed it outright.

The registry now keeps a test-only count of threads parked in claim,
bumped under the lock just before the condvar wait. The test spins until
that count is one before releasing. The release needs the same lock, so
it can only reach a waiter that is already waiting. Passed 500 runs in a
row.
2026-09-22 21:12:45 -04:00
dtourolle 7fa3176f88 Release 0.14.0
Benchmarks / Frame budget (on demand) (push) Skipped
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m25s
Build and test / Desktop (Linux) (push) Failing after 26m11s
Build and test / Layer separation (push) Successful in 33s
🐳 Android image / Build and push (push) Successful in 10m41s
Build and test / android-image (push) Successful in 10m42s
🐳 Windows image / Build and push (push) Successful in 4m17s
Build and test / windows-image (push) Successful in 4m17s
Traceability / Requirement traces (push) Successful in 59s
Build and test / Android (aarch64) (push) Successful in 40m53s
Build and test / Windows (x86_64, cross) (push) Successful in 46m51s
2026-09-21 11:32:41 +02:00
dtourolle af162dd010 Merge: origin's sync ordering and adoption work, its scan-complete trigger ported into library_ui/ 2026-09-21 11:32:05 +02:00
dtourolle f5300a9f43 Merge: the UI crate restructured so a feature owns a function, not a region
Nine agent branches, merged one at a time on ui-wiring and gated at each
step: develop.rs, library.rs, collections_ui.rs and library_ui.rs become
module directories; every screen's wire() and lib.rs::run() become lists
of named functions, with the develop screen's callbacks in develop_ui.rs;
the six view booleans become View and Page enums; the collections sidebar
and the library grid get their own Slint globals, taking 155 members off
AppWindow. No behaviour change: a multiset audit of code lines over every
moved region lost nothing, the workspace gate is clean, and the manual
recorded from this build and from master's from the same library snapshot
matches picture for picture, apart from a panorama stall that master
shows too when a merge starts during the engine's TensorRT compile queue.
2026-09-20 23:41:25 +02:00
dtourolle b19d470189 Merge: the library grid on its own Slint global, the last of the screen state off the root 2026-09-20 22:43:09 +02:00
dtourolleandClaude Opus 5 bfadd9c409 Ship the border filler in the Windows installer, and count what is staged
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m2s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 25m17s
Build and test / Layer separation (push) Failing after 0s
Traceability / Requirement traces (push) Failing after 0s
🐳 Android image / Build and push (push) Failing after 0s
Build and test / android-image (push) Failing after 0s
Build and test / Android (aarch64) (push) Skipped
🐳 Windows image / Build and push (push) Failing after 0s
Build and test / windows-image (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Skipped
The installer smoke test asserted seven model files, the number on the
day it was written; models/face has since gained the eye-state trio's
companions and the int8 detector forms, and the run on f71d7ba failed
with thirteen installed. The test now expects as many files as
package.sh's directories hold, so the next model needs no edit here.

package.sh also stages models/inpaint, which the APK and the Arch
package already carry and the Windows build did not: without
migan-512.onnx the panorama's border fill has no model on Windows.
xfeat needs nothing, it is embedded in the binary. windows.md §5.2
lists the result.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 22:35:04 +02:00
dtourolle 050b2a3914 Collapse the blank runs the library global's move left behind
Moving each group of properties and callbacks out of AppWindow left
several two- and three-line gaps where a removed block's neighbours no
longer needed separating. No declarations changed.
2026-09-20 22:32:47 +02:00
dtourolleandClaude Opus 5 195388b2e3 Adopt faces a hundred images per commit, and hold one generation per image in the shards
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 4h42m34s
Build and test / Layer separation (push) Successful in 59s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The shard import recorded each adopted image in its own transaction:
fourteen thousand commits, and fourteen thousand turns at the write lock
that every read on the UI thread queued behind — the sync was felt as a
laggy grid and as "database is locked" from whichever writer lost the
wait. `record_detections_within` takes the caller's transaction, and the
import commits every hundred images.

The store carried every detector generation of an image — 24,123 entries
for 19,089 images on the reference library, a third of its 293 MB — when
only the strongest is ever adopted. A put now skips a pass a held one
outranks, and retires the passes it outranks from the index; sealed
shards keep their bytes, but nothing is written twice from here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 22:31:47 +02:00
dtourolle e166b64ee9 Format fill.rs, which the last release left past the width limit 2026-09-20 22:29:42 +02:00
dtourolle a773ad5c27 Merge: the developer docs under docs/dev, and the folder indexed for users first 2026-09-20 22:27:48 +02:00
dtourolle 1e472fd251 Move the library's routes and status onto its own Slint global
The scan/opening/status/error lines, offline mode and its retry, pinning a
collection offline, the sync and thumbnail-sweep state, and the callbacks
that route the grid to a rescan, a sync, another library, a panorama merge
or an export — the last of what AppWindow still carried under the
library- prefix — move onto the `Library` global started earlier on this
branch. Rust reaches them through window.global::<Library>() rather than
window.set_/get_/on_/invoke_ on the root.

library-visible is the one name that stays: it is computed from
active-page and active-view, the shell's own routing state, which a
global cannot read. AppWindow now declares no other library- property or
callback.
2026-09-20 22:22:55 +02:00
dtourolle 6bf67cefc4 Move the filter bar and the timeline onto the library's Slint global
The date-range fields and their band on the capture-time axis, the
timeline's bars, labels, scrub/pinch/pan/zoom callbacks and its
sweep/current-bucket/anchored state, and the filter bar's people chips,
mode, eyes-open toggle and gesture reference move from AppWindow onto the
`Library` global. Rust reaches them through window.global::<Library>()
rather than window.set_/get_/on_ on the root, as the earlier commits on
this branch did for the grid's cells, selection, ratings and keywords.
2026-09-20 22:07:12 +02:00
dtourolle 00e2fe6aaf Move ratings, flags and keywording onto the library's Slint global
The keywording sheet's rows and its open/assign/unassign callbacks, the
star and flag callbacks a cell click or a judgement key fires, the burst
toggle and representative-chosen callbacks, the trash-selection shortcut,
and the rating/unjudged/flag filter chips with their rating-counts model
move from AppWindow onto the `Library` global started in the previous
commit. Rust reaches them through window.global::<Library>() rather than
window.set_/get_/on_ on the root.
2026-09-20 21:57:40 +02:00
dtourolle 402dcdc24c Move the library grid's cells and selection onto their own Slint global
AppWindow carried the grid's loaded window of cells, the keyboard cursor,
drag and drop, the held-row long-press state, columns and cell size, the
scroll and viewport bookkeeping, the photo roll's pick and centre-request,
and the local-only/reorder/collection-filing gestures that act on a
selection, as properties and callbacks on the root component. That state now
lives in the `Library` global declared in library.slint, next to the structs
(LibraryCell, TimelineBar, KeywordRow, PersonChip) it and the grid's other
components already share; Rust reaches it through
window.global::<Library>() instead of window.set_/get_/on_/invoke_ on the
root, the same change collections.slint's `Collections` global made for the
sidebar.

library-visible stays on AppWindow: it is computed from active-page and
active-view, the shell's own routing state, which a global cannot read.
Everything else still prefixed library- — the timeline, the filter bar,
ratings and flags, keywording, and the routes and status lines — stays on
the window for now and moves in the commits that follow.
2026-09-20 21:42:48 +02:00
dtourolle 94b39410bc Tidy the ported drag block, and format fill.rs as master left it 2026-09-20 21:40:20 +02:00
dtourolle 2014c80e62 Merge: master at 0.13.6, with the drag-ghost file and the shared model lookup ported into the split modules 2026-09-20 21:32:21 +02:00
dtourolleandClaude Opus 5 6fd342680b Format set_indexed_at's signature
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m15s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Build and test / Desktop (Linux) (push) Successful in 1h8m29s
🐳 Windows image / Build and push (push) Successful in 5s
Build and test / windows-image (push) Successful in 5s
Build and test / Layer separation (push) Successful in 34s
Traceability / Requirement traces (push) Successful in 50s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
baed1c4 landed it over rustfmt's width; `cargo fmt --check` is the
first gate the Desktop job runs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:31:07 +02:00
dtourolleandClaude Opus 5 78acc73dad Regenerate the traceability matrix for the sync commits
f71d7ba and 34ac2f1 added tags without re-running the report, which
the Traceability job's "is it committed" step rejects.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:28:55 +02:00
dtourolleandClaude Opus 5 954246b969 Register the inference requirements, and format the fill test
Two CI gates have failed on every push since 0.13.4 and both are
fixed here.

The traceability gate rejected `FR-INF-1` as an orphan: settings.slint
and dr-ui tag it, but the four register entries inference.md §12 wrote
were never carried into requirements.md, which is the only file the
extractor reads. §3.12 and §4.10 now hold FR-INF-1..3 and NFR-INF-1
verbatim, with the acceptance milestones pointed back at inference.md.
The matrix is regenerated (188 defined, 155 covered) and the README's
"where it stands" line, which the 0.13.6 release commit skipped, says
0.13.6 and the new figures.

`cargo fmt --check` failed on the `fill_border` call in dr-pano's
padding test, which is the first thing the Desktop job runs after
installing the toolchain and why it failed within a minute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:28:55 +02:00
dtourolleandClaude Opus 5 baed1c4782 Keep the peer's run marker on adopted faces, so they are not re-exported as ours
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 32s
Build and test / Layer separation (push) Successful in 29s
Traceability / Requirement traces (push) Failing after 42s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
Adopting an image from a peer's face shard stamped its run marker as now,
and the export reads a catalog marker newer than the shard's as a
re-index. So every adopted image went straight back out under this
device's client id: 14,100 adopted, 15,457 "newly indexed" on the next
pass, twenty-two shards of a peer's faces uploaded a second time.

merge_shard now carries the peer's indexed_at into the local index, and
the import writes that marker into face_index; where an older peer's shard
carries none, the store takes the catalog's, so the two agree either way
and the export finds nothing to send.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:25:53 +02:00
dtourolle fbfa891296 Regenerate the traceability matrix after the rebase 2026-09-20 21:23:23 +02:00
dtourolle aee62dc7f2 Wrap the line the docs move pushed past the width limit 2026-09-20 21:16:03 +02:00
dtourolle 84fade99ec Put the developer docs under docs/dev and index the folder for users first
docs/ had 26 developer documents flat beside the manual, and the two
audiences are very differently sized: most readers want the manual and
the gesture reference, a few want the register, the designs and the
measurements. The manual and gestures.md stay at the top; everything for
someone changing the code moves to docs/dev/, and the two documents that
name their own successors — the v0.1 milestone and the UI-refinement plan
— go to docs/dev/archive/ rather than being deleted, since both are still
cited. docs/README.md is the index, users first.

Every reference follows: code comments, Cargo manifests, the workflows,
the pre-commit hook, the bench and traceability tools (which locate the
repo root by docs/dev/requirements.md now), packaging, the Docker READMEs,
CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level
deeper and is regenerated. Links out of the moved documents into the tree
gain a level; a link checker over every Markdown file finds none broken.
2026-09-20 21:16:03 +02:00
dtourolle 3bfa73d1e1 Merge: the collections sidebar on its own Slint global 2026-09-20 21:14:00 +02:00
dtourolle 4616cb0a23 Merge: library_ui split into a module directory 2026-09-20 21:12:36 +02:00
dtourolle cc73ea3153 Split library_ui.rs into a module directory by area of behaviour
controller holds LibraryController and the window-sizing constants every
other module reads and writes through pub(super) fields, the same shape
collections_ui and develop already use. open is the launch-to-scan cycle and
the worker that checks the catalog file before either touches it. offline is
what of a collection is on this device and the prompt that offers to change
it. window fills the grid model from the catalog and drains the thumbnail
fetch, which is the piece the catalog-reads-are-proportional-to-what-changed
rule (docs/catalog.md §1) bears on most directly. sync is the background
passes that reach beyond the loaded window: the metadata sweep, the
whole-library thumbnail pass, and the exchange with the server.
ratings_keywords applies a judgement or a keyword to a selection and queues
the sidecar and XMP writes behind it. timeline is the capture-time sidebar
and the photographer's place together, kept in one file because a restored
place ends by moving the timeline marker and a scrub is a restore of one
instant, so most calls between the two would otherwise cross a module
boundary. grid wires the grid's own callbacks — the keyboard cursor,
cell-size zoom, the routes into and out of develop — and filter_bar wires
the rating, people, date and offline-scope filters, calling back into
whichever of the above owns the work a filter change triggers.

Extracted by item rather than by line range, so every doc comment and
TRACES/GESTURE annotation stayed attached to the code it describes; the
sorted set of TRACES/GESTURE lines in the new directory is identical to the
original file's. Tests moved with the code they exercise, including the
handful of fixtures — settle, model_of, with_catalog, zoom_cell, pinch_step
— that only one target module needed and so were not worth sharing through
a test_support module the way the other splits use one. Items that crossed
a new module boundary were widened from private to pub(super), narrower
than the whole-file access the original gave them; a few items already
pub(crate) for recovery_ui or presets stayed there rather than being
narrowed, since nothing needed them tightened further.

mod.rs re-exports the same surface library_ui:: callers used before, so
lib.rs and every other caller needed no change.
2026-09-20 21:10:02 +02:00
dtourolle f6ff5eabd9 Give the collections sidebar its own Slint global
AppWindow carried the sidebar's tree, its row menu, renaming, drag and
drop between rows, the trash row, and the membership sheet as ~40
properties and callbacks on the root component, in the pattern CH-1
describes and the develop screen's globals (Adjustments, Framing, Steps,
...) already replaced. Collections.* in collections.slint now holds that
state, declared next to the MembershipRow struct it and the membership
sheet both use; Rust reaches it through window.global::<Collections>()
instead of window.set_/get_/on_/invoke_ on the root.

collection-selected, collection-select, and collection-offline-menu stay
on AppWindow: library_ui.rs invokes collection-select directly and
registers collection-offline-menu's handler, and lib.rs reads
collection-selected for back-navigation, so moving them would have meant
editing library_ui.rs, which another change on this branch is splitting
into a module directory. collections-visible stays too — it is
lib.rs's panel-layout state, seeded from the saved layout before the
sidebar exists, and collections_ui never touches it. Everything prefixed
library- (the grid's drag, selection and keyword state that the sidebar's
Rust also wires for cross-feature gestures like filing a selection into
a collection) stays on the window as well, since it belongs to the
library screen, not the sidebar.
2026-09-20 21:07:25 +02:00
dtourolleandClaude Opus 5 34ac2f14d3 Sync faces and the catalog before thumbnails, so a fresh device sees its names and collections first
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 4s
Build and test / Desktop (Linux) (push) Failing after 35s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 30s
Traceability / Requirement traces (push) Failing after 48s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The thumbnail stage ran first and, on a device that had just adopted its
peers' shards, spent its time re-uploading hundreds of megabytes under its
own client id while faces, people, collections and dates waited behind it.
Faces go first — the catalog merge assigns identities to faces this device
holds — then the catalog, then thumbnails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:04:10 +02:00
dtourolleandClaude Opus 5 f71d7bacc6 Take the server's shards and dates when the scan completes, not after the sweep
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 36s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 43m11s
Build and test / Windows (x86_64, cross) (push) Failing after 41m4s
The derived sync fired only after the metadata sweep, so a fresh device
re-derived every thumbnail it scrolled past, re-detected faces and re-read
every header for hours before adopting the shards and snapshot that held
all of it. It now fires as soon as the scan completes — the first moment
the rows the merges key on exist — and the sweep starts behind it. In
steady state that pass is one listing.

The catalog merge gains a fourth half: capture metadata (captured_at,
offset, camera, lens, ISO) for images still at metadata_state < 2, matched
by oc:fileid from a remote row at 2. A date is a fact about the file's
bytes, not local state, and the snapshot already carried it. The sweep's
per-chunk query then finds nothing left, and the timeline is whole on a
fresh device without a header fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 20:37:59 +02:00
dtourolle 14ed1dc410 Merge: View and Page enums in place of the six view booleans 2026-09-20 20:32:41 +02:00
dtourolle 38819da222 Replace the six view booleans with View and Page enums
app.slint carried show-launch, show-library, show-identity, show-settings,
show-import and show-merge as separate booleans, so the root component chose
what to draw with five- and six-term conjunctions and nothing stopped two of
them being true at once. Replaced with two enums: View { develop, library,
identity, launch } for which top-level screen is showing, and Page { none,
settings, import, merge } for which page, if any, is drawn over it.

Two values rather than one, because the two questions are genuinely
different. Settings, Import and Merge are reachable from more than one View
and are drawn outermost without touching it — closing one has to return to
whichever View was already current, and today that works because the
underlying property is left alone while the page sits over it. A single
View with five or more variants would need a second field remembering what
to return to; Page needs nothing to remember, since View was never
overwritten in the first place. Identity, by contrast, genuinely replaces
the window the way Launch and Library do (see the existing "like the launch
screen" comment on its `if`), so it is a View variant, not a Page.

Every `if` chain in app.slint that used to compare four, five or six
booleans now compares active-view and active-page to at most one variant
each. library-visible collapsed from a six-term conjunction to
`active-page == Page.none && active-view == View.library`.

The Rust side follows: every set_show_*/get_show_* call in library_ui.rs,
identity_ui.rs, settings_ui.rs, merge_ui.rs, import_ui.rs, launch_ui.rs and
lib.rs now reads or writes active-view or active-page instead, including
lib.rs's startup match (View.launch vs View.develop, since a Startup that
skips the launch screen used to leave both old booleans false and fall
through the chain to develop) and identity_ui's close handler, which now
writes View.library or View.develop in one call where it used to write
show-library then show-identity separately.

back_one_step needed one deliberate adjustment beyond the mechanical
rename. Identity was never represented in NavState: back had nothing to do
when Identity was opened from the library (show-library stayed true,
unread by IdentityScreen's own condition) and could only reach ToLibrary
when opened from develop, which likewise wrote a property IdentityScreen
never read — so escaping out of Identity was invisible in both cases before
this change. With a single active-view, falling into the general case
would instead overwrite the value IdentityScreen's `if` does read and close
it as an unintended side effect. back_one_step now swallows the gesture
while View.identity is current, reproducing the same "nothing visible
happens" outcome for both origins without threading identity_ui's private
came-from-library state through lib.rs for one screen.

Verified with tools/manual/drive.py against a private Xvfb and the debug
build: launch screen to library, Settings opened and closed, Identity
opened and closed (including Escape doing nothing while it is open),
develop opened from a cell and closed both by the back button and by
Escape. Screenshots under verify/.
2026-09-20 20:30:56 +02:00
dtourolle d6e9c7dc94 Merge: collections_ui split into a module directory 2026-09-20 20:30:12 +02:00
dtourolle 9c8f21b754 Split collections_ui.rs into a module directory by area of behaviour
collections_ui.rs had grown to 4,591 lines covering the sidebar controller,
the click/drag selection policy, tree refresh, the drag gesture, the trash
worker, twelve wiring functions, and the row's rename/create/context menu,
all in one file. Split into collections_ui/ with one module per area, the
way develop/ and library/ were already split on this branch:

- controller.rs: CollectionsController and the pure drop/delete/release
  decisions (decide_drop, decide_delete, decide_release, menu_detail,
  delete_warning) that a test can drive without a window.
- press.rs: PressUndo and the click-and-release selection policy
  (apply_press, select_row, commit_press, cancel_press).
- tree_sync.rs: rebuilding the sidebar from the catalog and pushing
  catalog-derived state into the grid (refresh_tree, offline_state,
  sync_lifted/sync_selection/sync_reorderable/sync_badges,
  refresh_membership, direct_holdings).
- drag.rs: the cursor bitmap (compose_drag_image, blit_scaled) and the
  hold/spring timers (arm_hold, arm_spring, should_spring,
  collapse_spring_opened) plus their delay constants.
- trash.rs: the soft delete (start_trash, start_restore, drain_trash,
  stop_trash, refresh_trash, format_bytes).
- wiring_grid.rs / wiring_tree.rs: the wire() entry point and its twelve
  wire_* functions, split in two because together they were the largest
  single piece (grid-facing selection/drag/trash vs. sidebar-facing
  navigation/create/rename/row-drag/menu/membership).
- rename_menu.rs: creating, naming and renaming collections, and the row's
  context menu (apply_rename, create_child, unique_name, open_row_menu,
  close_row_menu, close_rename).

mod.rs carries the module's own top-level doc comment, the `pub use`
re-exports for the eight items the rest of the crate reaches by
`collections_ui::` path (CollectionsController, wire, refresh_tree,
sync_badges, sync_selection, select_row, commit_press, cancel_press), and a
shared `test_support` for the one fixture (`ids`) more than one file's
tests needed. Every item that only crossed a boundary within this module,
not out of it, was narrowed to `pub(super)` rather than kept at the
crate-wide `pub` a single file gave it for free.

Extracted with a brace-aware pass that kept each item's own leading doc
comment and attributes attached to it, and tests moved with the code they
exercise; every TRACES/GESTURE comment lands on the same code it did
before. No file outside the new directory changed — lib.rs's `mod
collections_ui;` resolves to the directory automatically, and every
outside caller's `collections_ui::` path still resolves through mod.rs's
re-exports.
2026-09-20 20:29:21 +02:00
dtourolle bd7d75522d Merge: the format fix from docs-layout 2026-09-20 20:26:56 +02:00
dtourolle 4ef1b74f2f Wrap the line the docs move pushed past the width limit 2026-09-20 20:26:54 +02:00
dtourolle 681486196e Release 0.13.6
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m17s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Build and test / Desktop (Linux) (push) Failing after 29s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 42s
Build and test / Android (aarch64) (push) Failing after 2m48s
Build and test / Windows (x86_64, cross) (push) Failing after 4m22s
2026-09-20 20:19:48 +02:00
dtourolle f5d0d57574 Regenerate the traceability matrix after the rebase 2026-09-20 20:19:43 +02:00
dtourolle c96e670356 Re-record the panorama for the trained filler; the scene waits for the preview and for the DNG instead of guessing 2026-09-20 20:19:01 +02:00
dtourolle 9c556364fa Pad an open void's canvas to a tile: the merge page's preview is shorter than one, and filled nothing 2026-09-20 20:19:01 +02:00
dtourolle 8d72cabff5 Ship the border filler trained against MI-GAN's own discriminator: texture in the deep bands, level with stock on LPIPS 2026-09-20 20:19:01 +02:00
dtourolleandClaude Opus 5 8a90d888d5 Probe and compile for a package's models, not only the user's
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 53s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m19s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
`inference::init` listed the models from the user's shared directory
alone, while the app loads them from there or from the package's
`/usr/share/darkroom/models`. On a fresh package install the probe found
"no model to probe with", stayed on the CPU, and compiled nothing. Both
now resolve each file with the same search, `library::shared_model`.

The PKGBUILD names the ONNX Runtime packages as optional dependencies,
since the app loads one from /usr/lib if present.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 19:39:05 +02:00
dtourolle e86edef47c Merge: run() reduced to construction and startup, the develop wiring in its own module 2026-09-20 19:28:23 +02:00
dtourolle 0aff5e6c8c Merge: the collections, identity, merge, launch and import wiring lifted into named functions 2026-09-20 19:28:12 +02:00
dtourolle 5458314083 Split import_ui::wire into one function per section
The 188-line wire() registered the import page's callbacks in four
comment-delimited sections. Lift each into its own fn: wire_opening_and_closing,
wire_choosing_a_source, wire_options, wire_running. context stays generic
over C on each of the three functions that use it, matching how survey()
and start() already take it (impl Fn, implicitly Sized) rather than coercing
it to a trait object, which would have needed ?Sized added to those two
unrelated functions for no benefit.
2026-09-20 19:23:21 +02:00
dtourolleandClaude Opus 5 39a22875b1 Add the MIGraphX rung for AMD GPUs
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m20s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 45s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 46s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m19s
Build and test / Windows (x86_64, cross) (push) Failing after 3m2s
Measured on a Radeon RX 7900 XT against Arch's onnxruntime-rocm 1.29
(docs/inference.md §1.3): MIGraphX fp16 runs the detectors at 2.4–3.4 ms
against 10–58 ms on the CPU provider, the inpainter at 8 ms against 514,
with a 15–135 s compile per graph the first time and under a second from
its cache after. A compiling rung on TensorRT's terms, wired the same way.

The ROCm execution provider is gone (removed in ONNX Runtime 1.23), so the
AMD ladder is MIGraphX then the CPU, with no non-compiling rung between.

MIGraphX is registered through the runtime's generic key/value entry
point rather than ort's builder: 1.29 reads the legacy options struct for
its precision flags only, and the compiled-program cache directory
(`migraphx_model_cache_dir`) only travels the generic way. The provider's
cache key omits the precision, so f32 and fp16 programs get their own
directories. The probe fingerprint now includes the provider libraries
beside the runtime and the ROCm version, since a distribution's CPU and
ROCm builds are the same file at the same path.

`status().failed` reports only the rungs above the selection, so an AMD
desktop's About line says why MIGraphX won rather than that the NVIDIA
providers are not in the build.

Two examples: `ep_probe` times each provider cold and from cache, and
`ladder` drives `init` as the app does to watch the first-run sequence.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 19:23:00 +02:00
dtourolle cb7b9d71c4 Split lib.rs::run into named construction and wiring functions
run() was 2,724 lines that built every controller, owned the develop
session and its render loop, and registered every develop-screen callback
inline — the state CH-1 in docs/dev/code-health.md describes. This is the
mechanical split CH-1 calls for, done in one pass rather than section by
section since develop_ui.rs only compiles once lib.rs stops registering
those callbacks itself.

develop_ui.rs is new: a DevelopWiring struct holding the session, the rows
model, the redraw/render closures and the other controllers' handles, and
one wire_* function per section run() used to contain — presets, export,
the adjustment panel, film, undo/redo, zoom/pan/crop, rotation/flips/
straightening, navigation and peaking — called in the order run()
registered them. window travels as its own parameter throughout rather
than living on the struct, because the generated AppWindow type is not
Clone; every other field is an owned clone so each function's body could
be pasted from run() unchanged.

lib.rs::run is now under 300 lines: construction and startup only, calling
named functions for the window and its diagnostics/inference wiring, the
launch/library/collections/identity/settings screens, import and merge,
the render loop (render_now/redraw/show), the remote open path, the
window chrome (resize, layout class, panel toggles, back gesture), and
develop_ui::wire for the rest. Every TRACES and GESTURE comment moved with
the code it annotates.

Two latent type errors surfaced while restructuring rather than being
introduced by it: activity and display were being passed around as bare
ActivityLog/DisplayWatch instead of the Rc<...> their constructors
actually return, which only worked before because nothing needed to name
the type explicitly.
2026-09-20 19:17:22 +02:00
dtourolle beb7a5eac0 Split launch_ui::wire into one function per section
The 225-line wire() registered the launch screen's callbacks in nine
comment-delimited sections. Lift them into fn's, merging a few adjacent
ones that were only a handful of lines each: wire_sign_in covers both the
browser flow and the app-password fallback (the same button in two forms),
wire_choose_folder_and_open covers opening the folder browser and the
final "Open library" press, since both are short and sit back to back.
wire_use_folder, wire_sign_out, wire_formats, wire_folder_picker_navigation
and wire_copy_url stay as they were sectioned. wire() calls each in the
original order and keeps the closing render() call, which every section
relies on having run once at startup.
2026-09-20 19:11:43 +02:00
dtourolle 262ed2553c Split merge_ui::wire into one function per section
The 312-line wire() registered the panorama page's callbacks in three
comment-delimited sections. Lift each into its own fn: wire_start (the
"Merge to panorama" press, its fetch, and the DARKROOM_START_MERGE dev
entry point — all three only ever used together), wire_decision (confirm,
projection and border chips, the fill knobs), and wire_stop_and_leave
(abandon, close). wire() itself now just calls the three in order; S, C
and F stay generic on wire_start since sources/context/on_done are used
nowhere else.
2026-09-20 19:00:31 +02:00
dtourolle 8d08ffd7b7 Split identity_ui::wire into one function per feature
The 837-line wire() had almost no section comments, unlike its siblings, so
the seams had to be found by reading it rather than following markers. Lift
each into its own fn: wire_dials (the two grouping sliders), wire_navigation
(open/close/switch person — needs models() too, for the missing-model
banner), wire_rename_and_merge (a rename and the namesake offer it can
raise), wire_face_actions (pick/confirm/reject/split, the grid's own
actions), wire_grouping_preview, wire_recluster, wire_indexing (the shared
launcher behind Index/Re-index plus Stop), and wire_coverage_and_ignore.
wire() keeps the generic-to-trait-object coercions and the eyes_available
closure, since most of the above need it, and calls each function in the
original order. The reload! macro moved from inside wire() to module scope,
dedented, since macro_rules is scoped textually and every extracted function
uses it.
2026-09-20 18:49:43 +02:00
dtourolle 8540518022 Split collections_ui::wire into one function per section
The 1,457-line wire() registered every collections-sidebar and grid-drag
callback in one function, sectioned only by comment. Lift each section
into its own fn: wire_selection (split further into wire_selection and
wire_selection_filing, since the original section ran to 375 lines),
wire_drag, wire_trash, wire_trash_from_grid, wire_tree_navigation,
wire_create, wire_rename, wire_remove, wire_row_drag (the tree row's own
hold-drag, which the original "remove" comment's span covered but which
is really a separate feature), wire_row_menu, and wire_membership.
wire() itself now just coerces the shared closures to trait objects and
calls each in the original order. visible_ids is coerced to
Rc<dyn Fn() -> Vec<ImageId>> at the top, alongside on_scope_changed and
session, so the new functions take plain trait objects instead of
threading a generic parameter through every one of them.
2026-09-20 18:36:18 +02:00
dtourolle b952f5976a Merge: the wire sections lifted into named functions, beside the module splits 2026-09-20 18:26:24 +02:00
dtourolle a1d511fd4b Split library.rs into library/ by area of behaviour
library.rs was 7,729 lines wiring together everything "open a remote
library" touches: scanning, pulling other devices' judgements out of
sidecars found along the way, writing local edits back out to the
sidecar outbox, pushing/reloading XMP by hand, fetching and prefetching
thumbnails and originals, generating thumbnails locally, the metadata
and thumbnail background sweeps, on-disk paths for the catalog and
model files, and reading the grid's cells, spans and rating filter.
Same motivation as the develop.rs split (docs/dev/code-health.md CH-1):
a pure, no-behaviour-change move into one file per area, each under
about 1,500 lines.

Tracing actual call sites rather than trusting the file's physical
layout mattered here: `persist`, `load_folder_etags`, `pull_sidecars`,
`load_sidecar_etags`, `record_sidecar_read` and `apply_judgement` sit
textually beside the XMP push/reload functions but are called only
from `run_scan` (pulling a device's own past judgements out of the
sidecars a scan just walked), so they went to scan.rs and not xmp.rs.
`cells` came out at over 1,800 lines once its tests moved with it and
split further into cells.rs (windowed reads, trash, ordinals) and
spans.rs (collection scope, manual reordering, the capture-time
histogram) -- ten submodules rather than the nine first planned.

Previously-private items reached from a sibling module became
`pub(super)`, narrower than the whole-crate reachability one file gave
them. Tests moved with the code they test; the two test fixtures used
across more than one file (`scanned`, and develop.rs's
`session_with_a_left_half_subject` in the matching commit) joined the
shared `test_support` module alongside the existing `entry`/
`with_images`/`image_ids` helpers. `mod.rs` re-exports every module's
public items under `library::`, including the `pub(crate)`
`test_support` module `repairs.rs` reads its fixtures from, so no file
outside `library` needed a change.

The previous commit split develop.rs the same way; taken alone it left
dr-ui without library.rs, so that intermediate commit does not build on
its own. This one restores it.
2026-09-20 18:21:43 +02:00
dtourolle 050c2c9d16 Split develop.rs into develop/ by area of behaviour
develop.rs had grown to 9,327 lines covering everything the develop
session does: opening a photograph, the parameter-row and curve-widget
panel model, mask viewing and editing, mask creation and the rasteriser
that turns a mask stack into GPU arrays, spot repairs, scene
segmentation, framing and zoom, white-balance sampling, rendering and
film choice, and the undo/snapshot history. docs/dev/code-health.md
CH-1 names dr-ui's lack of a view layer as the reason every feature
kept landing in a handful of files; this is the first of the two pure
splits it recommends as easy, no-behaviour-change wins independent of
that larger rework.

The boundaries follow the file's own sections (several were already
marked off with comment headers) and the seams a full read turned up
underneath them -- mask storage/rasterisation turned out to be a
distinct concern from mask viewing and editing, and rows/tabs/curves
from each other, so those split further than the headers alone
suggested. Each module stays under about 1,500 lines. Struct fields
and the handful of helper methods now called from a sibling module
became `pub(super)`, which is strictly narrower than the whole-crate
reachability a single file gave them; nothing gained visibility outside
`develop`. Tests moved with the code they test, including the few
cases where a helper one file's tests needed was itself only defined
in another's -- those became shared fixtures in `mod.rs` alongside the
`headless`/`read_back`/`grey_session` helpers that already worked that
way. `mod.rs` re-exports every item `develop::` callers outside this
module used before, so lib.rs, masks_ui.rs and the rest needed no
changes.
2026-09-20 18:21:26 +02:00
dtourolle bd3b993b90 Split library_ui::wire into one function per section
wire() registered every grid callback in one 1,214-line function behind
four section comments, two of which were themselves far over 300 lines
with no further markers. Each fenced section becomes its own function,
called from wire() in the original order with the section's own comment
kept as its doc comment:

- "the keyboard cursor (FR-CULL-4)" (496 lines) splits at its own topic
  breaks into wire_grid_cursor_and_zoom (cursor movement, cell zoom and
  pinch), wire_grid_sync_and_load (explicit sync/thumbnail requests and
  the reloads a changed viewport, column count or scroll position
  trigger), wire_timeline (the capture-time sidebar) and wire_grid_routes
  (grid/launch/develop navigation and a manual rescan).
- "ratings and flags" and "keywords" were already under 300 lines and
  become one function each.
- "the filter bar" (447 lines) splits into wire_filter_ratings_and_people
  (which keeps the section's own comment), wire_filter_dates and
  wire_filter_scope_and_offline.

The three callbacks registered before the first section comment
(on_settings_xmp_reload, on_library_cell_clicked, on_library_roll_pick)
and the trailing crate::recovery_ui::wire call stay directly in wire(),
since neither is inside a fenced section.
2026-09-20 17:25:00 +02:00
dtourolle 59605f9fbb Split masks_ui::wire into one function per section
wire() registered every mask-panel callback in one 778-line function
behind section comments. Each of the seven fenced sections (computing
the region map, refining a subject's mask, dragging a gradient,
selecting on the photograph, the stack, the edge treatment, adding
layers) becomes its own function, called from wire() in the original
order with the section's own comment kept as its doc comment. The
"adding layers" section was itself over 300 lines and had no further
section markers inside it, so it is split at its own natural seam
between painting/viewing a mask (wire_layers_paint) and working the
parts and add-mask buttons (wire_layers_parts); the second half gets an
introductory doc line since there was no comment of its own to reuse.
Locals declared just for one section's closures (running, refining,
dragging) move into that section's function instead of staying in
wire().
2026-09-20 17:24:42 +02:00
dtourolle 04949741c1 Split settings_ui::wire into one function per section
wire() registered every settings-page callback in one 416-line function,
fenced only by section comments. Each fenced section (opening and
closing, cache, faces, export, reset) is now its own private function
that wire() calls in the same order, with the section's own comment kept
as its doc comment. on_budget_changed and on_open are coerced to trait
objects at the top of wire() so the new functions take a plain
Rc<dyn Fn> rather than needing their own generic parameter, with no
change in the closures registered or the order they are registered in.
2026-09-20 17:24:28 +02:00
dtourolle 5b4ad11853 Manual: nested collections, and the ghost drawn as it should be
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 42s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 38s
Build and test / Android (aarch64) (push) Failing after 2m20s
Build and test / Windows (x86_64, cross) (push) Failing after 3m3s
A scene that makes a parent, nests two collections in it by drag and by
the menu, files frames into a child and opens the parent to see it count
both; stills of the tree and of the menu. The collections recording is
re-made now that the bitmap under the cursor is the photograph.

drive.py grows a multi-leg drag: a diagonal with much vertical in it is
taken by the grid's Flickable as a scroll before the DragArea can claim
it, so a drag to the sidebar goes sideways first.
2026-09-20 16:27:18 +02:00
dtourolle 6b1aac477d Put the developer docs under docs/dev and index the folder for users first
docs/ had 26 developer documents flat beside the manual, and the two
audiences are very differently sized: most readers want the manual and
the gesture reference, a few want the register, the designs and the
measurements. The manual and gestures.md stay at the top; everything for
someone changing the code moves to docs/dev/, and the two documents that
name their own successors — the v0.1 milestone and the UI-refinement plan
— go to docs/dev/archive/ rather than being deleted, since both are still
cited. docs/README.md is the index, users first.

Every reference follows: code comments, Cargo manifests, the workflows,
the pre-commit hook, the bench and traceability tools (which locate the
repo root by docs/dev/requirements.md now), packaging, the Docker READMEs,
CLAUDE.md, CONTRIBUTING.md and the README. The matrix links one level
deeper and is regenerated. Links out of the moved documents into the tree
gain a level; a link checker over every Markdown file finds none broken.
2026-09-20 16:20:15 +02:00
dtourolle 2afc2a7890 Report the grid's column count on creation, not only on change
`changed columns` fires on a change, and a first evaluation is not one:
a grid built after the window had settled at its size never said how
wide it was, so Rust placed month headings for the one column it was
told about at start-up — every month began a row, and was announced
wherever its first cell fell, mid-row included. Opening a collection
showed "October 2025" stranded over a row of August.
2026-09-20 15:58:44 +02:00
dtourolle 6507593715 Hand the drag ghost to the renderer through a file, so it draws
The bitmap under the cursor was a solid red rectangle. Slint's drag
overlay uploads the image as a texture, draws it and drops the texture in
one call; with the wgpu FemtoVG renderer the drop is immediate and the
draw is deferred to the flush, so the frame binds femtovg's placeholder —
which is red. An image with a cache key survives in the texture cache
until after the flush, and only a path gives one. So the composite goes
to the data directory's scratch as a PNG and comes back through
load_from_path; one file per drag, removed when the drag ends. A
workaround for Slint 1.17.1, written up as one beside the code.
2026-09-20 15:58:44 +02:00
dtourolle 08727cff5a Release 0.13.5
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m16s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 45s
Build and test / Layer separation (push) Successful in 25s
Traceability / Requirement traces (push) Failing after 38s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m20s
Build and test / Windows (x86_64, cross) (push) Failing after 3m1s
2026-09-20 15:24:47 +02:00
dtourolle f4c3f425dd Show the picker correcting something in the manual
The recording sampled a red brick wall and moved the sliders by three
units, which at GIF size is a click that does nothing. The scene now
drags the frame cold first and picks a white air conditioner, so the
correction is visible and the picker's being absolute - set from the
photograph, not from where the sliders were - is what the picture
shows. The text says so, and says a blown highlight is refused.
2026-09-20 15:24:27 +02:00
dtourolle 12f8990e09 Average a patch under the white balance picker, not one photosite
The probe's comment said a 192px render "averages a small neighbourhood
into each of its pixels". It does not: the composed shader fetches the
source at one position per output pixel - nearest for an unrotated
frame, four photosites blended otherwise - so the probe was a point
sample of a noisy sensor, and two painted-white air conditioners on the
same wall answered +37 and -50.

The tap is now narrowed to the patch of the canvas around the click, a
couple of percent of its width and square on screen, and rendered at
64x64 with interpolation forced on, which puts a sample on every sensor
pixel under it at any ordinary zoom. The samples are averaged, with the
void and clipped ones left out rather than allowed to pull the mean, and
fewer than half surviving is refused. compose_camera_probe takes the
patch; the merge's compose_camera_linear keeps its nearest sampling. The
readback shrinks from six megabytes to sixty-four kilobytes.

A frame of alternating warm and cool columns, averaging neutral, moves
the controls by at most two units; a point sample swung them to sixty.
2026-09-20 15:24:25 +02:00
dtourolle 8a897bbc01 Release 0.13.4
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m22s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 29s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m22s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
2026-09-20 14:03:58 +02:00
dtourolle a03e082fe2 Point the manual's white balance scene at "pick", and record it again
The scene clicked 40px to the right of "pick", on "reset", so the
recording showed a neutral group being reset and a click on the wall
that panned. Re-recorded with the picker fixed: the word lights, the
sample moves temperature and tint, and Before shows what it corrected.
2026-09-20 13:41:17 +02:00
dtourolle 4576499c3b Refuse a clipped highlight as a neutral
Sampling the overcast sky on a Canon 6D frame set tint to -100 and
temperature to -15 for a patch the canvas showed as pure white. A clipped
photosite is sensor white, not a colour: every channel stopped counting,
so what the tap hands back is the as-shot multipliers themselves, which
are strongly magenta, and the solver dutifully drove green to its stop.
The display shader already fades such a pixel to a neutral of the same
brightness before any operation runs, so the picker was balancing against
something the photographer could not see.

The probe now refuses a sample with any channel at or above the onset the
shader fades from, the way the solver already refuses black. The threshold
is one constant, CLIP_ONSET, formatted into the shader and read by the
probe, so the two cannot drift apart.
2026-09-20 13:41:17 +02:00
dtourolle 2f47087223 Measure the white balance probe in camera RGB, where the gains multiply
Pressing "pick" and clicking a near-neutral wall on a Canon 6D frame set
tint to -77 and turned the whole photograph green. The white balance
operation runs first in the chain, on camera RGB, before the body's base
curve and colour matrix; the probe was read off a display render after
all three, and the solve treated that sRGB triple as if the gains
multiplied it directly. On a JPEG the two spaces coincide, which is why
the existing tests passed while the picker was broken on every raw file.

The probe now reads the camera-space tap a merge stitches from, composed
under the edit's own framing so a fraction of the canvas is a fraction of
the probe, and puts the as-shot balance on itself - exactly the value the
operation's gains are about to multiply. No operations run in the tap, so
nothing has to be stripped and restored, and the display target is left
alone, so a sample that found nothing usable no longer needs a redraw.

A raw-frame test with the 6D's matrix and a typical as-shot balance
samples a warm grey and asserts the rendered pixel comes back neutral; it
fails on the previous probe.
2026-09-20 13:41:11 +02:00
dtourolle 764ad55ead Stand each person in the grouping pass by at most 100 references
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m23s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 55s
Build and test / Layer separation (push) Successful in 27s
Traceability / Requirement traces (push) Failing after 54s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Every face the user has ruled on entered the pass as an anchor, and the
scan is exhaustive by design (`dr_face::neighbours`), so a person with
750 confirmed faces cost 750 comparisons against every other face in
the library — and the cost of a library grew with how well it was
named. Most of those comparisons said nothing new: thirty frames from
one afternoon are one point of view, not thirty, and a face that
matches one of them matches the rest.

Each person now enters through at most 100 of their anchored faces
(`dr_face::references`). Eligible are those whose raw embedding is at
least 15 long — one above the gallery floor, since a reference speaks
for someone rather than merely being admitted — with an unmeasured
length admitted as it is everywhere else. From those, the set spanning
the greatest volume is chosen greedily: the longest vector first, then
at each step the face with the largest component orthogonal to the
chosen so far. That is pivoted Gram–Schmidt, and the product of the
residuals it picks is the Gram determinant, so the greedy step is the
exact greedy on the objective. A near-duplicate of a chosen face has
no residual and is passed over; the one profile shot among two hundred
frontal frames is taken early; faces inside the span of the chosen add
no volume and are not taken to fill the cap.

The faces not chosen keep their confirmations and are not touched by
the pass — they stay in the anchor map, so it never releases them —
they are simply not compared. A person none of whose faces is long
enough is still stood for, by their longest, rather than losing their
anchor and having their next face filed as a stranger. Under the cap
nothing changes: every eligible face stands, and the short ones stay
in as the probes they were.

At the reference library's 3,851 confirmations the scan shrinks by
about a fifth; at 15,000 it is a fifth of what it was.
2026-09-20 13:29:16 +02:00
dtourolle 301e6f3828 Release 0.13.3
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m21s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m8s
2026-09-20 13:01:41 +02:00
dtourolle a437363bd6 Schema V20: put the mis-spelled run markers right
The markers the previous commit stops writing are already in the
catalogs — 2 on the desktop, 429 on the tablet — and in the shards
both have exchanged. Renaming them to the faces' own id with a fresh
time is what makes the export send each image again, under an entry
newer than the empty one `held_model` would otherwise pick. Where the
old write had inserted its marker beside the right one, the wrong one
goes and the right one is refreshed for the same reason: its entry in
the shards is older than the empty one.

Images V14 left with faces and no marker are not touched. That state is
the quality pass's cue, and the fixed write marks them correctly when
it reaches them.

Checked against copies of both real catalogs: the desktop renames 2,
the tablet deletes 429, both in under 200 ms.
2026-09-20 13:00:59 +02:00
dtourolle 065872bec5 Keep a run marker under the detector that found the faces
`record_updates` — the write behind the quality, eye and crop passes —
re-marked the image as indexed under the pipeline the pass ran as, and
left the faces it had updated under the id of the detector that found
them. On a desktop set to Thorough that put `scrfd_10g+w600k_mbf` over
faces spelled `w600k_mbf`; on the tablet, `scrfd_10g_i8+w600k_mbf` over
faces it had adopted from the desktop's thorough pass.

Every reader takes the marker and the faces to agree. `marker_under`
reads the marker as the detector having examined the image, so the
upgrade repair never revisits it. The shard store keys each face by
its pipeline id, so `export_to_shards` selects an image's faces by the
marker's id, finds none, and sends an entry that says the thorough
detector looked and found nothing — over photographs with named faces
on them. The desktop's shard index holds 54 such entries beside real
faces; the tablet's eye pass over the faces it had adopted made 430
more, and both devices have exchanged them. `held_model` takes the
newest entry for an image, which is the empty one. Nothing has been
lost yet only because the two spellings of the thorough detector rank
equal and neither side adopts the other's; a third device, or either
one after a reinstall, would adopt "nothing here" for 484 images. And
the desktop's eye pass is 4,739 images from doing the same to every
face from before V14 — which are the ones that only exist on the
desktop, and would then never reach anywhere.

The marker now takes the id the faces carry; the pass's own id is used
only when it dropped the last of them and there is no detector left to
name. A stale marker under another spelling of the same embedder is
removed in the same transaction, so one embedder has one marker.
2026-09-20 13:00:58 +02:00
dtourolle 0ed38ada28 Adopt a peer's unmeasured faces instead of refusing them
The tablet showed a fraction of each person: 681 of the desktop's 3,851
confirmations, and none of Ian's 746, Catherine's 626 or my own 480.
Every face that existed on both devices agreed on who it was, and the
people rows were identical — the merge was fine. The missing 3,170
confirmations were on faces the tablet did not hold at all: the
desktop's 16,080 faces from the original detector, on 4,310 images,
detected before schema V14 kept the quality reading.

Those faces were in shards the tablet had already downloaded, in
August's export. `import_from_shards` looked at them on every sync pass
and declined each one, because a face without a quality reading was
"work this device cannot finish": adopting it would write the run
marker, and the marker was what stopped an image being looked at again.
That was true when it was written and has not been since the quality
repair existed — that pass lists its work by `f.quality IS NULL`, not by
the marker, exactly as the eye pass does, and faces without an eye
reading were already adopted on that reasoning.

The refusal had no exit. V14 had deleted the markers of every image
holding such faces so the quality pass would find them, and
`export_to_shards` walks the markers, so the desktop never re-exported
them either; the unmeasured August copies were the only ones there
would ever be. The tablet's answer was to queue all 17,727 images for a
re-detection of its own, a fetch of the whole library, while holding
the faces on disk.

Adopt them. The receiving device's quality pass measures them when it
reaches them, and the desktop's confirmations match onto them by box
overlap on the next catalog merge. The test that asserted the refusal
now asserts the adoption and that the image is still owed to the pass.
2026-09-20 12:58:41 +02:00
dtourolle 695d5ec304 Correct four claims in the README against the tree
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m26s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Eighteen declared operations, not fifteen; JPEG XL is an export format;
the grid does not filter by keyword, only the catalog's query can; and a
panorama's provenance is a sidecar beside the composite, not a history
step in it.
2026-09-20 11:57:17 +02:00
dtourolle ef1afc254d Release 0.13.2
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m36s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 34s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m26s
Build and test / Windows (x86_64, cross) (push) Failing after 3m7s
2026-09-20 11:06:54 +02:00
dtourolle 103c6e7fc0 Rewrite the README for someone arriving, not someone already here
Benchmarks / CPU and I/O (per commit) (push) Failing after 30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 53s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 41s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m25s
Build and test / Windows (x86_64, cross) (push) Failing after 3m11s
It said 0.9.0 against a 0.13.1 tree, listed focus peaking and burst
grouping as unbuilt when both have shipped, and opened with a page of
prose about the display path before saying what the application does.
Lead with what it is and a picture of it, say how to get it on each
platform and what state each channel is in, keep the honest account of
what is missing, and put the manual first in the documentation table.
2026-09-20 11:06:10 +02:00
dtourolle e9398de9c1 Ship the fine-tuned border filler: MI-GAN 512 trained on projection-shaped voids from the maintainer's library 2026-09-20 10:56:38 +02:00
dtourolle b1c99b5796 Let the fill show the model an open void: mirror depth 0 means no ring and nothing known beyond the band 2026-09-20 10:56:38 +02:00
dtourolle 3de109fbd1 Write down the catalog and screen-refresh patterns the Identity fixes exposed
A CLAUDE.md at the root, for anyone changing this code: the ways a redraw
and a catalog read came to cost half a second per click, what each fix
looked like, and how to measure the next one against a copy of a real
catalog.
2026-09-20 10:56:37 +02:00
dtourolle 388bda6af3 Count the outstanding repairs from the faces, on partial indexes
"How many images still owe a quality reading" was a correlated EXISTS per
image over `faces`, and the face row is 8 KB of embedding and crop before
the column it looks at, so each count opened every row. Six such counts
run on every open of the Identity screen and at the end of every sweep:
160 ms on the reference library.

V19 adds three partial indexes holding only the faces still owing each
pass, keyed on the image and carrying the model id the predicate reads,
and replaces `faces_image` with `(image_id, model_id)` so "does this image
hold this embedder's faces" is answered from the index too. The planner
takes a partial index when the count is driven from `faces` and ignores it
inside the EXISTS, so `Needs::Face` carries the per-face fragment and
`repairs::count` spells the query from the faces' side; the list and the
per-image check keep the EXISTS. A test holds the two spellings to the
same answer for every repair.
2026-09-20 10:56:37 +02:00
dtourolle 73845d8a77 Ask the server about a collection once per job, not once per file
Every `move_to` guaranteed its destination's parent with a `MKCOL` for
each ancestor down from the account root, and a trash folder under a
library root several levels deep meant three round trips answering
`405 Method Not Allowed` before the one `MOVE` that did anything — for
every image of a delete, on a connection built for that job.

The backend now records the collections it has confirmed exist and asks
about each once. It lives for one job, so a folder another client removes
mid-batch is the one case this misses, and the `MOVE` then reports the
`409` rather than hiding it.
2026-09-20 10:56:37 +02:00
dtourolle b4821ee1ab Filter the people rail in the query, and count the unassigned faces
`faces::people` grouped `face_person` after a LEFT JOIN over every person
and sorted the lot by name; the rail then discarded the empty, unnamed
groups a regrouping pass leaves behind — 17,000 of 19,000 rows on the
reference library. `people_in_use` filters them in the WHERE and joins
`people` to face counts aggregated first (2,000 groups), so the sort sees
only the rows that will be drawn. `count_unassigned` replaces fetching
2,400 ids to take their length. `load_people` 22 ms → 10 ms.
2026-09-20 10:56:36 +02:00
dtourolle 9d1aa5735b Confirm a group, and split one, in one transaction
`confirm_all` called `faces::confirm` per face, and `split_off` called
`reject` then `confirm` per face: each opens and commits its own
transaction, so a click on a group of several hundred was several hundred
commits. `faces::confirm_all` is two statements — clear the rejections the
confirmations override, then flip the rows — and `faces::reassign` does a
split's reject-and-confirm for every face under one commit. 16 ms → 2 ms
and 22 ms → 4 ms on the largest group.
2026-09-20 10:56:36 +02:00
dtourolle 97a854833d Reuse the face grid's decoded crops across a redraw
A confirm or a reject changes one row and redraws the whole grid, and the
redraw re-read every crop blob of the selected person (4 MB for the
largest) and decoded every one — 316 ms per click on the reference
library's 754-face person, to arrive at the pixels already on screen.

`load_faces` now takes the crops the previous load decoded, keyed by face,
and moves each into its new cell; the blob read is skipped when every face
is already in hand. `refresh` drains the old cells into it rather than
cloning them. The redraw is 2.6 ms.
2026-09-20 10:56:36 +02:00
dtourolle e0e193efb4 Do not recount face coverage on every confirm, and count it without listing
Every click on the Identity screen's face grid — confirm, reject, split,
rename, merge — redrew the whole screen, and the redraw recomputed the
coverage line. That line lists every repair's outstanding images to count
them: six scans of the images table with a correlated EXISTS over the
8 KB face rows, an ORDER BY the job's visiting order, a Target with its
path per row, and a thumbnail-index query per image with faces. On the
reference library (24k images, 19k faces) that was ~200 ms of the
~540 ms each click cost, spent computing a figure a confirm cannot change.

`refresh` now takes what changed: `Changed::Identities` re-reads the rail
and the grid and leaves the coverage line alone; `Changed::Library` — an
open, a sweep ending or stopped, the face data deleted — re-reads it too.

For the times it does run, `repairs::counts` counts instead of building
and dropping the lists, and the thumbnail store is read once
(`ThumbStore::held`) rather than probed once per image in the audit, the
outstanding list and the proxy repair.

`identity_bench` is the measurement: the reads a click performs and the
batch writes, timed against a copy of a real catalog.
2026-09-20 10:56:36 +02:00
dtourolle d790961b28 Add the manual: every feature pictured from the application itself
Benchmarks / CPU and I/O (per commit) (push) Failing after 30s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 47s
Build and test / Layer separation (push) Successful in 27s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 4s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Traceability / Requirement traces (push) Failing after 38s
Build and test / Android (aarch64) (push) Failing after 2m24s
Build and test / Windows (x86_64, cross) (push) Failing after 3m21s
docs/manual/README.md is a tour for a photographer opening DarkRoom for
the first time — one picture per thing, moving where movement is the
point. tools/manual/ is how the pictures are made: drive.py puppeteers the
desktop build on a private Xvfb (launch, click, drag, type, screenshot,
record), scenes.py is each picture as a script, and record.sh runs them
all over a folder and writes the results into docs/manual/media/.

The media is in LFS, with the CI pulls excluding it as they exclude the
fixtures; a screenshot changes wholesale when the interface does.

Nothing in the pictures shows a person, by design: the demo library is
seventy urban and alpine frames, chosen from the catalog's rows that face
detection found nobody in.

The traceability matrix is regenerated here after the rebase that
brought this branch up to master.
2026-09-20 00:26:00 +02:00
dtourolle 14ac41bee0 Give the keyword sheet's field focus, and the empty trash its own words
Without focus the keys typed into the keywords sheet went to the grid
behind the scrim, and the Return meant for the keyword opened a
photograph. The naming sheet already takes focus on open; do the same.

An empty trash said "No images found — check the library folder and which
formats are ticked", which sends someone off to fix a library that is
fine.
2026-09-20 00:21:14 +02:00
dtourolle 86410def88 Title the gesture book for the whole application, not the grid 2026-09-20 00:21:14 +02:00
dtourolle df8be10c7d Stop promising to ask for an export folder
An empty device destination read "Ask each time" on the settings page,
and nothing asks: an export made with the field blank is refused with
"no export folder is set". Say what will happen.
2026-09-20 00:21:14 +02:00
dtourolle f176043632 Report a merge worker that dies rather than leaving the page on Stop
wgpu reports a device out of memory by panicking, and a twelve-frame
merge on a GPU another process is using is where that happens. The panic
unwound the worker, the sender went with it, and the page sat on "Stop"
with every control disabled and nothing to say why — the crash record on
disk was the only sign. Catch the panic and send it as a failure, and
treat a closed channel with no final event as a dead worker too.
2026-09-20 00:21:14 +02:00
dtourolle 9c2cd73337 Show a mask's tint only while masking
The eyes are per layer and outlive the mode, so a photographer coming
back finds the layers they were looking at still lit. But the tint is a
way of looking at a mask, and outside Local there is no mask being looked
at: the sky stayed red through Repair and back in Photo, a mode that had
been left leaving its overlay behind — the fault ui-navigation.md D-N1
exists to prevent.
2026-09-20 00:21:14 +02:00
dtourolle e3acbdf4a3 Wrap the falloff and edge chips so a category mask cannot widen the column
Five chips in one row declare 440px, and the develop column takes the
widest panel's request — so selecting a category mask levered the sidebar
past the window's edge, clipping the histogram, the group strip and the
subject list. The same trap ChipGrid's comment records for film formats.
2026-09-20 00:21:14 +02:00
dtourolle eddaa44cd3 Announce a month at the next row it opens, not only if it begins one
A heading was drawn only on a cell that both began a month and began a
row, so at seven columns most months were never named, and the one
heading on screen — always on the window's first cell — was wrong about
every row below it. Worse, two headings drawn on the same cell overprinted
each other. Now a row carries a heading whenever its first cell's month is
not the one last announced: a month starting mid-row is named on the next
row it opens, one row late and right about everything under it.
2026-09-20 00:21:14 +02:00
dtourolle 7fba28f7d8 Upload a snapshot of a thumbnail shard, never the live file
Every shard is in WAL mode and every put opens its own connection, so
while thumbnails are being generated on several threads — which is when
the first sync pass runs — the log is never checkpointed and the main
file holds whatever the last quiet moment left in it. For a shard created
seconds earlier that is nothing: a zero-byte file with the schema still
in the log. The sync read that file and uploaded it, and every other
device merging it failed with "no such table: thumbs" on every pass.

Copy the shard through SQLite's backup API into scratch first, which
serialises against writers and carries the log, and upload that.
2026-09-20 00:21:14 +02:00
dtourolle 0fa9003e54 Let the top level be chosen as the library root
Confirming "/" in the folder picker set an empty root, which the launch
model read as no root at all: "Open library" stayed disabled after the
question had plainly been answered, and a folder library — whose folder
is the whole library — could never be opened without first descending
into a subfolder of it. The empty string was carrying two meanings.

Record the choice as its own fact on the account (`root_chosen`, defaulted
so existing configuration loads unchanged), treat a folder endpoint as
chosen by definition, and let the launch screen say so: a folder is shown
as a LIBRARY rather than an ACCOUNT, the second question becomes an
optional "scan only a subfolder", and the library header names the folder
instead of calling it "· whole account".
2026-09-20 00:21:13 +02:00
dtourolle e750bcdb8c Date the composite at the mean of its frames' capture times
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 31m52s
Build and test / Layer separation (push) Successful in 47s
Traceability / Requirement traces (push) Failing after 55s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Benchmarks / CPU and I/O (per commit) (push) Failing after 32s
Build and test / Android (aarch64) (push) Failing after 2m37s
Build and test / Windows (x86_64, cross) (push) Failing after 3m10s
It carried the first frame's, so it sorted before the sweep it was made
from. The middle of the sweep puts it among them.
2026-09-19 22:36:09 +02:00
dtourolle 48c4b403d2 Date a DNG whose IFDs follow its pixels: read the head and the tail
The scan reads the first 256 KB of a file for its metadata. A camera
writes its IFDs at the front, so that is the whole structure; the linear
DNG a merge writes puts its first IFD after the pixels, and rawler,
given the head alone, finds no decoder in it. The composite was
catalogued without a date and sorted to the very end of the grid, after
every dated photograph — which is where a panorama merged on the tablet
went unfound.

dr-decode's own TIFF reader now reads through a head and a tail at a
known offset; trailing_ifd says where the tail starts and
metadata_split reads the two together. The scan, when the head fails
and points beyond itself, fetches from the IFD to the end — kilobytes —
and dates the file from both. Tested against the writer's own output.
2026-09-19 22:35:58 +02:00
dtourolle 9d04ff2154 Release 0.13.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 12m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 5s
Build and test / Desktop (Linux) (push) Failing after 31m25s
🐳 Windows image / Build and push (push) Successful in 4s
Build and test / windows-image (push) Successful in 4s
Build and test / Layer separation (push) Successful in 46s
Build and test / Android (aarch64) (push) Failing after 6h25m37s
Build and test / Windows (x86_64, cross) (push) Failing after 3m17s
Traceability / Requirement traces (push) Failing after 48s
2026-09-19 22:06:06 +02:00
dtourolle c6cfb2a02a Put the -1 on the greens along the chroma axis, not across it
The Malvar "R at green in R row" kernel weights the two greens two
sites away along the row at -1 and the pair up and down the column at
+1/2. The shader had the two swapped, in the comment as well as the
code, so the transcription checked against itself. Both sum to zero
and reconstruct a flat patch exactly, which is all the tests fed it.

On an edge the correction at green sites is half strength and the
false colour doubles: 0.375 against 0.19 on a grey step, and a
blue/yellow zipper around every clipped highlight at 1:1. The other
three kernels and the CFA tables were right.

A grey vertical step now runs through the pass; the transposed kernel
fails it at 0.375.
2026-09-19 22:05:39 +02:00
dtourolle b83f192847 Package release 2 of 0.13.0: the inference engine and the user runtime directory 2026-09-19 21:20:53 +02:00
dtourolle ecb648818b Search the user's own runtime directory before the system library
The reference desktop's only system ONNX Runtime is Arch's
onnxruntime-opt-cuda: 1.29, built without TensorRT and against cuDNN 8
on a cuDNN 9 machine. The probe rejects both providers correctly and
the app runs on the CPU provider, which is right and not what anyone
wants. runtime/ beside the models is now searched ahead of /usr/lib,
tools/fetch-desktop-runtime.sh fills it with the four libraries from
the current onnxruntime-gpu wheel (cuDNN 9, TensorRT 10), and the
About caption lists every rung that lost and why, not only the first.
Verified: the app selects TensorRT from that directory with no
environment variable set.
2026-09-19 21:15:55 +02:00
dtourolle 5fbf8944d7 Count the filler in the APK's bundled-model array
Benchmarks / CPU and I/O (per commit) (push) Successful in 3m35s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 31m59s
Build and test / Layer separation (push) Successful in 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Traceability / Requirement traces (push) Failing after 57s
Build and test / Android (aarch64) (push) Failing after 54m6s
Build and test / Windows (x86_64, cross) (push) Failing after 1h4m55s
The unpack list gained migan-512.onnx without its length following;
nothing on the desktop compiles that crate, and the first Android build
of 0.13.0 stopped there.
2026-09-19 20:55:38 +02:00
261 changed files with 41803 additions and 33759 deletions
+6
View File
@@ -20,3 +20,9 @@
# reasoning as the models, with the opposite default: the model is not # reasoning as the models, with the opposite default: the model is not
# optional and the fixtures are. # optional and the fixtures are.
fixtures/** filter=lfs diff=lfs merge=lfs -text fixtures/** filter=lfs diff=lfs merge=lfs -text
# The manual's pictures live in LFS for the same reason the models do: a
# screenshot or a GIF changes wholesale when the interface it shows changes,
# and every re-recording would otherwise stay in every clone for good. CI's
# pulls exclude the directory; nothing built or tested reads it.
docs/manual/media/** filter=lfs diff=lfs merge=lfs -text
+4 -4
View File
@@ -1,6 +1,6 @@
name: Benchmarks name: Benchmarks
# The suite docs/requirements.md §8 has been promising since it was written: # The suite docs/dev/requirements.md §8 has been promising since it was written:
# "an automated benchmark suite against a synthetic 50k catalog, run per-commit # "an automated benchmark suite against a synthetic 50k catalog, run per-commit
# … A regression beyond stated tolerance fails the build." # … A regression beyond stated tolerance fails the build."
# #
@@ -29,7 +29,7 @@ name: Benchmarks
# every commit to establish, every time, that this runner has no GPU. It # every commit to establish, every time, that this runner has no GPU. It
# runs on demand (Actions → Run workflow) so that a runner that *does* # runs on demand (Actions → Run workflow) so that a runner that *does*
# have one can be pointed at it, and the numbers it produces belong in # have one can be pointed at it, and the numbers it produces belong in
# docs/frame-budget.md by hand, as they already are. # docs/dev/frame-budget.md by hand, as they already are.
on: on:
push: push:
@@ -148,7 +148,7 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true | while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \ git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs" "https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
git lfs pull --exclude="fixtures/**" git lfs pull --exclude="fixtures/**,docs/manual/media/**"
- name: Cache cargo - name: Cache cargo
uses: actions/cache@v4 uses: actions/cache@v4
@@ -183,7 +183,7 @@ jobs:
- name: Frame budget (FR-DSP-3) - name: Frame budget (FR-DSP-3)
run: cargo test --release -p dr-gpu --test frame_budget -- --nocapture run: cargo test --release -p dr-gpu --test frame_budget -- --nocapture
# The instrument behind docs/frame-budget.md. It exits non-zero with no # The instrument behind docs/dev/frame-budget.md. It exits non-zero with no
# adapter, which is right for a tool a person runs deliberately and wrong # adapter, which is right for a tool a person runs deliberately and wrong
# for a job that usually has none — hence continue-on-error. Its table is # for a job that usually has none — hence continue-on-error. Its table is
# in the log for whoever asked for this run; the committed numbers are # in the log for whoever asked for this run; the committed numbers are
+69 -6
View File
@@ -7,6 +7,10 @@ name: Build and test
on: on:
push: push:
branches: [main, master, develop] branches: [main, master, develop]
# A release tag builds again and publishes what it built (the `release`
# job at the end). The master push of the same commit has usually filled
# the caches, so the second run is the warm one.
tags: ['v*']
pull_request: pull_request:
branches: [main, master, develop] branches: [main, master, develop]
@@ -96,7 +100,7 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true | while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \ git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs" "https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
git lfs pull --exclude="fixtures/**" git lfs pull --exclude="fixtures/**,docs/manual/media/**"
ls -lR models/ ls -lR models/
- name: Cache cargo - name: Cache cargo
@@ -154,6 +158,16 @@ jobs:
- name: Build - name: Build
run: cargo build --workspace --release run: cargo build --workspace --release
# Only on a release tag: the binary is 150 MB and nothing but the
# release job wants it.
- name: Upload the desktop binary
if: startsWith(github.ref, 'refs/tags/v')
uses: actions/upload-artifact@v3
with:
name: darkroom-desktop-x86_64-linux
path: target/release/darkroom-desktop
if-no-files-found: error
- name: Disk after - name: Disk after
if: always() if: always()
run: df -h /workspace 2>/dev/null || df -h . run: df -h /workspace 2>/dev/null || df -h .
@@ -213,7 +227,7 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true | while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \ git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs" "https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
git lfs pull --exclude="fixtures/**" git lfs pull --exclude="fixtures/**,docs/manual/media/**"
ls -lR models/ ls -lR models/
- name: Cache cargo - name: Cache cargo
@@ -322,7 +336,7 @@ jobs:
env: env:
CARGO_TARGET_DIR: target-android CARGO_TARGET_DIR: target-android
# Absent secrets mean a debug signature, which is what a fork or a # Absent secrets mean a debug signature, which is what a fork or a
# branch build should get. Set all three (see docs/android-signing.md) # branch build should get. Set all three (see docs/dev/android-signing.md)
# and the same job produces a release-signed APK instead. # and the same job produces a release-signed APK instead.
ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }} ANDROID_KEYSTORE_BASE64: ${{ secrets.ANDROID_KEYSTORE_BASE64 }}
KEYSTORE_PASS: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }} KEYSTORE_PASS: ${{ secrets.ANDROID_KEYSTORE_PASSWORD }}
@@ -370,7 +384,7 @@ jobs:
# TRACES: FR-PLAT-WIN-3 # TRACES: FR-PLAT-WIN-3
# The Windows executable and its installer, cross-built from Linux # The Windows executable and its installer, cross-built from Linux
# (docs/windows.md §7). No Windows machine anywhere in this job: what it # (docs/dev/windows.md §7). No Windows machine anywhere in this job: what it
# can prove is that the binary links, is a Windows executable with no # can prove is that the binary links, is a Windows executable with no
# MinGW runtime imports, starts under Wine, and that the installer installs # MinGW runtime imports, starts under Wine, and that the installer installs
# and uninstalls under Wine. What it cannot prove — a Vulkan device, a # and uninstalls under Wine. What it cannot prove — a Vulkan device, a
@@ -406,7 +420,7 @@ jobs:
| while read -r key; do git config --local --unset-all "$key"; done || true | while read -r key; do git config --local --unset-all "$key"; done || true
git config --local lfs.url \ git config --local lfs.url \
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs" "https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
git lfs pull --exclude="fixtures/**" git lfs pull --exclude="fixtures/**,docs/manual/media/**"
ls -l models/face models/scene ls -l models/face models/scene
- name: Cache cargo - name: Cache cargo
@@ -454,7 +468,12 @@ jobs:
wine "$SETUP" /S 2>/dev/null wine "$SETUP" /S 2>/dev/null
INST=$(echo "$HOME"/.wine/drive_c/users/*/AppData/Local/Programs/DarkRoom) INST=$(echo "$HOME"/.wine/drive_c/users/*/AppData/Local/Programs/DarkRoom)
ls "$INST" ls "$INST"
[ "$(ls "$INST/models" | wc -l)" = 7 ] || { echo "FAIL: expected 7 model files"; exit 1; } # As many files as package.sh stages: everything but the READMEs in
# the directories it copies. A literal here went stale the first
# time a model was added.
WANT=$(find models/face models/scene models/inpaint -maxdepth 1 -type f ! -name README.md | wc -l)
GOT=$(ls "$INST/models" | wc -l)
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT model files, installed $GOT"; exit 1; }
wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \ wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \
| grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; } | grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; }
wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \ wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \
@@ -514,3 +533,47 @@ jobs:
fi fi
done done
exit $FAILED exit $FAILED
# A v* tag becomes a Gitea Release carrying the three builds and their
# SHA256SUMS, titled and described by the tag's message. Until this job
# existed every release was made by hand, and most tags never got one.
#
# It needs all three platform jobs, so a tag whose tests fail publishes
# nothing; re-run the failed job and this one follows. The work is
# tools/publish-release.sh, which is also how a release is finished by hand.
release:
if: startsWith(github.ref, 'refs/tags/v')
needs: [desktop, android, windows]
runs-on: linux/amd64
name: Publish the release
container:
image: catthehacker/ubuntu:act-latest
permissions:
contents: write
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Fetch the builds
uses: actions/download-artifact@v3
with:
path: dist
# Named for the download page, with the version in each name the way
# the hand-made releases had them. The installer already carries its
# version from package.sh.
- name: Publish
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN || github.token }}
TAG: ${{ github.ref_name }}
run: |
set -e
V="${TAG#v}"
ls -lR dist
mkdir -p out
cp dist/darkroom-arm64-v8a-apk/darkroom.apk "out/darkroom-${V}-arm64-v8a.apk"
cp dist/darkroom-desktop-x86_64-linux/darkroom-desktop "out/darkroom-desktop-${V}-x86_64-linux"
chmod +x "out/darkroom-desktop-${V}-x86_64-linux"
cp dist/darkroom-windows-x86_64-setup/DarkRoom-${V}-x86_64-setup.exe out/
bash tools/publish-release.sh "$TAG" out/*
+5 -5
View File
@@ -7,7 +7,7 @@ name: Traceability
# fail its own threshold. Two rules follow, and the extractor's own tests # fail its own threshold. Two rules follow, and the extractor's own tests
# enforce both: # enforce both:
# #
# 1. Denominators are parsed from docs/requirements.md at run time. # 1. Denominators are parsed from docs/dev/requirements.md at run time.
# 2. Coverage is |traced ∩ defined| / |defined|, never a raw traced count. # 2. Coverage is |traced ∩ defined| / |defined|, never a raw traced count.
# #
# This job is static analysis of source comments plus markdown parsing, so it # This job is static analysis of source comments plus markdown parsing, so it
@@ -74,11 +74,11 @@ jobs:
run: | run: |
set -e set -e
cargo run -q -p traceability -- report cargo run -q -p traceability -- report
if ! git diff --quiet docs/traceability.md; then if ! git diff --quiet docs/dev/traceability.md; then
echo "" echo ""
echo "docs/traceability.md is out of date." echo "docs/dev/traceability.md is out of date."
echo "Run: cargo run -p traceability -- report" echo "Run: cargo run -p traceability -- report"
git diff --stat docs/traceability.md git diff --stat docs/dev/traceability.md
exit 1 exit 1
fi fi
@@ -127,4 +127,4 @@ jobs:
- name: Summary - name: Summary
if: always() if: always()
run: head -30 docs/traceability.md || true run: head -30 docs/dev/traceability.md || true
+4 -4
View File
@@ -26,7 +26,7 @@ fi
# The artefacts are generated from the tree, so regenerating them because one # The artefacts are generated from the tree, so regenerating them because one
# was itself edited would be circular. # was itself edited would be circular.
case "$(tr -d '[:space:]' <<< "${staged}")" in case "$(tr -d '[:space:]' <<< "${staged}")" in
docs/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs) docs/dev/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs)
exit 0 exit 0
;; ;;
esac esac
@@ -41,9 +41,9 @@ if ! cargo run -q -p traceability -- report >/dev/null 2>&1; then
exit 0 exit 0
fi fi
if ! git diff --quiet -- docs/traceability.md; then if ! git diff --quiet -- docs/dev/traceability.md; then
git add docs/traceability.md git add docs/dev/traceability.md
echo "pre-commit: regenerated docs/traceability.md and staged it" echo "pre-commit: regenerated docs/dev/traceability.md and staged it"
fi fi
# The gesture vocabulary, same discipline. # The gesture vocabulary, same discipline.
+132
View File
@@ -0,0 +1,132 @@
# Working in this repository
Notes for anyone — person or agent — changing this code. They record what
went wrong once and what the fix looked like, so the same shape is not
written again. Requirements live in `docs/dev/requirements.md`; this file is
about habits, not features.
## Catalog reads: work is proportional to what changed, never to library size
`docs/dev/catalog.md §1` states the rule. These are the ways it was broken on
the Identity screen, found when every confirm click cost half a second on a
24k-image library (2026-09-19), and what each fix looked like.
**A redraw must know what changed.** A click handler that calls "refresh
everything" pays for everything. `identity_ui::refresh` takes a `Changed`:
a confirm re-reads the rail and the grid and *not* the coverage line,
because moving a face between people cannot alter how many images are
indexed. Before adding a read to a shared refresh, ask which events can
change its answer, and gate it on those.
**Count with `COUNT(*)`, never with `.len()` on a list you then drop.**
`repairs::counts` used to build every repair's work list — a `Target` with
its path per row, sorted into visiting order — to report its length. Six
repairs, 350 ms, nothing kept. If the caller wants a number, the query
returns a number.
**One query, not one per row.** `ThumbStore::contains` in a filter over
5,000 rows is 5,000 prepared statements; `ThumbStore::held(size)` reads the
index once into a set. The same applies to any `query_row` inside a loop
over a result set — including `deep_count` per sidebar row, which is fine
at sidebar scale and would not be at grid scale. Aggregate in one
statement and look up in memory.
**Filter and aggregate in SQL, and aggregate the small side first.**
`faces::people` read 19,000 rows, grouped, sorted them by name, and the
screen threw 17,000 away (empty unnamed groups). `people_in_use` filters in
the `WHERE`, and joins `people` to a pre-aggregated `face_person` (2,000
groups) rather than grouping after a `LEFT JOIN` over every person. The
sort then sees only the rows that will be drawn.
**Wide rows make "just check one column" a table scan.** A `faces` row is
~8 KB (a 1 KB embedding and a ~5 KB crop, then the columns added later).
Any predicate that reads `quality`, `crop` or an eye column for every face
reads every row. V17 learned this for the eye filter; V19 applies it to the
repair counts with partial indexes (`faces_owed_*`) that hold only the rows
still owing, keyed on what the predicate joins on and carrying `model_id`
because the predicate reads it. Two things to know about them:
- **Drive the count from the small side.** SQLite uses a partial index
when the query starts from `faces` (`repairs::count`, `Needs::Face`) and
ignores it inside a correlated `EXISTS (... WHERE f.image_id = i.id ...)`.
That is why `Needs::Face` carries the per-face fragment and spells it two
ways.
- **Spell the predicate as the index's `WHERE` is spelled.** `NEEDS_EYES`
is `(f.eye_right IS NULL OR f.landmarks_dense IS NULL)` because
`faces_owed_eyes` is `WHERE eye_right IS NULL OR landmarks_dense IS NULL`.
Change one, change both, and `counts_are_the_sizes_of_the_lists` will
tell you if they drift.
Check a query's plan with `EXPLAIN QUERY PLAN` against a copy of a real
catalog before trusting an index exists for it: "SEARCH ... USING COVERING
INDEX" is the answer you want, "SEARCH f USING INDEX faces_image" on a wide
table means every probe opens a row.
## Catalog writes: one transaction per user action
`faces::confirm` opens a transaction. Calling it in a loop over a group is
a commit per face; `faces::confirm_all` is two statements and one commit,
`faces::reassign` one transaction for a whole split. When a UI action
touches N rows, give the catalog a function that takes the N, not a loop
that calls the one-row function N times — `unchecked_transaction` cannot
nest, so this has to be designed in at the catalog layer, not wrapped
from above.
## Screens: keep what is already decoded
`identity::load_faces` takes the crops the grid is currently showing and
hands them back into the new cells. Before that, a click re-read 4 MB of
crop blobs and decoded 700 JPEGs to produce the pixels already on screen.
When a redraw replaces a model, the expensive parts of the old model — a
decoded image, a cut portrait — are the first thing to reuse; only the row
that changed needs new work. Drain the old cells rather than cloning them.
## Remote calls: one round trip per file, not one per ancestor
`NextcloudBackend::move_to` guaranteed its destination's parent with a
`MKCOL` per ancestor from the account root, on every file of a batch —
three `405`s before each `MOVE`. The backend now remembers the collections
it has confirmed (`known_dirs`) for its lifetime, which is one job. When a
per-file operation has a per-batch precondition, satisfy it once.
## Providers: read the runtime's source for the version on disk, not the binding
Two things the MIGraphX rung (2026-09-20) got wrong before it was measured
right, both because `ort`'s builder was trusted to mean what its method
names say.
**A binding's option builder may fill a struct the runtime no longer
reads.** `ep::MIGraphX::with_save_model` sets fields of the legacy
`OrtMIGraphXProviderOptions`; ONNX Runtime 1.29 reads that struct for the
precision flags and ignores the rest, so every session compiled for 40 s
and the cache directory went nowhere. The option that works
(`migraphx_model_cache_dir`) exists only in the generic key/value
registration, which `session::migraphx` calls on the API table directly.
Before wiring a provider option, fetch the provider's source at the
runtime's exact version and find where the option is *read*.
**A provider's cache key may leave out what you are varying.** MIGraphX
keys a compiled program on graph, GPU and its own version — not precision.
The first fp16 measurement built in 0.3 s and matched f32 to the tenth of a
millisecond, because it had loaded the f32 program. A "from cache" build
that is suspiciously fast on the first run of a new configuration is a key
collision, not a fast provider; give each precision its own directory (the
engine does) and check the cache directory gained a file.
## Measuring
`cargo run --release -p dr-ui --example identity_bench -- CATALOG THUMBS`
times what one click on the Identity screen reads and what the batch
operations write. Run it against a **copy** of a real catalog (it writes),
never the library's own file; `sqlite3 catalog.sqlite ".backup copy.sqlite"`
takes a consistent one while the app runs. Compare the `cpu` column when
other builds are running on the machine — the wall clock doubles under
load, the CPU figure does not. Keep the binary from before the change and
run both back to back rather than trusting numbers taken an hour apart.
Reference figures from the 2026-09-19 fixes, largest person (754 faces),
24k images, 19k faces, before → after. What one click read: `load_people`
22 ms → 12 ms, `load_faces` 316 ms → 2.4 ms, `audit` 190 ms → not run
(66 ms when it is, on open and at the end of a sweep). What one click
wrote: `confirm_all` 16 ms → 2 ms, `split_off` 23 ms → 4.5 ms. A click on
the face grid went from ~530 ms of catalog work to ~15 ms.
+11 -11
View File
@@ -90,18 +90,18 @@ break it by accident:
cargo run --release -p dr-bench -- check cargo run --release -p dr-bench -- check
``` ```
That is the benchmark suite (`docs/requirements.md` §8), which builds a That is the benchmark suite (`docs/dev/requirements.md` §8), which builds a
synthetic 50,000-image catalog and fails the build if a performance target is synthetic 50,000-image catalog and fails the build if a performance target is
missed or a measurement has drifted past its tolerance. It runs on every push in missed or a measurement has drifted past its tolerance. It runs on every push in
its own workflow. [`docs/benchmarks.md`](docs/benchmarks.md) says what it its own workflow. [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) says what it
measures, what it deliberately does not, and how to read a failure. If you have measures, what it deliberately does not, and how to read a failure. If you have
touched the catalog, the decoder, the thumbnail store or the exporter, run it touched the catalog, the decoder, the thumbnail store or the exporter, run it
before you send. before you send.
## Requirements and traceability ## Requirements and traceability
[`requirements.md`](docs/requirements.md) is the register of record. [`requirements.md`](docs/dev/requirements.md) is the register of record.
[`traceability.md`](docs/traceability.md) is generated from `TRACES:` tags in [`traceability.md`](docs/dev/traceability.md) is generated from `TRACES:` tags in
the source and must never be hand-edited: the source and must never be hand-edited:
```rust ```rust
@@ -124,7 +124,7 @@ Note that it tracks line numbers, so a change that only moves code still moves
the matrix. Never regenerate it with a stale prebuilt binary. the matrix. Never regenerate it with a stale prebuilt binary.
**One convention that the tooling cannot enforce.** A tag proves that a tag **One convention that the tooling cannot enforce.** A tag proves that a tag
exists, not that the code under it does the thing — `docs/code-health.md` exists, not that the code under it does the thing — `docs/dev/code-health.md`
CH-4 has the details, and two requirements currently read as covered on the CH-4 has the details, and two requirements currently read as covered on the
strength of plumbing a future feature would use. So: **close a requirement strength of plumbing a future feature would use. So: **close a requirement
with a test that would fail if the behaviour were removed.** Coverage that with a test that would fail if the behaviour were removed.** Coverage that
@@ -163,12 +163,12 @@ One commit per change. If you fixed two things, that is two commits.
| Document | Read it when | | Document | Read it when |
|---|---| |---|---|
| [`core/dr-pipeline/ops/README.md`](core/dr-pipeline/ops/README.md) | Adding or changing a develop operation — start here regardless | | [`core/dr-pipeline/ops/README.md`](core/dr-pipeline/ops/README.md) | Adding or changing a develop operation — start here regardless |
| [`docs/architecture.md`](docs/architecture.md) | Anything touching the render path, catalog or sync | | [`docs/dev/architecture.md`](docs/dev/architecture.md) | Anything touching the render path, catalog or sync |
| [`docs/code-health.md`](docs/code-health.md) | Deciding what to work on; grades each seam by what it costs | | [`docs/dev/code-health.md`](docs/dev/code-health.md) | Deciding what to work on; grades each seam by what it costs |
| [`docs/benchmarks.md`](docs/benchmarks.md) | A change that could plausibly cost time or memory | | [`docs/dev/benchmarks.md`](docs/dev/benchmarks.md) | A change that could plausibly cost time or memory |
| [`docs/technical-debt.md`](docs/technical-debt.md) | Something looks wrong — check it was not chosen | | [`docs/dev/technical-debt.md`](docs/dev/technical-debt.md) | Something looks wrong — check it was not chosen |
| [`docs/distribution.md`](docs/distribution.md) | Packaging a build, or adding a permission to one | | [`docs/dev/distribution.md`](docs/dev/distribution.md) | Packaging a build, or adding a permission to one |
| [`docs/requirements.md`](docs/requirements.md) | Reference, not reading | | [`docs/dev/requirements.md`](docs/dev/requirements.md) | Reference, not reading |
`technical-debt.md` is the one to check before "fixing" anything surprising. `technical-debt.md` is the one to check before "fixing" anything surprising.
It records compromises that were deliberate, each with the reasoning and a It records compromises that were deliberate, each with the reasoning and a
Generated
+27 -25
View File
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]] [[package]]
name = "darkroom-android" name = "darkroom-android"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"android_logger", "android_logger",
"dr-plat", "dr-plat",
@@ -1234,7 +1234,7 @@ dependencies = [
[[package]] [[package]]
name = "darkroom-desktop" name = "darkroom-desktop"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-plat", "dr-plat",
@@ -1408,7 +1408,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]] [[package]]
name = "dr-bench" name = "dr-bench"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"dr-catalog", "dr-catalog",
@@ -1425,7 +1425,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-catalog" name = "dr-catalog"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-face", "dr-face",
"dr-plat", "dr-plat",
@@ -1440,7 +1440,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-decode" name = "dr-decode"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"env_logger", "env_logger",
@@ -1454,7 +1454,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-export" name = "dr-export"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-decode", "dr-decode",
"dr-gpu", "dr-gpu",
@@ -1473,7 +1473,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-face" name = "dr-face"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-inference-engine", "dr-inference-engine",
"env_logger", "env_logger",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-film" name = "dr-film"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"log", "log",
"serde", "serde",
@@ -1495,7 +1495,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-gpu" name = "dr-gpu"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"bytemuck", "bytemuck",
"dr-decode", "dr-decode",
@@ -1513,8 +1513,9 @@ dependencies = [
[[package]] [[package]]
name = "dr-inference-engine" name = "dr-inference-engine"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"env_logger",
"libloading", "libloading",
"log", "log",
"ort", "ort",
@@ -1527,7 +1528,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ingest" name = "dr-ingest"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-plat", "dr-plat",
"dr-types", "dr-types",
@@ -1539,7 +1540,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-lens" name = "dr-lens"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"lensfun", "lensfun",
"log", "log",
@@ -1547,7 +1548,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-pano" name = "dr-pano"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-decode", "dr-decode",
"dr-inference-engine", "dr-inference-engine",
@@ -1561,7 +1562,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-pipeline" name = "dr-pipeline"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -1570,7 +1571,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-plat" name = "dr-plat"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"android-native-keyring-store", "android-native-keyring-store",
"dr-types", "dr-types",
@@ -1586,7 +1587,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-preset-xmp" name = "dr-preset-xmp"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-pipeline", "dr-pipeline",
"log", "log",
@@ -1596,7 +1597,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-segment" name = "dr-segment"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-inference-engine", "dr-inference-engine",
"env_logger", "env_logger",
@@ -1609,7 +1610,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync" name = "dr-sync"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-plat", "dr-plat",
@@ -1623,7 +1624,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-folder" name = "dr-sync-folder"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-sync", "dr-sync",
@@ -1635,7 +1636,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-sync-nextcloud" name = "dr-sync-nextcloud"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"async-trait", "async-trait",
"dr-decode", "dr-decode",
@@ -1657,7 +1658,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-thumbs" name = "dr-thumbs"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"jpeg-encoder", "jpeg-encoder",
@@ -1669,7 +1670,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-types" name = "dr-types"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"serde", "serde",
"serde_json", "serde_json",
@@ -1678,7 +1679,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-ui" name = "dr-ui"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"async-trait", "async-trait",
@@ -1706,6 +1707,7 @@ dependencies = [
"jni 0.22.4", "jni 0.22.4",
"log", "log",
"ndk-context", "ndk-context",
"png",
"pollster", "pollster",
"reqwest", "reqwest",
"rusqlite", "rusqlite",
@@ -1720,7 +1722,7 @@ dependencies = [
[[package]] [[package]]
name = "dr-xmp" name = "dr-xmp"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"dr-types", "dr-types",
"log", "log",
@@ -7021,7 +7023,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]] [[package]]
name = "traceability" name = "traceability"
version = "0.13.0" version = "0.14.1"
dependencies = [ dependencies = [
"anyhow", "anyhow",
"serde", "serde",
+2 -2
View File
@@ -29,7 +29,7 @@ members = [
] ]
[workspace.package] [workspace.package]
version = "0.13.0" version = "0.14.1"
edition = "2021" edition = "2021"
rust-version = "1.92" rust-version = "1.92"
license = "GPL-3.0-or-later" license = "GPL-3.0-or-later"
@@ -48,7 +48,7 @@ dr-export = { path = "core/dr-export" }
dr-face = { path = "core/dr-face", default-features = false } dr-face = { path = "core/dr-face", default-features = false }
dr-film = { path = "core/dr-film" } dr-film = { path = "core/dr-film" }
# `tract` on by default so a test binary can open a session with nothing # `tract` on by default so a test binary can open a session with nothing
# installed; the apps add `native` to look for a runtime file (docs/inference.md §3). # installed; the apps add `native` to look for a runtime file (docs/dev/inference.md §3).
dr-inference-engine = { path = "core/dr-inference-engine" } dr-inference-engine = { path = "core/dr-inference-engine" }
dr-ingest = { path = "core/dr-ingest" } dr-ingest = { path = "core/dr-ingest" }
dr-gpu = { path = "core/dr-gpu" } dr-gpu = { path = "core/dr-gpu" }
+109 -59
View File
@@ -1,84 +1,134 @@
# DarkRoom # DarkRoom
A cross-platform, non-destructive RAW photo editor for Linux and Android. A non-destructive RAW photo editor and library for Linux and Android, with a
GPU develop pipeline, a catalog that syncs between devices, and no account,
no telemetry and no cloud of its own.
**Status:** 0.9.0, and no longer a spike. A library opens, culls, develops and [![The library: seventy frames, the timeline beside them, the filter bar above](docs/manual/media/library.png)](docs/manual/README.md)
exports on both platforms, across eight tagged releases. What is *not*
built is written down rather than merely absent — see
[docs/outstanding.md](docs/outstanding.md) for the requirements that have no
implementation and why, and [docs/technical-debt.md](docs/technical-debt.md)
for the compromises that were chosen.
## Documentation **[The manual](docs/manual/README.md)** shows every feature, pictured from
the application itself. This page says what it is, how to get it, and what
is still missing.
| Document | Contents | ## What it does
|---|---|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
| [requirements.md](docs/requirements.md) | What the software must do — 179 numbered requirements |
| [architecture.md](docs/architecture.md) | How it is built — crates, GPU pipeline, data model, sync |
| [technical-debt.md](docs/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
| [outstanding.md](docs/outstanding.md) | What is not built, and whether that is a decision or a gap |
| [code-health.md](docs/code-health.md) | What a contribution costs, per seam, measured |
| [traceability.md](docs/traceability.md) | Generated: which requirement is claimed by which file |
| [faces.md](docs/faces.md) | Face detection and identity — the models, the licence problem, and what S14 measured |
## Building **A library.** Point it at a folder — on this machine, on a network mount,
or one a Nextcloud client keeps in virtual-files mode, where a placeholder
is treated as the photograph rather than as a one-byte file — or at a
Nextcloud account directly. The grid is virtualised, ordered by capture
time with a timeline beside it, and filtered by rating, flag, person and
whether the file is here. Ratings, keywords, collections and a
trash that survives a crash mid-operation. Card ingest. Bursts fold. Face
detection and identity, with the index syncing between devices.
Desktop: **Developing.** Eighteen declared operations fused into one compute
dispatch, plus the neighbourhood work that cannot be: clarity, texture,
capture sharpening, noise reduction, lens correction, spectral film
simulation. Crop and straighten, spot repair, and local adjustments over
masks the model draws — click a subject or a category, then paint, subtract
a gradient, grow or shrink the edge. Focus peaking and a raw histogram for
judging what is recoverable. Named presets; XMP sidecars other editors read.
[![Segmenting an urban scene and choosing the sky as a mask](docs/manual/media/local-segment.png)](docs/manual/README.md#local-adjustments)
**Panoramas.** Select the frames, align, choose a projection, fill the
ragged border rather than crop it, and the composite lands beside its
sources as a DNG, with a sidecar recording what it was merged from.
[![Twelve hand-held frames aligned on a cylinder](docs/manual/media/panorama-aligned.png)](docs/manual/README.md#merging-a-panorama)
**Export.** JPEG, PNG, AVIF, JPEG XL, 8- and 16-bit TIFF, with resize, output
sharpening, a naming template and a colour space — to a folder here or back
into the library.
**On both platforms.** The same core runs on a desktop and a 12-inch
tablet; the interface is one layout, tuned for a wide viewport with touch
targets throughout. On desktop the develop view draws the compute pass's
texture directly — no readback between the GPU and the screen.
## Getting it
| Platform | How | State |
|---|---|---|
| Arch Linux | [`packaging/PKGBUILD`](packaging/PKGBUILD) — `makepkg -si` | Built from every release |
| Android | The APK from each CI run, or `./docker/android/package.sh --install` | Runs on a tablet; F-Droid not yet submitted |
| Windows | `DarkRoom-<version>-x86_64-setup.exe`, cross-built by CI ([windows.md](docs/dev/windows.md)) | Verified under Wine only; unsigned |
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; choosing a library does not yet work in the sandbox |
Or build it. Git LFS is required for the model weights, and the toolchain
pins itself to 1.92.0:
```bash ```bash
cargo run -p darkroom-desktop git lfs install && git lfs pull
cargo run --release -p darkroom-desktop
``` ```
Android (containerised toolchain, see [docker/android](docker/android/README.md)): Android, through the containerised toolchain ([docker/android](docker/android/README.md)):
```bash ```bash
./docker/android/build.sh cargo ndk -t arm64-v8a build --release ./docker/android/build.sh cargo ndk -t arm64-v8a build --release
``` ```
Git LFS is required for the model weights, and the toolchain pins itself. [CONTRIBUTING.md](CONTRIBUTING.md) has the system packages, the four
[CONTRIBUTING.md](CONTRIBUTING.md) has the details and the four commands CI commands CI runs against what you send, and the shortest useful
will run against what you send. contribution — a develop operation is one YAML file, and it arrives with its
controls, its place in the chain and its tests.
## Current state ## Where it stands
**Working.** A catalog over a local folder, a Nextcloud account, or a folder a **0.14.1**, twenty-two tagged releases in. 188 numbered requirements in
sync client keeps in virtual-files mode — where a placeholder is treated as the scope, 82% of them claimed by code and [traced to it](docs/dev/traceability.md);
photograph rather than as a one-byte file. A virtualised library grid with a the rest are written down rather than merely absent.
capture-time timeline, ratings, labels, keywords, collections and a trash that
survives a crash mid-operation. Card ingest. Face detection and identity, with
the index syncing between devices. A develop pipeline of fifteen declared
operations fused into a single compute dispatch, plus the neighbourhood
operations that cannot be — clarity, texture, capture sharpening, noise
reduction, lens correction, spectral film simulation. Crop, straighten, spot
removal, gradient and subject-segmentation masks, named presets, and a
generated panel that no operation in `ui/` is allowed to name. Export to JPEG,
PNG and 8- or 16-bit TIFF with resize and output sharpening.
**The zero-copy display path works on desktop.** The compute pass writes a **Not built:** plugins (post-v1, [D12](docs/dev/requirements.md)), compare and
texture that Slint composites directly, which is what survey culling, AI denoise, tiled and progressive rendering, HDR merge and
[ARCH §6.1](docs/architecture.md) requires; the readback it forbids costs 96% focus stacking, most of the Android platform integration beyond running,
of frame time at 4K, and and the Flatpak's library chooser. The performance targets are half
verified: the per-commit benchmark suite §8 requires exists for everything
that does not need a frame — the catalog, the scan, the thumbnails — and
not yet for the render path, so a regression there fails nothing.
[outstanding.md](docs/dev/outstanding.md) is the list, with the reasoning for
each.
```bash **The one deliberate compromise worth knowing about before reading
cargo run -p dr-gpu --example bench --features readback anything else:** the Android develop view reads its frame back through the
``` CPU, because zero-copy there needs wgpu's Vulkan swapchain and that tears a
portrait window on a tablet whose panel is mounted landscape. It is debt,
not a revision of the rule — [technical-debt.md TD-1](docs/dev/technical-debt.md)
has the measurements and the three things any one of which would remove it.
still reproduces that measurement. **The one exception is the Android develop ## Documentation
view**, which reads the frame back through the CPU because zero-copy there
needs wgpu's Vulkan swapchain, and that tears a portrait window on a tablet
whose panel is mounted landscape. It is debt, not a revision of the rule: the
reasoning, the on-device measurements that forced it, and the three separate
things any one of which would remove it are in
[technical-debt.md TD-1](docs/technical-debt.md).
**Not built.** Plugins, compare and survey culling, focus peaking, burst [docs/README.md](docs/README.md) is the index. The short version, for someone using it:
grouping, AI denoise, tiled and progressive rendering, and most of the Android
platform integration beyond running. The performance targets in §4.1 are | | |
unverified rather than unmet — the per-commit benchmark suite §8 requires does |---|---|
not exist, so nothing fails a build on a regression. | [manual](docs/manual/README.md) | Every feature, pictured |
[docs/outstanding.md](docs/outstanding.md) is the list, with the reasoning. | [gestures.md](docs/gestures.md) | How it is driven — generated from the code, so it cannot describe a gesture that does not exist |
For someone changing it:
| | |
|---|---|
| [CONTRIBUTING.md](CONTRIBUTING.md) | How to land a first change without reading the rest |
| [requirements.md](docs/dev/requirements.md) | What the software must do — the numbered register, and the decisions |
| [architecture.md](docs/dev/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
| [technical-debt.md](docs/dev/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
| [outstanding.md](docs/dev/outstanding.md) | What is not built, and whether that is a decision or a gap |
| [code-health.md](docs/dev/code-health.md) | What a contribution costs, per seam, measured |
| [traceability.md](docs/dev/traceability.md) | Generated: which requirement is claimed by which file |
Designs, one per subsystem:
[segmentation](docs/dev/segmentation.md) and [mask editing](docs/dev/mask-editing.md) ·
[spot removal](docs/dev/spot-removal.md) · [panorama](docs/dev/panorama.md) ·
[faces](docs/dev/faces.md) · [inference](docs/dev/inference.md) ·
[storage and sync](docs/dev/storage.md) · [catalog](docs/dev/catalog.md) ·
[display and extension](docs/dev/display-and-extension.md) ·
[navigation](docs/dev/ui-navigation.md) · [distribution](docs/dev/distribution.md) ·
[windows](docs/dev/windows.md) · [benchmarks](docs/dev/benchmarks.md).
## Licence ## Licence
GPL-3.0-or-later. GPL-3.0-or-later. The photographs in the manual and the test fixtures are
the author's and are there to show and test this project, nothing else.
The model weights carry their own licences — [models/LICENCE.md](models/LICENCE.md).
+6 -6
View File
@@ -240,7 +240,7 @@ fn android_main(app: slint::android::AndroidApp) {
/// ///
/// **Face weights are absent from the repository by design.** The InsightFace /// **Face weights are absent from the repository by design.** The InsightFace
/// grant is research-only and incompatible with this project's licence /// grant is research-only and incompatible with this project's licence
/// (docs/faces.md §2), so a desktop user fetches them, runs /// (docs/dev/faces.md §2), so a desktop user fetches them, runs
/// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build /// `tools/fix-face-model-shapes.sh` over them, and drops the result in. A build
/// that carries none is the ordinary case and face indexing simply stays off. /// that carries none is the ordinary case and face indexing simply stays off.
/// ///
@@ -321,19 +321,19 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// before it reports the tab available. // before it reports the tab available.
// //
// Three detectors, because which one runs is a setting // Three detectors, because which one runs is a setting
// (`FaceDetector`, docs/faces.md §12.3) and a tablet has no other way to // (`FaceDetector`, docs/dev/faces.md §12.3) and a tablet has no other way to
// obtain the one it was not shipped with. Twenty megabytes of APK for // obtain the one it was not shipped with. Twenty megabytes of APK for
// the choice; the embedder is the same for all three. // the choice; the embedder is the same for all three.
// //
// Then the three eye-state models (docs/faces.md §17): landmarks, open // Then the three eye-state models (docs/dev/faces.md §17): landmarks, open
// or closed, sunglasses. The app indexes without them; with them the // or closed, sunglasses. The app indexes without them; with them the
// eyes-open filter has something to read, and a tablet has no other way // eyes-open filter has something to read, and a tablet has no other way
// to get them either. // to get them either.
// //
// The int8 forms beside the three detectors are what the Hexagon runs // The int8 forms beside the three detectors are what the Hexagon runs
// (docs/inference.md §5); the engine loads the sibling when the probe // (docs/dev/inference.md §5); the engine loads the sibling when the probe
// chose that rung and ignores it otherwise. // chose that rung and ignores it otherwise.
const BUNDLED: [(&std::ffi::CStr, &str); 13] = [ const BUNDLED: [(&std::ffi::CStr, &str); 14] = [
(c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"), (c"models/scrfd_500m_640.onnx", "scrfd_500m_640.onnx"),
( (
c"models/scrfd_500m_640.int8.onnx", c"models/scrfd_500m_640.int8.onnx",
@@ -420,7 +420,7 @@ fn unpack_bundled_models(app: &slint::android::AndroidApp) {
// on a first launch they were not on disk until this line. The runtime // on a first launch they were not on disk until this line. The runtime
// is in the APK's native library directory beside `libdarkroom.so`, // is in the APK's native library directory beside `libdarkroom.so`,
// which is also where Qualcomm's DSP loader has to be pointed for the // which is also where Qualcomm's DSP loader has to be pointed for the
// Hexagon skel (docs/inference.md §3, §8). // Hexagon skel (docs/dev/inference.md §3, §8).
dr_ui::inference::init(native_library_dir().into_iter().collect()); dr_ui::inference::init(native_library_dir().into_iter().collect());
} }
+12 -7
View File
@@ -21,7 +21,7 @@ fn main() -> anyhow::Result<()> {
// build made on a machine that cannot run the application — the Linux CI // build made on a machine that cannot run the application — the Linux CI
// producing the Windows binary, checked under Wine — has an exit that // producing the Windows binary, checked under Wine — has an exit that
// proves the executable starts without opening a window or touching the // proves the executable starts without opening a window or touching the
// user's directories (docs/windows.md §6). // user's directories (docs/dev/windows.md §6).
if std::env::args().nth(1).as_deref() == Some("--version") { if std::env::args().nth(1).as_deref() == Some("--version") {
println!("darkroom-desktop {}", env!("CARGO_PKG_VERSION")); println!("darkroom-desktop {}", env!("CARGO_PKG_VERSION"));
return Ok(()); return Ok(());
@@ -63,7 +63,7 @@ fn main() -> anyhow::Result<()> {
// Before the window: the probe runs on its own thread and the first // Before the window: the probe runs on its own thread and the first
// frame does not wait for it, but the models a background job asks for // frame does not wait for it, but the models a background job asks for
// should already know where the runtime is (docs/inference.md §4). // should already know where the runtime is (docs/dev/inference.md §4).
dr_ui::inference::init(runtime_dirs()); dr_ui::inference::init(runtime_dirs());
dr_ui::run(paths)?; dr_ui::run(paths)?;
@@ -78,15 +78,19 @@ fn main() -> anyhow::Result<()> {
/// Where a desktop package may have put `libonnxruntime`, most specific /// Where a desktop package may have put `libonnxruntime`, most specific
/// first. None of these existing is the tract build, which is a complete /// first. None of these existing is the tract build, which is a complete
/// application and not an error (docs/inference.md §3). /// application and not an error (docs/dev/inference.md §3).
/// ///
/// `DARKROOM_ORT_DIR` is for a developer pointing at a runtime that is not /// `DARKROOM_ORT_DIR` is for a developer pointing at a runtime that is not
/// installed — the wheel's `capi` directory, say. Then beside the executable /// installed — the wheel's `capi` directory, say. Then beside the executable
/// and in the package's private library directory, for a package that /// and in the package's private library directory, for a package that
/// bundles its own; then the Flatpak prefix; then the system library /// bundles its own; then the user's own `runtime/` beside the models, where
/// directory, for a distribution that ships ONNX Runtime as a package of its /// `tools/fetch-desktop-runtime.sh` puts one; then the Flatpak prefix; then
/// own. A system copy whose GPU providers do not load is not a problem: the /// the system library directory, for a distribution that ships ONNX Runtime
/// probe builds a real session before believing a provider. /// as a package of its own. The user's copy outranks the system's because
/// the system's is the one most likely to be built without the GPU
/// providers, or against the wrong cuDNN — and a system copy whose providers
/// do not load is not a problem, only a slower app: the probe builds a real
/// session before believing a provider.
fn runtime_dirs() -> Vec<PathBuf> { fn runtime_dirs() -> Vec<PathBuf> {
let mut dirs = Vec::new(); let mut dirs = Vec::new();
if let Some(dir) = std::env::var_os("DARKROOM_ORT_DIR") { if let Some(dir) = std::env::var_os("DARKROOM_ORT_DIR") {
@@ -98,6 +102,7 @@ fn runtime_dirs() -> Vec<PathBuf> {
dirs.push(bin.join("../lib/darkroom")); dirs.push(bin.join("../lib/darkroom"));
} }
} }
dirs.push(dr_ui::inference::user_runtime_dir());
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
dirs.extend([ dirs.extend([
PathBuf::from("/app/lib/darkroom"), PathBuf::from("/app/lib/darkroom"),
+1 -1
View File
@@ -315,7 +315,7 @@ fn full_library(
// The three phases, separately, because "a regroup takes n seconds" does // The three phases, separately, because "a regroup takes n seconds" does
// not tell anyone which half to optimise — and the answer differs between // not tell anyone which half to optimise — and the answer differs between
// a desktop and a tablet (docs/faces.md §9). // a desktop and a tablet (docs/dev/faces.md §9).
{ {
let dim = candidates.first().map(|c| c.embedding.len()).unwrap_or(0); let dim = candidates.first().map(|c| c.embedding.len()).unwrap_or(0);
let flat: Vec<f32> = candidates let flat: Vec<f32> = candidates
+1 -1
View File
@@ -82,7 +82,7 @@
//! Grouping has no natural `subject_id`: it is a property of a *run* of frames, //! Grouping has no natural `subject_id`: it is a property of a *run* of frames,
//! so a per-image job would rebuild the world once per photograph. It is //! so a per-image job would rebuild the world once per photograph. It is
//! therefore a debounced library-level pass, for exactly the reasons //! therefore a debounced library-level pass, for exactly the reasons
//! docs/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole //! docs/dev/catalog.md §10.2 gives for face clustering, and [`regroup`] is the whole
//! of it — one ordered walk, no per-pair comparison beyond adjacent frames. //! of it — one ordered walk, no per-pair comparison beyond adjacent frames.
//! //!
//! # Grouping is not hiding //! # Grouping is not hiding
+210 -33
View File
@@ -2,7 +2,7 @@
//! Face data as sealed shards, so a second device does not re-index the library. //! Face data as sealed shards, so a second device does not re-index the library.
//! //!
//! Indexing a 23,500-image library is on the order of two hours of CPU //! Indexing a 23,500-image library is on the order of two hours of CPU
//! (docs/faces.md §12.2). It is also **byte-identical on every device**: the //! (docs/dev/faces.md §12.2). It is also **byte-identical on every device**: the
//! same model over the same proxy produces the same embedding. Paying for it //! same model over the same proxy produces the same embedding. Paying for it
//! once per account rather than once per device is the whole point of this //! once per account rather than once per device is the whole point of this
//! module, and it is the same bargain the thumbnail store already makes. //! module, and it is the same bargain the thumbnail store already makes.
@@ -178,6 +178,67 @@ impl FaceShardStore {
.flatten() .flatten()
} }
/// The other pipelines this file is held under that share `model_id`'s
/// embedder — the generations a put of `model_id` may supersede.
fn siblings(&self, file_id: u64, model_id: &str) -> Vec<String> {
let mut stmt = match self.index.prepare(&format!(
"SELECT model_id FROM entries
WHERE file_id = ?1 AND model_id != ?2 AND {} = ?3",
crate::faces::embedder_sql("model_id")
)) {
Ok(s) => s,
Err(_) => return Vec::new(),
};
stmt.query_map(
rusqlite::params![
file_id as i64,
model_id,
crate::faces::embedder_of(model_id)
],
|r| r.get::<_, String>(0),
)
.map(|rows| rows.filter_map(|r| r.ok()).collect())
.unwrap_or_default()
}
/// Whether a pass this file is already held under outranks `model_id`,
/// so a put of `model_id` would add a generation nobody would adopt.
pub fn outranked(&self, file_id: u64, model_id: &str) -> bool {
use dr_types::FaceDetector;
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
return false;
};
self.siblings(file_id, model_id)
.iter()
.filter_map(|m| FaceDetector::for_model_id(m))
.any(|held| held.outranks(incoming))
}
/// Forget the index entries for generations of this file that `model_id`
/// outranks. The bytes stay where they are — a sealed shard is
/// immutable — but the store stops offering them, and a later export or
/// merge writes nothing for them again.
fn supersede(&self, file_id: u64, model_id: &str) -> Result<(), CatalogError> {
use dr_types::FaceDetector;
let Some(incoming) = FaceDetector::for_model_id(model_id) else {
return Ok(());
};
for held in self.siblings(file_id, model_id) {
let weaker = FaceDetector::for_model_id(&held).is_some_and(|h| incoming.outranks(h));
if weaker {
self.index.execute(
"DELETE FROM entries WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, held],
)?;
self.index.execute(
"DELETE FROM faces_meta WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, held],
)?;
}
}
Ok(())
}
pub fn contains(&self, file_id: u64, model_id: &str) -> bool { pub fn contains(&self, file_id: u64, model_id: &str) -> bool {
self.index self.index
.query_row( .query_row(
@@ -233,6 +294,15 @@ impl FaceShardStore {
faces: &[SharedFace], faces: &[SharedFace],
indexed_at: Option<i64>, indexed_at: Option<i64>,
) -> Result<u32, CatalogError> { ) -> Result<u32, CatalogError> {
// One generation per image per embedder. A store carried every pass
// — 24,123 entries for 19,089 images on the reference library, a
// third of its 293 MB — and only the strongest was ever adopted.
// A weaker pass arriving after a stronger one is not written; a
// stronger one arriving retires the weaker from the index.
if self.outranked(file_id, model_id) {
return Ok(0);
}
self.supersede(file_id, model_id)?;
let incoming = faces let incoming = faces
.iter() .iter()
.map(|f| BYTES_PER_FACE + if f.crop.is_empty() { 0 } else { BYTES_PER_CROP }) .map(|f| BYTES_PER_FACE + if f.crop.is_empty() { 0 } else { BYTES_PER_CROP })
@@ -474,15 +544,28 @@ impl FaceShardStore {
rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX, rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_NO_MUTEX,
)?; )?;
let mut q = // The peer's marker travels with the image: it is what lets
src.prepare("SELECT file_id, model_id, faces_found, source_edge FROM indexed")?; // `import_from_shards` record the adoption under the time the peer
let images: Vec<(i64, String, i64, i64)> = q // indexed it, and so what keeps `export_to_shards` from reading the
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))? // adoption as a re-index and sending the peer's faces back out under
// this device's name. A shard from before the column has none.
let mut q = src.prepare(&format!(
"SELECT file_id, model_id, faces_found, source_edge, {} FROM indexed",
match has_column(&src, "indexed", "indexed_at") {
Ok(true) => "indexed_at",
_ => "NULL",
}
))?;
let images: Vec<(i64, String, i64, i64, Option<i64>)> = q
.query_map([], |r| {
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
})?
.collect::<Result<_, _>>()?; .collect::<Result<_, _>>()?;
let mut adopted = 0; let mut adopted = 0;
for (file_id, model_id, _found, edge) in images { for (file_id, model_id, _found, edge, indexed_at) in images {
if self.contains(file_id as u64, &model_id) { if self.contains(file_id as u64, &model_id) || self.outranked(file_id as u64, &model_id)
{
continue; continue;
} }
let mut fq = src.prepare(&format!( let mut fq = src.prepare(&format!(
@@ -501,12 +584,27 @@ impl FaceShardStore {
let faces: Vec<SharedFace> = fq let faces: Vec<SharedFace> = fq
.query_map(rusqlite::params![file_id, &model_id], read_shared_face)? .query_map(rusqlite::params![file_id, &model_id], read_shared_face)?
.collect::<Result<_, _>>()?; .collect::<Result<_, _>>()?;
self.put_image(file_id as u64, &model_id, edge as u32, &faces)?; self.put_image_at(file_id as u64, &model_id, edge as u32, &faces, indexed_at)?;
adopted += 1; adopted += 1;
} }
Ok(adopted) Ok(adopted)
} }
/// Record when the catalog indexed a held image, for an entry that
/// arrived without a marker — a peer's shard from before the column.
pub fn set_indexed_at(
&self,
file_id: u64,
model_id: &str,
at: i64,
) -> Result<(), CatalogError> {
self.index.execute(
"UPDATE entries SET indexed_at = ?3 WHERE file_id = ?1 AND model_id = ?2",
rusqlite::params![file_id as i64, model_id, at],
)?;
Ok(())
}
/// Read back everything held for one image. /// Read back everything held for one image.
pub fn get_image( pub fn get_image(
&self, &self,
@@ -798,8 +896,21 @@ pub fn import_from_shards(
})? })?
.collect::<Result<_, _>>()?; .collect::<Result<_, _>>()?;
/// Images per write transaction. Large enough that fourteen thousand
/// adoptions are a hundred and forty commits rather than fourteen
/// thousand; small enough that a read on the UI thread, queued behind
/// the lock, waits a fraction of a second and not the whole import.
const CHUNK: usize = 100;
let mut adopted = 0; let mut adopted = 0;
let mut tx = conn.unchecked_transaction()?;
let mut in_chunk = 0;
for (file_id, image_id, local) in candidates { for (file_id, image_id, local) in candidates {
if in_chunk == CHUNK {
tx.commit()?;
tx = conn.unchecked_transaction()?;
in_chunk = 0;
}
let Some(held) = store.held_model(file_id as u64, model_id) else { let Some(held) = store.held_model(file_id as u64, model_id) else {
continue; continue;
}; };
@@ -820,20 +931,22 @@ pub fn import_from_shards(
let Some((faces, edge)) = store.get_image(file_id as u64, &held)? else { let Some((faces, edge)) = store.get_image(file_id as u64, &held)? else {
continue; continue;
}; };
// A peer that embedded before the quality was kept has done work this // A face the peer embedded before its quality was kept (schema V14)
// device cannot finish: the number exists only at embedding time, and // is adopted with the reading missing, exactly as one without an eye
// adopting the faces would write the run marker that keeps them from // reading is. The measuring passes find their work by the NULL
// ever being measured (schema V14). Left for this device's own pass — // column, not by the run marker (`dr_ui::repairs`, `faces_needing`),
// or for the peer's, whose re-export replaces these. // so adopting costs the reading nothing and this device's own pass
// fills it.
// //
// A missing *eye* reading is not the same case and is adopted. The // This used to refuse such faces, on the reasoning that the marker
// measuring pass finds those by the NULL, not by the marker, so // would stop them ever being measured — true before the quality
// adopting the faces costs the reading nothing (schema V16) — and a // repair existed, and wrong after. What it cost: V14 had dropped the
// peer that has no eye models may be the only one that has done the // markers of every image holding such faces, so the peer never
// detection at all. // re-exported them, and the only copies in the shards were the
if faces.iter().any(|f| f.quality.is_none()) { // unmeasured ones. A tablet holding shards with 4,310 of the
continue; // desktop's images and 3,170 of its confirmations declined every one
} // of them, showed a fraction of each person, and queued the whole
// library for a re-detection of its own instead.
let local: Vec<crate::faces::DetectedFace> = faces let local: Vec<crate::faces::DetectedFace> = faces
.into_iter() .into_iter()
.map(|f| crate::faces::DetectedFace { .map(|f| crate::faces::DetectedFace {
@@ -856,15 +969,42 @@ pub fn import_from_shards(
}) })
.collect(); .collect();
crate::faces::record_detections( crate::faces::record_detections_within(
conn, &tx,
dr_types::ImageId(image_id as u64), dr_types::ImageId(image_id as u64),
&held, &held,
edge, edge,
&local, &local,
)?; )?;
// The peer's marker, not this moment. `record_detections` stamps the
// run as now, and `export_to_shards` reads a marker newer than the
// shard's as a re-index — so every adopted image went straight back
// out as this device's own work: 14,100 adopted, 15,457 "newly
// indexed" on the next pass, and twenty-two shards of a peer's faces
// uploaded again under a second name. Where the peer's shard carried
// no marker, the store takes the catalog's, so the two agree either
// way and the export sees nothing to send.
match store.indexed_at(file_id as u64, &held) {
Some(theirs) => {
tx.execute(
"UPDATE face_index SET indexed_at = ?3
WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id, held, theirs],
)?;
}
None => {
let ours: i64 = tx.query_row(
"SELECT indexed_at FROM face_index WHERE image_id = ?1 AND model_id = ?2",
rusqlite::params![image_id, held],
|r| r.get(0),
)?;
store.set_indexed_at(file_id as u64, &held, ours)?;
}
}
adopted += 1; adopted += 1;
in_chunk += 1;
} }
tx.commit()?;
Ok(adopted) Ok(adopted)
} }
@@ -1100,6 +1240,32 @@ mod tests {
assert!(!s.contains(1, "lvface")); assert!(!s.contains(1, "lvface"));
} }
/// One generation per image per embedder: a stronger detector's pass
/// retires a weaker one from the index, and a weaker pass arriving after
/// a stronger is not written at all.
#[test]
fn a_stronger_pass_retires_a_weaker_one_and_a_weaker_is_not_added() {
let dir = tempdir();
let mut s = FaceShardStore::open(&dir).unwrap();
s.put_image(1, "w600k_mbf", 1024, &[face(1, 1)]).unwrap();
s.put_image(1, "scrfd_10g+w600k_mbf", 1024, &[face(1, 2)])
.unwrap();
assert!(s.contains(1, "scrfd_10g+w600k_mbf"));
assert!(!s.contains(1, "w600k_mbf"), "the fast pass was not retired");
assert_eq!(s.len(), 1, "faces_meta still counts the retired pass");
s.put_image(1, "scrfd_2.5g+w600k_mbf", 1024, &[face(1, 3)])
.unwrap();
assert!(
!s.contains(1, "scrfd_2.5g+w600k_mbf"),
"a weaker pass was added"
);
assert_eq!(
s.held_model(1, "w600k_mbf").as_deref(),
Some("scrfd_10g+w600k_mbf")
);
}
#[test] #[test]
fn re_storing_an_image_replaces_rather_than_doubling_it() { fn re_storing_an_image_replaces_rather_than_doubling_it() {
let dir = tempdir(); let dir = tempdir();
@@ -1320,6 +1486,11 @@ mod catalog_round_trip {
assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5); assert!((got[0].landmarks[2].0 - 0.15).abs() < 1e-5);
let emb = faces::embeddings(&b, "w600k_mbf").unwrap(); let emb = faces::embeddings(&b, "w600k_mbf").unwrap();
assert!(emb.iter().any(|e| e.embedding[0] == 1)); assert!(emb.iter().any(|e| e.embedding[0] == 1));
// And what B adopted is not B's work: its next export sends nothing.
// Adopting used to stamp the run as now, so every adopted image went
// back out under B's name as a re-index.
assert_eq!(export_to_shards(&b, &mut store_b, "w600k_mbf").unwrap(), 0);
} }
/// The desktop switched to a stronger detector part-way through the /// The desktop switched to a stronger detector part-way through the
@@ -1365,11 +1536,12 @@ mod catalog_round_trip {
); );
} }
/// A face a peer embedded without measuring it is work this device /// A face a peer embedded without measuring it is adopted all the same,
/// cannot finish, and adopting it would write the marker that stops it /// and left on this device's quality pass by its missing reading. Refusing
/// ever being measured. The image stays outstanding instead. /// it was what stranded every confirmation the desktop had made on faces
/// from before V14: the tablet held the shards and would not use them.
#[test] #[test]
fn a_peers_unmeasured_faces_are_left_for_this_device_to_index() { fn a_peers_unmeasured_faces_are_adopted_and_left_for_the_quality_pass() {
let b = device(&[(90, 5001), (91, 5002)]); let b = device(&[(90, 5001), (91, 5002)]);
let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap(); let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap();
let shared = |file_id: u64, quality: Option<f32>| SharedFace { let shared = |file_id: u64, quality: Option<f32>| SharedFace {
@@ -1395,13 +1567,18 @@ mod catalog_round_trip {
.put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))]) .put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))])
.unwrap(); .unwrap();
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 1); assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 2);
let cov = faces::coverage(&b, "w600k_mbf").unwrap(); let cov = faces::coverage(&b, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1); assert_eq!(cov.indexed, 2);
assert_eq!(cov.outstanding(), 1, "the unmeasured image was adopted"); assert_eq!(cov.outstanding(), 0, "the unmeasured image was refused");
assert!(faces::for_image(&b, dr_types::ImageId(90)) let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap();
.unwrap() assert_eq!(got.len(), 1);
.is_empty()); assert_eq!(got[0].quality, None, "a reading was invented");
// Still owed to the measuring pass, which lists by the column.
assert_eq!(
faces::count_needing(&b, "w600k_mbf", "f.quality IS NULL").unwrap(),
1
);
} }
#[test] #[test]
+252 -19
View File
@@ -1,7 +1,7 @@
//! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5 //! TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
//! People and faces: what was detected, who it is, and who said so. //! People and faces: what was detected, who it is, and who said so.
//! //!
//! The storage half of docs/faces.md. `dr-face` finds faces and turns them into //! The storage half of docs/dev/faces.md. `dr-face` finds faces and turns them into
//! 512 numbers; this module is where those numbers acquire an identity, and //! 512 numbers; this module is where those numbers acquire an identity, and
//! where the user's corrections outrank the model's guesses. //! where the user's corrections outrank the model's guesses.
//! //!
@@ -110,7 +110,7 @@ pub struct DetectedFace {
/// Raw rather than unit length, so the length ([`Self::quality`]) is in /// Raw rather than unit length, so the length ([`Self::quality`]) is in
/// the blob and not only beside it. Readers re-normalise on load. /// the blob and not only beside it. Readers re-normalise on load.
pub embedding: Vec<u8>, pub embedding: Vec<u8>,
/// Source pixels across the aligned crop (docs/faces.md §7). /// Source pixels across the aligned crop (docs/dev/faces.md §7).
pub crop_px: f32, pub crop_px: f32,
/// Length of the raw embedding before normalisation — the model's own /// Length of the raw embedding before normalisation — the model's own
/// reading of how recognisable the crop was, and the gate on whether /// reading of how recognisable the crop was, and the gate on whether
@@ -211,7 +211,7 @@ pub use dr_face::Calibration;
/// the same face in the same photograph, for carrying an identity across a /// the same face in the same photograph, for carrying an identity across a
/// re-detection. /// re-detection.
/// ///
/// Set at the reference library's P≈0.95 line (docs/faces.md §9's table: /// Set at the reference library's P≈0.95 line (docs/dev/faces.md §9's table:
/// 0.449), which is far above anything two different people in one frame /// 0.449), which is far above anything two different people in one frame
/// reach and below what one face re-embedded from a better crop of itself /// reach and below what one face re-embedded from a better crop of itself
/// does. The number is only ever asked about *overlapping* boxes on *one* /// does. The number is only ever asked about *overlapping* boxes on *one*
@@ -279,7 +279,25 @@ pub fn record_detections(
faces: &[DetectedFace], faces: &[DetectedFace],
) -> Result<Vec<FaceId>, CatalogError> { ) -> Result<Vec<FaceId>, CatalogError> {
let tx = conn.unchecked_transaction()?; let tx = conn.unchecked_transaction()?;
let ids = record_detections_within(&tx, image_id, model_id, source_edge, faces)?;
tx.commit()?;
Ok(ids)
}
/// [`record_detections`] inside a transaction the caller owns.
///
/// For a caller recording many images at once — the shard import adopts
/// fourteen thousand in one pass — where a commit per image is fourteen
/// thousand fsyncs and fourteen thousand turns at the write lock that every
/// read on the UI thread queues behind. `unchecked_transaction` cannot nest,
/// so the batching has to be offered here rather than wrapped from above.
pub fn record_detections_within(
tx: &Connection,
image_id: ImageId,
model_id: &str,
source_edge: u32,
faces: &[DetectedFace],
) -> Result<Vec<FaceId>, CatalogError> {
// Everything the old faces knew, so it can be carried across the // Everything the old faces knew, so it can be carried across the
// replacement. Read only when there is something to carry it onto: a // replacement. Read only when there is something to carry it onto: a
// pass that found nothing has nothing to match, and decoding a vector // pass that found nothing has nothing to match, and decoding a vector
@@ -287,7 +305,7 @@ pub fn record_detections(
let prior = if faces.is_empty() { let prior = if faces.is_empty() {
Vec::new() Vec::new()
} else { } else {
read_priors(&tx, image_id)? read_priors(tx, image_id)?
}; };
tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?; tx.execute("DELETE FROM faces WHERE image_id = ?1", [image_id.0 as i64])?;
@@ -389,7 +407,6 @@ pub fn record_detections(
], ],
)?; )?;
tx.commit()?;
Ok(ids) Ok(ids)
} }
@@ -556,6 +573,19 @@ impl FaceUpdate {
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose /// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how what was written /// marker is newer than the store's copy, which is how what was written
/// here reaches the other devices. /// here reaches the other devices.
///
/// It is re-written under the pipeline id the **faces carry**, not the one
/// this pass ran as. `model_id` names the pass only through its embedder;
/// the detector half of a marker is a statement about who drew the boxes,
/// and this pass drew none. Every reader takes the two to agree: the export
/// selects an image's faces by the marker's id, `marker_under` takes a
/// marker as proof the detector has been over the image, and the shard
/// store keys each face by it. When the marker was written as
/// `scrfd_10g+w600k_mbf` over faces still spelled `w600k_mbf`, the export
/// found no faces under it and sent the other devices an entry saying the
/// thorough detector had looked and found nothing — over photographs with
/// named faces on them. With no faces left, the pass's own id is the only
/// one there is, and the marker says so.
pub fn record_updates( pub fn record_updates(
conn: &Connection, conn: &Connection,
image_id: ImageId, image_id: ImageId,
@@ -602,13 +632,25 @@ pub fn record_updates(
for f in dropped { for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?; tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
} }
let remaining: i64 = tx.query_row( let (remaining, found_by): (i64, Option<String>) = tx.query_row(
&format!( &format!(
"SELECT COUNT(*) FROM faces WHERE image_id = ?1 AND {} = ?2", "SELECT COUNT(*), MIN(model_id) FROM faces WHERE image_id = ?1 AND {} = ?2",
embedder_sql("model_id") embedder_sql("model_id")
), ),
rusqlite::params![image_id.0 as i64, embedder_of(model_id)], rusqlite::params![image_id.0 as i64, embedder_of(model_id)],
|r| r.get(0), |r| Ok((r.get(0)?, r.get(1)?)),
)?;
let marker = found_by.as_deref().unwrap_or(model_id);
// One marker per embedder: a stale one under another spelling would
// keep saying that detector had been here, which is the claim the
// faces' own id is now making in its place.
tx.execute(
&format!(
"DELETE FROM face_index
WHERE image_id = ?1 AND model_id != ?2 AND {} = ?3",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, marker, embedder_of(model_id)],
)?; )?;
tx.execute( tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge) "INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
@@ -619,7 +661,7 @@ pub fn record_updates(
source_edge = excluded.source_edge", source_edge = excluded.source_edge",
rusqlite::params![ rusqlite::params![
image_id.0 as i64, image_id.0 as i64,
model_id, marker,
now_secs(), now_secs(),
remaining, remaining,
source_edge as i64, source_edge as i64,
@@ -838,6 +880,22 @@ pub fn unassigned(conn: &Connection, model_id: &str) -> Result<Vec<FaceId>, Cata
rows.collect::<Result<_, _>>().map_err(Into::into) rows.collect::<Result<_, _>>().map_err(Into::into)
} }
/// How many faces have no identity yet — [`unassigned`] counted rather than
/// listed, for a screen that only shows the number.
pub fn count_unassigned(conn: &Connection, model_id: &str) -> Result<u64, CatalogError> {
let n: i64 = conn.query_row(
&format!(
"SELECT COUNT(*) FROM faces f
LEFT JOIN face_person fp ON fp.face_id = f.id
WHERE fp.face_id IS NULL AND {} = ?1",
embedder_sql("f.model_id")
),
[embedder_of(model_id)],
|r| r.get(0),
)?;
Ok(n as u64)
}
/// One face's stored embedding, as the clustering pass consumes it. /// One face's stored embedding, as the clustering pass consumes it.
/// ///
/// A struct rather than a tuple because it crosses a crate boundary and "the /// A struct rather than a tuple because it crosses a crate boundary and "the
@@ -910,17 +968,46 @@ pub fn rename_person(conn: &Connection, person: PersonId, name: &str) -> Result<
/// Merged-away people are excluded: they exist as redirects so a sync does not /// Merged-away people are excluded: they exist as redirects so a sync does not
/// resurrect them, not as entries in a list. /// resurrect them, not as entries in a list.
pub fn people(conn: &Connection) -> Result<Vec<Person>, CatalogError> { pub fn people(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
let mut q = conn.prepare( people_where(conn, false)
}
/// Everyone who holds a face, carries a name, or was set aside — the people a
/// screen has a row for.
///
/// The rest are the empty, unnamed groups a regrouping pass leaves behind
/// (`prune_empty_unnamed`), and on the reference library they were 17,000 of
/// 19,000 rows: read, counted, sorted by name and then thrown away by the
/// caller on every redraw. Filtered here, in the query, they are never
/// sorted. The filter is SQL's `trim`, which strips spaces and not every
/// whitespace character, so a name that is only a tab is listed rather than
/// hidden — the safe direction for a row the user typed something into.
pub fn people_in_use(conn: &Connection) -> Result<Vec<Person>, CatalogError> {
people_where(conn, true)
}
/// [`people`], with face counts aggregated once per person *before* the join
/// rather than grouped after it: the face table is joined to the 2,000
/// people it names, not the 19,000 rows of the people table.
fn people_where(conn: &Connection, in_use: bool) -> Result<Vec<Person>, CatalogError> {
let filter = if in_use {
"AND (c.person_id IS NOT NULL OR trim(p.name) <> '' OR p.ignored)"
} else {
""
};
let mut q = conn.prepare(&format!(
"SELECT p.id, p.uuid, p.name, "SELECT p.id, p.uuid, p.name,
COALESCE(SUM(fp.confirmed = 1), 0), COALESCE(c.confirmed, 0),
COALESCE(SUM(fp.confirmed = 0), 0), COALESCE(c.suggested, 0),
p.ignored p.ignored
FROM people p FROM people p
LEFT JOIN face_person fp ON fp.person_id = p.id LEFT JOIN (SELECT person_id,
WHERE p.merged_into IS NULL SUM(confirmed = 1) AS confirmed,
GROUP BY p.id SUM(confirmed = 0) AS suggested
ORDER BY 4 DESC, 5 DESC, p.name", FROM face_person
)?; GROUP BY person_id) c ON c.person_id = p.id
WHERE p.merged_into IS NULL {filter}
ORDER BY 4 DESC, 5 DESC, p.name"
))?;
let rows = q.query_map([], |r| { let rows = q.query_map([], |r| {
Ok(Person { Ok(Person {
id: PersonId(r.get::<_, i64>(0)? as u64), id: PersonId(r.get::<_, i64>(0)? as u64),
@@ -1057,6 +1144,72 @@ pub fn confirm(conn: &Connection, face: FaceId, person: PersonId) -> Result<(),
Ok(()) Ok(())
} }
/// The user says every suggested face of this person is right.
///
/// What "Confirm all" runs, and the reason it is not a loop over [`confirm`]:
/// that is a transaction per face, and on a group of several hundred it was
/// several hundred commits for one click. Two statements, one commit, and the
/// same two rules `confirm` applies face by face — an earlier rejection of
/// the pair is overridden, and a face already confirmed is left alone.
///
/// Returns how many suggestions became confirmations.
pub fn confirm_all(conn: &Connection, person: PersonId) -> Result<u64, CatalogError> {
let tx = conn.unchecked_transaction()?;
tx.execute(
"DELETE FROM face_person_rejected
WHERE person_id = ?1
AND face_id IN (SELECT face_id FROM face_person
WHERE person_id = ?1 AND confirmed = 0)",
[person.0 as i64],
)?;
let n = tx.execute(
"UPDATE face_person SET confirmed = 1, probability = 1.0
WHERE person_id = ?1 AND confirmed = 0",
[person.0 as i64],
)?;
tx.commit()?;
Ok(n as u64)
}
/// The user says these faces are `to`, not `from`.
///
/// [`reject`] from one and [`confirm`] onto the other, for every face, in one
/// transaction — what a split commits. Rejecting first is what stops the
/// split being undone: without it the next pass sees a face that looks like
/// `from` and suggests it straight back. Confirmed rather than suggested on
/// `to`, because the user has just asserted these belong together.
pub fn reassign(
conn: &Connection,
faces: &[FaceId],
from: PersonId,
to: PersonId,
) -> Result<(), CatalogError> {
let tx = conn.unchecked_transaction()?;
{
let mut reject = tx.prepare(
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
VALUES (?1, ?2)",
)?;
let mut unrejected =
tx.prepare("DELETE FROM face_person_rejected WHERE face_id = ?1 AND person_id = ?2")?;
let mut confirm = tx.prepare(
"INSERT INTO face_person (face_id, person_id, probability, confirmed)
VALUES (?1, ?2, 1.0, 1)
ON CONFLICT(face_id) DO UPDATE SET
person_id = excluded.person_id,
probability = 1.0,
confirmed = 1",
)?;
for face in faces {
reject.execute(rusqlite::params![face.0 as i64, from.0 as i64])?;
unrejected.execute(rusqlite::params![face.0 as i64, to.0 as i64])?;
confirm.execute(rusqlite::params![face.0 as i64, to.0 as i64])?;
}
}
tx.commit()?;
Ok(())
}
/// The user says this face is **not** this person. /// The user says this face is **not** this person.
/// ///
/// Stored rather than implied by removal, so the next clustering pass does not /// Stored rather than implied by removal, so the next clustering pass does not
@@ -1446,7 +1599,7 @@ fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
} }
} }
fn now_secs() -> i64 { pub(crate) fn now_secs() -> i64 {
std::time::SystemTime::now() std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH) .duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64) .map(|d| d.as_secs() as i64)
@@ -1779,6 +1932,86 @@ mod tests {
assert!(at >= marked_at, "the marker was not refreshed"); assert!(at >= marked_at, "the marker was not refreshed");
} }
/// The marker a per-face pass leaves names the detector that drew the
/// boxes, whatever pipeline the pass itself ran as. A marker under the
/// pass's id over faces spelled another way is one the export finds no
/// faces under — and it sent every other device "nothing here".
#[test]
fn an_update_keeps_the_marker_under_the_detector_that_found_the_faces() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(
&c,
img,
"w600k_mbf",
1024,
&[DetectedFace {
quality: None,
..face(1)
}],
)
.unwrap();
// The state V14 leaves: the faces, and no marker at all.
c.execute("DELETE FROM face_index", []).unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let markers: Vec<(String, i64)> = c
.prepare("SELECT model_id, faces_found FROM face_index")
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(markers, vec![("w600k_mbf".to_string(), 1)]);
// A marker already there under the pass's own id is replaced, not
// kept beside the right one.
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (1, 'scrfd_10g+w600k_mbf', 0, 0, 6000)",
[],
)
.unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(vec![1, 2, 3]),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let n: i64 = c
.query_row("SELECT COUNT(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 1, "a second marker survived");
// With every face dropped there is no detector left to name, and
// the pass's own id records that it looked.
record_updates(&c, img, "scrfd_10g+w600k_mbf", 6000, &[], &ids).unwrap();
let marker: (String, i64) = c
.query_row("SELECT model_id, faces_found FROM face_index", [], |r| {
Ok((r.get(0)?, r.get(1)?))
})
.unwrap();
assert_eq!(marker, ("scrfd_10g+w600k_mbf".to_string(), 0));
}
/// Re-detection is coalesced per image, so it must replace rather than /// Re-detection is coalesced per image, so it must replace rather than
/// append — otherwise every re-index doubles the library's face count. /// append — otherwise every re-index doubles the library's face count.
#[test] #[test]
@@ -2253,7 +2486,7 @@ mod tests {
} }
/// The reference implementation's fitted MBF curve puts the P=0.5 boundary /// The reference implementation's fitted MBF curve puts the P=0.5 boundary
/// at cosine 0.267 (docs/faces.md §1). Our own first end-to-end run scored /// at cosine 0.267 (docs/dev/faces.md §1). Our own first end-to-end run scored
/// 0.596 between distinct photographs of one person and 0.05 between /// 0.596 between distinct photographs of one person and 0.05 between
/// different people, so those two must land either side. /// different people, so those two must land either side.
#[test] #[test]
+112 -1
View File
@@ -119,6 +119,8 @@ pub struct MergeReport {
pub keywords_fused: usize, pub keywords_fused: usize,
/// Keyword assignments taken from the remote. /// Keyword assignments taken from the remote.
pub keywords_assigned: usize, pub keywords_assigned: usize,
/// Images whose capture metadata was taken from the remote.
pub metadata_adopted: usize,
} }
impl MergeReport { impl MergeReport {
@@ -133,6 +135,7 @@ impl MergeReport {
|| self.keywords_deleted > 0 || self.keywords_deleted > 0
|| self.keywords_fused > 0 || self.keywords_fused > 0
|| self.keywords_assigned > 0 || self.keywords_assigned > 0
|| self.metadata_adopted > 0
} }
/// Whether the local catalog holds anything the remote did not, and so /// Whether the local catalog holds anything the remote did not, and so
@@ -193,10 +196,67 @@ pub fn merge_all(conn: &Connection) -> Result<MergeReport, CatalogError> {
merge_collections_within(&tx, &mut report)?; merge_collections_within(&tx, &mut report)?;
merge_keywords_within(&tx, &mut report)?; merge_keywords_within(&tx, &mut report)?;
merge_people_within(&tx, &mut report)?; merge_people_within(&tx, &mut report)?;
merge_metadata_within(&tx, &mut report)?;
tx.commit()?; tx.commit()?;
Ok(report) Ok(report)
} }
/// Adopt capture metadata from an attached catalog, on its own.
pub fn merge_metadata(conn: &Connection) -> Result<MergeReport, CatalogError> {
let tx = conn.unchecked_transaction()?;
let mut report = MergeReport::default();
merge_metadata_within(&tx, &mut report)?;
tx.commit()?;
Ok(report)
}
/// Capture metadata a peer's sweep already read, for images this device has
/// not dated yet.
///
/// The `images` table is local state and the merge leaves it alone — except
/// for these columns, which are not: a capture time, an offset, a camera, a
/// lens and an ISO are facts about the file's bytes, identical on every
/// device, and read by fetching a header per image across the whole library
/// (`dr_ui::library::spawn_sweep`). A fresh device inherits its peers'
/// thumbnails and faces from the shards and then spent hours re-reading
/// every header for the timeline; the snapshot it had just merged held
/// every one of those dates.
///
/// Matched by `oc:fileid`, as collection membership is. Only rows still at
/// `metadata_state < 2` take anything, and only from a remote row at 2: a
/// date this device read for itself is never overwritten, and a peer that
/// has not read one has nothing to give. The sweep's own query
/// (`metadata_state < 2`) then finds nothing left to do for them.
const METADATA_BY_FILE_ID: &str = "
UPDATE main.images
SET captured_at = r.captured_at,
captured_offset = coalesce(main.images.captured_offset, r.captured_offset),
camera = coalesce(main.images.camera, r.camera),
lens = coalesce(main.images.lens, r.lens),
iso = coalesce(main.images.iso, r.iso),
metadata_state = 2
FROM (SELECT lr.image_id, ri.captured_at, ri.captured_offset,
ri.camera, ri.lens, ri.iso
FROM remote_cat.images ri
JOIN remote_cat.remote rr ON rr.image_id = ri.id
JOIN main.remote lr ON lr.file_id = rr.file_id
WHERE ri.metadata_state >= 2 AND ri.captured_at IS NOT NULL) AS r
WHERE main.images.id = r.image_id
AND main.images.metadata_state < 2";
fn merge_metadata_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
// A snapshot from before these columns, or from a library with no server
// behind it, has nothing to join on.
if !remote_has(tx, "remote")?
|| !remote_has_column(tx, "images", "metadata_state")?
|| !remote_has_column(tx, "images", "captured_offset")?
{
return Ok(());
}
report.metadata_adopted = tx.execute(METADATA_BY_FILE_ID, [])?;
Ok(())
}
/// Merge people and identity judgements from an attached catalog. /// Merge people and identity judgements from an attached catalog.
/// ///
/// The people half of [`merge_all`], on its own, for the same reason the other /// The people half of [`merge_all`], on its own, for the same reason the other
@@ -729,7 +789,7 @@ fn attached_has_table(conn: &Connection, schema: &str, table: &str) -> Result<bo
/// # What travels, and what is recomputed /// # What travels, and what is recomputed
/// ///
/// The rule this module already follows for the rest of the catalog: user /// The rule this module already follows for the rest of the catalog: user
/// judgements travel, inference is rebuilt. Concretely (docs/faces.md, and the /// judgements travel, inference is rebuilt. Concretely (docs/dev/faces.md, and the
/// asymmetry `crate::faces` opens with): /// asymmetry `crate::faces` opens with):
/// ///
/// - **People** — uuid, name, and whether the user set them aside. Merged by /// - **People** — uuid, name, and whether the user set them aside. Merged by
@@ -1168,6 +1228,57 @@ mod tests {
// ---- integration over two real catalogs ------------------------------ // ---- integration over two real catalogs ------------------------------
/// A fresh device takes the capture dates a peer's sweep read, matched by
/// `oc:fileid`, and never overwrites a date it read for itself.
#[test]
fn capture_metadata_arrives_for_undated_images_only() {
let c = two_catalogs();
// Three photographs on both devices: 1 undated here and dated there;
// 2 dated on both, differently; 3 undated on both.
for id in 1..=3 {
add_image_without_hash(&c, "main", id);
add_image_without_hash(&c, "remote_cat", id + 10);
add_remote_id(&c, "main", id, 100 + id);
add_remote_id(&c, "remote_cat", id + 10, 100 + id);
}
c.execute(
"UPDATE remote_cat.images
SET captured_at = 1000, captured_offset = 60, camera = 'X', metadata_state = 2
WHERE id = 11",
[],
)
.unwrap();
c.execute(
"UPDATE remote_cat.images SET captured_at = 2000, metadata_state = 2 WHERE id = 12",
[],
)
.unwrap();
c.execute(
"UPDATE main.images SET captured_at = 2222, metadata_state = 2 WHERE id = 2",
[],
)
.unwrap();
let report = merge_metadata(&c).unwrap();
assert_eq!(report.metadata_adopted, 1);
let row = |id: i64| -> (Option<i64>, Option<i64>, Option<String>, i64) {
c.query_row(
"SELECT captured_at, captured_offset, camera, metadata_state
FROM main.images WHERE id = ?1",
[id],
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
)
.unwrap()
};
assert_eq!(row(1), (Some(1000), Some(60), Some("X".into()), 2));
assert_eq!(row(2), (Some(2222), None, None, 2));
assert_eq!(row(3), (None, None, None, 0));
// Idempotent: a second pass finds nothing left to take.
assert_eq!(merge_metadata(&c).unwrap().metadata_adopted, 0);
}
fn two_catalogs() -> Connection { fn two_catalogs() -> Connection {
attached_remote(schema::for_attached("remote_cat")) attached_remote(schema::for_attached("remote_cat"))
} }
+2 -2
View File
@@ -16,7 +16,7 @@
//! # The one thing a rebuild does not recover //! # The one thing a rebuild does not recover
//! //!
//! **Collections.** A manual collection is a set of images the user assembled //! **Collections.** A manual collection is a set of images the user assembled
//! by hand and nothing in the filesystem records it (`docs/catalog.md` §8.1) — //! by hand and nothing in the filesystem records it (`docs/dev/catalog.md` §8.1) —
//! which is the whole reason the catalog file itself syncs. So the two offers //! which is the whole reason the catalog file itself syncs. So the two offers
//! are not interchangeable, and the interface must not present them as if they //! are not interchangeable, and the interface must not present them as if they
//! were: a restore keeps the user's collections, a rebuild does not. //! were: a restore keeps the user's collections, a rebuild does not.
@@ -570,7 +570,7 @@ mod tests {
// The first NFR-R6 branch, asserted on the thing that distinguishes it // The first NFR-R6 branch, asserted on the thing that distinguishes it
// from the second: a collection exists nowhere but the catalog, so it // from the second: a collection exists nowhere but the catalog, so it
// is the evidence that the *contents* came back and not merely a // is the evidence that the *contents* came back and not merely a
// readable file (docs/catalog.md §8.1). // readable file (docs/dev/catalog.md §8.1).
let dir = tempdir("restore"); let dir = tempdir("restore");
let path = dir.join("catalog.sqlite"); let path = dir.join("catalog.sqlite");
fixture(&path, 500); fixture(&path, 500);
+221 -3
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError; use crate::error::CatalogError;
/// Schema version this build writes and understands. /// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 18; pub const SCHEMA_VERSION: i64 = 20;
/// Apply migrations up to [`SCHEMA_VERSION`]. /// Apply migrations up to [`SCHEMA_VERSION`].
/// ///
@@ -181,9 +181,105 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?; tx.commit()?;
} }
if from < 19 {
let tx = conn.unchecked_transaction()?;
tx.execute_batch(V19)?;
tx.pragma_update(None, "user_version", 19)?;
tx.commit()?;
}
if from < 20 {
let tx = conn.unchecked_transaction()?;
v20_markers_name_the_detector_that_found_the_faces(&tx)?;
tx.pragma_update(None, "user_version", 20)?;
tx.commit()?;
}
Ok(from) Ok(from)
} }
// V20 -- TRACES: FR-CAT-7
//
// Run markers that named the wrong detector, put right.
//
// `faces::record_updates` -- the write behind the quality, eye and crop
// passes -- re-marked an image under the pipeline the pass ran as, while
// the faces it had updated kept the id of the detector that found them.
// A marker of `scrfd_10g+w600k_mbf` over faces spelled `w600k_mbf` reads,
// to every consumer, as the thorough detector having examined the image:
// the upgrade repair skips it, and `face_shard::export_to_shards` selects
// its faces by the marker's id, finds none, and tells every other device
// that the thorough detector found nothing there. The desktop's shard index
// held 54 such entries over photographs with named faces, and the tablet's
// eye pass over faces it had adopted from the desktop had made 430 more.
//
// The write is fixed to keep the marker under the faces' own id. This puts
// the markers already written right, with a fresh time so the export sends
// each image again under an entry newer than the empty one -- which is what
// `held_model` orders by. Where the right marker is still there beside the
// wrong one (the old write inserted rather than replaced), the wrong one
// goes and the right one is refreshed for the same reason: its entry in
// the shards is older than the empty one, and a device that has neither
// would take the empty one. An image V14 left with faces and no marker at
// all is not touched: that state is the quality pass's cue, and the fixed
// write marks it correctly when the pass reaches it.
//
// Restated in Rust rather than SQL because the embedder half of a pipeline
// id is `faces::embedder_sql`, which this must agree with.
fn v20_markers_name_the_detector_that_found_the_faces(tx: &Connection) -> Result<(), CatalogError> {
let fi = crate::faces::embedder_sql("face_index.model_id");
let f = crate::faces::embedder_sql("f.model_id");
// A marker is wrong when the image holds faces of its embedder under
// another id. First the wrong ones that sit beside a right one -- the
// update below would collide with it -- then the rest are renamed.
let wrong = format!(
"EXISTS (SELECT 1 FROM faces f
WHERE f.image_id = face_index.image_id
AND {f} = {fi}
AND f.model_id != face_index.model_id)"
);
let found_by = format!(
"(SELECT MIN(f.model_id) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi})"
);
let now = crate::faces::now_secs();
tx.execute(
&format!(
"UPDATE face_index
SET indexed_at = ?1
WHERE model_id = {found_by}
AND EXISTS (SELECT 1 FROM face_index w
WHERE w.image_id = face_index.image_id
AND w.model_id != face_index.model_id
AND {} = {fi})",
crate::faces::embedder_sql("w.model_id")
),
[now],
)?;
tx.execute(
&format!(
"DELETE FROM face_index
WHERE {wrong}
AND EXISTS (SELECT 1 FROM face_index o
WHERE o.image_id = face_index.image_id
AND o.model_id = {found_by})"
),
[],
)?;
tx.execute(
&format!(
"UPDATE face_index
SET model_id = {found_by},
faces_found = (SELECT COUNT(*) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi}),
indexed_at = ?1
WHERE {wrong}"
),
[now],
)?;
Ok(())
}
/// The seven columns V16 adds to `faces`, in the order the readers name them. /// The seven columns V16 adds to `faces`, in the order the readers name them.
/// ///
/// Named once because three places have to agree on them: this migration, /// Named once because three places have to agree on them: this migration,
@@ -766,6 +862,44 @@ CREATE TABLE IF NOT EXISTS xmp_conflicts (
); );
"#; "#;
// V19 -- TRACES: NFR-P9
//
// The indexes the repair counts are served from, and V17's lesson applied
// to the rest of the face columns.
//
// "How many images still owe a quality reading" was answered per image: a
// correlated EXISTS over `faces` that had to open each face's row to look
// at one nullable column -- the row being eight kilobytes of embedding and
// crop. Six such counts run every time the Identity screen opens and every
// time a sweep ends, 160 ms of them on the reference library. Three
// partial indexes hold only the faces still owing each pass, keyed by the
// image and carrying the model id the predicate also reads, so the count
// walks a few thousand index entries and touches no row at all -- and each
// index shrinks to nothing as its pass completes. The planner takes them
// when the count is driven from `faces` (`repairs::count`) and ignores
// them inside the per-image EXISTS, which is why that function has two
// spellings of the same predicate.
//
// `faces_image_model` replaces `faces_image`: the same key with the model
// id beside it, so "does this image hold this embedder's faces" -- asked in
// the audit, the proxy repair and the outstanding-detection count -- is an
// index-only probe where it used to read the row for the model id. Every
// lookup that used `faces_image` is served by its prefix.
//
// Not applied to attached catalogs, like V7 and V17: an index is a local
// concern, and a merge never runs these queries across an attachment.
const V19: &str = r#"
CREATE INDEX IF NOT EXISTS faces_image_model ON faces(image_id, model_id);
DROP INDEX IF EXISTS faces_image;
CREATE INDEX IF NOT EXISTS faces_owed_quality ON faces(image_id, model_id)
WHERE quality IS NULL;
CREATE INDEX IF NOT EXISTS faces_owed_crop ON faces(image_id, model_id)
WHERE crop IS NULL;
CREATE INDEX IF NOT EXISTS faces_owed_eyes ON faces(image_id, model_id)
WHERE eye_right IS NULL OR landmarks_dense IS NULL;
"#;
// V18 -- TRACES: FR-CULL-8a | FR-CULL-12 // V18 -- TRACES: FR-CULL-8a | FR-CULL-12
// //
// The 106 dense landmarks the eye pass reads its eye boxes from, kept beside // The 106 dense landmarks the eye pass reads its eye boxes from, kept beside
@@ -875,7 +1009,7 @@ CREATE INDEX face_index_model ON face_index(model_id);
const V8: &str = r#" const V8: &str = r#"
-- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5 -- TRACES: FR-CULL-8 | FR-CULL-9 | FR-CULL-10 | FR-CULL-11 | FR-CULL-12 | NFR-SEC-5
-- People and faces (docs/faces.md, docs/catalog.md §10). -- People and faces (docs/dev/faces.md, docs/dev/catalog.md §10).
-- --
-- Everything here is **derived data** except one column. Faces, landmarks, -- Everything here is **derived data** except one column. Faces, landmarks,
-- embeddings, cluster assignments and suggestions are all reproducible by -- embeddings, cluster assignments and suggestions are all reproducible by
@@ -912,7 +1046,7 @@ CREATE TABLE faces (
landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise landmarks BLOB NOT NULL, -- 5 x (x, y) f32, normalised likewise
detector_confidence REAL NOT NULL, detector_confidence REAL NOT NULL,
embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since embedding BLOB NOT NULL, -- 512 x f16; unit length until V14, raw since
-- Source pixels across the aligned 112x112 crop (docs/faces.md §7). -- Source pixels across the aligned 112x112 crop (docs/dev/faces.md §7).
-- --
-- Not cosmetic: it is the honest quality signal for the UI, a feature in -- Not cosmetic: it is the honest quality signal for the UI, a feature in
-- the §8 calibration -- FR-CULL-9 names face size as an axis along which an -- the §8 calibration -- FR-CULL-9 names face size as an axis along which an
@@ -1809,6 +1943,90 @@ mod tests {
assert_eq!(faces, 2); assert_eq!(faces, 2);
} }
#[test]
fn v20_renames_markers_to_the_detector_that_found_the_faces() {
let c = mem();
c.pragma_update(None, "user_version", 0).unwrap();
migrate(&c).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'test')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1,1,'a',0),(2,1,'b',0),(3,1,'c',0),(4,1,'d',0),(5,1,'e',0)",
[],
)
.unwrap();
// 1: the desktop's case -- old faces, re-marked as thorough.
// 2: the tablet's case -- adopted thorough faces, re-marked int8,
// and the right marker still beside it (refreshed, so it is
// exported again over the empty entry).
// 3: right already. 4: examined and empty. 5: V14's state, faces
// and no marker.
for (image, model) in [
(1, "scrfd_10g+w600k_mbf"),
(2, "scrfd_10g_i8+w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(4, "scrfd_10g+w600k_mbf"),
] {
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, 100, 0, 6000)",
rusqlite::params![image, model],
)
.unwrap();
}
for (image, model) in [
(1, "w600k_mbf"),
(1, "w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(5, "w600k_mbf"),
] {
c.execute(
"INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, embedding,
crop_px, model_id, detected_at)
VALUES (?1, 0.1, 0.1, 0.2, 0.2, X'00', 0.9, X'00', 180.0, ?2, 0)",
rusqlite::params![image, model],
)
.unwrap();
}
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let markers: Vec<(i64, String, i64, bool)> = c
.prepare(
"SELECT image_id, model_id, faces_found, indexed_at > 100
FROM face_index ORDER BY image_id, model_id",
)
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(
markers,
vec![
(1, "w600k_mbf".to_string(), 2, true),
(2, "scrfd_10g+w600k_mbf".to_string(), 0, true),
(3, "scrfd_10g+w600k_mbf".to_string(), 0, false),
(4, "scrfd_10g+w600k_mbf".to_string(), 0, false),
]
);
// Re-enterable: nothing left to rename.
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let n: i64 = c
.query_row("SELECT count(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 4);
}
#[test] #[test]
fn job_uniqueness_coalesces_rather_than_duplicating() { fn job_uniqueness_coalesces_rather_than_duplicating() {
let c = mem(); let c = mem();
+20
View File
@@ -304,6 +304,26 @@ pub fn metadata(bytes: &[u8]) -> Result<Metadata, DecodeError> {
error::guarded("metadata", || metadata_unguarded(bytes)) error::guarded("metadata", || metadata_unguarded(bytes))
} }
/// TRACES: FR-CAT-5
/// Where a TIFF-shaped file keeps its first IFD, when the head handed to
/// [`metadata`] does not reach it — the linear DNG a merge writes puts its
/// IFDs after the pixels, and rawler, given the head alone, finds no
/// decoder in it. The caller fetches from this offset to the end and
/// reads the two ranges with [`metadata_split`].
pub fn trailing_ifd(head: &[u8]) -> Option<u64> {
locate::trailing_ifd(head)
}
/// TRACES: FR-CAT-5
/// [`metadata`] for a file read in two ranges: `head` from offset 0 and
/// `tail` from `tail_at`. The EXIF sub-IFD such a file wrote before its
/// pixels is in the head; the first IFD and its values are in the tail.
pub fn metadata_split(head: &[u8], tail: &[u8], tail_at: u64) -> Result<Metadata, DecodeError> {
error::guarded("metadata", || {
locate::tiff_metadata_split(head, tail, tail_at)
})
}
fn metadata_unguarded(bytes: &[u8]) -> Result<Metadata, DecodeError> { fn metadata_unguarded(bytes: &[u8]) -> Result<Metadata, DecodeError> {
use rawler::rawsource::RawSource; use rawler::rawsource::RawSource;
+82 -13
View File
@@ -183,22 +183,65 @@ struct Entry {
value: u32, value: u32,
} }
/// The bytes a [`TiffReader`] reads: the head of a file, and optionally a
/// second range from further in, at a known offset.
///
/// A camera writes its IFDs at the front, so the first 256 KB of a file
/// is the whole structure. A file written strip by strip — the linear
/// DNG a merge produces — has its first IFD at the *end*, after the
/// pixels, and a reader that only has the head sees a pointer into
/// nothing. Rather than fetch 800 MB to read a date, the caller fetches
/// the head, asks [`crate::trailing_ifd`] where the IFD is, fetches that
/// tail, and reads through both. Offsets are the file's own throughout;
/// a read that falls in neither range is simply absent.
#[derive(Clone, Copy)]
struct Src<'a> {
head: &'a [u8],
tail: &'a [u8],
/// Where `tail` starts in the file.
tail_at: usize,
}
impl<'a> Src<'a> {
fn whole(data: &'a [u8]) -> Self {
Src {
head: data,
tail: &[],
tail_at: 0,
}
}
fn get(&self, start: usize, len: usize) -> Option<&'a [u8]> {
let end = start.checked_add(len)?;
if let Some(b) = self.head.get(start..end) {
return Some(b);
}
let s = start.checked_sub(self.tail_at)?;
self.tail.get(s..s.checked_add(len)?)
}
}
/// A minimal TIFF structure reader. /// A minimal TIFF structure reader.
/// ///
/// Deliberately not a general TIFF parser: it reads the IFD chain and entry /// Deliberately not a general TIFF parser: it reads the IFD chain and entry
/// values and nothing else, because that is all locating a preview needs. /// values and nothing else, because that is all locating a preview needs.
struct TiffReader<'a> { struct TiffReader<'a> {
data: &'a [u8], data: Src<'a>,
little_endian: bool, little_endian: bool,
first_ifd: u32, first_ifd: u32,
} }
impl<'a> TiffReader<'a> { impl<'a> TiffReader<'a> {
fn new(data: &'a [u8]) -> Option<Self> { fn new(data: &'a [u8]) -> Option<Self> {
if data.len() < 8 { Self::over(Src::whole(data))
}
fn over(data: Src<'a>) -> Option<Self> {
let head = data.head;
if head.len() < 8 {
return None; return None;
} }
let little_endian = match &data[0..2] { let little_endian = match &head[0..2] {
b"II" => true, b"II" => true,
b"MM" => false, b"MM" => false,
_ => return None, _ => return None,
@@ -362,9 +405,7 @@ impl<'a> TiffReader<'a> {
}; };
raw[..len.min(4)].to_vec() raw[..len.min(4)].to_vec()
} else { } else {
self.data self.data.get(e.value as usize, len)?.to_vec()
.get(e.value as usize..e.value as usize + len)?
.to_vec()
}; };
let s = String::from_utf8_lossy(&bytes); let s = String::from_utf8_lossy(&bytes);
@@ -396,8 +437,7 @@ impl<'a> TiffReader<'a> {
// same way to recover the original byte order. // same way to recover the original byte order.
return None; return None;
} }
let start = e.value as usize; self.data.get(e.value as usize, len)
self.data.get(start..start.checked_add(len)?)
} }
fn offsets(&self, e: &Entry) -> Vec<u32> { fn offsets(&self, e: &Entry) -> Vec<u32> {
@@ -420,8 +460,8 @@ impl<'a> TiffReader<'a> {
} }
} }
fn read_u16(data: &[u8], at: usize, le: bool) -> Option<u16> { fn read_u16(data: Src<'_>, at: usize, le: bool) -> Option<u16> {
let b = data.get(at..at + 2)?; let b = data.get(at, 2)?;
Some(if le { Some(if le {
u16::from_le_bytes([b[0], b[1]]) u16::from_le_bytes([b[0], b[1]])
} else { } else {
@@ -429,8 +469,8 @@ fn read_u16(data: &[u8], at: usize, le: bool) -> Option<u16> {
}) })
} }
fn read_u32(data: &[u8], at: usize, le: bool) -> Option<u32> { fn read_u32(data: Src<'_>, at: usize, le: bool) -> Option<u32> {
let b = data.get(at..at + 4)?; let b = data.get(at, 4)?;
Some(if le { Some(if le {
u32::from_le_bytes([b[0], b[1], b[2], b[3]]) u32::from_le_bytes([b[0], b[1], b[2], b[3]])
} else { } else {
@@ -460,7 +500,36 @@ pub fn jpeg_metadata(bytes: &[u8]) -> Result<crate::Metadata, crate::DecodeError
/// no `DateTimeOriginal` for some DNGs whose tag sits plainly at byte 826 — /// no `DateTimeOriginal` for some DNGs whose tag sits plainly at byte 826 —
/// and without this fallback those images are silently undated. /// and without this fallback those images are silently undated.
pub fn tiff_metadata(tiff_data: &[u8]) -> Result<crate::Metadata, crate::DecodeError> { pub fn tiff_metadata(tiff_data: &[u8]) -> Result<crate::Metadata, crate::DecodeError> {
let reader = TiffReader::new(tiff_data) tiff_metadata_over(Src::whole(tiff_data))
}
/// Where a TIFF-shaped file's first IFD is, when the head does not reach
/// it: the offset to fetch from, so [`tiff_metadata_split`] can read it.
/// `None` for a file that is not TIFF, or whose IFD the head already holds.
pub fn trailing_ifd(head: &[u8]) -> Option<u64> {
let r = TiffReader::new(head)?;
let at = r.first_ifd as u64;
(at >= head.len() as u64).then_some(at)
}
/// [`tiff_metadata`] over a head and a tail fetched separately: the head
/// from offset 0, the tail from `tail_at`. For the file whose IFDs follow
/// its pixels.
pub fn tiff_metadata_split(
head: &[u8],
tail: &[u8],
tail_at: u64,
) -> Result<crate::Metadata, crate::DecodeError> {
tiff_metadata_over(Src {
head,
tail,
tail_at: usize::try_from(tail_at)
.map_err(|_| crate::DecodeError::Metadata("tail offset out of range".into()))?,
})
}
fn tiff_metadata_over(src: Src<'_>) -> Result<crate::Metadata, crate::DecodeError> {
let reader = TiffReader::over(src)
.ok_or_else(|| crate::DecodeError::Metadata("malformed EXIF header".into()))?; .ok_or_else(|| crate::DecodeError::Metadata("malformed EXIF header".into()))?;
let mut md = crate::Metadata::default(); let mut md = crate::Metadata::default();
+28
View File
@@ -241,6 +241,8 @@ mod tests {
let source = SourceMetadata { let source = SourceMetadata {
make: Some("Canon".into()), make: Some("Canon".into()),
model: Some("Canon EOS 6D".into()), model: Some("Canon EOS 6D".into()),
captured_at: Some(1_754_398_664),
captured_offset: Some(120),
..Default::default() ..Default::default()
}; };
write_linear_dng( write_linear_dng(
@@ -301,6 +303,32 @@ mod tests {
assert_eq!((crop.p.x, crop.p.y, crop.d.w, crop.d.h), (2, 1, 15, 10)); assert_eq!((crop.p.x, crop.p.y, crop.d.w, crop.d.h), (2, 1, 15, 10));
} }
#[test]
fn the_catalog_reads_the_date_from_a_head_and_a_tail() {
// TRACES: FR-CAT-5
// The IFDs follow the pixels, so a scan that has the first bytes of
// the file has a pointer into nothing; rawler finds no decoder in
// that, and the composite would sit undated at the end of the grid.
// The scan's second range — from the first IFD to the end — with
// the head is enough to date it, and to name the camera.
let bytes = write(640, 400, 64);
let head = &bytes[..4096];
assert!(
dr_decode::metadata(head).is_err(),
"the head alone must not read"
);
let at = dr_decode::trailing_ifd(head).expect("the IFD is beyond the head");
assert!(at as usize > head.len());
let tail = &bytes[at as usize..];
assert!(tail.len() < 4096, "the tail is the IFD, not the pixels");
let md = dr_decode::metadata_split(head, tail, at).expect("read from two ranges");
assert_eq!(md.captured_at, Some(1_754_398_664));
assert_eq!(md.captured_offset, Some(120));
assert_eq!(md.model.as_deref(), Some("Canon EOS 6D"));
// A head that holds everything is not a trailing-IFD file.
assert_eq!(dr_decode::trailing_ifd(&bytes), None);
}
#[test] #[test]
fn a_strip_of_the_wrong_length_is_refused() { fn a_strip_of_the_wrong_length_is_refused() {
let mut bytes = std::io::Cursor::new(Vec::new()); let mut bytes = std::io::Cursor::new(Vec::new());
+2 -2
View File
@@ -11,7 +11,7 @@ log.workspace = true
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s # Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
# business** — tract, or an ONNX Runtime the app found on disk, on whichever # business** — tract, or an ONNX Runtime the app found on disk, on whichever
# provider the device has (docs/inference.md). This crate never names either. # provider the device has (docs/dev/inference.md). This crate never names either.
ort = { workspace = true, optional = true } ort = { workspace = true, optional = true }
dr-inference-engine = { workspace = true, optional = true } dr-inference-engine = { workspace = true, optional = true }
ndarray = { workspace = true, optional = true } ndarray = { workspace = true, optional = true }
@@ -37,7 +37,7 @@ required-features = ["inference"]
[features] [features]
# Nothing on by default, and in particular **no `embedded-model`**: the weights # Nothing on by default, and in particular **no `embedded-model`**: the weights
# are not a build input and never become one (docs/faces.md §2.2). A feature # are not a build input and never become one (docs/dev/faces.md §2.2). A feature
# flag that *could* embed them is a flag someone eventually sets in a packaging # flag that *could* embed them is a flag someone eventually sets in a packaging
# script, and the InsightFace grant does not survive that. # script, and the InsightFace grant does not survive that.
default = [] default = []
+1 -1
View File
@@ -1,4 +1,4 @@
//! Detect the faces in a JPEG and read each one's eyes (docs/faces.md §17). //! Detect the faces in a JPEG and read each one's eyes (docs/dev/faces.md §17).
//! //!
//! The thing worth looking at is whether the eye boxes land on eyes and //! The thing worth looking at is whether the eye boxes land on eyes and
//! whether soft ones are refused — so with `--dump DIR` the crops the //! whether soft ones are refused — so with `--dump DIR` the crops the
+1 -1
View File
@@ -7,7 +7,7 @@
//! DET.onnx EMB.onnx photo.jpg [photo.jpg ...] //! DET.onnx EMB.onnx photo.jpg [photo.jpg ...]
//! //!
//! The models must have had their input dims frozen first; see //! The models must have had their input dims frozen first; see
//! `tools/fix-face-model-shapes.sh` and docs/faces.md §12 M1. //! `tools/fix-face-model-shapes.sh` and docs/dev/faces.md §12 M1.
use std::time::Instant; use std::time::Instant;
+1 -1
View File
@@ -1,4 +1,4 @@
//! M1 (docs/faces.md §12) — will tract load these graphs at all? //! M1 (docs/dev/faces.md §12) — will tract load these graphs at all?
//! //!
//! The one measurement everything else in the face subsystem is conditional //! The one measurement everything else in the face subsystem is conditional
//! on. `det_500m.onnx` has a dynamic H/W input, which is exactly what tract //! on. `det_500m.onnx` has a dynamic H/W input, which is exactly what tract
+1 -1
View File
@@ -9,7 +9,7 @@
//! //!
//! # What it is for //! # What it is for
//! //!
//! docs/faces.md §9 has the desktop numbers and the question they leave open: //! docs/dev/faces.md §9 has the desktop numbers and the question they leave open:
//! a GPU GEMM is worth roughly 1.5× of a regroup on a twenty-core desktop, //! a GPU GEMM is worth roughly 1.5× of a regroup on a twenty-core desktop,
//! because the scan is under a third of the pass there. On a tablet the CPU is //! because the scan is under a third of the pass there. On a tablet the CPU is
//! several times slower and the GPU is not, so the same optimisation is worth //! several times slower and the GPU is not, so the same optimisation is worth
+5 -5
View File
@@ -1,4 +1,4 @@
//! Five-point face alignment (docs/faces.md §5). //! Five-point face alignment (docs/dev/faces.md §5).
//! //!
//! ArcFace embeddings are trained on faces warped to a canonical 112×112 //! ArcFace embeddings are trained on faces warped to a canonical 112×112
//! arrangement. Feeding the model a plain bounding-box crop *works* — it //! arrangement. Feeding the model a plain bounding-box crop *works* — it
@@ -208,7 +208,7 @@ impl Similarity {
/// ///
/// # Why least squares and not RANSAC /// # Why least squares and not RANSAC
/// ///
/// The reference C++ implementation (docs/faces.md §1.1) fits this with /// The reference C++ implementation (docs/dev/faces.md §1.1) fits this with
/// OpenCV's `estimateAffinePartial2D` under RANSAC. RANSAC over five points is /// OpenCV's `estimateAffinePartial2D` under RANSAC. RANSAC over five points is
/// a strange fit: the minimal sample for a similarity is two, so it can discard /// a strange fit: the minimal sample for a similarity is two, so it can discard
/// landmarks it judges outliers and solve from a subset — and on a profile face /// landmarks it judges outliers and solve from a subset — and on a profile face
@@ -427,7 +427,7 @@ fn sample_window(
// ── eyes ────────────────────────────────────────────────────────────────── // ── eyes ──────────────────────────────────────────────────────────────────
/// Width of an eye crop as the classifier reads it, in pixels. Fixed by the /// Width of an eye crop as the classifier reads it, in pixels. Fixed by the
/// OCEC input (`docs/faces.md` §17): 40 wide, 24 high. /// OCEC input (`docs/dev/faces.md` §17): 40 wide, 24 high.
pub const EYE_PATCH_WIDTH: usize = 40; pub const EYE_PATCH_WIDTH: usize = 40;
/// Height of an eye crop as the classifier reads it, in pixels. /// Height of an eye crop as the classifier reads it, in pixels.
pub const EYE_PATCH_HEIGHT: usize = 24; pub const EYE_PATCH_HEIGHT: usize = 24;
@@ -438,7 +438,7 @@ pub const EYE_PATCH_HEIGHT: usize = 24;
/// The classifier was trained on a whole-body detector's *eye* boxes — tight /// The classifier was trained on a whole-body detector's *eye* boxes — tight
/// round the palpebral fissure — and measured on 25 open-eyed faces from the /// round the palpebral fissure — and measured on 25 open-eyed faces from the
/// reference library, a tight box is what it wants: 22 of 25 read open at /// reference library, a tight box is what it wants: 22 of 25 read open at
/// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/faces.md §17.2). A tenth, so a /// 0 and 0.1, 18 at 0.4, 14 at 0.6 (docs/dev/faces.md §17.2). A tenth, so a
/// contour landing a pixel short of the lashes still holds them. /// contour landing a pixel short of the lashes still holds them.
pub const EYE_BOX_MARGIN: f32 = 0.1; pub const EYE_BOX_MARGIN: f32 = 0.1;
@@ -571,7 +571,7 @@ pub const SUNGLASSES_EDGE: usize = 48;
/// clear glasses, at 0.68. Erring towards "sunglasses" is the safe direction /// clear glasses, at 0.68. Erring towards "sunglasses" is the safe direction
/// for what this feeds: a face called sunglasses is left alone by the /// for what this feeds: a face called sunglasses is left alone by the
/// eyes-open filter, where a pair of sunglasses missed hands the eye /// eyes-open filter, where a pair of sunglasses missed hands the eye
/// classifier a lens to guess at (docs/faces.md §17). /// classifier a lens to guess at (docs/dev/faces.md §17).
pub const SUNGLASSES_WINDOWS: [(f32, f32, f32, f32); 2] = pub const SUNGLASSES_WINDOWS: [(f32, f32, f32, f32); 2] =
[(0.0, 0.0, 112.0, 112.0), (-5.0, -14.0, 122.0, 122.0)]; [(0.0, 0.0, 112.0, 112.0), (-5.0, -14.0, 122.0, 122.0)];
+3 -3
View File
@@ -1,4 +1,4 @@
//! Cosine to probability (docs/faces.md §8, FR-CULL-9). //! Cosine to probability (docs/dev/faces.md §8, FR-CULL-9).
//! //!
//! FR-CULL-9 is a hard requirement rather than an implementation detail: no //! FR-CULL-9 is a hard requirement rather than an implementation detail: no
//! code path may threshold a bare cosine, every threshold in the subsystem is //! code path may threshold a bare cosine, every threshold in the subsystem is
@@ -28,7 +28,7 @@
//! calibration to the belief it was supposed to test — and that is the whole //! calibration to the belief it was supposed to test — and that is the whole
//! of the alternative. //! of the alternative.
//! //!
//! docs/faces.md §8.1 names one more that would cost no labelling at all: two //! docs/dev/faces.md §8.1 names one more that would cost no labelling at all: two
//! faces in adjacent frames of one burst are near-certainly the same person, //! faces in adjacent frames of one burst are near-certainly the same person,
//! and FR-CULL-5's grouping is sitting there. Nothing draws on it. This crate //! and FR-CULL-5's grouping is sitting there. Nothing draws on it. This crate
//! cannot see a catalog, let alone the bursts in one — it is handed cosines by //! cannot see a catalog, let alone the bursts in one — it is handed cosines by
@@ -83,7 +83,7 @@ pub struct Calibration {
} }
impl Default for Calibration { impl Default for Calibration {
/// The reference implementation's fitted MBF curve (docs/faces.md §1): /// The reference implementation's fitted MBF curve (docs/dev/faces.md §1):
/// steepness 16.2, P=0.5 at cosine 0.267. /// steepness 16.2, P=0.5 at cosine 0.267.
/// ///
/// **`valid` is false**, and that is the point. It is a documented /// **`valid` is false**, and that is the point. It is a documented
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-CULL-8a //! TRACES: FR-CULL-8a
//! The two small classifiers behind a face's eye state (docs/faces.md §17). //! The two small classifiers behind a face's eye state (docs/dev/faces.md §17).
//! //!
//! **OCEC** — *open closed eyes classification*, Hyodo 2025 — reads one //! **OCEC** — *open closed eyes classification*, Hyodo 2025 — reads one
//! 40×24 eye and answers P(open). **SGC** — *sunglasses classification*, //! 40×24 eye and answers P(open). **SGC** — *sunglasses classification*,
+1 -1
View File
@@ -1,4 +1,4 @@
//! Grouping faces into people (docs/faces.md §9, FR-CULL-10). //! Grouping faces into people (docs/dev/faces.md §9, FR-CULL-10).
//! //!
//! Model-free: this is arithmetic over embeddings, and it is where the //! Model-free: this is arithmetic over embeddings, and it is where the
//! subsystem's accuracy actually lives, so it is testable with no weights on //! subsystem's accuracy actually lives, so it is testable with no weights on
+4 -4
View File
@@ -1,4 +1,4 @@
//! SCRFD face detection (docs/faces.md §4). //! SCRFD face detection (docs/dev/faces.md §4).
//! //!
//! One forward pass produces a box, a confidence and **five landmarks** per //! One forward pass produces a box, a confidence and **five landmarks** per
//! face — the landmarks being the reason for this detector rather than a //! face — the landmarks being the reason for this detector rather than a
@@ -138,7 +138,7 @@ impl Detection {
pub struct Detector { pub struct Detector {
session: Model, session: Model,
/// f32 or int8 — the int8 form finds a different set of faces and is a /// f32 or int8 — the int8 form finds a different set of faces and is a
/// different detector in `model_id` (docs/inference.md §7). /// different detector in `model_id` (docs/dev/inference.md §7).
form: Form, form: Form,
/// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}. /// Feature-map count: 3 for strides {8,16,32}, 4 for {8,16,32,64}.
/// ///
@@ -346,7 +346,7 @@ fn iou(a: &(f32, f32, f32, f32), b: &(f32, f32, f32, f32)) -> f32 {
/// How the image is fitted into the graph's fixed square input. /// How the image is fitted into the graph's fixed square input.
/// ///
/// The forward and inverse mappings live in one struct on purpose: /// The forward and inverse mappings live in one struct on purpose:
/// docs/faces.md §4.1 notes that what matters is not *where* the padding goes /// docs/dev/faces.md §4.1 notes that what matters is not *where* the padding goes
/// but that the two agree. A mismatch offsets every box and landmark by the /// but that the two agree. A mismatch offsets every box and landmark by the
/// padding, producing detections that look plausible and embeddings that /// padding, producing detections that look plausible and embeddings that
/// quietly cluster badly three stages later. /// quietly cluster badly three stages later.
@@ -372,7 +372,7 @@ impl Letterbox {
/// ///
/// `(x·255 − 127.5) / 128` — note `/128`, not `/127.5`. The reference /// `(x·255 − 127.5) / 128` — note `/128`, not `/127.5`. The reference
/// implementation this is ported from uses `/128` for both models, and /// implementation this is ported from uses `/128` for both models, and
/// every measured number in docs/faces.md §1 came from it. /// every measured number in docs/dev/faces.md §1 came from it.
/// ///
/// Padding is grey, matching the reference's `114`: the value the network /// Padding is grey, matching the reference's `114`: the value the network
/// reads least as an edge, where black would draw a hard border across the /// reads least as an edge, where black would draw a hard border across the
+2 -2
View File
@@ -1,4 +1,4 @@
//! ArcFace / MobileFaceNet inference (docs/faces.md §6). //! ArcFace / MobileFaceNet inference (docs/dev/faces.md §6).
//! //!
//! Takes an aligned crop and returns 512 L2-normalised floats. The alignment is //! Takes an aligned crop and returns 512 L2-normalised floats. The alignment is
//! not optional and cannot be skipped by accident: [`Embedder::embed`] takes an //! not optional and cannot be skipped by accident: [`Embedder::embed`] takes an
@@ -66,7 +66,7 @@ impl Embedder {
pub fn from_bytes(bytes: &[u8], model: ModelId) -> Result<Self, FaceError> { pub fn from_bytes(bytes: &[u8], model: ModelId) -> Result<Self, FaceError> {
// Always the f32 form: an embedding must compare across devices // Always the f32 form: an embedding must compare across devices
// (docs/inference.md §7), and the engine pins this role to it. // (docs/dev/inference.md §7), and the engine pins this role to it.
let loaded = dr_inference_engine::open(Role::Embedder, Form::F32, bytes)?; let loaded = dr_inference_engine::open(Role::Embedder, Form::F32, bytes)?;
let acquired = loaded.acquire()?; let acquired = loaded.acquire()?;
let session = acquired.lock(); let session = acquired.lock();
+2 -2
View File
@@ -1,4 +1,4 @@
//! What an embedder produces, and how it is stored (docs/faces.md §6). //! What an embedder produces, and how it is stored (docs/dev/faces.md §6).
//! //!
//! Deliberately **model-free**: the vector, its identity, its comparison and //! Deliberately **model-free**: the vector, its identity, its comparison and
//! its storage encoding are arithmetic, and `calibrate` and `cluster` are built //! its storage encoding are arithmetic, and `calibrate` and `cluster` are built
@@ -273,7 +273,7 @@ mod tests {
); );
} }
/// The claim docs/faces.md §6 makes about the storage format: the f16 /// The claim docs/dev/faces.md §6 makes about the storage format: the f16
/// round-trip costs ~1e-3 of cosine, three orders below the separation /// round-trip costs ~1e-3 of cosine, three orders below the separation
/// between a match and a non-match. /// between a match and a non-match.
#[test] #[test]
+3 -3
View File
@@ -67,14 +67,14 @@ pub const SUNGLASSES_THRESHOLD: f32 = 0.5;
/// The classifier was trained on eyes down to about a dozen pixels wide /// The classifier was trained on eyes down to about a dozen pixels wide
/// (its reference footage averaged 15–21); below that the 40-pixel patch is /// (its reference footage averaged 15–21); below that the 40-pixel patch is
/// an interpolation of nothing, and the answer is noise that reads as /// an interpolation of nothing, and the answer is noise that reads as
/// "closed". docs/faces.md §17.3 has the measurement behind the number. /// "closed". docs/dev/faces.md §17.3 has the measurement behind the number.
pub const MIN_EYE_PX: f32 = 12.0; pub const MIN_EYE_PX: f32 = 12.0;
/// Least [`Eye::sharpness`] for the eye to be read. /// Least [`Eye::sharpness`] for the eye to be read.
/// ///
/// The same measure as the face's `min_sharpness`, over the eye patch, and /// The same measure as the face's `min_sharpness`, over the eye patch, and
/// chosen the same way: the value under which the open-eyed faces of the /// chosen the same way: the value under which the open-eyed faces of the
/// reference sample were being called closed. docs/faces.md §17.3. /// reference sample were being called closed. docs/dev/faces.md §17.3.
pub const MIN_EYE_SHARPNESS: f32 = 0.02; pub const MIN_EYE_SHARPNESS: f32 = 0.02;
/// An eye narrower than this fraction of its partner is the far eye of a /// An eye narrower than this fraction of its partner is the far eye of a
@@ -82,7 +82,7 @@ pub const MIN_EYE_SHARPNESS: f32 = 0.02;
/// ///
/// A landmark model's contour for a hidden eye collapses towards the nose. /// A landmark model's contour for a hidden eye collapses towards the nose.
/// Measured on twenty native renders of the reference library /// Measured on twenty native renders of the reference library
/// (docs/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near /// (docs/dev/faces.md §17.4): profiles put the far eye at 0.02–0.43 of the near
/// one, two three-quarter faces whose far eye read closed sat at 0.54, and /// one, two three-quarter faces whose far eye read closed sat at 0.54, and
/// every face looking at the camera — winks included, since a shut eye's /// every face looking at the camera — winks included, since a shut eye's
/// box keeps its width — sat at 0.78 or more. 0.6 splits the gap. /// box keeps its width — sat at 0.78 or more. 0.6 splits the gap.
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-CULL-8a //! TRACES: FR-CULL-8a
//! Dense facial landmarks — InsightFace's `2d106det` (docs/faces.md §17.2). //! Dense facial landmarks — InsightFace's `2d106det` (docs/dev/faces.md §17.2).
//! //!
//! SCRFD's five points place a face; they do not place an eye. Its eye //! SCRFD's five points place a face; they do not place an eye. Its eye
//! point is loose enough that a window centred on it left the eye in a //! point is loose enough that a window centred on it left the eye in a
+5 -4
View File
@@ -1,4 +1,4 @@
//! Faces and identity (S14, docs/faces.md). //! Faces and identity (S14, docs/dev/faces.md).
//! //!
//! Two models, run over the native render, producing per face a box, five //! Two models, run over the native render, producing per face a box, five
//! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the //! landmarks, a confidence and a 512-d embedding (FR-CULL-8) — and then the
@@ -21,9 +21,9 @@
//! for a packaging script to switch on. The application obtains a model at //! for a packaging script to switch on. The application obtains a model at
//! runtime; this crate takes bytes and never fetches anything. //! runtime; this crate takes bytes and never fetches anything.
//! //!
//! docs/faces.md §2 is the full reading, including what would have to change //! docs/dev/faces.md §2 is the full reading, including what would have to change
//! for that to stop being true. The eye-state models are the exception: MIT, //! for that to stop being true. The eye-state models are the exception: MIT,
//! weights and all, and shipped in `models/face/` (docs/faces.md §17). //! weights and all, and shipped in `models/face/` (docs/dev/faces.md §17).
//! //!
//! # Why the runtime is split behind a feature //! # Why the runtime is split behind a feature
//! //!
@@ -50,11 +50,12 @@ pub mod eyes;
pub mod landmarks; pub mod landmarks;
pub mod naming; pub mod naming;
pub mod neighbours; pub mod neighbours;
pub mod references;
/// Smallest long edge a face crop may be sampled from. /// Smallest long edge a face crop may be sampled from.
/// ///
/// **A floor on the crop source, not on the detector input.** The distinction /// **A floor on the crop source, not on the detector input.** The distinction
/// is the whole of FR-CULL-8 and `docs/faces.md` §7: detection letterboxes /// is the whole of FR-CULL-8 and `docs/dev/faces.md` §7: detection letterboxes
/// every buffer into 640×640, so its input resolution decides nothing, while /// every buffer into 640×640, so its input resolution decides nothing, while
/// [`warp`] samples the 112×112 the embedder sees and so converts source /// [`warp`] samples the 112×112 the embedder sees and so converts source
/// resolution directly into embedding quality. FR-CULL-8 requires that crop to /// resolution directly into embedding quality. FR-CULL-8 requires that crop to
+1 -1
View File
@@ -115,7 +115,7 @@ pub struct Faces<'a> {
/// Source pixels across the aligned crop, for the calibration's size term. /// Source pixels across the aligned crop, for the calibration's size term.
pub crop_px: &'a [f32], pub crop_px: &'a [f32],
/// Which photograph each face came from. Two faces in one frame are not /// Which photograph each face came from. Two faces in one frame are not
/// the same person, so those pairs are never returned (docs/faces.md §9). /// the same person, so those pairs are never returned (docs/dev/faces.md §9).
pub images: &'a [u64], pub images: &'a [u64],
/// Which faces may be compared *against* — the gallery /// Which faces may be compared *against* — the gallery
/// ([`crate::embedding::MIN_GALLERY_QUALITY`]). /// ([`crate::embedding::MIN_GALLERY_QUALITY`]).
+232
View File
@@ -0,0 +1,232 @@
//! TRACES: FR-CULL-10 | NFR-P9
//! Which of a person's faces stand for them in a grouping pass.
//!
//! # Why not all of them
//!
//! Every face the user has ruled on enters [`crate::cluster`] as an anchor,
//! and the pass compares every face against every other
//! ([`crate::neighbours`] is exhaustive by design). So a person with 750
//! confirmed faces costs 750 comparisons against each of the library's other
//! faces, and the cost of naming a library well grows with how well it is
//! named: a fully confirmed library of 25,000 faces spends almost the whole
//! scan re-comparing faces whose identity is already settled against each
//! other.
//!
//! Most of those comparisons say nothing new. A person's confirmed faces are
//! heavily redundant — thirty frames from one afternoon are one point of
//! view, not thirty — and a new face that matches one of them matches the
//! others too. What a new face needs to be measured against is the person's
//! *range*: the angles, ages and lights they have been photographed in, each
//! represented once.
//!
//! # The choice: the most diverse of the good ones
//!
//! Two rules, in order.
//!
//! **Good enough to vouch.** Only faces whose raw embedding was at least
//! [`MIN_REFERENCE_QUALITY`] long are eligible — a stricter floor than the
//! gallery's ([`crate::embedding::MIN_GALLERY_QUALITY`]), because a reference
//! is asked to speak *for* a person rather than merely be admitted to the
//! comparison. A face whose length was never recorded is admitted, as it is
//! everywhere else: a rule that cannot be checked admits rather than excludes.
//!
//! **As far apart as possible.** From the eligible pool, up to
//! [`MAX_REFERENCES`] faces are chosen to maximise the volume they span —
//! the determinant of their Gram matrix — greedily: start from the longest
//! vector, and at each step add the face with the largest component
//! orthogonal to everything chosen so far. That is Gram–Schmidt with a
//! pivot, and the product of the squared residuals it picks *is* the
//! determinant, so the greedy step is the exact greedy on the objective.
//! The effect is that a near-duplicate of a chosen face has almost no
//! residual and is passed over, while the one profile shot among two
//! hundred frontal frames is taken early.
//!
//! What is not chosen still belongs to the person. Those faces keep their
//! confirmations and are not touched by the pass; they are simply not
//! compared, which is the whole saving.
/// The most faces that stand for one person.
///
/// A hundred is far more points of view than a person has. What it bounds
/// is the cost: with every person at the cap, a scan against the named part
/// of a library is `people × 100` comparisons per face rather than
/// `confirmations`, and the two part company as soon as a library is used.
pub const MAX_REFERENCES: usize = 100;
/// The shortest raw embedding that may stand for a person.
///
/// One above the gallery floor: a reference vouches for someone, and the
/// margin keeps the faces that only just cleared the gallery — the ones
/// nearest the middle of the sphere — out of the set that speaks for a
/// person.
pub const MIN_REFERENCE_QUALITY: f32 = 15.0;
/// Whether a face of this quality may stand for a person.
///
/// `None` is "never measured" and is admitted, as in
/// [`crate::embedding::in_gallery`].
pub fn eligible(quality: Option<f32>) -> bool {
quality.is_none_or(|q| q >= MIN_REFERENCE_QUALITY)
}
/// Choose which of one person's faces stand for them.
///
/// `embeddings` and `quality` are one entry per face, the embeddings unit
/// length and all of one dimension. Returns the indices chosen, in the order
/// chosen — the first is the longest eligible vector, and each after it is
/// the one furthest from the span of those before. Every eligible face is
/// returned when there are `max` or fewer of them, so a person under the
/// cap loses nothing.
///
/// Deterministic: equal residuals break on the longer vector, then the lower
/// index, so two devices holding the same faces choose the same references
/// and group the same way (`cluster::clustering_is_deterministic`).
pub fn select(embeddings: &[&[f32]], quality: &[Option<f32>], max: usize) -> Vec<usize> {
debug_assert_eq!(embeddings.len(), quality.len());
let mut pool: Vec<usize> = (0..embeddings.len())
.filter(|&i| eligible(quality[i]))
.collect();
if pool.len() <= max {
return pool;
}
// Longest first, so the seed is the pool's front and a tie on residual
// resolves to the earlier position. A missing reading ranks below any
// measured one for this purpose only: it is admitted, but a face that
// was measured and found long is the better seed.
pool.sort_by(|&a, &b| {
let qa = quality[a].unwrap_or(0.0);
let qb = quality[b].unwrap_or(0.0);
qb.total_cmp(&qa).then(a.cmp(&b))
});
// Residuals: what remains of each pool vector outside the span of the
// chosen ones. Copied, since they are rewritten in place.
let mut residual: Vec<Vec<f32>> = pool.iter().map(|&i| embeddings[i].to_vec()).collect();
let mut taken = vec![false; pool.len()];
let mut chosen = Vec::with_capacity(max);
while chosen.len() < max {
// The face with the most left outside the span. The seed is the
// pool's front by construction: every unit vector has the same
// residual before anything is chosen, up to rounding, and rounding
// is not a reason to prefer one. After that `> best` and not `>=`,
// so a genuine tie keeps the earlier (longer) candidate.
let mut pick = None;
let mut best = 0.0_f32;
if chosen.is_empty() {
pick = Some(0);
best = residual[0].iter().map(|x| x * x).sum();
} else {
for (k, r) in residual.iter().enumerate() {
if taken[k] {
continue;
}
let n2: f32 = r.iter().map(|x| x * x).sum();
if n2 > best {
best = n2;
pick = Some(k);
}
}
}
// Nothing left outside the span: every remaining face is a
// combination of the chosen ones and adds no volume.
let Some(k) = pick.filter(|_| best > 1e-6) else {
break;
};
taken[k] = true;
chosen.push(pool[k]);
// Project the chosen direction out of every remaining residual.
let inv = best.sqrt().recip();
let q: Vec<f32> = residual[k].iter().map(|x| x * inv).collect();
for (j, r) in residual.iter_mut().enumerate() {
if taken[j] {
continue;
}
let d: f32 = r.iter().zip(&q).map(|(a, b)| a * b).sum();
for (x, y) in r.iter_mut().zip(&q) {
*x -= d * y;
}
}
}
chosen
}
#[cfg(test)]
mod tests {
use super::*;
fn unit(v: &[f32]) -> Vec<f32> {
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect()
}
#[test]
fn a_person_under_the_cap_keeps_every_eligible_face() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0]), unit(&[1.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), None, Some(16.0)];
assert_eq!(select(&refs, &q, 100), vec![0, 1, 2]);
}
#[test]
fn a_short_vector_never_stands_for_a_person() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), Some(MIN_REFERENCE_QUALITY - 0.01)];
assert_eq!(select(&refs, &q, 100), vec![0]);
}
/// Two hundred frames from one afternoon and one profile shot: the
/// profile is the second choice, not the two-hundred-and-first.
#[test]
fn the_odd_one_out_is_chosen_before_any_duplicate() {
let mut e: Vec<Vec<f32>> = Vec::new();
let mut q = Vec::new();
for i in 0..200 {
// Near-duplicates of one direction, with a little noise.
let t = (i as f32) * 1e-3;
e.push(unit(&[1.0, t, t * 0.5]));
q.push(Some(20.0 + (i % 7) as f32));
}
e.push(unit(&[0.0, 0.0, 1.0]));
q.push(Some(16.0));
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let chosen = select(&refs, &q, 3);
assert_eq!(chosen.len(), 3);
assert_eq!(
chosen[1], 200,
"the profile shot was not second: {chosen:?}"
);
// Seeded on the longest vector.
assert_eq!(q[chosen[0]], Some(26.0));
}
/// Faces inside the span of the chosen ones add no volume and are not
/// taken to fill the cap.
#[test]
fn the_cap_is_not_filled_from_inside_the_span() {
let e = [
unit(&[1.0, 0.0]),
unit(&[0.0, 1.0]),
unit(&[1.0, 1.0]),
unit(&[2.0, -1.0]),
];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0); 4];
assert_eq!(select(&refs, &q, 3).len(), 2);
}
#[test]
fn the_choice_is_deterministic() {
let e: Vec<Vec<f32>> = (0..50)
.map(|i| {
let a = (i as f32) * 0.37;
unit(&[a.cos(), a.sin(), (a * 3.0).sin(), 0.2])
})
.collect();
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = vec![Some(18.0); 50];
assert_eq!(select(&refs, &q, 5), select(&refs, &q, 5));
}
}
+2 -2
View File
@@ -1,6 +1,6 @@
//! What a frame actually costs — the measurement FR-DSP-2 is waiting on. //! What a frame actually costs — the measurement FR-DSP-2 is waiting on.
//! //!
//! `docs/display-and-extension.md` §2 argues that tiled computation predates //! `docs/dev/display-and-extension.md` §2 argues that tiled computation predates
//! the fused-shader design and may not need to exist: the composer folds every //! the fused-shader design and may not need to exist: the composer folds every
//! active operation into **one dispatch over a viewport-sized target**, so the //! active operation into **one dispatch over a viewport-sized target**, so the
//! problem tiles were invented to solve may already be solved. That argument //! problem tiles were invented to solve may already be solved. That argument
@@ -28,7 +28,7 @@
//! the per-frame CPU half is dominated by shader-source assembly, which is //! the per-frame CPU half is dominated by shader-source assembly, which is
//! string formatting and is several times slower unoptimised. //! string formatting and is several times slower unoptimised.
//! //!
//! The committed numbers live in `docs/frame-budget.md`. Rerun this and diff //! The committed numbers live in `docs/dev/frame-budget.md`. Rerun this and diff
//! that file; a regression should be a diff rather than somebody's memory. //! that file; a regression should be a diff rather than somebody's memory.
//! //!
//! # Why the 99th percentile and not the mean //! # Why the 99th percentile and not the mean
+1 -1
View File
@@ -1,6 +1,6 @@
//! Segment an image and write the granularity ladder as false-coloured PPMs. //! Segment an image and write the granularity ladder as false-coloured PPMs.
//! //!
//! The whole point of S15 step 2 (docs/segmentation.md §11): look at the //! The whole point of S15 step 2 (docs/dev/segmentation.md §11): look at the
//! ladder and decide whether clicking through it would land on the things a //! ladder and decide whether clicking through it would land on the things a
//! person means. No amount of design settles that — the pictures do. //! person means. No amount of design settles that — the pictures do.
//! //!
+47
View File
@@ -1347,6 +1347,53 @@ mod tests {
} }
} }
#[test]
fn a_grey_step_edge_stays_grey() {
// A flat patch cannot tell the Malvar kernels from any other set of
// weights that sum to zero. An edge can. A grey vertical step, so
// every photosite records the same profile, must come back with the
// three channels close together on both sides; any spread is false
// colour from interpolating across the edge.
//
// The bound is set by the paper's kernels, which peak at 0.19 here.
// With the ±2 terms of the green-site kernels transposed — the bug
// this test was written against — the peak is 0.375.
let Some(ctx) = ctx() else { return };
let d = Demosaicer::new(&ctx).expect("demosaicer");
let size = 32u32;
let white = 16383u16;
let mut raw = flat_cfa(CfaPattern::Rggb, size, [0, 0, 0], 0, white);
for y in 0..size {
for x in size / 2..size {
raw.data[(y * size + x) as usize] = white;
}
}
let img = d.run(&raw).expect("demosaic");
let px = read_rgba(&ctx, &img);
let (w, _) = img.size();
let mut worst = (0.0f32, 0u32, 0u32);
for y in 2..size - 2 {
for x in 2..size - 2 {
let p = px[(y * w + x) as usize];
let spread = (p[0] - p[1]).abs().max((p[2] - p[1]).abs());
if spread > worst.0 {
worst = (spread, x, y);
}
}
}
assert!(
worst.0 < 0.25,
"false colour of {} at ({}, {}) on a grey edge — the green-site \
kernels are interpolating across the edge",
worst.0,
worst.1,
worst.2
);
}
#[test] #[test]
fn output_is_free_of_nan_and_negatives() { fn output_is_free_of_nan_and_negatives() {
// f16 NaN propagates silently through every later stage; a negative // f16 NaN propagates silently through every later stage; a negative
+7 -7
View File
@@ -1,4 +1,4 @@
//! Watershed segmentation — arm A's GPU half (S15, docs/segmentation.md). //! Watershed segmentation — arm A's GPU half (S15, docs/dev/segmentation.md).
//! //!
//! Runs the five passes in `shaders/watershed.wgsl` over a demosaiced image //! Runs the five passes in `shaders/watershed.wgsl` over a demosaiced image
//! and leaves a basin label per pixel on the GPU. The hierarchy built from //! and leaves a basin label per pixel on the GPU. The hierarchy built from
@@ -20,7 +20,7 @@
//! one AC-8 forbids is per frame in the render loop, and sharing a switch //! one AC-8 forbids is per frame in the render loop, and sharing a switch
//! would force a build wanting local masking to unlock the other. //! would force a build wanting local masking to unlock the other.
//! //!
//! It is still a real cost and still unfinished. F3 in docs/segmentation.md //! It is still a real cost and still unfinished. F3 in docs/dev/segmentation.md
//! §12 stands: the adjacency accumulation belongs GPU-side with atomics, and //! §12 stands: the adjacency accumulation belongs GPU-side with atomics, and
//! until it moves there every segmentation pays a full-resolution transfer. //! until it moves there every segmentation pays a full-resolution transfer.
//! Read the feature name as a description of a known gap rather than as //! Read the feature name as a description of a known gap rather than as
@@ -36,7 +36,7 @@ pub struct SegmentOptions {
/// Longest proxy edge. The segmentation runs here, not at sensor /// Longest proxy edge. The segmentation runs here, not at sensor
/// resolution: a 24 MP watershed costs 12× the memory to place boundaries /// resolution: a 24 MP watershed costs 12× the memory to place boundaries
/// a person cannot see, and the boundary refinement that matters at 1:1 /// a person cannot see, and the boundary refinement that matters at 1:1
/// is a separate stage (docs/segmentation.md §4). /// is a separate stage (docs/dev/segmentation.md §4).
pub max_edge: u32, pub max_edge: u32,
/// Pre-smoothing radius in proxy pixels. The caller's to raise with ISO — /// Pre-smoothing radius in proxy pixels. The caller's to raise with ISO —
/// this is the single knob that decides whether a noisy file segments /// this is the single knob that decides whether a noisy file segments
@@ -69,7 +69,7 @@ impl Default for SegmentOptions {
w_chroma: 0.5, w_chroma: 0.5,
// **Zero: the pass is off.** It is implemented, dispatched // **Zero: the pass is off.** It is implemented, dispatched
// correctly and measurably changes nothing — see the ignored test // correctly and measurably changes nothing — see the ignored test
// below and §12 of docs/segmentation.md. Until that is understood, // below and §12 of docs/dev/segmentation.md. Until that is understood,
// running it would buy 64 dispatches per segmentation and no // running it would buy 64 dispatches per segmentation and no
// improvement, so the default declines to pay. // improvement, so the default declines to pay.
plateau_iterations: 0, plateau_iterations: 0,
@@ -486,7 +486,7 @@ impl Segmentation {
/// a region graph of a few thousand nodes that every later interaction /// a region graph of a few thousand nodes that every later interaction
/// reads from the CPU anyway. /// reads from the CPU anyway.
/// ///
/// What it is *not* is finished. F3 in docs/segmentation.md §12 stands: /// What it is *not* is finished. F3 in docs/dev/segmentation.md §12 stands:
/// the adjacency accumulation belongs on the GPU with atomics, and until /// the adjacency accumulation belongs on the GPU with atomics, and until
/// it moves there a segmentation costs one full-resolution transfer of the /// it moves there a segmentation costs one full-resolution transfer of the
/// label and gradient buffers. That is a real cost on a phone and the /// label and gradient buffers. That is a real cost on a phone and the
@@ -724,7 +724,7 @@ mod tests {
px px
} }
#[test] #[test]
#[ignore = "the plateau pass is a measured no-op; see docs/segmentation.md §12"] #[ignore = "the plateau pass is a measured no-op; see docs/dev/segmentation.md §12"]
fn lower_completion_drains_a_plateau_instead_of_shattering_it() { fn lower_completion_drains_a_plateau_instead_of_shattering_it() {
// F1, asserted rather than eyeballed, and asserted at the level where // F1, asserted rather than eyeballed, and asserted at the level where
// it matters. // it matters.
@@ -741,7 +741,7 @@ mod tests {
// with no exit anywhere — cannot be drained by a distance that has // with no exit anywhere — cannot be drained by a distance that has
// nowhere to descend to, and collapsing it fully would need connected // nowhere to descend to, and collapsing it fully would need connected
// component labelling rather than a local rule. It is not worth it: // component labelling rather than a local rule. It is not worth it:
// see docs/segmentation.md §12. // see docs/dev/segmentation.md §12.
let Some(ctx) = ctx() else { return }; let Some(ctx) = ctx() else { return };
let (w, h) = (96u32, 96u32); let (w, h) = (96u32, 96u32);
let src = DemosaicedImage::from_rgba8(&ctx, &ramp(w, h), w, h).expect("source"); let src = DemosaicedImage::from_rgba8(&ctx, &ramp(w, h), w, h).expect("source");
+10 -4
View File
@@ -160,12 +160,18 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
// Green is measured. Red and blue are interpolated from their own // Green is measured. Red and blue are interpolated from their own
// axis, with a correction from the green Laplacian. // axis, with a correction from the green Laplacian.
// //
// Malvar "G at R/B locations" kernels, transposed per axis: // Malvar "R at green in R row" kernel, and its transpose:
// chroma along the row: (5c + 4(w1+e1) - (nw+ne+sw+se) - (n2+s2) + 0.5(w2+e2)) / 8 // chroma along the row: (5c + 4(w1+e1) - (nw+ne+sw+se) - (w2+e2) + 0.5(n2+s2)) / 8
//
// The -1 goes on the two greens *along* the chroma axis and the +0.5
// on the pair across it. Transposed, both kernels still sum to zero
// and reconstruct a flat patch exactly, but on an edge the correction
// at green sites is half strength and the false colour doubles: a
// blue/yellow zipper around every clipped highlight.
let along_row = let along_row =
(5.0 * c + 4.0 * (w1 + e1) - diag1 - vert2 + 0.5 * horiz2) * 0.125; (5.0 * c + 4.0 * (w1 + e1) - diag1 - horiz2 + 0.5 * vert2) * 0.125;
let along_col = let along_col =
(5.0 * c + 4.0 * (n1 + s1) - diag1 - horiz2 + 0.5 * vert2) * 0.125; (5.0 * c + 4.0 * (n1 + s1) - diag1 - vert2 + 0.5 * horiz2) * 0.125;
let red_horizontal = red_is_horizontal(gid.x, gid.y); let red_horizontal = red_is_horizontal(gid.x, gid.y);
let r = select(along_col, along_row, red_horizontal); let r = select(along_col, along_row, red_horizontal);
+2 -2
View File
@@ -1,4 +1,4 @@
// Watershed segmentation — the passes behind arm A of S15 (docs/segmentation.md). // Watershed segmentation — the passes behind arm A of S15 (docs/dev/segmentation.md).
// //
// Seven entry points forming one chain: // Seven entry points forming one chain:
// //
@@ -197,7 +197,7 @@ fn gradient(@builtin(global_invocation_id) gid: vec3<u32>) {
// lowest-indexed neighbour, which is up and to the left. Each pixel therefore // lowest-indexed neighbour, which is up and to the left. Each pixel therefore
// walks diagonally until it falls off the plateau, and one flat region becomes // walks diagonally until it falls off the plateau, and one flat region becomes
// a fan of diagonal chains rather than one basin — visible as hatching across // a fan of diagonal chains rather than one basin — visible as hatching across
// what should be a single area (docs/segmentation.md §12, F1). // what should be a single area (docs/dev/segmentation.md §12, F1).
// //
// The fix is the standard lower-completion: give each plateau pixel its // The fix is the standard lower-completion: give each plateau pixel its
// geodesic distance to the nearest pixel that *does* have a lower neighbour, // geodesic distance to the nearest pixel that *does* have a lower neighbour,
+5 -5
View File
@@ -2,11 +2,11 @@
//! //!
//! FR-DSP-3 says a slider updates the visible region within one frame budget at //! FR-DSP-3 says a slider updates the visible region within one frame budget at
//! proxy resolution. Until this file existed nothing checked it, which made it //! proxy resolution. Until this file existed nothing checked it, which made it
//! a wish — `docs/display-and-extension.md` §3 is blunt about that, and §7 is //! a wish — `docs/dev/display-and-extension.md` §3 is blunt about that, and §7 is
//! blunt about what tagging an unchecked requirement does to the coverage //! blunt about what tagging an unchecked requirement does to the coverage
//! figure. //! figure.
//! //!
//! The measurements this guards are in [`docs/frame-budget.md`], produced by //! The measurements this guards are in [`docs/dev/frame-budget.md`], produced by
//! `examples/frame_budget.rs`. This file is the part of them that has to keep //! `examples/frame_budget.rs`. This file is the part of them that has to keep
//! being true: it renders the **whole point-operation chain** through the real //! being true: it renders the **whole point-operation chain** through the real
//! `render_detailed` for a hundred frames, moving a slider between each, and //! `render_detailed` for a hundred frames, moving a slider between each, and
@@ -17,7 +17,7 @@
//! **The neighbourhood stage is deliberately not in the asserted chain.** It is //! **The neighbourhood stage is deliberately not in the asserted chain.** It is
//! over the budget today — clarity alone is 34 ms at 4K, because its kernel is //! over the budget today — clarity alone is 34 ms at 4K, because its kernel is
//! a fraction of the frame and reaches a 52-pixel radius there — and //! a fraction of the frame and reaches a 52-pixel radius there — and
//! `docs/frame-budget.md` records that, names the fix (a base computed at //! `docs/dev/frame-budget.md` records that, names the fix (a base computed at
//! reduced resolution) and does not pretend otherwise. Asserting a budget the //! reduced resolution) and does not pretend otherwise. Asserting a budget the
//! code does not meet would produce a red suite that everyone learns to ignore; //! code does not meet would produce a red suite that everyone learns to ignore;
//! asserting it on a chain that quietly excluded the expensive stage *without //! asserting it on a chain that quietly excluded the expensive stage *without
@@ -81,7 +81,7 @@ const SOURCE: (u32, u32) = (6000, 4000);
/// The viewport the budget is asserted at: a 16:10 desktop display. /// The viewport the budget is asserted at: a 16:10 desktop display.
/// ///
/// Not 4K, and the reason is worth stating. At 4K the fused chain still passes /// Not 4K, and the reason is worth stating. At 4K the fused chain still passes
/// with room to spare (4.5 ms of GPU; see `docs/frame-budget.md`), but a test /// with room to spare (4.5 ms of GPU; see `docs/dev/frame-budget.md`), but a test
/// that renders 8.3 M pixels a hundred times twice over is four seconds of /// that renders 8.3 M pixels a hundred times twice over is four seconds of
/// suite time to re-establish a conclusion 4.1 M pixels already establishes. /// suite time to re-establish a conclusion 4.1 M pixels already establishes.
const VIEWPORT: (u32, u32) = (2560, 1600); const VIEWPORT: (u32, u32) = (2560, 1600);
@@ -166,7 +166,7 @@ impl Run {
judged <= BUDGET_MS, judged <= BUDGET_MS,
"{case} at {}x{}: p99 of {FRAMES} frames was {judged:.2} ms, over the \ "{case} at {}x{}: p99 of {FRAMES} frames was {judged:.2} ms, over the \
{BUDGET_MS:.0} ms budget (cpu {:.2} ms, gpu {:.2} ms, total {:.2} ms). \ {BUDGET_MS:.0} ms budget (cpu {:.2} ms, gpu {:.2} ms, total {:.2} ms). \
FR-DSP-3 is what this violates; docs/frame-budget.md holds the \ FR-DSP-3 is what this violates; docs/dev/frame-budget.md holds the \
numbers it used to be.", numbers it used to be.",
viewport.0, viewport.0,
viewport.1, viewport.1,
+1 -1
View File
@@ -326,7 +326,7 @@ fn a_proxy_and_an_export_agree_about_the_effect() {
#[test] #[test]
fn crossing_the_reduction_threshold_does_not_change_the_picture() { fn crossing_the_reduction_threshold_does_not_change_the_picture() {
// TRACES: FR-DSP-3 — `docs/technical-debt.md` TD-4, held in pixels. // TRACES: FR-DSP-3 — `docs/dev/technical-debt.md` TD-4, held in pixels.
// //
// Clarity's base is computed on a reduced grid, and how reduced depends on // Clarity's base is computed on a reduced grid, and how reduced depends on
// the viewport: `LocalContrast::reduction` steps 4 -> 2 -> 1 as sigma // the viewport: `LocalContrast::reduction` steps 4 -> 2 -> 1 as sigma
+1 -1
View File
@@ -10,7 +10,7 @@
//! code path — the zoom is the full-resolution path — which is why the //! code path — the zoom is the full-resolution path — which is why the
//! requirement has been satisfied for some time without anyone tagging it. //! requirement has been satisfied for some time without anyone tagging it.
//! //!
//! `docs/display-and-extension.md` §7 is the reason this file exists rather //! `docs/dev/display-and-extension.md` §7 is the reason this file exists rather
//! than a tag on `framing.rs`: a requirement counts as covered when a `TRACES` //! than a tag on `framing.rs`: a requirement counts as covered when a `TRACES`
//! comment names it, and nothing checks that the code under the tag does the //! comment names it, and nothing checks that the code under the tag does the
//! thing. `FR-DEV-8` is tagged against plumbing a future operation would use. //! thing. `FR-DEV-8` is tagged against plumbing a future operation would use.
+10 -2
View File
@@ -6,7 +6,7 @@ rust-version.workspace = true
license.workspace = true license.workspace = true
# The one crate that names a runtime, a provider, a vendor library or a # The one crate that names a runtime, a provider, a vendor library or a
# device (docs/inference.md §8). `dr-face` and `dr-segment` ask it for a # device (docs/dev/inference.md §8). `dr-face` and `dr-segment` ask it for a
# session by role and never see which of these answered. # session by role and never see which of these answered.
[dependencies] [dependencies]
@@ -28,7 +28,10 @@ ort-sys = { version = "2.0.0-rc.13", default-features = false, features = ["disa
# The NVIDIA rungs exist on the desktop only. These features add `ort`'s # The NVIDIA rungs exist on the desktop only. These features add `ort`'s
# option builders and nothing else — no linking under `alternative-backend` — # option builders and nothing else — no linking under `alternative-backend` —
# but an Android binary has no business carrying even the option names, and # but an Android binary has no business carrying even the option names, and
# the packaging must never be tempted to (§2, §3.1). # the packaging must never be tempted to (§2, §3.1). The AMD rung needs no
# feature: MIGraphX is registered through the runtime's generic key/value
# entry point (`session::migraphx`), because `ort`'s own builder cannot
# name the compiled-program cache.
[target.'cfg(not(target_os = "android"))'.dependencies] [target.'cfg(not(target_os = "android"))'.dependencies]
ort = { workspace = true, features = ["cuda", "tensorrt"] } ort = { workspace = true, features = ["cuda", "tensorrt"] }
@@ -42,3 +45,8 @@ default = ["tract"]
tract = ["dep:ort-tract"] tract = ["dep:ort-tract"]
# Look for `libonnxruntime` on disk and hand its table to `ort`. # Look for `libonnxruntime` on disk and hand its table to `ort`.
native = ["dep:libloading", "dep:ort-sys"] native = ["dep:libloading", "dep:ort-sys"]
[dev-dependencies]
# The `ep_probe` example prints the provider's own diagnostics, which is most
# of what a failed rung tells you.
env_logger.workspace = true
@@ -0,0 +1,207 @@
//! Time each execution provider a runtime offers, on the models this
//! repository ships — the measurement docs/inference.md §1 requires before a
//! rung is added to §2's ladder.
//!
//! DARKROOM_ORT_DIR=/usr/lib \
//! cargo run --release -p dr-inference-engine --features native,tract \
//! --example ep_probe -- models/face/scrfd_500m_640.onnx ...
//!
//! Prints one row per (model, provider): the median of timed runs after
//! warm-ups, and the build time, which for a compiling provider is the
//! number that decides whether it needs an engine cache. MIGraphX is built
//! twice per precision — cold, then again from the cache it just wrote —
//! so both numbers are on the page.
//!
//! The ROCm provider is not in the list: ONNX Runtime removed it in 1.23,
//! and 1.29's `onnxruntime-rocm` ships `libonnxruntime_providers_migraphx.so`
//! and nothing else for AMD.
use std::path::{Path, PathBuf};
use std::time::Instant;
#[derive(Clone, Copy, PartialEq)]
enum Ep {
Cpu,
MiGraphX,
MiGraphXFp16,
}
impl Ep {
fn label(self) -> &'static str {
match self {
Ep::Cpu => "CPU",
Ep::MiGraphX => "MIGraphX f32",
Ep::MiGraphXFp16 => "MIGraphX fp16",
}
}
}
fn build(ep: Ep, bytes: &[u8], threads: usize, cache: &Path) -> ort::Result<ort::session::Session> {
let mut b = ort::session::Session::builder()?.with_intra_threads(threads)?;
match ep {
Ep::Cpu => {}
Ep::MiGraphX => migraphx(&mut b, false, &cache.join("f32"))?,
Ep::MiGraphXFp16 => migraphx(&mut b, true, &cache.join("fp16"))?,
}
b.commit_from_memory(bytes)
}
/// Register MIGraphX through the generic key/value API. `ort`'s own
/// builder fills the legacy `OrtMIGraphXProviderOptions`, which 1.29 reads
/// for its precision flags and nothing else: the model cache directory —
/// the difference between a 40 s load and a 0.3 s one — only travels this
/// way. The cache key is the graph, the GPU and the MIGraphX version, not
/// the precision, so each precision gets its own directory.
fn migraphx(
b: &mut ort::session::builder::SessionBuilder,
fp16: bool,
cache: &Path,
) -> ort::Result<()> {
use ort::AsPointer;
use std::ffi::CString;
std::fs::create_dir_all(cache).map_err(|e| ort::Error::new(e.to_string()))?;
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
let values = [
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
CString::new(cache.to_string_lossy().as_bytes()).unwrap(),
];
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
// SAFETY: the documented C call, over arrays that outlive it; the
// runtime copies the strings into its own options map.
unsafe {
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
b.ptr_mut(),
c"MIGraphX".as_ptr(),
key_ptrs.as_ptr(),
value_ptrs.as_ptr(),
keys.len(),
);
ort::Error::result_from_status(status)
}
}
/// Median of `runs` timed runs over zeros, in milliseconds, after warm-ups.
fn time(session: &mut ort::session::Session, warmups: usize, runs: usize) -> Result<f64, String> {
let shape: Vec<usize> = session.inputs()[0]
.dtype()
.tensor_shape()
.ok_or("input is not a tensor")?
.iter()
.map(|&d| if d > 0 { d as usize } else { 1 })
.collect();
let zeros = vec![0f32; shape.iter().product()];
let once = |s: &mut ort::session::Session| -> Result<f64, String> {
let input = ort::value::Tensor::from_array((shape.clone(), zeros.clone()))
.map_err(|e| e.to_string())?;
let t = Instant::now();
let out = s.run(ort::inputs![input]).map_err(|e| e.to_string())?;
let _ = out[0]
.try_extract_tensor::<f32>()
.map_err(|e| e.to_string())?;
Ok(t.elapsed().as_secs_f64() * 1e3)
};
for _ in 0..warmups {
once(session)?;
}
let mut times = Vec::with_capacity(runs);
for _ in 0..runs {
times.push(once(session)?);
}
times.sort_by(|a, b| a.partial_cmp(b).unwrap());
Ok(times[times.len() / 2])
}
fn first_line(s: &str) -> String {
s.lines().next().unwrap_or("").chars().take(120).collect()
}
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let models: Vec<PathBuf> = std::env::args_os().skip(1).map(PathBuf::from).collect();
if models.is_empty() {
eprintln!("usage: ep_probe MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
}
dr_inference_engine::ensure_runtime();
let runtime = dr_inference_engine::status().runtime;
println!("runtime: {}", runtime.label());
if !runtime.is_native() {
println!("(tract: no provider to compare; set DARKROOM_ORT_DIR)");
}
let threads = std::thread::available_parallelism()
.map(|n| n.get().saturating_sub(2).max(1))
.unwrap_or(1);
println!("intra-op threads: {threads}");
let cache = std::env::temp_dir().join("darkroom-ep-probe");
let _ = std::fs::remove_dir_all(&cache);
println!("compiled-program cache: {}\n", cache.display());
println!(
"{:<28} {:<15} {:>10} {:>10}",
"model", "provider", "build s", "median ms"
);
for model in &models {
let bytes = match std::fs::read(model) {
Ok(b) => b,
Err(e) => {
println!("{:<28} read failed: {e}", name(model));
continue;
}
};
// A compiling provider is built twice: the second build reads the
// program the first wrote, and its time is what a launch after the
// first costs.
let plan = [
(Ep::Cpu, false),
(Ep::MiGraphX, false),
(Ep::MiGraphX, true),
(Ep::MiGraphXFp16, false),
(Ep::MiGraphXFp16, true),
];
for (ep, cached) in plan {
let started = Instant::now();
match build(ep, &bytes, threads, &cache) {
Ok(mut session) => {
let built = started.elapsed().as_secs_f64();
match time(&mut session, 3, 15) {
Ok(ms) => println!(
"{:<28} {:<15} {:>10.1} {:>10.1}{}",
name(model),
ep.label(),
built,
ms,
if cached { " (from cache)" } else { "" }
),
Err(e) => println!(
"{:<28} {:<15} {:>10.1} {:>10} {}",
name(model),
ep.label(),
built,
"ran ✗",
first_line(&e)
),
}
}
Err(e) => println!(
"{:<28} {:<15} {:>21} {}",
name(model),
ep.label(),
"build ✗",
first_line(&e.to_string())
),
}
}
println!();
}
}
fn name(p: &Path) -> String {
p.file_name()
.unwrap_or(p.as_os_str())
.to_string_lossy()
.into_owned()
}
+100
View File
@@ -0,0 +1,100 @@
//! Walk the ladder as the app does — probe, engines, then a session — and
//! say what each step chose. The M5 check of docs/inference.md §6 without
//! the app around it.
//!
//! DARKROOM_ORT_DIR=/usr/lib \
//! cargo run --release -p dr-inference-engine --features native,tract \
//! --example ladder -- CACHE_DIR models/face/scrfd_500m_640.onnx [MODEL.onnx ...]
//!
//! Every model named is a `Detector` for the config's purposes, which is
//! enough to see the rung taken, the engines compiled and a session land
//! on it. Delete `CACHE_DIR` to see the first run again; keep it to see the
//! second.
use std::path::PathBuf;
use std::time::{Duration, Instant};
fn main() {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
let mut args = std::env::args_os().skip(1).map(PathBuf::from);
let (Some(cache_dir), models) = (args.next(), args.collect::<Vec<_>>()) else {
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
};
if models.is_empty() {
eprintln!("usage: ladder CACHE_DIR MODEL.onnx [MODEL.onnx ...]");
std::process::exit(2);
}
let runtime_dirs: Vec<PathBuf> = std::env::var_os("DARKROOM_ORT_DIR")
.map(PathBuf::from)
.into_iter()
.collect();
let started = Instant::now();
dr_inference_engine::init(dr_inference_engine::Config {
runtime_dirs,
cache_dir: cache_dir.clone(),
models: models
.iter()
.map(|p| (dr_inference_engine::Role::Detector, p.clone()))
.collect(),
embedded: Vec::new(),
ceiling: None,
threads: 0,
decay: Duration::ZERO,
});
let mut last = String::new();
loop {
let s = dr_inference_engine::status();
let line = format!(
"{} · {} · engines {}/{}{}",
s.line(),
if s.probing {
"probing"
} else {
s.reason.as_str()
},
s.engines.0,
s.engines.1,
if s.failed.is_empty() {
String::new()
} else {
format!(
" · tried {}",
s.failed
.iter()
.map(|(r, why)| format!("{}: {why}", r.label()))
.collect::<Vec<_>>()
.join(" · ")
)
}
);
if line != last {
println!("{:>6.1} s {line}", started.elapsed().as_secs_f64());
last = line;
}
if !s.probing && s.engines.0 >= s.engines.1 {
break;
}
std::thread::sleep(Duration::from_millis(500));
}
for path in &models {
let bytes = std::fs::read(path).expect("read model");
let t = Instant::now();
let model = dr_inference_engine::open(
dr_inference_engine::Role::Detector,
dr_inference_engine::Form::F32,
&bytes,
)
.expect("open model");
let acquired = model.acquire().expect("acquire session");
println!(
"{} on {} in {:.2} s",
path.file_name().unwrap().to_string_lossy(),
acquired.rung().label(),
t.elapsed().as_secs_f64()
);
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
//! The API table `ort` runs on, chosen once (docs/inference.md §3). //! The API table `ort` runs on, chosen once (docs/dev/inference.md §3).
//! //!
//! `ort` with `alternative-backend` links no runtime and asks, on first use, //! `ort` with `alternative-backend` links no runtime and asks, on first use,
//! for an `OrtApi` — a struct of function pointers. Two things can fill it: //! for an `OrtApi` — a struct of function pointers. Two things can fill it:
+1 -1
View File
@@ -1,5 +1,5 @@
//! Compiled engines: what a rung builds once per device, and the thread that //! Compiled engines: what a rung builds once per device, and the thread that
//! builds them before anyone asks (docs/inference.md §5, §6). //! builds them before anyone asks (docs/dev/inference.md §5, §6).
//! //!
//! TensorRT keeps its own engine cache keyed by graph hash; QNN writes a //! TensorRT keeps its own engine cache keyed by graph hash; QNN writes a
//! context model. Both are opaque to this crate, which tracks only *that* a //! context model. Both are opaque to this crate, which tracks only *that* a
+58 -13
View File
@@ -1,10 +1,10 @@
//! Which runtime, which provider and which model form — decided once per //! Which runtime, which provider and which model form — decided once per
//! device, and the only crate that knows the answer (docs/inference.md). //! device, and the only crate that knows the answer (docs/dev/inference.md).
//! //!
//! Consumers ask for a session by [`Role`] and get `ort`'s `Session` back; //! Consumers ask for a session by [`Role`] and get `ort`'s `Session` back;
//! what built it — tract on one core, ONNX Runtime's CPU pool, a TensorRT //! what built it — tract on one core, ONNX Runtime's CPU pool, a TensorRT
//! engine, the Hexagon — is this crate's business and shows up in //! engine, a MIGraphX program, the Hexagon — is this crate's business and
//! [`status`] for the settings row and nowhere else. //! shows up in [`status`] for the settings row and nowhere else.
//! //!
//! The shape follows §3 of the spec: `ort` links nothing (`alternative-backend`), //! The shape follows §3 of the spec: `ort` links nothing (`alternative-backend`),
//! and the first call hands it an API table from either a `libonnxruntime` //! and the first call hands it an API table from either a `libonnxruntime`
@@ -35,13 +35,13 @@ pub enum Role {
Embedder, Embedder,
Segmenter, Segmenter,
Scene, Scene,
/// The dense landmark model behind the eye reading (docs/faces.md §7c). /// The dense landmark model behind the eye reading (docs/dev/faces.md §7c).
Landmarks, Landmarks,
/// The eye-state and sunglasses classifiers, a few hundred kilobytes. /// The eye-state and sunglasses classifiers, a few hundred kilobytes.
EyeClassifier, EyeClassifier,
/// XFeat, the panorama keypoint detector (docs/panorama.md). /// XFeat, the panorama keypoint detector (docs/dev/panorama.md).
Keypoints, Keypoints,
/// MI-GAN, the panorama border filler (docs/panorama.md §12). Plain /// MI-GAN, the panorama border filler (docs/dev/panorama.md §12). Plain
/// convolutions, so any rung serves it; fp16 on TensorRT and int8 on /// convolutions, so any rung serves it; fp16 on TensorRT and int8 on
/// the Hexagon are the point of it. /// the Hexagon are the point of it.
Inpainter, Inpainter,
@@ -60,7 +60,9 @@ pub enum Form {
/// A rung of the ladder (§2). Ordered: a user override names the highest rung /// A rung of the ladder (§2). Ordered: a user override names the highest rung
/// the probe may take, and a compiling rung falls back to the one below it /// the probe may take, and a compiling rung falls back to the one below it
/// until its engine exists. /// until its engine exists. The order is within a vendor's ladder — a
/// machine has NVIDIA rungs or an AMD rung, never both — so a ceiling is
/// read as "no higher than this on whichever ladder the device has".
#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
pub enum Rung { pub enum Rung {
/// ONNX Runtime's CPU provider, or tract when no runtime file was found. /// ONNX Runtime's CPU provider, or tract when no runtime file was found.
@@ -69,6 +71,11 @@ pub enum Rung {
Cuda, Cuda,
/// NVIDIA, through a TensorRT engine compiled on this device. Desktop only. /// NVIDIA, through a TensorRT engine compiled on this device. Desktop only.
TensorRt, TensorRt,
/// AMD, through a MIGraphX program compiled on this device. Desktop
/// only. ONNX Runtime's ROCm provider, the CUDA provider's twin, was
/// removed in ONNX Runtime 1.23, so there is no non-compiling AMD rung
/// to fall back to: this one falls back to the CPU.
MiGraphX,
/// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only. /// Qualcomm's Hexagon NPU through QNN, int8 models only. Android only.
Hexagon, Hexagon,
} }
@@ -79,6 +86,7 @@ impl Rung {
Rung::Cpu => "CPU", Rung::Cpu => "CPU",
Rung::Cuda => "CUDA", Rung::Cuda => "CUDA",
Rung::TensorRt => "TensorRT", Rung::TensorRt => "TensorRT",
Rung::MiGraphX => "MIGraphX",
Rung::Hexagon => "Hexagon NPU", Rung::Hexagon => "Hexagon NPU",
} }
} }
@@ -88,13 +96,13 @@ impl Rung {
fn fallback(self) -> Rung { fn fallback(self) -> Rung {
match self { match self {
Rung::TensorRt => Rung::Cuda, Rung::TensorRt => Rung::Cuda,
Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu, Rung::MiGraphX | Rung::Hexagon | Rung::Cuda | Rung::Cpu => Rung::Cpu,
} }
} }
/// Whether a session on this rung needs an engine built first. /// Whether a session on this rung needs an engine built first.
fn compiles(self) -> bool { fn compiles(self) -> bool {
matches!(self, Rung::TensorRt | Rung::Hexagon) matches!(self, Rung::TensorRt | Rung::MiGraphX | Rung::Hexagon)
} }
/// The model form this rung wants for a role. /// The model form this rung wants for a role.
@@ -155,6 +163,8 @@ pub struct Status {
/// Engines compiled and engines wanted, for a compiling rung; `(0, 0)` /// Engines compiled and engines wanted, for a compiling rung; `(0, 0)`
/// otherwise. /// otherwise.
pub engines: (usize, usize), pub engines: (usize, usize),
/// Every rung above the selected one that was tried, and why it lost.
pub failed: Vec<(Rung, String)>,
} }
impl Status { impl Status {
@@ -162,7 +172,7 @@ impl Status {
pub fn line(&self) -> String { pub fn line(&self) -> String {
let form = match self.rung { let form = match self.rung {
Rung::Hexagon => " · int8", Rung::Hexagon => " · int8",
Rung::TensorRt => " · fp16", Rung::TensorRt | Rung::MiGraphX => " · fp16",
_ => "", _ => "",
}; };
format!("{}{} · {}", self.rung.label(), form, self.runtime.label()) format!("{}{} · {}", self.rung.label(), form, self.runtime.label())
@@ -267,8 +277,8 @@ fn acquire(role: Role, form: Form, bytes: &Arc<[u8]>, hash: u64) -> Result<Acqui
return Ok(Acquired { entry }); return Ok(Acquired { entry });
} }
// Built outside the registry lock: a TensorRT engine load is long enough // Built outside the registry lock: a TensorRT or MIGraphX engine load
// that another role's acquire should not wait on it. // is long enough that another role's acquire should not wait on it.
let session = session::build(rung, role, bytes, &cfg)?; let session = session::build(rung, role, bytes, &cfg)?;
log::debug!("inference: {role:?} loaded on {}", rung.label()); log::debug!("inference: {role:?} loaded on {}", rung.label());
let entry = Arc::new(Loaded { let entry = Arc::new(Loaded {
@@ -399,6 +409,16 @@ pub fn status() -> Status {
runtime: api::runtime(), runtime: api::runtime(),
rung, rung,
reason: s.cache.reason.clone(), reason: s.cache.reason.clone(),
// Only what explains the selection: on an AMD machine the NVIDIA
// rungs "not enabled in this build" say nothing about why MIGraphX
// was taken. With the floor selected, everything tried is above it.
failed: s
.cache
.failed
.iter()
.filter(|(r, _)| *r > rung)
.cloned()
.collect(),
probing: s.probing, probing: s.probing,
engines: if rung.compiles() { engines: if rung.compiles() {
(s.cache.compiled.len(), s.wanted) (s.cache.compiled.len(), s.wanted)
@@ -496,7 +516,7 @@ mod tests {
/// The smallest shipped graph, if this checkout has the weights; a test /// The smallest shipped graph, if this checkout has the weights; a test
/// suite that needs a research-licensed download is one that does not /// suite that needs a research-licensed download is one that does not
/// run in CI (docs/faces.md §3), so absence is a skip. /// run in CI (docs/dev/faces.md §3), so absence is a skip.
fn probe_bytes() -> Option<Vec<u8>> { fn probe_bytes() -> Option<Vec<u8>> {
let path = concat!( let path = concat!(
env!("CARGO_MANIFEST_DIR"), env!("CARGO_MANIFEST_DIR"),
@@ -611,8 +631,33 @@ mod tests {
); );
} }
#[test]
fn the_status_reports_only_the_rungs_above_the_selection() {
let _serial = serial();
let failed = vec![
(Rung::TensorRt, "not enabled".to_string()),
(Rung::Cuda, "not enabled".to_string()),
];
let before = state().lock().unwrap().cache.clone();
state().lock().unwrap().cache = Cache {
rung: Some(Rung::MiGraphX),
failed: failed.clone(),
..Cache::default()
};
// An AMD desktop: the NVIDIA rungs below MIGraphX are not the story.
assert!(status().failed.is_empty());
// An NVIDIA desktop on the CUDA provider: TensorRT's failure is.
state().lock().unwrap().cache.rung = Some(Rung::Cuda);
assert_eq!(status().failed, vec![failed[0].clone()]);
// The floor: everything tried explains it.
state().lock().unwrap().cache.rung = Some(Rung::Cpu);
assert_eq!(status().failed.len(), 2);
state().lock().unwrap().cache = before;
}
#[test] #[test]
fn the_status_line_reads_as_the_floor_before_init() { fn the_status_line_reads_as_the_floor_before_init() {
let _serial = serial();
let s = status(); let s = status();
assert_eq!(s.rung, Rung::Cpu); assert_eq!(s.rung, Rung::Cpu);
assert!(s.line().starts_with("CPU"), "{}", s.line()); assert!(s.line().starts_with("CPU"), "{}", s.line());
+46 -8
View File
@@ -1,4 +1,4 @@
//! Walk the ladder, once, by building real sessions (docs/inference.md §4). //! Walk the ladder, once, by building real sessions (docs/dev/inference.md §4).
//! //!
//! A rung is taken when a session builds on it, runs, and is faster than //! A rung is taken when a session builds on it, runs, and is faster than
//! the floor. Both halves matter: a provider can register and then fail at //! the floor. Both halves matter: a provider can register and then fail at
@@ -16,8 +16,11 @@ use crate::{api::Runtime, state, Cache, Config, Form, Role, Rung};
fn ladder(ceiling: Option<Rung>) -> Vec<Rung> { fn ladder(ceiling: Option<Rung>) -> Vec<Rung> {
#[cfg(target_os = "android")] #[cfg(target_os = "android")]
let all = [Rung::Hexagon]; let all = [Rung::Hexagon];
// A desktop has one vendor's GPU; the other vendor's providers are
// "not enabled in this build" or a library that fails to load, and
// either answer arrives in milliseconds.
#[cfg(not(target_os = "android"))] #[cfg(not(target_os = "android"))]
let all = [Rung::TensorRt, Rung::Cuda]; let all = [Rung::TensorRt, Rung::Cuda, Rung::MiGraphX];
all.into_iter() all.into_iter()
.filter(|r| ceiling.is_none_or(|c| *r <= c)) .filter(|r| ceiling.is_none_or(|c| *r <= c))
.collect() .collect()
@@ -206,15 +209,22 @@ fn first_line(s: &str) -> String {
line[start..].chars().take(200).collect() line[start..].chars().take(200).collect()
} }
/// Everything a change of which should re-probe: the runtime and where it /// Everything a change of which should re-probe: the runtime, where it
/// came from, this crate, the platform, the driver or SoC, and the models. /// came from and which providers sit beside it, this crate, the platform,
/// the driver or SoC, and the models.
fn fingerprint(runtime: &Runtime, cfg: &Config) -> String { fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
let mut parts = vec![ let mut parts = vec![
format!("engine {}", env!("CARGO_PKG_VERSION")), format!("engine {}", env!("CARGO_PKG_VERSION")),
format!("{} {}", std::env::consts::OS, std::env::consts::ARCH), format!("{} {}", std::env::consts::OS, std::env::consts::ARCH),
match runtime { match runtime {
Runtime::Tract => "tract".to_string(), Runtime::Tract => "tract".to_string(),
Runtime::OnnxRuntime { path, version } => format!("ort {version} {}", path.display()), Runtime::OnnxRuntime { path, version } => {
format!(
"ort {version} {} [{}]",
path.display(),
providers_beside(path)
)
}
}, },
device_identity(), device_identity(),
]; ];
@@ -237,13 +247,41 @@ fn fingerprint(runtime: &Runtime, cfg: &Config) -> String {
parts.join("\n") parts.join("\n")
} }
/// The `libonnxruntime_providers_*.so` files in the runtime's directory.
/// A distribution's CPU-only and ROCm builds are the same version at the
/// same path; the provider libraries beside them are what differs.
fn providers_beside(runtime: &Path) -> String {
let Some(dir) = runtime.parent() else {
return String::new();
};
let mut names: Vec<String> = std::fs::read_dir(dir)
.into_iter()
.flatten()
.filter_map(|e| e.ok())
.filter_map(|e| e.file_name().into_string().ok())
.filter(|n| {
n.starts_with("libonnxruntime_providers_") || n.starts_with("onnxruntime_providers_")
})
.collect();
names.sort();
names.join(" ")
}
#[cfg(target_os = "linux")] #[cfg(target_os = "linux")]
fn device_identity() -> String { fn device_identity() -> String {
// The NVIDIA driver's version line; absent means no NVIDIA driver. // The NVIDIA driver's version line, or the ROCm release the AMD stack
std::fs::read_to_string("/proc/driver/nvidia/version") // came from (`rocm-core` writes it; the kernel driver has no version
// of its own). Absent means neither.
if let Some(line) = std::fs::read_to_string("/proc/driver/nvidia/version")
.ok() .ok()
.and_then(|s| s.lines().next().map(str::to_string)) .and_then(|s| s.lines().next().map(str::to_string))
.unwrap_or_else(|| "no nvidia driver".into()) {
return line;
}
if let Ok(rocm) = std::fs::read_to_string("/opt/rocm/.info/version") {
return format!("rocm {}", rocm.trim());
}
"no nvidia driver, no rocm".into()
} }
#[cfg(target_os = "android")] #[cfg(target_os = "android")]
+59 -2
View File
@@ -1,4 +1,4 @@
//! One session builder per rung (docs/inference.md §2, §7, §9). //! One session builder per rung (docs/dev/inference.md §2, §7, §9).
use ort::session::Session; use ort::session::Session;
@@ -83,10 +83,65 @@ fn providers(
ep::CUDA::default().build(), ep::CUDA::default().build(),
])?) ])?)
} }
Rung::MiGraphX => {
// fp16 on the same terms as TensorRT (§7). MIGraphX compiles a
// program per graph — 20–60 s here — and keeps it in the cache
// directory, keyed on the graph, the GPU and its own version
// but not the precision: hence one directory per precision.
// The CPU takes any node it declines.
let fp16 = role != Role::Embedder;
let cache = cfg
.cache_dir
.join("migraphx")
.join(if fp16 { "fp16" } else { "f32" });
let _ = std::fs::create_dir_all(&cache);
let mut b = b;
migraphx(&mut b, fp16, &cache)?;
Ok(b)
}
Rung::Hexagon => unreachable!("the Hexagon rung is not on a desktop ladder"), Rung::Hexagon => unreachable!("the Hexagon rung is not on a desktop ladder"),
} }
} }
/// Register MIGraphX through ONNX Runtime's generic key/value entry point.
///
/// `ort`'s own builder (`ep::MIGraphX`) fills the legacy
/// `OrtMIGraphXProviderOptions`, and 1.29 reads that struct for its
/// precision flags and nothing else — the compiled-program cache directory
/// is only a key in the generic map (`migraphx_model_cache_dir`), and
/// without it every session is a full compile. Registration through the
/// generic entry point needs no `ort` feature: it is one call on the API
/// table, which is why the crate's `ort` dependency names no AMD feature.
#[cfg(not(target_os = "android"))]
fn migraphx(
b: &mut ort::session::builder::SessionBuilder,
fp16: bool,
cache: &std::path::Path,
) -> ort::Result<()> {
use ort::AsPointer;
use std::ffi::CString;
let keys = [c"migraphx_fp16_enable", c"migraphx_model_cache_dir"];
let values = [
CString::new(if fp16 { "1" } else { "0" }).unwrap(),
CString::new(cache.to_string_lossy().as_bytes())
.map_err(|e| ort::Error::new(e.to_string()))?,
];
let key_ptrs: Vec<_> = keys.iter().map(|k| k.as_ptr()).collect();
let value_ptrs: Vec<_> = values.iter().map(|v| v.as_ptr()).collect();
// SAFETY: the documented C call over arrays that outlive it; the
// runtime copies the strings into its own options map before returning.
unsafe {
let status = (ort::api().SessionOptionsAppendExecutionProvider)(
b.ptr_mut(),
c"MIGraphX".as_ptr(),
key_ptrs.as_ptr(),
value_ptrs.as_ptr(),
keys.len(),
);
ort::Error::result_from_status(status)
}
}
#[cfg(target_os = "android")] #[cfg(target_os = "android")]
fn providers( fn providers(
b: ort::session::builder::SessionBuilder, b: ort::session::builder::SessionBuilder,
@@ -120,6 +175,8 @@ fn providers(
.build() .build()
.error_on_failure()])?) .error_on_failure()])?)
} }
Rung::Cuda | Rung::TensorRt => unreachable!("no NVIDIA rung on Android"), Rung::Cuda | Rung::TensorRt | Rung::MiGraphX => {
unreachable!("no desktop GPU rung on Android")
}
} }
} }
+1 -1
View File
@@ -13,7 +13,7 @@ log.workspace = true
# Inference for the learned keypoint detector, on the same footing as # Inference for the learned keypoint detector, on the same footing as
# `dr-segment`: `ort` is the API, `dr-inference-engine` decides what runs # `dr-segment`: `ort` is the API, `dr-inference-engine` decides what runs
# it (docs/inference.md), and both are optional so that the geometry — # it (docs/dev/inference.md), and both are optional so that the geometry —
# matching, the rotation solve, the projections — is a dependency-free crate # matching, the rotation solve, the projections — is a dependency-free crate
# that tests without a model. # that tests without a model.
ort = { workspace = true, optional = true } ort = { workspace = true, optional = true }
+142 -18
View File
@@ -89,12 +89,19 @@ pub struct Params {
pub coarse: usize, pub coarse: usize,
/// The fine passes' band width. /// The fine passes' band width.
pub band: usize, pub band: usize,
/// How deep into the picture the mirrored context reaches. A plain /// How deep into the picture the mirrored context reaches, or **zero
/// reflection of a deep hole pulls in whatever is that far from the /// for no mirrored context at all**: the void is then shown to the
/// edge — a ridge, a peak — and the model, told that is what lies /// model as it is — reaching the picture's edge with nothing beyond,
/// beyond, paints it upside down. Folding the reflection within this /// and, beyond the band being filled, still unknown. That is what the
/// band keeps the ring looking like the edge it continues (sky beside /// shipped model was trained on (a fine-tune of MI-GAN on voids cut
/// sky, grass beside grass) and nothing further away. /// from photographs the way a cylindrical merge cuts them, see
/// `docs/dev/panorama.md` §14); a ring would give it a fold to continue.
///
/// Non-zero is the stock model's crutch: a plain reflection of a deep
/// hole pulls in whatever is that far from the edge — a ridge, a peak —
/// and the model, told that is what lies beyond, paints it upside down.
/// Folding the reflection within this band keeps the ring looking like
/// the edge it continues and nothing further away.
pub mirror_depth: usize, pub mirror_depth: usize,
/// How far inside the real edge the fill also regenerates, the two /// How far inside the real edge the fill also regenerates, the two
/// blended by distance. A hard cut between real pixels and invented /// blended by distance. A hard cut between real pixels and invented
@@ -108,9 +115,9 @@ pub struct Params {
impl Default for Params { impl Default for Params {
fn default() -> Self { fn default() -> Self {
Params { Params {
coarse: 4, coarse: 1,
band: 96, band: 192,
mirror_depth: 48, mirror_depth: 0,
feather: 24, feather: 24,
stride: 384, stride: 384,
} }
@@ -141,7 +148,10 @@ pub fn fill_border(
} = params; } = params;
let q = q.max(1); let q = q.max(1);
let band = band.max(8); let band = band.max(8);
let mirror_depth = mirror_depth.max(1); // No ring: the void beyond the band stays unknown, as in the model's
// training; with a ring the far side is the coarse fill, presented as
// known, which the stock model needed to see something there.
let open = mirror_depth == 0;
if width == 0 || height == 0 || rgb.len() != width * height * 3 || known.len() != width * height if width == 0 || height == 0 || rgb.len() != width * height * 3 || known.len() != width * height
{ {
return Err(PanoError::Input("fill: buffer sizes disagree".into())); return Err(PanoError::Input("fill: buffer sizes disagree".into()));
@@ -227,7 +237,11 @@ pub fn fill_border(
let mut any = false; let mut any = false;
for i in 0..width * height { for i in 0..width * height {
let in_band = !known[i] && dist[i] > lo && dist[i] <= hi; let in_band = !known[i] && dist[i] > lo && dist[i] <= hi;
band_known[i] = !in_band; band_known[i] = if open {
known[i] || dist[i] <= lo
} else {
!in_band
};
any |= in_band; any |= in_band;
} }
if !any { if !any {
@@ -289,7 +303,7 @@ fn fill_once(
} }
// The padded canvas with mirrored context, and the hole within it. // The padded canvas with mirrored context, and the hole within it.
let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth); let ctx = MirroredContext::build(rgb, width, height, known, mirror_depth, t);
let (pw, ph) = (ctx.width, ctx.height); let (pw, ph) = (ctx.width, ctx.height);
// Tiles that touch the hole, on a grid that reaches both far edges. // Tiles that touch the hole, on a grid that reaches both far edges.
@@ -369,7 +383,7 @@ fn fill_once(
if known[i] { if known[i] {
continue; continue;
} }
let p = (yy + RING) * pw + (xx + RING); let p = (yy + ctx.ring) * pw + (xx + ctx.ring);
if wsum[p] > 0.0 { if wsum[p] > 0.0 {
for ch in 0..3 { for ch in 0..3 {
rgb[i * 3 + ch] = (acc[p * 3 + ch] / wsum[p]).clamp(0.0, 1.0); rgb[i * 3 + ch] = (acc[p * 3 + ch] / wsum[p]).clamp(0.0, 1.0);
@@ -478,12 +492,45 @@ fn fold(d: usize, depth: usize) -> usize {
struct MirroredContext { struct MirroredContext {
width: usize, width: usize,
height: usize, height: usize,
/// The padding on every side: `RING` with mirrored context, 0 without.
ring: usize,
rgb: Vec<f32>, rgb: Vec<f32>,
hole: Vec<bool>, hole: Vec<bool>,
} }
impl MirroredContext { impl MirroredContext {
fn build(rgb: &[f32], width: usize, height: usize, known: &[bool], depth: usize) -> Self { fn build(
rgb: &[f32],
width: usize,
height: usize,
known: &[bool],
depth: usize,
tile: usize,
) -> Self {
if depth == 0 {
// Open: the picture as it is, the hole as it is. What the hole
// holds does not matter — the model masks it out. A picture
// smaller than a tile (the merge page's preview) sits at the
// origin of a tile-sized canvas whose rest is hole: still the
// void as it is, and the only way a tile fits at all.
let (pw, ph) = (width.max(tile), height.max(tile));
let mut canvas = vec![0.0f32; pw * ph * 3];
let mut hole = vec![true; pw * ph];
for y in 0..height {
canvas[y * pw * 3..(y * pw + width) * 3]
.copy_from_slice(&rgb[y * width * 3..(y + 1) * width * 3]);
for x in 0..width {
hole[y * pw + x] = !known[y * width + x];
}
}
return MirroredContext {
width: pw,
height: ph,
ring: 0,
rgb: canvas,
hole,
};
}
let fold = |d: usize| fold(d, depth); let fold = |d: usize| fold(d, depth);
let (pw, ph) = (width + 2 * RING, height + 2 * RING); let (pw, ph) = (width + 2 * RING, height + 2 * RING);
let mut canvas = vec![0.0f32; pw * ph * 3]; let mut canvas = vec![0.0f32; pw * ph * 3];
@@ -534,6 +581,7 @@ impl MirroredContext {
MirroredContext { MirroredContext {
width: pw, width: pw,
height: ph, height: ph,
ring: RING,
rgb: canvas, rgb: canvas,
hole, hole,
} }
@@ -634,7 +682,10 @@ mod tests {
200, 200,
&known, &known,
&mut model, &mut model,
test_params(0), Params {
mirror_depth: 48,
..test_params(0)
},
&mut |_, _| {}, &mut |_, _| {},
) )
.unwrap(); .unwrap();
@@ -648,6 +699,74 @@ mod tests {
} }
} }
#[test]
fn an_open_void_reaches_the_tile_edge_and_stays_unknown_beyond_the_band() {
// A 150-tall hole above and below; bands of 96. With no ring the
// first band's tiles sit at the picture's edge, so a tile's top
// row is unknown, and the rows deeper than the band are unknown
// too — not "known" coarse fill — exactly as the model was trained.
let (mut rgb, known) = picture(200, 500, 150);
let mut model = Flat {
tile: 64,
seen: Vec::new(),
};
fill_border(
&mut rgb,
200,
500,
&known,
&mut model,
Params {
band: 96,
..test_params(0)
},
&mut |_, _| {},
)
.unwrap();
// The first pass's tile at the picture's top edge is unknown
// through and through: the hole is 150 deep, the tile 64, and
// nothing beyond the band was presented as known. With a ring, or
// with the far side shown as coarse fill, no tile is ever all hole.
assert!(model.seen.iter().any(|(_, k)| k.iter().all(|&v| !v)));
for i in 0..200 * 500 {
if !known[i] {
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
}
}
}
#[test]
fn a_picture_smaller_than_the_tile_is_still_filled_when_the_void_is_open() {
// The merge page's preview is 1600 wide and a few hundred tall —
// shorter than a 512 tile. With no ring the canvas is padded to a
// tile, the padding hole, and the border is still filled.
let (mut rgb, known) = picture(300, 40, 8);
let mut model = Flat {
tile: 64,
seen: Vec::new(),
};
let tiles = fill_border(
&mut rgb,
300,
40,
&known,
&mut model,
test_params(0),
&mut |_, _| {},
)
.unwrap();
assert!(tiles > 0, "no tile fitted a picture shorter than the tile");
for i in 0..300 * 40 {
if !known[i] {
assert!((rgb[i * 3] - 0.5).abs() < 1e-4, "pixel {i}");
}
}
// And the model saw the padding as hole, never as black content.
for (_, k) in &model.seen {
assert_eq!(k.len(), 64 * 64);
}
}
#[test] #[test]
fn the_fine_passes_run_in_bands_after_the_coarse_one() { fn the_fine_passes_run_in_bands_after_the_coarse_one() {
// A 150-tall hole above and below a picture: the coarse pass sees // A 150-tall hole above and below a picture: the coarse pass sees
@@ -663,7 +782,12 @@ mod tests {
500, 500,
&known, &known,
&mut model, &mut model,
test_params(0), Params {
coarse: 4,
band: 96,
mirror_depth: 48,
..test_params(0)
},
&mut |_, _| {}, &mut |_, _| {},
) )
.unwrap(); .unwrap();
@@ -752,7 +876,7 @@ mod tests {
#[test] #[test]
fn the_context_mirrors_the_top_rows_upward() { fn the_context_mirrors_the_top_rows_upward() {
let (rgb, known) = picture(40, 30, 5); let (rgb, known) = picture(40, 30, 5);
let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48); let ctx = MirroredContext::build(&rgb, 40, 30, &known, 48, 64);
let x = RING + 10; let x = RING + 10;
let first = RING + 5; let first = RING + 5;
for k in 1..=4 { for k in 1..=4 {
@@ -774,7 +898,7 @@ mod tests {
for x in 0..40 { for x in 0..40 {
rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]); rgb[(ridge * 40 + x) * 3..(ridge * 40 + x) * 3 + 3].copy_from_slice(&[0.9, 0.1, 0.1]);
} }
let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48); let ctx = MirroredContext::build(&rgb, 40, 400, &known, 48, 64);
let x = RING + 10; let x = RING + 10;
for y in 0..RING + 5 { for y in 0..RING + 5 {
let p = (y * ctx.width + x) * 3; let p = (y * ctx.width + x) * 3;
+1 -1
View File
@@ -9,7 +9,7 @@
//! on every rung, and what they cost is the whole story of whether a fill //! on every rung, and what they cost is the whole story of whether a fill
//! is interactive: 7.4 s a tile under tract, 0.4 s under ONNX Runtime's //! is interactive: 7.4 s a tile under tract, 0.4 s under ONNX Runtime's
//! CPU pool, 23 ms in fp16 and 13 ms in int8 on a laptop's TensorRT //! CPU pool, 23 ms in fp16 and 13 ms in int8 on a laptop's TensorRT
//! (2026-09-19, docs/panorama.md §12). //! (2026-09-19, docs/dev/panorama.md §12).
//! //!
//! The model's contract, from the reference `export_inference_model.py`: //! The model's contract, from the reference `export_inference_model.py`:
//! input `1×4×512×512` float — channel 0 is `mask − 0.5` with 1 where the //! input `1×4×512×512` float — channel 0 is `mask − 0.5` with 1 where the
+2 -2
View File
@@ -4,7 +4,7 @@
//! Apache-2.0 weights (`models/LICENCE.md`), exported at a fixed shape by //! Apache-2.0 weights (`models/LICENCE.md`), exported at a fixed shape by
//! `tools/export-xfeat.sh` and loaded through the same `dr-inference-engine` //! `tools/export-xfeat.sh` and loaded through the same `dr-inference-engine`
//! `dr-segment` and `dr-face` use, so this adds no runtime and no C to the //! `dr-segment` and `dr-face` use, so this adds no runtime and no C to the
//! tree; what runs it is the device's business (docs/inference.md). ~300 ms //! tree; what runs it is the device's business (docs/dev/inference.md). ~300 ms
//! per frame on tract on the reference desktop, ~400 ms on the tablet //! per frame on tract on the reference desktop, ~400 ms on the tablet
//! (S15.2, S15.4). //! (S15.2, S15.4).
@@ -37,7 +37,7 @@ pub struct XFeat {
} }
/// The bytes of both exports compiled into the binary, for whoever compiles /// The bytes of both exports compiled into the binary, for whoever compiles
/// engines ahead of the first request (docs/inference.md §6). /// engines ahead of the first request (docs/dev/inference.md §6).
#[cfg(feature = "embedded-model")] #[cfg(feature = "embedded-model")]
pub fn embedded_model_bytes() -> [&'static [u8]; 2] { pub fn embedded_model_bytes() -> [&'static [u8]; 2] {
[EMBEDDED_LANDSCAPE, EMBEDDED_PORTRAIT] [EMBEDDED_LANDSCAPE, EMBEDDED_PORTRAIT]
+2 -2
View File
@@ -315,7 +315,7 @@ pub struct DetailPass {
/// 52 render pixels at 4K — holds no spatial frequency a quarter-scale /// 52 render pixels at 4K — holds no spatial frequency a quarter-scale
/// grid cannot represent. Computing it at the render size therefore buys /// grid cannot represent. Computing it at the render size therefore buys
/// nothing and costs everything: 105 taps over 8.3 M pixels, twice, which /// nothing and costs everything: 105 taps over 8.3 M pixels, twice, which
/// measured at 34 ms and is where `docs/technical-debt.md` TD-4 came from. /// measured at 34 ms and is where `docs/dev/technical-debt.md` TD-4 came from.
/// At a quarter it is a sixteenth of the pixels at a quarter of the /// At a quarter it is a sixteenth of the pixels at a quarter of the
/// radius, and the result is not an approximation of the full-resolution /// radius, and the result is not an approximation of the full-resolution
/// base — it is the same band-limited function, sampled where it is still /// base — it is the same band-limited function, sampled where it is still
@@ -568,7 +568,7 @@ pub fn compose_detail(
/// photograph the photographer thinks they are sharpening. /// photograph the photographer thinks they are sharpening.
/// ///
/// It also means ARCH §5.2's stage list, which draws spot removal after /// It also means ARCH §5.2's stage list, which draws spot removal after
/// texture and clarity, is not what this does — see `docs/spot-removal.md` /// texture and clarity, is not what this does — see `docs/dev/spot-removal.md`
/// §5.1, which is where the disagreement is written down. /// §5.1, which is where the disagreement is written down.
pub fn compose_detail_with( pub fn compose_detail_with(
ops: &[Box<dyn Operation>], ops: &[Box<dyn Operation>],
+35 -2
View File
@@ -113,7 +113,7 @@ pub struct EditGraph {
/// a sidecar comes to name one stock while the shader draws another. /// a sidecar comes to name one stock while the shader draws another.
film: Option<Film>, film: Option<Film>,
/// TRACES: FR-DEV-8 /// TRACES: FR-DEV-8
/// The repairs (`docs/spot-removal.md`). /// The repairs (`docs/dev/spot-removal.md`).
/// ///
/// Apart from `ops` for the third time and the same reason: a spot is not /// Apart from `ops` for the third time and the same reason: a spot is not
/// a scalar, and a list of them is not a slider. It sits beside the masks /// a scalar, and a list of them is not a slider. It sits beside the masks
@@ -122,7 +122,7 @@ pub struct EditGraph {
/// photograph comes from. /// photograph comes from.
spots: SpotSet, spots: SpotSet,
/// The lens corrections that rewrite coordinates: distortion and lateral /// The lens corrections that rewrite coordinates: distortion and lateral
/// chromatic aberration (`docs/architecture.md` §5.2). /// chromatic aberration (`docs/dev/architecture.md` §5.2).
/// ///
/// Apart from `ops` for the fourth time, and this one is not about shape /// Apart from `ops` for the fourth time, and this one is not about shape
/// but about direction. Every [`Operation`] is a function from colour to /// but about direction. Every [`Operation`] is a function from colour to
@@ -857,6 +857,39 @@ impl EditGraph {
crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view) crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view)
} }
/// TRACES: FR-DEV-3
/// The camera-space tap over one patch of what is on the canvas.
///
/// `patch` is in fractions of the visible region — the coordinates a
/// click on the canvas arrives in — and is laid over this edit's own
/// framing: crop, view, rotation and all, so a fraction of the canvas is
/// a fraction of the probe. Nothing else of the edit: no operation, no
/// mask, no repair. Rendering only the patch is what lets a small target
/// cover every sensor pixel under it rather than sampling one in fifty;
/// see [`crate::operation::compose_camera_probe`] for why the white
/// balance picker reads from here and not from the display.
///
/// The patch is centred where asked and held to the view's own minimum
/// extent: at a deep zoom a patch a fraction of the view would be
/// smaller than a view may be, and letting `set_view` widen it from one
/// corner would move the sample off the point that was clicked.
pub fn compose_camera_probe(&self, patch: crate::framing::CropRect) -> ComposedShader {
use crate::framing::CropRect;
let mut framing = self.framing;
let view = framing.view();
let width = (patch.width * view.width).max(CropRect::MIN_EXTENT);
let height = (patch.height * view.height).max(CropRect::MIN_EXTENT);
let cx = view.x + (patch.x + patch.width * 0.5) * view.width;
let cy = view.y + (patch.y + patch.height * 0.5) * view.height;
framing.set_view(CropRect {
x: (cx - width * 0.5).clamp(0.0, 1.0 - width),
y: (cy - height * 0.5).clamp(0.0, 1.0 - height),
width,
height,
});
crate::operation::compose_camera_probe(&self.warps, &framing)
}
/// TRACES: FR-DEV-19c /// TRACES: FR-DEV-19c
/// [`Self::compose_for`], with one layer's mask drawn over the picture. /// [`Self::compose_for`], with one layer's mask drawn over the picture.
/// ///
+2 -1
View File
@@ -65,7 +65,8 @@ pub use history::{Edit, Entry as HistoryEntry, History, Step};
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp}; pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
pub use operation::{ pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation, compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS, OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
RESERVED_UNIFORM_FIELDS,
}; };
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope}; pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
pub use sidecar::{Sidecar, Version}; pub use sidecar::{Sidecar, Version};
+5 -5
View File
@@ -27,7 +27,7 @@
//! [`MaskSource::Regions`] stores integers naming regions in the segmentation //! [`MaskSource::Regions`] stores integers naming regions in the segmentation
//! hierarchy (`dr-segment`). That choice is what makes a mask diffable, cheap //! hierarchy (`dr-segment`). That choice is what makes a mask diffable, cheap
//! in a sidecar, and mergeable per-field under FR-NC-9 — three properties a //! in a sidecar, and mergeable per-field under FR-NC-9 — three properties a
//! stored raster has none of (docs/segmentation.md §1). Two devices that //! stored raster has none of (docs/dev/segmentation.md §1). Two devices that
//! select the same subject produce the same small sorted list, and a sync //! select the same subject produce the same small sorted list, and a sync
//! conflict between them is resolvable rather than a binary blob fight. //! conflict between them is resolvable rather than a binary blob fight.
//! //!
@@ -564,7 +564,7 @@ pub enum MaskSource {
/// This is what the watershed and the semantic model exist to produce. /// This is what the watershed and the semantic model exist to produce.
/// Selecting a subject means "the regions the model's instance covers", /// Selecting a subject means "the regions the model's instance covers",
/// and the resulting edge is the watershed's, which is to say the image's /// and the resulting edge is the watershed's, which is to say the image's
/// own (docs/segmentation.md §5). /// own (docs/dev/segmentation.md §5).
Regions { Regions {
/// Which segmentation these ids index into. /// Which segmentation these ids index into.
/// ///
@@ -590,7 +590,7 @@ pub enum MaskSource {
/// **The primary way a local adjustment is made.** The watershed hierarchy /// **The primary way a local adjustment is made.** The watershed hierarchy
/// this crate was first built around does not survive a photograph: its /// this crate was first built around does not survive a photograph: its
/// saddles are near zero almost everywhere, so a global cut collapses the /// saddles are near zero almost everywhere, so a global cut collapses the
/// frame into one region plus noise (docs/segmentation.md §15). A model /// frame into one region plus noise (docs/dev/segmentation.md §15). A model
/// instance is a whole object, found as one thing, and needs no ladder. /// instance is a whole object, found as one thing, and needs no ladder.
/// ///
/// The trade is that the boundary is the model's — a quarter-resolution /// The trade is that the boundary is the model's — a quarter-resolution
@@ -625,7 +625,7 @@ pub enum MaskSource {
/// reason both exist. A subject is *one* instance — this dog, not that one /// reason both exist. A subject is *one* instance — this dog, not that one
/// — found by a COCO-trained instance model. A category is *all* the sky, /// — found by a COCO-trained instance model. A category is *all* the sky,
/// or all the foliage, from an ADE20K-trained semantic model that has no /// or all the foliage, from an ADE20K-trained semantic model that has no
/// notion of instances at all (docs/segmentation.md §16). /// notion of instances at all (docs/dev/segmentation.md §16).
/// ///
/// So this is what a global grade attaches to: lift the sky, desaturate /// So this is what a global grade attaches to: lift the sky, desaturate
/// the vegetation, warm the architecture. Asking it for "that person /// the vegetation, warm the architecture. Asking it for "that person
@@ -2231,7 +2231,7 @@ fn reveal_block(slot: usize, layer: &MaskLayer, style: RevealStyle, colour: [f32
/// **not** a hash of the label field: that would be a readback on a path that /// **not** a hash of the label field: that would be a readback on a path that
/// must not have one (ARCH §6.1), and would also make the signature depend on /// must not have one (ARCH §6.1), and would also make the signature depend on
/// float arithmetic whose cross-vendor determinism is exactly the open /// float arithmetic whose cross-vendor determinism is exactly the open
/// question (docs/segmentation.md §6, M5). /// question (docs/dev/segmentation.md §6, M5).
pub fn segmentation_signature(width: u32, height: u32, regions: u32, tuning: u64) -> u64 { pub fn segmentation_signature(width: u32, height: u32, regions: u32, tuning: u64) -> u64 {
// FNV-1a over the four fields. Small, dependency-free, and adequate: this // FNV-1a over the four fields. Small, dependency-free, and adequate: this
// guards against accidental mismatch, not against a forged sidecar. // guards against accidental mismatch, not against a forged sidecar.
+15 -9
View File
@@ -69,20 +69,26 @@ const ROUNDS: u32 = 3;
/// Below this a channel carries no ratio worth balancing. /// Below this a channel carries no ratio worth balancing.
/// ///
/// A sample in the deep shadows, or one taken on a blown highlight where a /// A sample in the deep shadows has no white balance in it: the logarithms
/// channel has already clipped to nothing, has no white balance in it: the /// below would run away and the picker would slam a slider to its stop.
/// logarithms below would run away and the picker would slam a slider to its /// Refusing is the honest answer, and the caller reports that the point was
/// stop. Refusing is the honest answer, and the caller reports that the point /// not usable rather than moving the photograph. The other end — a blown
/// was not usable rather than moving the photograph. /// highlight, where every channel has stopped counting — is refused by the
/// caller before the sample is taken, because only the caller can see the
/// sensor value; see [`crate::operation::CLIP_ONSET`].
const FLOOR: f32 = 1e-4; const FLOOR: f32 = 1e-4;
/// TRACES: FR-DEV-3 /// TRACES: FR-DEV-3
/// Move the graph so that `sample` renders neutral. /// Move the graph so that `sample` renders neutral.
/// ///
/// `sample` is linear RGB, as the operation's own gains multiply it — that is, /// `sample` is the linear triple the operation's own gains multiply — camera
/// measured with the sampling operation at its defaults. Returns whether the /// RGB with the camera's as-shot balance on, *before* the body's base curve
/// graph was moved: `false` where the chain offers no white point widget, or /// and matrix, and with the sampling operation at its defaults. Not the
/// where the colour has no balance in it to correct. /// pixel on the screen: the matrix mixes the channels on the way there, so
/// a colour read after it does not answer to these gains, and a solve over
/// one lands somewhere no sample asked for. Returns whether the graph was
/// moved: `false` where the chain offers no white point widget, or where the
/// colour has no balance in it to correct.
/// ///
/// **Absolute, not relative.** The values written depend on the colour and not /// **Absolute, not relative.** The values written depend on the colour and not
/// on where the sliders happened to be, so sampling the same wall twice lands /// on where the sliders happened to be, so sampling the same wall twice lands
+68 -9
View File
@@ -54,7 +54,7 @@ pub enum Affects {
/// A pixel's *neighbourhood* — sharpening, noise reduction, clarity, /// A pixel's *neighbourhood* — sharpening, noise reduction, clarity,
/// texture, dehaze, spot removal. /// texture, dehaze, spot removal.
/// ///
/// The seam `docs/requirements.md` §3.3 designed and nothing cut until /// The seam `docs/dev/requirements.md` §3.3 designed and nothing cut until
/// [`crate::detail`] existed. It is a separate variant rather than a flavour /// [`crate::detail`] existed. It is a separate variant rather than a flavour
/// of `Colour` because it is a separate *dispatch*: a fragment in the fused /// of `Colour` because it is a separate *dispatch*: a fragment in the fused
/// pass is handed a colour and has no way back to a coordinate, so a /// pass is handed a colour and has no way back to a coordinate, so a
@@ -431,9 +431,11 @@ pub enum OutputMode {
/// no operations, and the caller fills the reserved uniforms neutral — /// no operations, and the caller fills the reserved uniforms neutral —
/// unit white balance, identity matrix, base curve off — so what is /// unit white balance, identity matrix, base curve off — so what is
/// stored is the sensor's own numbers, demosaiced and undistorted. Only /// stored is the sensor's own numbers, demosaiced and undistorted. Only
/// [`compose_camera_linear`] produces it, and only /// [`compose_camera_probe`] produces it — for a merge through
/// `AdjustPass::render_camera_linear` accepts it, so the neutral /// [`compose_camera_linear`], and for the white balance picker under the
/// uniforms cannot be forgotten by a caller that composed it by mistake. /// edit's own framing — and only `AdjustPass::render_camera_linear`
/// accepts it, so the neutral uniforms cannot be forgotten by a caller
/// that composed it by mistake.
/// ///
/// Thirty-two bits rather than sixteen because the composite is written /// Thirty-two bits rather than sixteen because the composite is written
/// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has /// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has
@@ -490,6 +492,19 @@ pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
/// selection in [`compose_full`]. /// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5; pub const BASE_CURVE_POINTS: usize = 5;
/// Where the highlight desaturation begins: the fraction of the white level
/// above which a photosite is treated as clipped.
///
/// A photosite this close to saturation has stopped counting, so its ratio
/// to its neighbours is not a colour. The generated prologue fades a pixel
/// above this toward a neutral of the same brightness before any operation
/// runs, and the white balance probe refuses to sample one: a blown sky is
/// sensor white, which the as-shot multipliers make magenta, and a solve
/// over that slams tint to its stop. One number, so the two cannot drift
/// apart — a probe that accepted what the shader had already desaturated
/// would be balancing against a pixel the photographer cannot see.
pub const CLIP_ONSET: f32 = 0.985;
/// Where an operation's own uniforms begin in the generated block. /// Where an operation's own uniforms begin in the generated block.
/// ///
/// The base fields, then framing's. Exported because `dr-gpu` writes the /// The base fields, then framing's. Exported because `dr-gpu` writes the
@@ -589,7 +604,9 @@ pub fn compose_full_revealing(
warps: &[Box<dyn crate::lens::Warp>], warps: &[Box<dyn crate::lens::Warp>],
reveal: Option<&crate::mask::Reveal>, reveal: Option<&crate::mask::Reveal>,
) -> ComposedShader { ) -> ComposedShader {
compose_inner(ops, framing, output, masks, spots, warps, reveal, None) compose_inner(
ops, framing, output, masks, spots, warps, reveal, None, false,
)
} }
/// TRACES: FR-MRG-2 /// TRACES: FR-MRG-2
@@ -614,15 +631,50 @@ pub fn compose_camera_linear(
// upright too, and `view` is a fraction of the upright frame. // upright too, and `view` is a fraction of the upright frame.
framing.set_baseline(baseline); framing.set_baseline(baseline);
framing.set_view(view); framing.set_view(view);
compose_camera_tap(warps, &framing, false)
}
/// TRACES: FR-DEV-3
/// The same tap under a framing the caller chose: the white balance probe.
///
/// A neutral picked off the canvas has to be measured in the space the
/// white balance gains multiply, and that is camera RGB — the operation
/// runs before the body's matrix, and a probe read after the matrix would
/// be solving the wrong equation on any body whose matrix mixes the
/// channels, which is every body. It also has to be measured at the pixel
/// the canvas is showing, which is why this takes the edit's own framing
/// where a merge passes the file's orientation and a tile.
///
/// **Interpolated whatever the framing says.** The point of rendering a
/// patch is to average what is under it, and the nearest sampling an
/// unrotated frame otherwise gets is a comb: at two source pixels per
/// probe pixel it lands on the same column of any pattern every time, and
/// the average of a thousand samples is then the average of nothing.
pub fn compose_camera_probe(
warps: &[Box<dyn crate::lens::Warp>],
framing: &Framing,
) -> ComposedShader {
compose_camera_tap(warps, framing, true)
}
/// The camera-space tap proper: no operations, `rgba32float`, and the
/// profile uniforms left for the GPU side to fill neutral. `smooth` forces
/// the interpolating sampler; see the two callers for who wants it and why.
fn compose_camera_tap(
warps: &[Box<dyn crate::lens::Warp>],
framing: &Framing,
smooth: bool,
) -> ComposedShader {
compose_inner( compose_inner(
&[], &[],
&framing, framing,
ColourSpace::Srgb, ColourSpace::Srgb,
&MaskStack::new(), &MaskStack::new(),
&crate::spot::SpotSet::new(), &crate::spot::SpotSet::new(),
warps, warps,
None, None,
Some(OutputMode::CameraLinear), Some(OutputMode::CameraLinear),
smooth,
) )
} }
@@ -636,6 +688,7 @@ fn compose_inner(
warps: &[Box<dyn crate::lens::Warp>], warps: &[Box<dyn crate::lens::Warp>],
reveal: Option<&crate::mask::Reveal>, reveal: Option<&crate::mask::Reveal>,
forced: Option<OutputMode>, forced: Option<OutputMode>,
smooth: bool,
) -> ComposedShader { ) -> ComposedShader {
// The lens corrections, composed into one coordinate transform. Beside // The lens corrections, composed into one coordinate transform. Beside
// `framing` because they are the other half of the same stage: framing // `framing` because they are the other half of the same stage: framing
@@ -668,7 +721,7 @@ fn compose_inner(
// twice and be bound to a texture of the wrong format. // twice and be bound to a texture of the wrong format.
// //
// `forced` is the one exception, and it is not a caller flag in the // `forced` is the one exception, and it is not a caller flag in the
// sense above: `compose_camera_linear` is the only function that passes // sense above: `compose_camera_probe` is the only function that passes
// it, with an empty operation list, and the mode it forces has its own // it, with an empty operation list, and the mode it forces has its own
// storage format and its own render entry on the GPU side. // storage format and its own render entry on the GPU side.
let output_mode = forced.unwrap_or( let output_mode = forced.unwrap_or(
@@ -842,7 +895,10 @@ fn compose_inner(
// framing alone — which is what this did before the warps existed — would // framing alone — which is what this did before the warps existed — would
// have nearest-neighboured a distortion correction on an unstraightened // have nearest-neighboured a distortion correction on an unstraightened
// frame, and the aliasing would have looked like a bad profile. // frame, and the aliasing would have looked like a bad profile.
let interpolate = framing.needs_interpolation() || warp.is_active(); // `smooth` is the third reason, and the only one a caller states: the
// white balance probe averages a patch and cannot do that through a
// nearest-neighbour comb (see `compose_camera_probe`).
let interpolate = smooth || framing.needs_interpolation() || warp.is_active();
// Declared ahead of the warp block, which assigns to them. They enter // Declared ahead of the warp block, which assigns to them. They enter
// equal to `p` so that a chain mixing a splitting warp with a // equal to `p` so that a chain mixing a splitting warp with a
@@ -997,6 +1053,9 @@ fn compose_inner(
.to_string() .to_string()
}; };
// Formatted with Rust's `Display` so the shader reads the same threshold
// the probe checks against; see `CLIP_ONSET`.
let clip_onset = CLIP_ONSET;
let source = format!( let source = format!(
"// GENERATED — do not edit. "// GENERATED — do not edit.
// //
@@ -1067,7 +1126,7 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
// A photosite at its white level carries no colour information — every // A photosite at its white level carries no colour information — every
// channel simply stopped counting — so the balance below must not be // channel simply stopped counting — so the balance below must not be
// allowed to tint it. // allowed to tint it.
let clipped = smoothstep(0.985, 1.0, max(c.r, max(c.g, c.b))); let clipped = smoothstep({clip_onset}, 1.0, max(c.r, max(c.g, c.b)));
c = c * u.as_shot_wb.rgb; c = c * u.as_shot_wb.rgb;
+1 -1
View File
@@ -99,7 +99,7 @@
//! A minimum over a patch is separable, as a Gaussian is: minimum along x, //! A minimum over a patch is separable, as a Gaussian is: minimum along x,
//! then along y. That alone is not enough. The patch is 1% of the shorter edge //! then along y. That alone is not enough. The patch is 1% of the shorter edge
//! — 61 taps across at 4K — and two passes of 61 taps is the arithmetic that //! — 61 taps across at 4K — and two passes of 61 taps is the arithmetic that
//! measured 34 ms for clarity and became `docs/technical-debt.md` TD-4. //! measured 34 ms for clarity and became `docs/dev/technical-debt.md` TD-4.
//! //!
//! A minimum has a property a Gaussian does not: **erosions compose by adding //! A minimum has a property a Gaussian does not: **erosions compose by adding
//! their structuring elements**. The minimum over a contiguous run of `d` //! their structuring elements**. The minimum over a contiguous run of `d`
+2 -2
View File
@@ -150,7 +150,7 @@
//! the artefact this control must not have. //! the artefact this control must not have.
//! //!
//! Run at the render size, that measured **34 ms at 4K** — seven times the //! Run at the render size, that measured **34 ms at 4K** — seven times the
//! entire fused point chain, for one slider — which is `docs/technical-debt.md` //! entire fused point chain, for one slider — which is `docs/dev/technical-debt.md`
//! TD-4 and is what [`Recipe::base_scale`] now answers. The base is computed on //! TD-4 and is what [`Recipe::base_scale`] now answers. The base is computed on
//! a grid a quarter the size on each axis: a sixteenth of the pixels at a //! a grid a quarter the size on each axis: a sixteenth of the pixels at a
//! quarter of the radius. //! quarter of the radius.
@@ -271,7 +271,7 @@ impl Band for Coarse {
threshold: 0.35, threshold: 0.35,
gain: 1.0, gain: 1.0,
midtone_taper: true, midtone_taper: true,
// A quarter, which is what `docs/technical-debt.md` TD-4 bought back. // A quarter, which is what `docs/dev/technical-debt.md` TD-4 bought back.
// //
// σ is 1.2% of the shorter edge — 26 px at 4K — so the base holds no // σ is 1.2% of the shorter edge — 26 px at 4K — so the base holds no
// spatial frequency anywhere near the quarter-scale Nyquist of one // spatial frequency anywhere near the quarter-scale Nyquist of one
+1 -1
View File
@@ -217,7 +217,7 @@ pub struct Version {
/// graph's film cleared and the caller re-bakes — see `EditGraph::set_film`. /// graph's film cleared and the caller re-bakes — see `EditGraph::set_film`.
pub film: Option<FilmRef>, pub film: Option<FilmRef>,
/// TRACES: FR-DEV-8 | FR-NC-9 /// TRACES: FR-DEV-8 | FR-NC-9
/// The repairs (`docs/spot-removal.md`). /// The repairs (`docs/dev/spot-removal.md`).
/// ///
/// A line per spot, keyed `spot.<id>`, rather than a block per spot as a /// A line per spot, keyed `spot.<id>`, rather than a block per spot as a
/// mask gets: a spot is eight numbers, and sixty-four blocks would bury the /// mask gets: a spot is eight numbers, and sixty-four blocks would bury the
+3 -3
View File
@@ -6,7 +6,7 @@
//! are blended. No pixels are stored, here or anywhere: the shader draws the //! are blended. No pixels are stored, here or anywhere: the shader draws the
//! repair from these numbers every time the photograph is rendered, which is //! repair from these numbers every time the photograph is rendered, which is
//! what makes it non-destructive, cheap to sync, and undoable //! what makes it non-destructive, cheap to sync, and undoable
//! (`docs/spot-removal.md`). //! (`docs/dev/spot-removal.md`).
//! //!
//! # Why this is not an operation //! # Why this is not an operation
//! //!
@@ -61,7 +61,7 @@ pub const MAX_SPOTS: usize = 64;
/// bounds something that is otherwise unbounded: a detail pass declares how far /// bounds something that is otherwise unbounded: a detail pass declares how far
/// it reads from the pixel it writes, and for a spot that is the offset plus /// it reads from the pixel it writes, and for a spot that is the offset plus
/// the radius. An unbounded offset is an unbounded halo, which is a pass the /// the radius. An unbounded offset is an unbounded halo, which is a pass the
/// tile scheduler cannot plan (ARCH §5.3, `docs/spot-removal.md` §5.3). /// tile scheduler cannot plan (ARCH §5.3, `docs/dev/spot-removal.md` §5.3).
pub const MAX_SOURCE_DISTANCE: f32 = 0.5; pub const MAX_SOURCE_DISTANCE: f32 = 0.5;
/// The radius a new spot starts at, in frame units. /// The radius a new spot starts at, in frame units.
@@ -210,7 +210,7 @@ impl Spot {
/// **FR-DEV-8 asks for automatic source placement, and this is the cheap /// **FR-DEV-8 asks for automatic source placement, and this is the cheap
/// half of it.** The good half searches the photograph for a patch whose /// half of it.** The good half searches the photograph for a patch whose
/// surroundings match — a compute dispatch scoring candidate offsets, and /// surroundings match — a compute dispatch scoring candidate offsets, and
/// one small readback when the spot is created (`docs/spot-removal.md` /// one small readback when the spot is created (`docs/dev/spot-removal.md`
/// §8). This is what stands in for it, and it is worth having on its own /// §8). This is what stands in for it, and it is worth having on its own
/// terms rather than as a placeholder: dust sits on skies, skies are /// terms rather than as a placeholder: dust sits on skies, skies are
/// smooth, and a patch two and a half radii away is nearly always the same /// smooth, and a patch two and a half radii away is nearly always the same
+1 -1
View File
@@ -13,7 +13,7 @@ log.workspace = true
# Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s # Inference. `ort` is the API; **what runs it is `dr-inference-engine`'s
# business** — tract, or an ONNX Runtime the app found on disk, on whichever # business** — tract, or an ONNX Runtime the app found on disk, on whichever
# provider the device has (docs/inference.md). This crate never names either. # provider the device has (docs/dev/inference.md). This crate never names either.
ort = { workspace = true, optional = true } ort = { workspace = true, optional = true }
dr-inference-engine = { workspace = true, optional = true } dr-inference-engine = { workspace = true, optional = true }
ndarray = { workspace = true, optional = true } ndarray = { workspace = true, optional = true }
+1 -1
View File
@@ -34,7 +34,7 @@
//! And doing it here buys two things a shader could not. It is **exactly //! And doing it here buys two things a shader could not. It is **exactly
//! deterministic**, which matters because masks reach the sidecar as indices //! deterministic**, which matters because masks reach the sidecar as indices
//! and a field that varied by vendor would mean a mask meaning one thing on //! and a field that varied by vendor would mean a mask meaning one thing on
//! the desktop and another on the phone (docs/segmentation.md §6, M5). And it //! the desktop and another on the phone (docs/dev/segmentation.md §6, M5). And it
//! is testable against hand-computed distances with no adapter present. //! is testable against hand-computed distances with no adapter present.
//! //!
//! # The transform //! # The transform
+1 -1
View File
@@ -40,7 +40,7 @@ pub struct Edge {
/// A partition of the image into labelled regions, plus how they adjoin. /// A partition of the image into labelled regions, plus how they adjoin.
/// ///
/// The shared interface from docs/segmentation.md §2: arm A produces this /// The shared interface from docs/dev/segmentation.md §2: arm A produces this
/// from a watershed, arm B would produce it from a class map, and the /// from a watershed, arm B would produce it from a class map, and the
/// consumers above cannot tell which. /// consumers above cannot tell which.
#[derive(Debug, Clone, PartialEq)] #[derive(Debug, Clone, PartialEq)]
+1 -1
View File
@@ -1,5 +1,5 @@
//! TRACES: FR-DEV-3i //! TRACES: FR-DEV-3i
//! Region segmentation for local masking (S15, docs/segmentation.md). //! Region segmentation for local masking (S15, docs/dev/segmentation.md).
//! //!
//! Local adjustments need to know where the image's regions are before they //! Local adjustments need to know where the image's regions are before they
//! can snap a mask to one. This crate is that map, and it is deliberately //! can snap a mask to one. This crate is that map, and it is deliberately
+2 -2
View File
@@ -1,6 +1,6 @@
//! Arm C — semantic instances as a prior over the watershed merge order. //! Arm C — semantic instances as a prior over the watershed merge order.
//! //!
//! docs/segmentation.md §5. The spec calls this the expected winner and it is //! docs/dev/segmentation.md §5. The spec calls this the expected winner and it is
//! what ships, for a reason that survives the model turning out to be narrower //! what ships, for a reason that survives the model turning out to be narrower
//! than §4 assumed: the two arms fail in *opposite* directions, so each one //! than §4 assumed: the two arms fail in *opposite* directions, so each one
//! covers the other's failure. //! covers the other's failure.
@@ -233,7 +233,7 @@ pub fn apply_semantic_prior(
/// This is the interaction the whole spike exists to enable, and the reason it /// This is the interaction the whole spike exists to enable, and the reason it
/// returns *region ids* rather than a raster: a mask that is a set of integers /// returns *region ids* rather than a raster: a mask that is a set of integers
/// is diffable, mergeable at node level under FR-NC-9, and cheap in a sidecar /// is diffable, mergeable at node level under FR-NC-9, and cheap in a sidecar
/// (docs/segmentation.md §1). A raster is none of those. /// (docs/dev/segmentation.md §1). A raster is none of those.
/// ///
/// The returned ids are sorted, so the same click always produces the same /// The returned ids are sorted, so the same click always produces the same
/// mask — which is what lets it be a cache key. /// mask — which is what lets it be a cache key.
+5 -5
View File
@@ -60,7 +60,7 @@
//! So the last step is a **marker-based watershed**. The mask is eroded to //! So the last step is a **marker-based watershed**. The mask is eroded to
//! give two markers — confidently inside, confidently outside — and the flood //! give two markers — confidently inside, confidently outside — and the flood
//! runs in the ribbon left between them, meeting along the most expensive line //! runs in the ribbon left between them, meeting along the most expensive line
//! it can find. The cost is a sum of terms, as docs/segmentation.md §2 says it //! it can find. The cost is a sum of terms, as docs/dev/segmentation.md §2 says it
//! should be: the photograph's own edges, and the colour model's disagreement. //! should be: the photograph's own edges, and the colour model's disagreement.
//! //!
//! Markers are what make this the right shape rather than the watershed §15 //! Markers are what make this the right shape rather than the watershed §15
@@ -244,7 +244,7 @@ pub struct RefineOptions {
/// How much the photograph's own edges count against the colour model in /// How much the photograph's own edges count against the colour model in
/// the flood's cost, `0.0..=1.0`. /// the flood's cost, `0.0..=1.0`.
/// ///
/// docs/segmentation.md §2 specifies the cost as *a sum of terms* — image /// docs/dev/segmentation.md §2 specifies the cost as *a sum of terms* — image
/// gradient always available, semantic evidence added when a model is /// gradient always available, semantic evidence added when a model is
/// present — and this is the mix. At one the boundary lands purely on the /// present — and this is the mix. At one the boundary lands purely on the
/// strongest edge in the band; at zero purely where the colour verdict /// strongest edge in the band; at zero purely where the colour verdict
@@ -378,7 +378,7 @@ const MAX_SAMPLES: usize = 20_000;
/// floating-point comparison is a stopping rule that can differ between /// floating-point comparison is a stopping rule that can differ between
/// machines, and a mask that differs between machines reaches the sidecar as /// machines, and a mask that differs between machines reaches the sidecar as
/// indices meaning one thing on the desktop and another on the phone /// indices meaning one thing on the desktop and another on the phone
/// (docs/segmentation.md §6). /// (docs/dev/segmentation.md §6).
const ITERATIONS: usize = 12; const ITERATIONS: usize = 12;
/// Half-width of the verdict scale, in nats. /// Half-width of the verdict scale, in nats.
@@ -676,7 +676,7 @@ impl Refinement {
/// fronts meet along the most expensive line in the ribbon — which is the /// fronts meet along the most expensive line in the ribbon — which is the
/// watershed, and which is where the boundary belongs. /// watershed, and which is where the boundary belongs.
/// ///
/// The cost is a sum of terms, as docs/segmentation.md §2 says it should /// The cost is a sum of terms, as docs/dev/segmentation.md §2 says it should
/// be: the photograph's own edges, and the colour model's disagreement. /// be: the photograph's own edges, and the colour model's disagreement.
/// Neither alone is right. An edge with no colour meaning is a texture, /// Neither alone is right. An edge with no colour meaning is a texture,
/// and a colour change with no edge is a gradient. /// and a colour change with no edge is a gradient.
@@ -889,7 +889,7 @@ fn neighbours(p: usize, w: usize, h: usize) -> impl Iterator<Item = usize> {
/// Edge strength over the opponent features, as one byte per pixel. /// Edge strength over the opponent features, as one byte per pixel.
/// ///
/// Sobel over the same three numbers the colour model is fitted on, rather /// Sobel over the same three numbers the colour model is fitted on, rather
/// than over plain luma — docs/segmentation.md §3 is explicit that a /// than over plain luma — docs/dev/segmentation.md §3 is explicit that a
/// channel-weighted RGB gradient reads a saturated red edge as weaker than it /// channel-weighted RGB gradient reads a saturated red edge as weaker than it
/// looks, and a flag against sky is exactly that edge. /// looks, and a flag against sky is exactly that edge.
/// ///
+3 -3
View File
@@ -1,4 +1,4 @@
//! Semantic segmentation — arm B (S15, docs/segmentation.md §4). //! Semantic segmentation — arm B (S15, docs/dev/segmentation.md §4).
//! //!
//! Runs a YOLO instance-segmentation graph over a proxy-resolution image and //! Runs a YOLO instance-segmentation graph over a proxy-resolution image and
//! returns the instances it found: a class, a score, a box, and a soft mask //! returns the instances it found: a class, a score, a box, and a soft mask
@@ -209,7 +209,7 @@ const EMBEDDED_MODEL: &[u8] = include_bytes!("../../../models/segment/yolo26n-se
const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json"); const EMBEDDED_CLASSES: &str = include_str!("../../../models/segment/yolo26n-seg.classes.json");
/// The bytes of the model that ships with this crate, for whoever compiles /// The bytes of the model that ships with this crate, for whoever compiles
/// engines ahead of the first request (docs/inference.md §6). /// engines ahead of the first request (docs/dev/inference.md §6).
#[cfg(feature = "embedded-model")] #[cfg(feature = "embedded-model")]
pub fn embedded_model_bytes() -> &'static [u8] { pub fn embedded_model_bytes() -> &'static [u8] {
EMBEDDED_MODEL EMBEDDED_MODEL
@@ -237,7 +237,7 @@ impl SemanticModel {
pub fn from_bytes(bytes: &[u8], classes: Vec<Arc<str>>) -> Result<Self, SegmentError> { pub fn from_bytes(bytes: &[u8], classes: Vec<Arc<str>>) -> Result<Self, SegmentError> {
// The f32 graph on whatever the device's backend is. An int8 form // The f32 graph on whatever the device's backend is. An int8 form
// for the Hexagon waits on docs/inference.md §10 M7 — the mask // for the Hexagon waits on docs/dev/inference.md §10 M7 — the mask
// boundary has to be measured before it moves. // boundary has to be measured before it moves.
let session = dr_inference_engine::open( let session = dr_inference_engine::open(
dr_inference_engine::Role::Segmenter, dr_inference_engine::Role::Segmenter,
+30 -3
View File
@@ -47,6 +47,20 @@ pub struct NextcloudBackend {
/// `/remote.php/dav/files/<user>/` — the prefix stripped from hrefs. /// `/remote.php/dav/files/<user>/` — the prefix stripped from hrefs.
dav_base: String, dav_base: String,
caps: Capabilities, caps: Capabilities,
/// Collections this backend has seen exist, so [`create_dir`] asks the
/// server about each one once.
///
/// Every `MOVE` guarantees its destination's parent, and did so with a
/// `MKCOL` for each ancestor down from the account root — for a trash
/// folder three levels deep that was three round trips of `405 Method
/// Not Allowed` before the one request that moved anything, on every
/// image of a batch. A backend lives for one job (`dr_ui::remote::
/// connect` builds one per worker), so a folder deleted by another
/// client mid-job is the one case this can get wrong, and it is reported
/// as the `409` the `MOVE` then earns rather than hidden.
///
/// [`create_dir`]: RemoteBackend::create_dir
known_dirs: std::sync::Mutex<std::collections::HashSet<String>>,
} }
impl NextcloudBackend { impl NextcloudBackend {
@@ -63,6 +77,7 @@ impl NextcloudBackend {
login: creds.login_name.clone(), login: creds.login_name.clone(),
password: creds.app_password.clone(), password: creds.app_password.clone(),
dav_base, dav_base,
known_dirs: Default::default(),
caps: Capabilities { caps: Capabilities {
// The property that makes a no-op sync one request (ARCH §8.1). // The property that makes a no-op sync one request (ARCH §8.1).
change_detection: ChangeDetection::PropagatingEtags, change_detection: ChangeDetection::PropagatingEtags,
@@ -567,6 +582,16 @@ impl RemoteBackend for NextcloudBackend {
chain.reverse(); chain.reverse();
for dir in chain { for dir in chain {
// Asked once per backend — see `known_dirs`. The lock is held
// across no await: it is taken to look, and again to record.
let known = self
.known_dirs
.lock()
.map(|k| k.contains(dir.as_str()))
.unwrap_or(false);
if known {
continue;
}
let url = self.url_for(&dir); let url = self.url_for(&dir);
let resp = self let resp = self
.client .client
@@ -581,10 +606,12 @@ impl RemoteBackend for NextcloudBackend {
// 405 is "already a collection here", which is exactly what the // 405 is "already a collection here", which is exactly what the
// caller wanted. Anything else is reported. // caller wanted. Anything else is reported.
if resp.status() == reqwest::StatusCode::METHOD_NOT_ALLOWED { if resp.status() != reqwest::StatusCode::METHOD_NOT_ALLOWED {
continue; map_status(resp.status(), &url)?;
}
if let Ok(mut k) = self.known_dirs.lock() {
k.insert(dir.as_str().to_string());
} }
map_status(resp.status(), &url)?;
} }
Ok(()) Ok(())
} }
+14
View File
@@ -127,6 +127,19 @@ pub struct Account {
#[serde(default)] #[serde(default)]
pub root: String, pub root: String,
/// Whether [`root`](Self::root) has been chosen at all.
///
/// An empty `root` is two different things: nothing picked yet, and the
/// endpoint itself picked on purpose — a user who keeps everything at
/// the top level, or a folder library, which is its own root. The string
/// cannot tell them apart, and reading empty as "not chosen" meant the
/// top level could be confirmed in the picker and still not open. So the
/// fact is recorded separately. Defaulted, so an account written before
/// it existed loads as it always did: a non-empty root is chosen by
/// virtue of being there, and an empty one asks again.
#[serde(default)]
pub root_chosen: bool,
/// Which formats the scan looks for (the tick-boxes). /// Which formats the scan looks for (the tick-boxes).
#[serde(default)] #[serde(default)]
pub formats: Vec<String>, pub formats: Vec<String>,
@@ -149,6 +162,7 @@ impl Account {
login: String::new(), login: String::new(),
user_id: String::new(), user_id: String::new(),
root: String::new(), root: String::new(),
root_chosen: false,
formats: Vec::new(), formats: Vec::new(),
last_scan: None, last_scan: None,
} }
+80
View File
@@ -240,6 +240,23 @@ impl ThumbStore {
.collect() .collect()
} }
/// Every file id stored at one size, read from the index in one query.
///
/// For a pass that asks about thousands of images at once — the face
/// audit, the repair lists. Each [`contains`](Self::contains) is a
/// prepared statement and a b-tree probe; asked ten thousand times over a
/// scan it costs more than the scan does, where one walk of the index is
/// a few milliseconds and answers every row.
pub fn held(&self, size: ThumbSize) -> Result<std::collections::HashSet<u64>, ThumbError> {
let mut stmt = self
.index
.prepare("SELECT file_id FROM entries WHERE size = ?1")?;
let ids = stmt
.query_map([size as i64], |r| r.get::<_, i64>(0))?
.collect::<Result<Vec<_>, _>>()?;
Ok(ids.into_iter().map(|id| id as u64).collect())
}
/// Store a thumbnail, opening a new shard if the active one is full. /// Store a thumbnail, opening a new shard if the active one is full.
/// ///
/// Re-storing an existing id overwrites in place rather than migrating it /// Re-storing an existing id overwrites in place rather than migrating it
@@ -481,6 +498,30 @@ impl ThumbStore {
self.dir.join(format!("shard-{shard:04}.sqlite")) self.dir.join(format!("shard-{shard:04}.sqlite"))
} }
/// Write a coherent copy of one shard to `dest`, ready to upload.
///
/// Not a file copy. Every shard is in WAL mode and every `put` opens its
/// own connection, so while thumbnails are being generated on several
/// threads at once — which is exactly when the first sync pass runs —
/// there is nearly always a connection open and the log is never
/// checkpointed. The main file then holds whatever the *last* quiet
/// moment left in it, which for a shard created seconds ago is nothing:
/// zero bytes, the schema still in the log. Reading it uploaded an empty
/// file, and every other device merging it failed with "no such table:
/// thumbs". The backup API serialises against writers and copies the
/// database as it is, log included.
pub fn snapshot_shard(&self, shard: u32, dest: &Path) -> Result<(), ThumbError> {
let source = self.open_shard(shard, false)?;
let _ = std::fs::remove_file(dest);
let mut out = Connection::open(dest)?;
let backup = rusqlite::backup::Backup::new(&source, &mut out)?;
// rusqlite asserts a positive page count where SQLite would take -1
// for "everything"; a shard is capped well under this many pages.
backup.run_to_completion(i32::MAX, std::time::Duration::ZERO, None)?;
drop(backup);
Ok(())
}
pub fn index_path(&self) -> PathBuf { pub fn index_path(&self) -> PathBuf {
self.dir.join("index.sqlite") self.dir.join("index.sqlite")
} }
@@ -726,6 +767,45 @@ mod tests {
} }
} }
#[test]
fn a_snapshot_carries_what_the_shard_file_does_not_yet() {
// A thumbnail worker holding the shard open keeps the log from being
// checkpointed; the file on disk is then not the database. The
// upload used to read that file.
let (mut s, _d) = store();
s.put(1, ThumbSize::Grid, &thumb(1024)).unwrap();
let path = s.shard_path(0);
let worker = Connection::open(&path).unwrap();
worker.pragma_update(None, "journal_mode", "WAL").unwrap();
s.put(2, ThumbSize::Grid, &thumb(2048)).unwrap();
s.put(3, ThumbSize::Grid, &thumb(2048)).unwrap();
// What a byte-for-byte reader sees is at most what the last
// checkpoint left; the log is the part a copy misses.
let copy = _d.join("copied.sqlite");
std::fs::copy(&path, &copy).unwrap();
let file_rows = Connection::open(&copy)
.ok()
.and_then(|c| {
c.query_row("SELECT count(*) FROM thumbs", [], |r| r.get::<_, i64>(0))
.ok()
})
.unwrap_or(0);
let snap = _d.join("upload.sqlite");
s.snapshot_shard(0, &snap).unwrap();
let snapshot = Connection::open(&snap).unwrap();
let rows: i64 = snapshot
.query_row("SELECT count(*) FROM thumbs", [], |r| r.get(0))
.unwrap();
assert_eq!(rows, 3, "the snapshot is the whole shard");
assert!(
file_rows < 3,
"the file alone lagged ({file_rows} rows), which is what the snapshot exists for"
);
drop(worker);
}
#[test] #[test]
fn a_forgotten_thumbnail_is_no_longer_served() { fn a_forgotten_thumbnail_is_no_longer_served() {
// The point of the whole method: a purged photograph must not keep a // The point of the whole method: a purged photograph must not keep a
+4
View File
@@ -115,6 +115,10 @@ pub enum PlaceScope {
#[serde(default)] #[serde(default)]
pub struct StoredFilter { pub struct StoredFilter {
pub min_rating: u8, pub min_rating: u8,
/// TRACES: FR-UI-5
/// The top of a star range. A record from before ranges existed has none,
/// which reads as no ceiling — what it meant when it was written.
pub max_rating: Option<u8>,
pub unjudged: bool, pub unjudged: bool,
pub flag: Option<FlagState>, pub flag: Option<FlagState>,
pub local_only: bool, pub local_only: bool,
+12 -6
View File
@@ -177,7 +177,7 @@ pub struct FaceSettings {
/// Which SCRFD graph the indexing pass detects with. /// Which SCRFD graph the indexing pass detects with.
/// ///
/// Three exports of one architecture, differing only in how much computation /// Three exports of one architecture, differing only in how much computation
/// they spend, and docs/faces.md §12.3 is the measurement that made this a /// they spend, and docs/dev/faces.md §12.3 is the measurement that made this a
/// choice rather than a constant: over the same photographs the cheapest one /// choice rather than a constant: over the same photographs the cheapest one
/// misses the small faces in a group and reports a dog a dozen times, the /// misses the small faces in a group and reports a dog a dozen times, the
/// middle one finds 14% more faces for 12% more time, and the largest a /// middle one finds 14% more faces for 12% more time, and the largest a
@@ -261,7 +261,7 @@ impl FaceDetector {
} }
} }
/// The id when the detector runs in its int8 form (docs/inference.md §7). /// The id when the detector runs in its int8 form (docs/dev/inference.md §7).
/// ///
/// A different detector: it finds a different set of faces, so it is a /// A different detector: it finds a different set of faces, so it is a
/// different population of detections. The embedder half is unchanged, /// different population of detections. The embedder half is unchanged,
@@ -885,9 +885,14 @@ impl ExportSettings {
/// ///
/// The library root has to read as a place rather than as a blank field, /// The library root has to read as a place rather than as a blank field,
/// or confirming the picker where it opens looks like it did nothing. /// or confirming the picker where it opens looks like it did nothing.
///
/// An empty device folder used to read "Ask each time", which nothing
/// does: no platform asks, and an export made with the field blank is
/// refused with "no export folder is set". The label now says what will
/// happen rather than what was once meant to.
pub fn destination_label(&self) -> &str { pub fn destination_label(&self) -> &str {
match self.target { match self.target {
ExportTarget::Device if self.destination.trim().is_empty() => "Ask each time", ExportTarget::Device if self.destination.trim().is_empty() => "Not set",
ExportTarget::Device => &self.destination, ExportTarget::Device => &self.destination,
ExportTarget::Remote if self.remote_destination.trim().is_empty() => "Library root", ExportTarget::Remote if self.remote_destination.trim().is_empty() => "Library root",
ExportTarget::Remote => &self.remote_destination, ExportTarget::Remote => &self.remote_destination,
@@ -1621,14 +1626,15 @@ mod tests {
} }
#[test] #[test]
fn an_empty_device_destination_still_means_ask() { fn an_empty_device_destination_is_not_set_and_says_so() {
// The asymmetry is deliberate: a filesystem has no folder worth // The asymmetry is deliberate: a filesystem has no folder worth
// assuming, so empty there is a question rather than an answer. // assuming, so empty there is a gap rather than an answer — and the
// label must not promise a question nobody will be asked.
let mut s = Settings::default(); let mut s = Settings::default();
s.export.target = ExportTarget::Device; s.export.target = ExportTarget::Device;
s.export.destination = String::new(); s.export.destination = String::new();
assert!(!s.export.destination_is_set()); assert!(!s.export.destination_is_set());
assert_eq!(s.export.destination_label(), "Ask each time"); assert_eq!(s.export.destination_label(), "Not set");
} }
#[test] #[test]
+1 -1
View File
@@ -48,7 +48,7 @@ pointers there; the build detects that and stops rather than shipping them.
This exists because Android offers no other route to a model: the directory the app reads from is This exists because Android offers no other route to a model: the directory the app reads from is
inside app-private storage, `run-as` needs a debuggable build, and the app has no picker and no inside app-private storage, `run-as` needs a debuggable build, and the app has no picker and no
fetch. docs/faces.md §2.2a is the decision and its limits — these files come back out before anything fetch. docs/dev/faces.md §2.2a is the decision and its limits — these files come back out before anything
is published. is published.
## Java in the APK ## Java in the APK
+2 -2
View File
@@ -258,7 +258,7 @@ fi
cp "${SO}" "${OUT}/staging/lib/${ABI}/libdarkroom.so" cp "${SO}" "${OUT}/staging/lib/${ABI}/libdarkroom.so"
cp "${DEX}" "${OUT}/staging/classes.dex" cp "${DEX}" "${OUT}/staging/classes.dex"
# The inference runtime (docs/inference.md §3): ONNX Runtime and Qualcomm's # The inference runtime (docs/dev/inference.md §3): ONNX Runtime and Qualcomm's
# Hexagon backend, beside libdarkroom.so so the app finds them in its own # Hexagon backend, beside libdarkroom.so so the app finds them in its own
# native library directory. The build links none of it — the app dlopens # native library directory. The build links none of it — the app dlopens
# `libonnxruntime.so` at launch and runs on tract if it is not there — so an # `libonnxruntime.so` at launch and runs on tract if it is not there — so an
@@ -278,7 +278,7 @@ else
fi fi
# The models. Android has no other route to one — app-private storage is not # The models. Android has no other route to one — app-private storage is not
# user-reachable and the in-app fetch is unbuilt (docs/faces.md §2.2a) — so # user-reachable and the in-app fetch is unbuilt (docs/dev/faces.md §2.2a) — so
# they go in the APK and `android_main` unpacks them on first launch. The # they go in the APK and `android_main` unpacks them on first launch. The
# sources are `models/face/` and `models/scene/`, shared with the Arch package # sources are `models/face/` and `models/scene/`, shared with the Arch package
# rather than living under this one platform's directory. # rather than living under this one platform's directory.
+1 -1
View File
@@ -73,7 +73,7 @@ SO="${CACHE}/target/jniLibs/${ABI}/libdarkroom.so"
# of KEYSTORE_PASS (see its header), and the keystore has to be reachable from # of KEYSTORE_PASS (see its header), and the keystore has to be reachable from
# inside the container, so a host path in KEYSTORE is copied under the mounted # inside the container, so a host path in KEYSTORE is copied under the mounted
# target directory for the duration of the build and removed after. The # target directory for the duration of the build and removed after. The
# passwords travel as environment, never as arguments -- docs/android-signing.md # passwords travel as environment, never as arguments -- docs/dev/android-signing.md
# has the incantation. # has the incantation.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
echo "==> packaging APK" echo "==> packaging APK"
+2 -2
View File
@@ -1,6 +1,6 @@
# DarkRoom — reproducible Windows cross-build environment # DarkRoom — reproducible Windows cross-build environment
# #
# Everything docs/windows.md §2 names: Rust with the GNU Windows target, the # Everything docs/dev/windows.md §2 names: Rust with the GNU Windows target, the
# MinGW-w64 cross compiler it links with, NSIS to build the installer, and Wine # MinGW-w64 cross compiler it links with, NSIS to build the installer, and Wine
# to smoke-test the result. Both CI and local builds use this image, so "works # to smoke-test the result. Both CI and local builds use this image, so "works
# on my machine" and "works in CI" are the same machine — the same argument # on my machine" and "works in CI" are the same machine — the same argument
@@ -77,7 +77,7 @@ RUN curl -fsSL https://sh.rustup.rs | sh -s -- \
# libwinpthread the Rust target's own MinGW pieces were built against, and # libwinpthread the Rust target's own MinGW pieces were built against, and
# picking the other produces link errors that read as if std were missing. # picking the other produces link errors that read as if std were missing.
# #
# The runtime is linked statically (docs/windows.md §2) so the installer # The runtime is linked statically (docs/dev/windows.md §2) so the installer
# carries one file. `-static-libgcc` is all it takes: rustc's windows-gnu # carries one file. `-static-libgcc` is all it takes: rustc's windows-gnu
# target links its own copy of winpthread in self-contained mode, so nothing # target links its own copy of winpthread in self-contained mode, so nothing
# imports libwinpthread-1.dll — the smoke test's objdump step is what checks # imports libwinpthread-1.dll — the smoke test's objdump step is what checks
+2 -2
View File
@@ -1,7 +1,7 @@
# Windows cross-build environment # Windows cross-build environment
Reproducible container for building the Windows executable and its installer from Linux. The Reproducible container for building the Windows executable and its installer from Linux. The
specification is [docs/windows.md](../../docs/windows.md); this directory is what it turned into, specification is [docs/dev/windows.md](../../docs/dev/windows.md); this directory is what it turned into,
and every departure from the spec's first draft is recorded in the Dockerfile's comments. and every departure from the spec's first draft is recorded in the Dockerfile's comments.
## Use ## Use
@@ -16,7 +16,7 @@ and every departure from the spec's first draft is recorded in the Dockerfile's
# Build the installer from that binary # Build the installer from that binary
./docker/windows/build.sh docker/windows/package.sh ./docker/windows/build.sh docker/windows/package.sh
# Smoke-test under Wine (docs/windows.md §6) # Smoke-test under Wine (docs/dev/windows.md §6)
./docker/windows/build.sh wine target-windows/x86_64-pc-windows-gnu/release/darkroom-desktop.exe --version ./docker/windows/build.sh wine target-windows/x86_64-pc-windows-gnu/release/darkroom-desktop.exe --version
./docker/windows/build.sh wine target-windows/installer/DarkRoom-0.12.0-x86_64-setup.exe /S ./docker/windows/build.sh wine target-windows/installer/DarkRoom-0.12.0-x86_64-setup.exe /S
+8 -2
View File
@@ -7,7 +7,7 @@
# to have run in the same target directory. Produces # to have run in the same target directory. Produces
# DarkRoom-<version>-x86_64-setup.exe in $OUT (default: target-windows/installer). # DarkRoom-<version>-x86_64-setup.exe in $OUT (default: target-windows/installer).
# #
# Runs inside the container, where makensis is; docs/windows.md §5 is the # Runs inside the container, where makensis is; docs/dev/windows.md §5 is the
# specification this implements. # specification this implements.
set -euo pipefail set -euo pipefail
@@ -48,7 +48,13 @@ sed 's/$/\r/' "${REPO}/LICENSE" > "${STAGE}/LICENSE"
# tract on the user's machine with a message about a broken graph rather than # tract on the user's machine with a message about a broken graph rather than
# a checkout that needed `git lfs pull`. Only the weights are checked; the # a checkout that needed `git lfs pull`. Only the weights are checked; the
# scene model's vocabulary and category descriptor are legitimately small. # scene model's vocabulary and category descriptor are legitimately small.
for dir in face scene; do #
# The directories are the ones the APK stages (assemble-apk.sh) and the Arch
# package installs: the face pair and its eye-state models, the scene model
# with its two descriptors, and the panorama border filler. The installer
# smoke test counts the same directories, so a model added here is expected
# there without a number to update.
for dir in face scene inpaint; do
for f in "${REPO}/models/${dir}"/*; do for f in "${REPO}/models/${dir}"/*; do
case "$(basename "${f}")" in case "$(basename "${f}")" in
README.md) continue ;; README.md) continue ;;
+86
View File
@@ -0,0 +1,86 @@
# Documentation
Two audiences, two folders. Most people want the first table and never the
second.
## Using DarkRoom
| | |
|---|---|
| [The manual](manual/README.md) | Every feature, pictured from the application itself — opening a library, rating and filing, developing, local masks, repair, film, panoramas, export |
| [How it is driven](gestures.md) | Every gesture and shortcut, by screen. Generated from the code, so it cannot describe one the application does not have |
The [top-level README](../README.md) says what DarkRoom is, how to get it on
each platform, and what is still missing.
## Changing DarkRoom
Everything under [`dev/`](dev/) is for someone working on the code. Start with
[CONTRIBUTING.md](../CONTRIBUTING.md), which says how to land a first change
without reading the rest.
**The register and the record.** What must be built, how it is built, and how
far along it is.
| | |
|---|---|
| [requirements.md](dev/requirements.md) | What the software must do — the numbered register, the decisions (D-numbers) and the spikes (S-numbers) |
| [architecture.md](dev/architecture.md) | How it is built — crates, the GPU pipeline, the data model, sync |
| [traceability.md](dev/traceability.md) | Generated: which requirement is claimed by which file. Never edited by hand |
| [outstanding.md](dev/outstanding.md) | What is specified and not built, and whether that is a decision or a gap |
| [technical-debt.md](dev/technical-debt.md) | Compromises taken deliberately, each with the condition that retires it |
| [code-health.md](dev/code-health.md) | What a contribution costs, per seam, measured |
**Designs, one per subsystem.** Each is the specification the code was built
to, kept current as the code moved.
| | |
|---|---|
| [catalog.md](dev/catalog.md) | The index, the library view, incremental scan, the job queue |
| [storage.md](dev/storage.md) | Storage backends: the seam a folder, a sync client and a Nextcloud account share |
| [faces.md](dev/faces.md) | Face detection, identity, clustering and the eye-state models |
| [segmentation.md](dev/segmentation.md) | How the application finds the regions a local mask snaps to |
| [mask-editing.md](dev/mask-editing.md) | Painting, erasing and combining masks |
| [spot-removal.md](dev/spot-removal.md) | Clone and heal as parameters in the edit graph |
| [panorama.md](dev/panorama.md) | Alignment, projection, the chunked composite and the border fill |
| [inference.md](dev/inference.md) | The neural runtime and model chosen per device, with the measurements |
| [display-and-extension.md](dev/display-and-extension.md) | The display contract, and why the fused pipeline is already most of a plugin format |
| [view-composition.md](dev/view-composition.md) | A controller for the display layer |
| [ui-navigation.md](dev/ui-navigation.md) | Finding things in the interface once there are many |
**Measurements.** Numbers committed so a regression is a diff rather than a
recollection.
| | |
|---|---|
| [benchmarks.md](dev/benchmarks.md) | The per-commit suite: what it covers, what it does not, how to read a failure |
| [bench-baseline.json](dev/bench-baseline.json) | The committed numbers the suite checks against |
| [frame-budget.md](dev/frame-budget.md) | What a frame costs on each device, and the decision those figures settled |
**Platforms and distribution.**
| | |
|---|---|
| [distribution.md](dev/distribution.md) | Which channels v1 targets and what each one constrains |
| [windows.md](dev/windows.md) | The Windows installer, cross-built from the Linux CI |
| [android-signing.md](dev/android-signing.md) | Which key signs the APK, and keeping it |
**Archive.** Kept as the record of what was asked for, not as plans.
| | |
|---|---|
| [milestone-v0.1.md](dev/archive/milestone-v0.1.md) | The first milestone, delivered 2026-08-30 and superseded |
| [ui-refinement.md](dev/archive/ui-refinement.md) | How the interface should look; succeeded by [ui-navigation.md](dev/ui-navigation.md) |
## Conventions
Two files here are generated and must not be edited by hand:
`gestures.md` and `dev/traceability.md`. Both come from
`cargo run -p traceability` and the pre-commit hook keeps them in step with
the tree. The manual's pictures are recorded by
[`tools/manual`](../tools/manual/README.md) and live in LFS.
A design document links to the requirements it satisfies and to the code
that satisfies them. When the code moves, the link moves with it; a document
that has stopped being true goes to `dev/archive/` with a note saying what
replaced it, rather than being deleted.
@@ -4,8 +4,8 @@
> Kept as the record of what the first milestone asked for, not as a plan. > Kept as the record of what the first milestone asked for, not as a plan.
> Everything below shipped, and the application went well past it — see > Everything below shipped, and the application went well past it — see
> [outstanding.md](outstanding.md) for what is still missing at 0.9.0. > [outstanding.md](../outstanding.md) for what is still missing at 0.9.0.
**Companion to:** [requirements.md](requirements.md) · [architecture.md](architecture.md) **Companion to:** [requirements.md](../requirements.md) · [architecture.md](../architecture.md)
The first buildable milestone: connect to a Nextcloud folder, index it locally, and display RAW The first buildable milestone: connect to a Nextcloud folder, index it locally, and display RAW
previews on both Linux and Android. previews on both Linux and Android.
@@ -39,7 +39,7 @@ building on sand.
## 2. The four assumptions under test ## 2. The four assumptions under test
Each maps to a spike in [requirements.md §9](requirements.md). Each maps to a spike in [requirements.md §9](../requirements.md).
| # | Assumption | If wrong | Spike | | # | Assumption | If wrong | Spike |
|---|---|---|---| |---|---|---|---|
@@ -55,7 +55,7 @@ all, so it should be proven in the first week, before catalog or sync work begin
## 3. Functional scope ## 3. Functional scope
Requirement IDs reference [requirements.md](requirements.md); a v0.1 suffix marks a reduced subset Requirement IDs reference [requirements.md](../requirements.md); a v0.1 suffix marks a reduced subset
of the full requirement. of the full requirement.
### 3.1 Account and connection ### 3.1 Account and connection
@@ -2,9 +2,9 @@
**Status:** Built, not yet recorded · 2026-08-30 **Status:** Built, not yet recorded · 2026-08-30
**Companion to:** [requirements.md](requirements.md) §4.1 (performance targets) · §8 (verification) **Companion to:** [requirements.md](requirements.md) §4.1 (performance targets) · §8 (verification)
**Instrument:** [`tools/bench`](../tools/bench) — `cargo run --release -p dr-bench -- check` **Instrument:** [`tools/bench`](../../tools/bench) — `cargo run --release -p dr-bench -- check`
**Committed numbers:** [`bench-baseline.json`](bench-baseline.json) **Committed numbers:** [`bench-baseline.json`](bench-baseline.json)
**GPU half:** [`core/dr-gpu/tests/frame_budget.rs`](../core/dr-gpu/tests/frame_budget.rs) · **GPU half:** [`core/dr-gpu/tests/frame_budget.rs`](../../core/dr-gpu/tests/frame_budget.rs) ·
[frame-budget.md](frame-budget.md) [frame-budget.md](frame-budget.md)
§8 has said since it was written that performance is verified by *"an automated §8 has said since it was written that performance is verified by *"an automated
@@ -60,7 +60,7 @@ Two of those rows carry a qualifier, and the qualifiers are the point.
library view cannot paint without: `Catalog::open` (which connects, migrates and library view cannot paint without: `Catalog::open` (which connects, migrates and
**backfills**, and the backfill is three passes over the images table on every **backfills**, and the backfill is three passes over the images table on every
open), `count`, the first 400-row `window`, and the monthly `timeline`. Tagged open), `count`, the first 400-row `window`, and the monthly `timeline`. Tagged
`TRACES: NFR-P1` in [`tools/bench/src/catalog_open.rs`](../tools/bench/src/catalog_open.rs), `TRACES: NFR-P1` in [`tools/bench/src/catalog_open.rs`](../../tools/bench/src/catalog_open.rs),
because a build that breaks it fails this gate. because a build that breaks it fails this gate.
**NFR-P3 — ≥ 100 images per second on the embedded preview path.** The **NFR-P3 — ≥ 100 images per second on the embedded preview path.** The
@@ -69,7 +69,7 @@ per-image work is exactly what `spawn_thumbnail_sweep` does — `decode_jpeg`,
`ThumbStore::put` — arranged in the same shape: chunks of 96, lanes owning `ThumbStore::put` — arranged in the same shape: chunks of 96, lanes owning
disjoint slices, and the single thread that owns the store writing the finished disjoint slices, and the single thread that owns the store writing the finished
chunk. Tagged `TRACES: NFR-P3` in chunk. Tagged `TRACES: NFR-P3` in
[`tools/bench/src/thumbnails.rs`](../tools/bench/src/thumbnails.rs). [`tools/bench/src/thumbnails.rs`](../../tools/bench/src/thumbnails.rs).
### Requirements this can only half-answer, and is not tagged for ### Requirements this can only half-answer, and is not tagged for
@@ -114,7 +114,7 @@ instead of ~2 TB, and neither half is flattered by that.
| Rows | 50,000 images, 50,000 default versions, 400 folders, one root | | Rows | 50,000 images, 50,000 default versions, 400 folders, one root |
| Capture times | Twelve years from a fixed epoch, so the timeline has ~144 monthly buckets | | Capture times | Twelve years from a fixed epoch, so the timeline has ~144 monthly buckets |
| Sources | 12 synthesised JPEGs at 1620 × 1080 — the size `dr-decode` records a CR2 carrying in IFD2 | | Sources | 12 synthesised JPEGs at 1620 × 1080 — the size `dr-decode` records a CR2 carrying in IFD2 |
| Seed | 20260829, in [`tools/bench/src/main.rs`](../tools/bench/src/main.rs) | | Seed | 20260829, in [`tools/bench/src/main.rs`](../../tools/bench/src/main.rs) |
| Location | `$DR_BENCH_DIR`, else the system temporary directory | | Location | `$DR_BENCH_DIR`, else the system temporary directory |
It is reproducible from the seed, and a `stamp.json` beside it records what it It is reproducible from the seed, and a `stamp.json` beside it records what it
+9
View File
@@ -530,6 +530,15 @@ Three invariants, each tested:
| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard | | Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced | | Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |
**When the exchange runs.** Corrected 2026-09-20. It fired only after the metadata sweep — hours
on a large library — so a fresh device re-derived every thumbnail it looked at, re-detected faces
and re-read every header before adopting the shards and snapshot that held all of it. It now also
fires the moment the scan completes, which is the first moment the rows the merges key on exist,
and the sweep starts behind it. In steady state that pass is one listing. The catalog merge also
takes **capture metadata** (`captured_at`, offset, camera, lens, ISO) for images still at
`metadata_state < 2`, matched by `oc:fileid` — a date is a fact about the file's bytes, not local
state, and the snapshot already carried it; the sweep's per-chunk query then finds nothing left.
**The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the **The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the
library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass
is one listing plus whichever shard is still open. is one listing plus whichever shard is still open.
@@ -17,8 +17,8 @@ permission to a package rather than after.
| Platform | Channel | State | What it constrains | | Platform | Channel | State | What it constrains |
|---|---|---|---| |---|---|---|---|
| Linux | Arch source package — [`packaging/PKGBUILD`](../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon | | Linux | Arch source package — [`packaging/PKGBUILD`](../../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon |
| Linux | Flatpak — [`packaging/flatpak/`](../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths | | Linux | Flatpak — [`packaging/flatpak/`](../../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths |
| Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all | | Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all |
| Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs | | Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs |
| Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering | | Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering |
@@ -39,7 +39,7 @@ somewhere:
that misses one of them costs the icon in the shell or the association in the that misses one of them costs the icon in the shell or the association in the
software centre, and neither failure announces itself. software centre, and neither failure announces itself.
- **The metainfo, not just the desktop entry.** - **The metainfo, not just the desktop entry.**
[`packaging/paris.tourolle.darkroom.metainfo.xml`](../packaging/paris.tourolle.darkroom.metainfo.xml) [`packaging/paris.tourolle.darkroom.metainfo.xml`](../../packaging/paris.tourolle.darkroom.metainfo.xml)
is the single description of the application, installed by every channel that is the single description of the application, installed by every channel that
has somewhere to put it. Its `metadata_license` is CC0-1.0 and its has somewhere to put it. Its `metadata_license` is CC0-1.0 and its
`project_license` is GPL-3.0-or-later; those differ on purpose — see the `project_license` is GPL-3.0-or-later; those differ on purpose — see the
@@ -173,7 +173,7 @@ Two changes, in this order:
chooser instead of a volume list. chooser instead of a volume list.
**Done when:** a Flatpak built from **Done when:** a Flatpak built from
[`packaging/flatpak/paris.tourolle.darkroom.yml`](../packaging/flatpak/paris.tourolle.darkroom.yml), [`packaging/flatpak/paris.tourolle.darkroom.yml`](../../packaging/flatpak/paris.tourolle.darkroom.yml),
with its `finish-args` unchanged and no `flatpak override` applied, can select a with its `finish-args` unchanged and no `flatpak override` applied, can select a
library root, scan it, and write a sidecar back into it. library root, scan it, and write a sidecar back into it.
+10
View File
@@ -821,6 +821,16 @@ here, and it is the phone and tablet story that should decide whether it gets bu
already has, the raw vector written over the old one and its id, box and identity untouched already has, the raw vector written over the old one and its id, box and identity untouched
(`faces::record_updates`). No detector runs and no suggestion is lost — the cost is the (`faces::record_updates`). No detector runs and no suggestion is lost — the cost is the
original fetched once more, since the length exists only at the moment of embedding. original fetched once more, since the length exists only at the moment of embedding.
- **A person is stood for by their references.** Every face the user has ruled on is an anchor,
and the scan is exhaustive, so a person with 750 confirmations would cost 750 comparisons
against every other face — and the cost of a library would grow with how well it was named.
Instead each person enters through at most `MAX_REFERENCES` (100) of their anchored faces,
chosen by `dr_face::references`: those whose raw embedding is at least `MIN_REFERENCE_QUALITY`
(15) long, and among them the set spanning the greatest volume — greedy max-determinant, the
longest vector first and then, at each step, the face with the largest component orthogonal to
the chosen so far. Thirty frames from one afternoon contribute one reference; the single profile
shot is taken early. The faces not chosen keep their confirmations and are not touched by the
pass; they are simply not compared.
**The algorithm.** Constrained average-link agglomeration over the probability graph, merging while **The algorithm.** Constrained average-link agglomeration over the probability graph, merging while
the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather

Some files were not shown because too many files have changed in this diff Show More