The backfill stamp read max(id) of images and versions and max(rowid)
of keywords. None of those tables is AUTOINCREMENT, so SQLite hands a
freed newest id out again: empty the trash of the newest photograph and
scan a new one, or let a local folder's walk delete a renamed file's row
and insert the new name in the same pass, and the new image takes the
old id. max(id) does not move, nor does count(*), and when a newer
version elsewhere keeps max(versions.id) still too, the stamp matched
and the open skipped the backfill.
That row is exactly one that needs it. Neither scan path creates the
default version: scan::persist and walk insert the image and leave the
version, the RAW/JPEG pairing and the keyword terms to the next open.
Skipped, the image went without them until the app restarted, so a
rating or a pulled sidecar judgement had no version to land on and a
JPEG beside its RAW showed twice.
The stamp now carries the newest row's content: the newest image's id,
path, added time and whether it has a version; the newest version's id
and image; the newest assignment's rowid, version and word. Whether the
newest image has a version is the part that cannot be fooled - after a
backfill every image has one, and a row that has just taken a freed id
has none - so the two stamps differ even when the same file comes back
at the same id in the same second. Still one statement: three reverse
rowid scans that stop at the first row, and one probe of versions_image.
An open that skips still costs ~1 ms on the reference catalog copy.
This closes the hole in the stamp itself rather than by a forget() at
each delete site, so a delete path added later, or one in another
process, cannot reopen it. Two tests delete the newest image and insert
another at the freed id on a separate connection, with a newer version
elsewhere holding max(versions.id); both fail against the old stamp.
holds_original, the prefetch worker's check that a neighbour's original
is already cached, opened the catalog for every neighbour it asked about.
Its own comment called it a row check; the open around it was four of
the five opens a develop landing made.
The worker now keeps one Catalog for the batch it is serving, opened at
the first check and reopened only if the batch names another catalog
file. The connection runs in autocommit, so each check still sees what
fetch_original committed in between. fetch_original is unchanged.
With the backfill no longer run on every open, a landing whose
neighbours are all cached goes from five opens to two, and from ~80 ms
of CPU to ~1-2 ms on a copy of the reference catalog.
Catalog::open ran schema::backfill every time, and every worker thread
opens its own connection. A develop landing made five opens, and each
paid the RAW/JPEG pairing, the default-version anti-join over every
image, the uuid pass over every default version and the keyword check:
17 ms of CPU an open on a copy of the reference catalog, ~80 ms a
landing, to confirm that nothing had changed since the open before.
Everything the backfill repairs is a row some write added: an image a
scan inserted, a version or keyword assignment a merge brought in. So
the open now reads a stamp - user_version, max(id) of images and
versions, max(rowid) of keywords, and the file's device and inode - and
skips the backfill when the stamp matches the one recorded at this
path's last backfill in this process. The maxima are each the last page
of a b-tree; an open that skips costs ~1 ms.
The backfill still runs:
- on the first open in a process (nothing recorded yet);
- on any open that migrated the schema, unconditionally;
- after a pull: merge_remote forgets the path, so the next open
backfills even when every incoming row collided and nothing moved;
- when the file is replaced under its name: the inode is in the stamp,
and recovery::set_aside, the first step of a restore and a rebuild,
forgets the path;
- when another process or thread adds rows, because the stamp is read
from the file, not from anything this process did.
The stamp is taken before the backfill, not after. Read after, it would
describe the backfill's own inserts, and could record an image another
connection inserted in between as covered when it was not. Read before,
the worst case is one redundant pass after a backfill that did real work.
Kept in memory rather than in the catalog: a stamp row would need a
table an older build does not have and would travel in the sync
snapshot, where a flag from another device's catalog says nothing about
this one. No schema version bump, so the tablet on 0.16.0 still reads
the snapshot. Tests cover the skip, a scan's new image, a migration, a
pull and a replaced file.
Landing on a photograph in develop opens the catalog once to fetch the
original and once more per prefetched neighbour to ask whether the cache
already holds it: five opens, each running the whole backfill. The bench
timed one open but not the landing, so the cost of the shape was not
visible and a fix to it could not be measured.
Two figures now, both against an empty cache so the question is asked the
same way whatever the answer: the five-open shape the app had, and the
two-open shape where the prefetch worker keeps one connection for its
batch. On a copy of the reference catalog (23,582 images) under load, the
five-open landing costs ~80 ms of CPU.
Dehaze cost 22.9 ms of a 2560x1600 frame on the reference laptop RTX 3050,
and 54.1 ms at 3840x2160, with the memory clock held at 810 MHz by the power
cap (graphics 1762 MHz). It ran five passes: a run and a span erosion along
x, the same along y, and the recovery. At those clocks a detail pass costs
what it reads and writes, not what it taps: a pass with an empty body -
one render-sized rgba16float read and write - measured 4.0 ms, and each
dehaze pass 4.4-4.6 ms, so the taps were about 2 ms of the 22 and the four
hand-offs between passes were the rest.
Each axis is now one pass that takes the minimum over the whole window
directly, and the recovery rides in the y pass, which already holds the
veil and the pixel's own colour. That is 36 texture reads per pixel at
2560x1600 in place of 12, nearly all of them cache hits, and two passes in
place of five.
The picture is the same bits. A minimum is exact in any order, and the
window is the one Split always covered, the surplus pixel on the far side
included (Split::first and Split::width). The veil crossing the removed
hand-offs was already exactly representable in rgba16float - a minimum of
channels read from rgba16float, floored at zero - so storing it between
passes never rounded anything that the fused form now keeps unrounded.
Measured with a scratch probe that renders the synthetic 60 MP frame from
examples/frame_budget.rs, only a detail parameter moving so the fused pass
is reused, 30 frames per scene after six of warm-up, five runs of each
binary alternated, median of the per-run p50:
scene before after
dehaze 2560 fit 22.88 ms 9.06 ms
dehaze 2560 1:1 23.41 ms 9.52 ms
dehaze 3840 fit 54.09 ms 28.12 ms
all detail 2560 fit 53.11 ms 39.97 ms (NR, sharpen, clarity,
all detail 2560 1:1 67.48 ms 56.42 ms texture, dehaze)
every op 2560 fit 57.59 ms 44.19 ms (with film)
every op 2560 1:1 71.83 ms 57.93 ms
controls without dehaze (NR, sharpen, clarity, texture): within +-2%
The rgba8 output hashed identically before and after for every scene -
dehaze alone, all five detail operations, every operation with film, and
each other detail operation alone - at fit and 1:1, at 2560x1600,
3840x2160, 1917x1203 and 333x211: 64 of 64.
rustfmt over the files the albums work touched, and the album merge's
incoming row as a named struct rather than an eight-field tuple, which
clippy's type_complexity refused.
FR-EXP-10 is the album: a named export destination beneath the
collections, whose folder holds only the exported files while the
catalog links each back to its original; how albums sync, why a device
folder does not, and why the tables are made on first use rather than
by a migration. FR-EXP-6 now says the destination is an album, never
inside the library, and that folders are chosen by pointing — the
portal or Windows dialogue, SAF's tree picker, the server browser —
each able to make a folder.
The manual's launch and export sections say the same in the words on
screen. Its pictures still show the 0.16.0 launch screen and export
settings; they are re-recorded with the rig, not edited by hand.
Android's only export destination was the library on the server
(ExportTarget::available), because writing to the device goes through
the Storage Access Framework and nothing did. An album's folder on the
tablet is now chosen in the system's tree picker — which has its own
"Create new folder" — and exports are written into it with
DocumentsContract.
The picker answers through onActivityResult, and the main activity is
NativeActivity, whose result is not ours. FolderPicker is a translucent
activity that only asks: it starts ACTION_OPEN_DOCUMENT_TREE, takes a
persistable grant (a folder is chosen once and exported to for months),
leaves the URI in a static, and finishes. Rust polls it from a Slint
timer — one static call, rather than a registered native method and a
thread to deliver on.
Two things the first build on the tablet got wrong, recorded where they
are fixed:
- Our classes must be loaded through Context.getClassLoader(). The
class of what ndk_context holds is a framework class from the boot
loader, which reports every class in the APK as not found.
- What ndk_context holds is the application context, not the activity,
and starting an activity from it throws without FLAG_ACTIVITY_NEW_TASK.
Saf.write creates the document (or, under Overwrite, reopens the one of
that name with "wt" so a shorter file does not keep the old tail) and
returns the name the provider actually gave it, since SAF renames on a
collision by itself; the album records that name. A tree URI reads in
the sidebar as its folder ("Pictures/Web"), not as a content:// string.
Export took a path typed into the settings page, or a folder inside the
library on the server. The first is how exports end up somewhere nobody
looks; the second put JPEGs into the tree a scan catalogues, where they
came back as photographs beside the RAWs they were made from.
The destination is now an album (FR-EXP-10), chosen by name in the
export sheet. Albums are listed under the collections in the sidebar;
"+" there, or "New album…" in the sheet, opens a sheet for its name and
its folder — on this device through the platform's dialogue, or on the
server through the browser with "New folder". A server folder inside
the library is refused, and the sheet says why. Selecting an album
narrows the grid to the photographs behind its files: library::Scope
is Collection or Album, and scope_clause is the one place the two are
spelled, which also retires the two copies of the collection predicate
total_images_scoped and read_cells_scoped had inlined.
A batch resolves the album when it starts, and refuses in words when
none is chosen, it has gone, or its folder is local to another device.
Each item reports the image it came from, and the files written are
recorded against the album in one transaction when the batch ends.
A server album lives outside the library, so its queued uploads are
relative to the account root. That is a third line in the outbox's
.dest record rather than a leading slash, because a record written
before albums may carry a stray slash and must keep the meaning it was
written with.
An export folder set before albums becomes an album called "Exports"
on first open, so upgrading does not lose where exports were going.
The old destination fields stay in ExportSettings so older settings
files still read.
The in-app browser that chooses a library folder on the server could
only open folders that already existed, so a library, or an export
destination, that was not on the server yet had to be made in the
Nextcloud web page first. It now has "New folder": a name, then MKCOL,
then the parent listed again and the new folder walked into — a folder
somebody has just named is the one they mean to choose.
The listing is the server's rather than the name inserted locally: the
server may have normalised or refused it. A name with a slash, "..",
or nothing at all is refused before any request, because a folder
typed with a slash in it is a path the user did not mean.
remote_folders holds the two WebDAV round trips (list, make) off the UI
thread, with the answer delivered through a Slint timer, so the album
sheet can use the same browser.
Every folder the desktop asked for was a text field: the library folder
at launch, an import's source and second copy, a preset folder brought
over from Lightroom. A typed path is how a destination silently becomes
a new folder nobody meant — one wrong letter three levels down and the
write succeeds somewhere the photographer will never look — and a field
cannot make the folder that is not there yet.
They now open the platform's own dialogue through rfd: the XDG desktop
portal on Linux, the common item dialogue on Windows. The portal rather
than GTK because it reaches the user's files from inside the Flatpak and
needs no GTK in a Slint application, and it draws whichever desktop's
chooser is running, "New folder" included. It is awaited on Slint's
event loop (spawn_local), so the window keeps drawing while it is open,
and parented to the window so it opens over it.
PathRow shows what is chosen, read-only, beside the button. Android has
no filesystem dialogue — only SAF, which returns document trees, not
paths — so there the same rows stay typed fields (Pickers.local-paths).
The launch screen keeps the folder used last on screen with "Open
folder" beside it, so reopening is one press. Presets get two buttons,
a folder and a single .xmp file, because no platform dialogue picks
"a file or a folder" in one go.
An album is a named export destination. Its folder holds only the
exported files; the catalog records, per file, the image it was
rendered from, so an album can show the originals behind its JPEGs
(FR-EXP-10).
The tables are created on first use (CREATE TABLE IF NOT EXISTS), the
way dedup_probes is, rather than by a schema migration: a new
user_version makes every older build refuse this catalog's snapshot at
sync, and the 0.16.0 tablet would stop merging collections, keywords
and people for a feature it does not have.
Albums merge as collections do: by uuid and revision, tombstones on
delete, exports as a set union keyed on the server's file id (content
hash for a folder library). A folder on the server lives on the album
row and syncs; a folder on this device lives in album_folders, which
the merge never reads and the upload snapshot drops, because a path or
a SAF grant on one device means nothing on another.
Exports are keyed on the file name, not the image: two crops of one
photograph are two files and two rows, and an overwrite re-points the
name at whatever wrote it last.
Each sync pass spent 0.8-2.0 s of CPU and 1.0-5.4 s wall on the upload
snapshot of the reference catalog (24k images, 18,871 faces), ahead of the
rest of the pass. The upload itself had been crop-less since the crops
moved to the face shards. The cost was in how it got that way. The backup
API copied all 158 MB of the catalog, 96 MB of it the ~5 KB JPEG crop on
every faces row. Then `UPDATE faces SET crop = NULL` rewrote 18.9k rows
and freed their overflow chains, and VACUUM rebuilt the file again. That
wrote the catalog about three times over to upload 50 MB.
The snapshot is now built rather than copied. An empty file attaches the
catalog, creates each table from the catalog's own sqlite_master and fills
it with INSERT ... SELECT, with faces.crop selected as NULL. Indexes,
triggers and views follow, and user_version, application_id, page size and
the WAL header flag are carried over. It all runs in one transaction on the
snapshot's connection, so the catalog is read as of one moment and
concurrent writers are serialised, not raced, as the backup API did. The
build journal is in memory with synchronous off, because the file is
scratch that is rebuilt every pass and quick_check'd before upload. Foreign
keys are off on that connection. The bundled SQLite enables them, and then
a multi-row INSERT into images scans images for children of each new row
(shadowed_by is a self-reference with no index), which cost 1.2 s alone.
Measured on a .backup copy of the reference catalog with catalog_bench,
old and new binaries back to back on a loaded machine:
before best 1.0-5.4 s wall, 0.84-1.98 s cpu, 49.8 MB
after best 0.40-2.1 s wall, 0.39-0.96 s cpu, 50.4 MB
With the machine quiet the new build takes 0.31-0.43 s.
What a receiving device gets is unchanged. It is the same schema, the same
rows and a NULL crop, which is what 0.16.0 already uploads and merges. The
merge reads only a remote face's box and model (merge::match_faces) and
never writes a local crop. No device adopts a downloaded catalog as its
own, and a fresh one takes faces and crops from the shards. There is no
schema bump, so older builds still merge it. NFR-R2 backups keep using the
backup API and keep their crops.
Tests: the snapshot matches the catalog in schema, row counts, pragmas and
WAL header. A leftover file is replaced. Merging a crop-less snapshot
carries a confirmed name across by box and leaves the local crop
untouched, and does so idempotently.
The presets sheet now lists the shipped collection beside the
photographer's own, a copy under a shipped name overrides it, and
shipped and imported presets are looks that leave a photograph's own
corrections alone. The manual says so where it introduces the sheet,
and FR-DEV-6 states the rule. The sheet's screenshot (media/presets.png)
predates the sections and needs re-recording.
A Lightroom preset changes the settings it was saved with and leaves every
other one where the photograph had it. Imported as a whole edit, a preset
holding only a grade reset the exposure, white balance and noise reduction
it was put on top of — the opposite of what the photographer had in
Lightroom.
Imported presets now reach only the operations they name
(`Reach::Named`), the rule the shipped presets already follow.
The six starter presets were copied into the photographer's own library
on a first run and were theirs from then on. That cannot grow into a
real collection: a copy is frozen at the release that wrote it, so an
improved preset reaches nobody who had the old one, and re-seeding would
overwrite a preset someone had tuned.
`dr_pipeline::bundled` now holds the shipped presets as `.drpl` files
compiled into the binary, in sections — Essentials (the former six) and
three sections of film presets, one per measured stock in dr-film,
printed on the paper its profile names — and never writes them to the
user's file. Every shipped preset is a look (`Reach::Named`), so applying
one keeps the corrections a photograph already has.
A name links a photographer's copy to a shipped preset. Saving over a
shipped name makes their version the one that name applies; it is listed
in the shipped section, marked as changed, and deleting it reverts to the
shipped one. Renaming it makes it one of their own and the shipped preset
reappears. Keyed on the name because that is what the photographer sees
and chooses by.
Copies an older first run seeded are forgotten on load where they are
still exactly as seeded — otherwise all six would list as changed and
stay frozen at their old values. A tuned one is kept and now overrides.
The sheet lists "Yours" first, then each shipped section, with headings.
Shipped rows apply and nothing else; a changed row offers Revert where
the photographer's own offer Delete. A dr-ui test checks every shipped
film names a stock this build can bake, on that stock's own paper,
because dr-pipeline does not link the profile database.
The film presets name stocks by id; the measurements behind them are
spektrafilm's (CC BY-SA 4.0), attributed in each file as in dr-film.
A preset could not choose a film stock. The stock is a choice of material
rather than a parameter, so `Preset` — a map of `op.param = value` — had
nowhere to hold it, and "Portra 400, printed" could not be saved, copied
or shipped as a look. Worse, the film node's own sliders *were*
parameters: a paste moved one stock's exposure and push onto whatever
stock the target was on, and left the target's tables baked from the
values it had just replaced.
A preset now carries a `FilmRef` beside its parameters. It travels under
whichever scope carries the film node, so the stock and its sliders are
never split, and by the replacement rule every other parameter follows:
applied at that scope, a preset without a film develops the target
without one. `Preset::apply` returns the `FilmRebake` it owes, as
`EditGraph::set_state` already did, because this crate cannot bake a
stock; the develop session pays it before recording the step, and the
batch paste writes the stock into each sidecar through `film_for`. The
library file spells it `film =` / `film_print =`, as a sidecar does, and
an older build keeps those lines as ones it does not understand.
`EditState` keeps the film in its own field only: the parameters it
captures leave it out, so one edit has one place to say which stock it
is on.
Second, a preset now has a reach. Replacement is right for a copy of a
whole edit — "make these match" — and wrong for a look: a stock-only
"Portra 400" applied that way would put the photograph's exposure, white
balance and noise reduction back to default. `Reach::Named` replaces only
the operations a preset names (whole operations, so a look that sets the
blacks resets the whites beside them) and the film only if it names one.
Saved edits and the clipboard keep `Reach::Whole`; the line `reach =
named` is written only for the other, so existing libraries write the
same bytes.
After 9cff677 the loop over the other device's confirmed and ignored faces
(13,000 on the reference library) still asked three cached statements per
face -- the person by uuid, the face's current assignment, and whether this
pair was rejected here. It was 51 ms of a steady-state merge.
The person is now resolved in the statement that reads the incoming rows,
by the local `people.uuid` key:
SCAN fp
SEARCH p USING INTEGER PRIMARY KEY (rowid=?)
SEARCH lp USING COVERING INDEX sqlite_autoindex_people_1 (uuid=?)
and the local `face_person` (16,800 rows) and `face_person_rejected` are
each read once into memory and looked up there. A write goes to the table
and to the map, so a second remote face matched to the same local face sees
what the first left, as it did when each face re-read the table. The
incoming rows are ordered by face id -- the order the table was already
walked in -- since which of two such faces is applied last decides the
answer. An inner join to `people` drops the rows the old loop skipped for
want of a local person, and the counts in the report are unchanged.
After: the loop 10-12 ms. The merge as a whole, with the two changes before
this, went from 228-231 ms to 135 ms best of 5, and every catalog table
checksums the same after the bench as after the old build's run.
`merge::match_faces` reads every local face's box and model to pair the
other device's faces with ours. It took 54 ms of a steady-state merge on the
reference library (19,000 faces).
A `faces` row is eight kilobytes -- the embedding, the crop, the dense
landmarks -- and `model_id` sits past the embedding, so reading it opened
each row's overflow pages:
SCAN f
SEARCH r USING INTEGER PRIMARY KEY (rowid=?)
`faces_box (image_id, model_id, x, y, w, h)` holds every column the scan
asks for:
SCAN f USING COVERING INDEX faces_box
SEARCH r USING INTEGER PRIMARY KEY (rowid=?)
The local scan went from 38 ms to 8 ms (sqlite3 on a copy, aggregated so
output formatting is not timed), and `match_faces` from 54 ms to 30-37 ms;
what remains is the other device's half. That is read from its snapshot,
which has whatever indexes its build made -- this one will carry
`faces_box` in its uploads -- and whose rows have had their crops stripped.
The bench merges a full copy with crops, so it overstates that half.
Created on first use in `match_faces`, with CREATE INDEX IF NOT EXISTS,
rather than by a migration, for the reason `keywords::ensure_term_index`
gives: a schema version bump makes older builds refuse the snapshot, and an
extra index is invisible to them. The first merge after the upgrade builds
it (about a second, once). Its prefix duplicates `faces_image_model`, which
is left alone; the planner takes either for an (image_id, model_id) probe.
Tables checksum the same after the bench run as after the old build's.
`merge_remote_catalog` on the reference library (catalog_bench, a copy
merged with itself: the steady state of a sync pass) cost 228-231 ms best
of 5. Timing its phases put 91 ms in the keyword half, not in the faces the
issue named.
Both assignment unions refuse a word this device holds only as a tombstone,
with a correlated `NOT EXISTS (... deleted = 1) OR EXISTS (... deleted = 0)`
per incoming assignment. The `deleted = 1` half has no index to use --
`keyword_terms_name` is partial on `deleted = 0` -- so it scanned the whole
vocabulary for each of the 10,800 rows:
SCAN rk
CORRELATED SCALAR SUBQUERY 1
SCAN t
CORRELATED SCALAR SUBQUERY 2
SEARCH t USING COVERING INDEX keyword_terms_name (name=?)
The refused words are one set for the whole statement, so it is asked once:
`rk.keyword NOT IN (tombstoned names EXCEPT live names)`, which is the same
condition -- refused exactly when deleted under some identity and live under
none -- and which SQLite builds as a list before the walk:
SCAN rk
LIST SUBQUERY 2
MERGE (EXCEPT) ...
The file-id union alone went from 72 ms to 11 ms (sqlite3 on a copy), and
the keyword phase of the merge from 91 ms to 28-35 ms. Every table of the
catalog checksums the same after the bench as after the old build's run,
and the merge tests for tombstones and renames pass unchanged.
The grid's total is read on every scroll reload (`load_window` compares it
to notice a delete). On the reference library it cost 1.3-1.5 ms best-of-50
by catalog_bench, 2-3.6 ms on a busy machine, and the issue measured 4 ms.
`uncollapsed` asked every visible image whether a collapsed burst stands in
for it -- two primary-key probes per image, 19,000 times, on a library with
no bursts at all:
SCAN i USING INDEX images_grid_order
CORRELATED SCALAR SUBQUERY
SEARCH bm USING INTEGER PRIMARY KEY (rowid=?)
CORRELATED SCALAR SUBQUERY
SEARCH be USING INTEGER PRIMARY KEY (rowid=?)
`total_images_filtered` now counts what the filter keeps and subtracts the
frames `bursts::collapsed_away_frames` lists, under the same filter:
SCALAR SUBQUERY: SCAN i USING INDEX images_grid_order
SCALAR SUBQUERY: SCAN bm; SEARCH be ...; SEARCH i USING INTEGER PRIMARY KEY
The second half walks only `burst_members`. Each image is in it at most
once (it is the key), and the filter is applied to both halves, so the
subtraction removes exactly the rows the predicate used to drop. The new
fragment sits beside `not_collapsed_away` in bursts.rs, and a test holds
the two to the same rows with bursts open and closed.
After: 0.3 ms, the same count (19,152). The cells query keeps the predicate:
it is a window with a LIMIT and needs the rows, not their number. The
rated grid count (3.5-4 ms with a one-star filter) is unchanged: its cost
is the rating subquery per image, and changing how `RatingFilter` spells
it changes every grid and timeline query, which is left for its own change.
`keywords::list` is the vocabulary with a per-word photograph count, and
`keywords::for_images` calls it on every selection change to redraw the
keyword panel. On the reference library (58 words, 10,800 assignments) it
cost 3.0-3.5 ms best-of-50 by catalog_bench, `for_images` 3.1-3.6 ms (6 and
5 ms on a busy machine).
Per word, the count walks `keywords_term (keyword)` and, for each
assignment, reads the `keywords` row to learn its version before probing
`versions` for the image:
SEARCH k USING INDEX keywords_term (keyword=?)
SEARCH v USING INTEGER PRIMARY KEY (rowid=?)
With `keywords_term_version (keyword, version_id)` the first step is
index-only:
SEARCH k USING COVERING INDEX keywords_term_version (keyword=?)
SEARCH v USING INTEGER PRIMARY KEY (rowid=?)
After: `list` 1.3 ms, `for_images` 1.5 ms, with the same answers (digests of
both outputs compared on the reference library).
The index is created on first use by `list`, with CREATE INDEX IF NOT EXISTS,
not by a migration: a new schema version makes every older build refuse this
catalog's snapshot at sync (`sync::remote_is_mergeable` compares
`user_version` and nothing else), and a build that meets an extra index
ignores it. Once the index exists the statement is a schema lookup, 8 us. A
failure to create it -- a read-only or busy catalog -- is logged and the
list is read without it, as before.
`library::local_original_count` feeds the "On this device" chip and runs
beside the rating counts on every star keystroke. On the reference library
it cost 1.3-1.4 ms best-of-50 (3 ms on a busy machine) to find 254
originals among 19,000 visible images.
It was a correlated EXISTS per visible image:
SCAN i USING INDEX images_grid_order
SEARCH ic EXISTS USING INTEGER PRIMARY KEY (rowid=?)
`image_cache` holds a row only for what has been fetched, so the question
is driven from it: `i.id IN (SELECT image_id FROM image_cache WHERE
tier_actual >= Original)`, which SQLite plans as the list first and a probe
of `images` by id for each entry:
SEARCH i USING INTEGER PRIMARY KEY (rowid=?)
LIST SUBQUERY 1
SCAN image_cache
`image_id` is the cache's primary key, so each image is in the list at most
once and the count is the one the EXISTS gave (254). After: 0.05 ms. On a
library whose every original is cached this is as much work as before,
which is the proportion the rule asks for.
The count stays on the keystroke path: dropping it there would leave the
chip stale after a background download until something else refreshed it,
and at this cost there is nothing left to save. catalog_bench spells the
query as dr-ui does, so its copy changes with it.
`label_histogram` runs on every label keystroke and after every batch of
judgements is saved. On the reference library it cost 7.2-8.4 ms best-of-50
by catalog_bench (13 ms on a busy machine), to report that none of 23,500
images carried a label.
The join was the rating histogram's, with one thing worse: the index does
not carry `label`, so each probe went on to read the version's row.
SCAN i USING COVERING INDEX images_folder
SEARCH v USING INDEX versions_judgement (image_id=?) LEFT-JOIN
USE TEMP B-TREE FOR GROUP BY
It now takes the rating histogram's shape: only labelled default versions
are grouped, and the unlabelled slot is what is left of `judged_rows`.
SCAN versions USING INDEX versions_judgement
USE TEMP B-TREE FOR GROUP BY (the labelled rows only)
That pass still reads each default version's row for `label`, but in the
index's order, which follows the table's; a partial index on the labelled
rows would make it index-only, and was not worth a new index for the
remaining 1 ms. After: 1.7-2.0 ms, the same answer on the reference library,
and a test that compares it with the old join over the awkward states the
rating test uses (a second default's label counted, unknown codes and zero
folded into unlabelled).
`rating_histogram` runs on every star keystroke. On the reference library
(24k images, 1,200 of them rated) it cost 6.3 ms best-of-50 by
catalog_bench, and up to 10-14 ms when the machine is busy.
It was `images LEFT JOIN versions ON ... AND is_default = 1 GROUP BY
rating`. The plan:
SCAN i USING COVERING INDEX images_folder
SEARCH v USING COVERING INDEX versions_judgement (image_id=?) LEFT-JOIN
USE TEMP B-TREE FOR GROUP BY
A probe of the index per image, then a sort of all 23,500 rows, to put
22,000 of them in slot zero.
Now the rated rows are grouped on their own (`rating != 0`: one pass over
`versions_judgement`, a sort of 1,200 rows), and slot zero is what is left
of the join's row count. That count is three index-only aggregates -- the
library size, the default versions, and the images holding one -- so an
image with no version is still unrated, and an image with two default
versions still counts twice, exactly as the join counted it:
SCAN versions USING COVERING INDEX versions_judgement (x3)
SCAN images USING COVERING INDEX images_folder
`count(DISTINCT image_id)` has its own statement because alone it reads the
distinct values off the index order; beside other aggregates SQLite builds a
temporary b-tree for it.
After: 1.2 ms. The histogram is the same on the reference library
([22364, 663, 19, 47, 115, 374]), and a new test compares it with the old
join on a catalog holding every state the schema allows: no version, only a
virtual copy, two defaults, ratings below zero and above five.
Issue #75 lists catalog reads paid on interactive paths rather than once:
the rating and label chip counts on every judgement keystroke, the "On this
device" count beside them, the keyword panel's vocabulary on every
selection change, and the grid's total on every scroll reload. catalog_bench
now times each of them against a real catalog and prints their answers, so
a change to any of them can be checked for giving the same numbers.
Two of them live in dr-ui's private `library` module; their SQL is spelled
in the bench as it is spelled there, which the module comment says.
Reference library (24k images), best of 50, CPU, on a loaded machine:
rating_histogram 9.0 ms, local_original_count 2.0, label_histogram 10.0,
keywords::list 4.0, keywords::for_images 5.0, grid count 1.9, grid count
with a one-star filter 4.0.
catalog.md §6 still described the design of 2026-08-09, where the grid
enqueued Thumbnail jobs at Interactive and a runner drained them. It was
never built that way: the grid asks a worker directly and the sweep's
work list is what the thumbnail store lacks. The one enqueue that did
exist fed a queue nobody claimed (#73).
§6.1 now states the decision and its evidence: the store is shared
between devices and is the only record that knows a thumbnail exists,
metadata is owed through metadata_state the same way, the retired rows
are dropped at open rather than by a migration so no older device loses
the synced catalog, and every_queued_kind_has_a_consumer holds the rule.
§6.3 notes that the priority ordering is had without the queue.
outstanding.md's FR-PLAT-AND-4 paragraph said the scan's thumbnail jobs
were the one reachable enqueue; it now says nothing enqueues, and that
feeding the runner means a handler and its enqueue in the same change.
Refs #73
The queue coalesces, so a producer with no consumer never fails: it
leaves one row per subject for ever. That is how 23,582 Thumbnail jobs
accumulated unnoticed (#73), and nothing at runtime would have said so.
every_queued_kind_has_a_consumer reads the shipping sources of every
crate under core/, ui/, apps/ and platform/ (cfg(test) items dropped)
and pairs the JobKind named at each enqueue( call with the kinds named
in a fn kinds( body or a claim_next_matching( call. An enqueue that does
not spell its kind is refused, since the pairing could not be checked.
It guards against passing over nothing: the queue's own files and the
scan must have been read. A second test runs the reader over fixed
snippets so a parsing bug shows up as a failure. Run against master's
scan.rs and walk.rs it names all three orphan enqueues.
Refs #73
Stopping the enqueue leaves the rows already queued: 23,582 on the
reference catalog, about 1 MB of table and indexes that every query over
jobs pays for.
A migration would be the usual tool and is the wrong one here. A schema
bump makes an older build refuse the synced catalog snapshot, and the
tablet is on 0.16.0. So the rows are dropped at runtime instead, by
jobs::drop_retired over a new JobKind::RETIRED list, from runner::recover
- which already runs exactly once per catalog open, before any worker.
It runs every open rather than once because an older build sharing the
catalog queues them again on its next scan. kind leads the
UNIQUE(kind, subject_id) index, so with nothing left it is one index
probe. Measured on a copy of the reference catalog: 23,582 rows dropped
in 40 ms on the first open, 0.07 ms after.
Thumbnail stays in the enum so its number is never reused for a kind
that would then inherit old rows. The runner tests that call recover
move to a live kind; the jobs.rs tests of queue mechanics never call
it and are unchanged.
Refs #73
walk::scan_root enqueued an ExtractMetadata and a Thumbnail job for every
image it inserted or found changed. No handler claims either kind. The
walk is only reachable from the scan_local example today, so no real
catalog holds these rows, but it is the same leftover the remote scan
carried (#73) and it is what a local library would inherit.
Both debts are already recorded where their consumers look: an inserted
or changed image is written at metadata_state 1, which is the metadata
sweep's work list, and the thumbnail store answers for itself.
The tests that used job rows as the measure of "this image owes work"
now read metadata_state, which is the record the sweep actually uses;
the no-requeue test marks the first image read before the second scan,
so it still proves an unchanged neighbour is not put back in debt.
Refs #73
The reference catalog held 23,582 Thumbnail jobs, one per image, and
every scan re-coalesced all of them. Nothing has ever claimed that kind:
no JobHandler is registered for it on desktop or Android, and
dr_catalog::sync never merges another device's jobs in.
Thumbnails are owed by the store, not the queue. The grid's worker and
the thumbnail sweep both find their work by asking ThumbStore what it
lacks, and the store is shared between devices, so it is the only record
that knows another device already made one. A queue row was a second,
staler copy of that debt that grew with the library and was read by
nothing.
persist still writes the images and their remote identities in the one
transaction; it just no longer adds a row to jobs for each of them. The
two tests that asserted the rows existed become one that asserts a
repeated scan queues nothing.
Refs #73
The download fix renamed the canvas gates to root.has-photo, and the
canvas-order test still searched for the old spelling, so it panicked
before checking anything. The order it guards is unchanged.
The develop view reported a remote original on its way through the
error message, so it read "Could not load image" over "Downloading…".
It did so on every step along the roll, including a cached frame that
was ready within a tick, so each step flashed the error.
Waiting is now its own state. On the step, the grid's thumbnail of the
photograph stands in at once. Only when a transfer is really on the
wire does it dim under "Not on this device yet", with a line like
"Downloading — 12.4 of 38.0 MB" and a progress bar.
The bytes come from a new RemoteBackend::get_reporting. The Nextcloud
backend overrides it to read the body chunk by chunk; the default
reports once at the end. Progress is kept in the in-flight registry by
path, because a step usually lands on a frame the prefetcher is already
fetching. The catalog's file length stands in when the server sends no
Content-Length.
Opening a photograph from the library starts a download and a timer that
polls for it. Every step along the roll started another, and each one put
its result on screen when it landed, so a frame stepped past earlier
could arrive last and replace the one whose name was showing. Each open
now takes a generation number; a download that lands for an older
generation is recorded in the activity list (its bytes are cached) and
goes no further.
The outgoing session also stayed live until the new download landed.
Its sliders kept working, and a second step before the first landed
saved that session's edit under the new photograph's identity. The
session is now dropped as soon as its edit is saved.
The manual described the review (7c9a4ee) without a picture, because the
demo library holds no duplicates. The new duplicates scene makes two:
it copies two New York frames into a bck folder beside their own,
restarts the app so the scan finds them, waits for the sidebar row the
sweep's dating brings (54aee50), opens the review from it, presses
Check and takes the page. It deletes the copies and restarts on the
library as it was; record.sh's snapshot restore would remove them too.
It is registered last, so no other scene sees the copies.
The picture shows both groups proved the same file, the camera-named
copy outside bck marked Stays, and "Move 2 copies to trash" ready.
Every scene was recorded again on a release build of this commit with
the automation feature. Master changed what nearly every picture shows
after they were taken: scrollbars on the develop column, the grid, the
sidebar and Settings; a "?" beside Settings in develop's top bar, with
its controls regrouped; the Film row opening its list as a popup.
The film scene pressed the list's rows through the develop column, which
no longer holds them: the list is a popup, and the automation hook
reports its contents relative to it. The scene now opens it with
film_list_open, turns the wheel down it and back so the popup and its
scrollbar are seen scrolling, and clicks Velvia at its popup position
plus the popup's origin. The caption says so. film_reach passed in the
same run: the last stock was reached by the wheel, a drag, the scrollbar
and the keys.
Looked at as contact sheets of each GIF's middle and last frames and
each PNG. launch, launch-folder, library-nesting.png and
library-collection-menu came out byte-identical after optipng and are
unchanged. develop-zoom's deepest frames are smooth on Xvfb as before;
its caption does not claim blocks. Settings shows version 0.15.0,
the build's own, until the release commit bumps it.
storage.md's BackendProvider listing and its notes predated #65: the
trait gained upgrade_endpoint (core/dr-sync/src/provider.rs:95), run at
launch by AccountStore::upgrade_endpoints to move an http:// account to
https:// with its keyring entry, and the Nextcloud client refuses plain
http below every URL it sends (adade27, ea31791, 5569a06). The listing
gains the method and a note says why it is not normalise_endpoint again.
catalog.md said content_hash is computed only for import duplicate
detection and reconnection, and left "the same image catalogued twice"
as unspecified. FR-CAT-11a now handles the within-root case: it proves a
group by content_hash where every copy has one, otherwise by first and
last megabyte digests kept in dedup_probes, a table created on first use
rather than by migration (core/dr-catalog/src/duplicates.rs:296). The
cross-root case stays open, and the bullet now says which half is done.
The manual described what the pictures show and missed what this round
changed around them:
- Rating and flagging gave stars only. The keys (0-5, P, X, U), Flag on
the selection bar, judging in develop without moving on, and the flag
and stars on the roll's cells had no sentence.
- Nothing said the desktop draws scrollbars on the grid, the sidebar,
the develop column and Settings, or how the grid's bar is used.
- Nothing said how to open Help: the header's Help or F1, and in
develop the "?" beside Settings (d9f2596, 380cfda), with its See it
links and its Manual button.
- Stepping along the roll now goes on past what the roll has loaded
(a030bfd); Settings gained the duplicate originals line and the
Manual row.
index.html is regenerated from the README.
The index described the manual's contents as they were before colour
labels and the duplicates review, and did not say the application
carries the manual or where the in-app copy of the gesture book is
(Help or F1 in the grid, "?" or F1 in develop since d9f2596 and
380cfda). Its conventions named two generated files; manual/index.html
is a third, with its own CI check.
The requirement count is 191, as traceability.md's summary now reads;
the 84% it quotes still rounds from 83.8%.
"Not built" left out two things the matrix lists as untagged and the
outstanding register describes: importing a Lightroom or darktable
catalog (FR-CAT-14) and translations past the launch screen
(NFR-A11Y-1).
The feature paragraphs predated this round: the library now filters and
sets colour labels and consolidates duplicate originals, develop corrects
converging verticals and intersects mask parts, and the keyboard
vocabulary, the help sheet (F1, or "?" in develop) and the bundled
manual it links into had no mention. The version line is left for the
release commit.
CONTRIBUTING named four CI commands and the traceability tag, and
nothing about the three checks added since 0.14.1 that a first UI change
is most likely to meet: gestures-check, which fails on a key bound
without a GESTURE block or a block naming an unbound key (9d1e31f);
manual-check, which fails when index.html is not the render of the
manual (1021635); and record.sh --check, which fails when the manual
shows a picture no scene makes (70583b9). Two short paragraphs say what
each holds and where the recording tools are.
distribution.md's list of what every channel must get right had the
face models as the one LFS trap. Since 0.15.0 (d8f26fb) the Arch
package, the Windows installer and the APK also carry the rendered
manual and its pictures, and refuse LFS pointers for the same reason;
the Flatpak manifest does not install it.
The Vulkan bullet still called NFR-R8 open. It was decided on
2026-09-19: no CPU pipeline, and a viewer on embedded previews with
develop and export withheld.
§3.2 sketches a RawDecoder over a seekable reader, and the crate map
named it. FR-RAW-2 was built in 0.15.0 as dr_decode::Decoder over bytes
(core/dr-decode/src/decoder.rs:33): the decoder states how much it needs
and where its preview is, and the storage layer fetches it. The sketch
stays as the argument for four entry points; a note under it says what
shipped, and the crate map uses the built name.
display-and-extension.md still called FR-DSP-4 absent, and
frame-budget.md ended its reading at "satisfied vacuously". 0.15.0 built
it (refine.rs, the provisional histogram, the fade), so the table row
says so, frame-budget.md gains a note under its FR-DSP-4 section, and the
register gains a status note like FR-RAW-2's and FR-UI-5's.
frame-budget.md also gains a short section with the figures 1dc7b45
(the fit view's gather cached per framing) and d430ec9 (an identity
detail pass dropped) measured, since its fit rows no longer describe the
fused path. They are quoted from those commits, on the machine they name,
and are marked as not re-run here.
Five things in it were no longer true of the tree:
- FR-DSP-4 said "Unbuilt" and that nothing tracked a provisional frame.
0.15.0 finished it: the settle debounce in ui/dr-ui/src/refine.rs,
the draft flag reaching the histogram (canvas-draft,
Levels.provisional), and the 150 ms fade of the last draft
(app.slint's canvas-previous).
- The note that dedup.rs is re-import detection stood alone. FR-CAT-11a
is now built beside it: dr_catalog::duplicates, dr_ui::duplicates and
the Duplicate originals review.
- Culling counted three unbuilt clauses and named two.
- Section 6 still counted zero @tr( and five accessible-* lines, figures
from before 2026-08-30. The launch screen is converted (build.rs holds
the mechanism, no .po exists), 127 accessible-* lines sit across eleven
files, and ui_controls_are_accessible.rs holds the structure.
- Section 11 said nothing of the panorama was built. It was, the same
week; FR-MRG-9, NFR-MRG-1 and NFR-MRG-2 are what carry no tag.
A new section 4a lists what the develop, mask and keyboard work left
open, so that its closed clauses are not looked for: FR-UI-5's wheel on
sliders, FR-RAW-2's second decoder, mask-editing M2's remainder, and
FR-DEV-17's deliberate blind spot for range and region parts. The
Android paragraph now says develop is zero-copy there since TD-1 was
paid off, and that the APK carries the manual.
The timeline beside the grid says *when* the view is. It does not say
how far through the library the view is, and scrubbing it jumps by
date. The bar gives the plain desktop answer: a thumb in proportion to
one screen of the whole grid, dragged or paged by clicking the track.
It reads the viewport that the rows are counted into, so it spans every
photograph and not only the loaded window of cells.
The Flickable keeps its id, its `interactive` arbitration with the
hold-to-pick-up and its pinch and zoom catchers. It now fills a
Rectangle that takes its place, and its stretch, in the grid's layout.
The gesture book runs to several screens inside a card that otherwise
looks complete. Same wrapping as the develop column. The bar is drawn
over the list's right edge, so it overlaps the last few pixels of each
"See it" button.
The page is several screens long, and nothing said so until something
had been scrolled. Same wrapping as the develop column; the bar sits at
the window's right edge, clear of the capped 680px form.
A tree longer than the panel looked, to a mouse, like the whole tree.
Same shape as the develop column: the Flickable fills a Rectangle that
takes its stretch, and a ScrollBar is drawn over its right edge. It is
drawn only when the tree overflows.
The column (histogram, compose, every adjustment group, history) runs
several screens past the window. A mouse without a wheel could only drag
the column's contents, and a drag that starts on a slider moves the
slider.
The Flickable now sits in a plain Rectangle with a ScrollBar beside it,
bound to its viewport. Nothing about how the column sizes itself
changes: the Flickable fills the Rectangle, and the Rectangle takes the
stretch the Flickable had in the layout under the group strip. The bar
is drawn over the column's right-hand 10px, so the mandated panel width
is not reduced. When the pointer is on the bar, the lit track covers
the value labels of the compose rows, which already sit flush against
the column's edge.
The film list bug gave no failure anywhere: the data was right, the
markup compiled, and the list rendered. Two guards now check that the
list can be walked to its end, both by driving input rather than by
reading markup.
tests/film_list_reaches_every_stock.rs runs in CI and needs no display.
It builds the real AppWindow on Slint's testing backend and gives it 28
stocks. It dispatches window events through the same routing a window
uses: popup, Flickables, arbitration. It then checks that the last stock
is on screen, that is, not clipped away:
- after Down past the end, and that Enter chooses it;
- after drags on the list;
- after a run of wheel events with a still pointer;
- after dragging the scrollbar thumb.
Element queries need the Slint compiler's debug tables, which build.rs
emitted only for the `automation` feature. It now emits them for every
debug build too. Release builds, the ones that ship, are unchanged. The
testing backend is a dev-dependency at the same pinned version the
automation feature already uses, so no new crate enters the lockfile.
The test is compiled out of release test runs.
film_reach in tools/manual/scenes.py is the same check on the recording
rig: a real X pointer from xdotool, the release build, and the demo
library. It makes no picture, so it adds nothing to the manual. It runs
with every recording, or alone with `record.sh LIBRARY film_reach`, and
fails the run if the last stock (Ilford HP5 Plus) is out of reach by
the wheel, a drag, the scrollbar or the keys.
Both have to add the popup's position back. The testing backend reports
anything inside a popup relative to the popup, and so does the
automation hook built on it. They take the popup's position from the
Film row and Slint's clamp into the window.
A list cut off at its edge looks, to a mouse, like a list that ends
there. The open film list shows ten rows of twenty-eight, and nothing
said there were more. The user asked for visible scrollbars on every
platform but Android.
ScrollBar (widgets.slint) is a vertical bar drawn over a Flickable's
right-hand edge. It shows the share on screen and the position, it can
be dragged by the thumb from wherever it was grabbed, a click on the
track moves a page toward the click, and the wheel over it scrolls. It
is the Flickable's sibling rather than a wrapper, bound to
`viewport-y <=> flick.viewport-y` and the two heights, so a scroller
keeps its own sizing. It is drawn over the content rather than beside
it, so the mandated column widths are not reduced. With nothing to
scroll it is not drawn and takes no input.
Whether to draw it is Scrolling.bars, which Rust sets from
dr_plat::is_touch_first(). That is the same function that puts the
develop groups in the rail, and it answers the same question: what is
the user pointing with? On Android, lists still scroll by flick only.
Placing the bar inside another scroller would put it back in that
scroller's arbitration. The film list can have one because it is now a
popup.
The open stock list showed ten rows, None to Kodak Kodachrome 64, and
the other eighteen - all seven black-and-white stocks among them - could
not be reached. The data was whole; the list could not be scrolled.
Reproduced on the manual rig (Xvfb, xdotool, the automation hook):
- a drag on the list scrolled the develop column, never the list;
- a wheel run over the list scrolled the column past it, whenever the
column had scrolled under that pointer in the last 800 ms - which is
how the list is reached, by wheeling the column down to it. After a
pause and a pointer move the wheel did reach the list;
- no key did anything.
The cause is Slint's routing, not the list. Since 2d878c2 the list was a
Flickable inside the develop column's Flickable, and Slint offers every
pointer event to the outermost Flickable first
(input_event_filter_before_children, i-slint-core 1.17.1 flickable.rs).
The column holds a press back (DelayForwarding) and intercepts the first
move past 8 px on the axis it can scroll, so the list never saw a drag.
For the wheel it intercepts while its own last wheel event is under
800 ms old and within 2 px, and always for a touchpad gesture that opens
with TouchPhase::Started - so on a touchpad the list could get no wheel
at all.
The list is now a PopupWindow under the Film row. A popup is its own
item tree: while it is open, events go to it and to nothing beneath it,
so the list scrolls by wheel, drag and flick however the panel is nested
and whatever the column did last. The alternative, standing the column
down while the pointer is over the list (the sliders' hover trick), fixes
the drag but not the wheel - `interactive: false` does not gate wheel
interception - so it would have left the bug for touchpad users.
The reason 2d878c2 bounded the list still holds: it is at most 320 px
and never lengthens the column, and now it covers the sliders instead of
pushing them down. The column cannot be scrolled while it is open, which
suits a one-click question; it closes on choosing, on Escape or Back, or
on a press outside it.
Keys, with the list open: Up and Down move along it from the chosen
stock and scroll it into view, Enter chooses, Escape or Back closes it
unchanged. A popup is its own focus tree, so the keys are taken when it
opens and Slint returns focus to the develop view's scope when it closes.
The Film row gains a button role, so a screen reader and the automation
hook can name it.
FR-CAT-11a records what #67 built: groups of the same file listed on
demand, proved the same before anything moves, merged onto one survivor
and the rest trashed a group at a time. The manual's new section under
The library says how to reach the page, how the copy that stays is
chosen, what the check reads and what merges.
A copy becomes a duplicate only once its capture time is read, and on a
fresh library that is the sweep, not the scan: the sidebar row stayed
hidden until the next launch. The count is refreshed when a sweep that
dated anything finishes.
Also says on a group left out of the plan that nothing will move, drops
an unused method, and names the review's completion callback type for
clippy.
"Duplicate originals" appears under the trash in the collections
sidebar while the catalog holds any, and Settings says how many there
are beside the other whole-library passes. Both open one page: every
group with its picture and paths, the copy that stays (tap another path
to change it), a per-group Include box, what the survivor will gain and
any flag, label or face conflict, and why a group was skipped.
The summary is the dry run -- "N groups, M files to trash, K skipped" --
and nothing moves until "Check" has read the copies and "Move M copies
to trash" is pressed. Both run on workers with progress on the page, in
the activity register and, for the move, on the library status line;
Stop ends a job between groups. When it ends the grid, the sidebar and
the trash are refreshed and the survivors' judgements are written to
their sidecars and XMP the way a rating keystroke writes them.
The page is paginated at 30 groups, so a redraw decodes 30 thumbnails
and previews 30 merges whatever the size of the library. Back and
Escape leave it like its own Back button.
dr_ui::duplicates is the half of #67 that touches files. The check reads
each copy's first and last megabyte by range through the backend and
hashes them (or compares stored content hashes where every copy has one),
keeps the probes in the catalog, and reads each copy's sidecar: a group
whose bytes differ, whose copies cannot be read, or whose develop edits
differ is left out and the review says why.
Consolidating a group carries the one edit onto the survivor's sidecar
where it has none, moves the other copies into the trash, and then
commits dr_catalog::duplicates::consolidate. A failure after the first
move puts the files and the sidecar back; a run that died between the
moves and the commit is finished by the next one, which finds each moved
file at its trash path.
Tested end to end on a folder library of real files: the copies land in
.darkroom-trash, the skipped groups are untouched, the edit reaches the
survivor, a catalog failure moves everything back, and restore returns
the copies byte for byte.
The library holds the same RAW in several folders: a dated folder, a
bck/ beside it, a renamed Darktable export tree. dr_catalog::duplicates
is the catalog half of consolidating them (#67).
candidates() is one grouped query over root, camera, capture instant and
size, joined back for the rows; count() is the same grouping under
COUNT. On a copy of the reference catalog (23,582 images) both take
10-35 ms and find 1,836 groups holding 3,379 spare copies.
survivor() prefers a copy outside a backup-looking folder, then one
still named the way the camera named it, then the oldest, then the
lowest id.
consolidate() re-checks the plan against the catalog, merges the copies'
judgements onto the survivor (highest rating, keywords unioned,
collections unioned with the survivor keeping its place, a flag or label
the copies agree on, faces via faces::carry_onto_copy) and records the
copies as trashed, all in one transaction, so a failure part way leaves
the group untouched. preview() runs the same code and rolls it back.
Sameness probes are kept in dedup_probes, created on first use rather
than by a migration: a schema bump would make older builds refuse this
catalog's snapshot at sync. trash::record_trashed_within lets the trash
write share the merge's transaction.
Capture sharpening at a scale too coarse to draw its radius emits one pass
with an empty body (`nothing_to_sharpen`), so that a chain still ends in
something that performs the output transform. At fit on any modern sensor
that is most of the time. When another neighbourhood operation follows it
- dehaze, clarity, texture - the pass is not last and does nothing: it
reads the rgba16float intermediate and writes the same texels to the other
one. It still cost a full render-sized read and write every frame:
scene before after
sharpen+clarity 2560x1600 fit 18.66 ms 14.16 ms
sharpen+clarity 3840x2160 fit 37.93 ms 27.88 ms
every operation 2560x1600 fit 71.80 ms 67.24 ms
clarity alone 2560x1600 fit 14.23 ms 14.20 ms (control)
sharpen+clarity 2560x1600 1:1 23.01 ms 22.97 ms (control: resolves)
(Laptop RTX 3050 held at 420/810 MHz by its power cap, synthetic 60 MP
source, median of five alternated runs of forty frames each.)
`compose_detail_with` now drops such a pass where dropping it is exact:
not the last pass, whose output transform would otherwise move onto the
previous pass's f32 result and round differently; and not a pass right
after a reduced one, because a full-resolution pass is what closes the
reduced chain for the operation after it. `DetailPass::is_identity` says
what "changes nothing" means: full size, nothing bound at binding 3, and a
body with no code in it.
The rgba8 output is bit-identical: every scene above hashed the same
before and after, and the sharpen+clarity frame hashes the same as clarity
on its own, which is the claim in one line. A new dr-pipeline test pins
the three cases - dropped ahead of another operation, kept when last, kept
when alone.
At fit, every output pixel of the fused pass loads one texel from a source
three or four times its width, on a stride. The memory system fetches the
texels it skips along with the one it wanted, so on a 60 MP rgba16float
source that gather was most of what the fused pass cost: 10.6 ms of a
2560x1600 frame against 3.8 ms for the same shader reading a contiguous
window (the 1:1 view). At 3840x2160 it was 21.1 ms. Those are the laptop
RTX 3050 with its clocks held at 420/810 MHz by the power cap; unthrottled
the same frames were about 2.0 and 3.2 ms, and the gather is the same
share of them.
Which texel an output pixel reads depends only on the framing prologue,
the framing and warp uniforms, the source and the render size. None of
those move during a slider drag, so the gather is the same work every
frame. The fused shader now takes a render-sized rgba16float cache of it
(bindings 6 and 7, declared in every generated shader like the masks) and
a pair of uniform flags: write what was gathered, or read it back at the
pixel's own coordinate. AdjustPass keeps the cache and decides per
dispatch. The composer supplies `ComposedShader::sample_key`, a hash of
the prologue and those uniforms, and AdjustPass adds the image and the
size; an image gets a process-unique id for this rather than being held
alive by the key.
The picture is bit-for-bit the same. The source is rgba16float and so is
the cache, so the stored texel is the texel, and only the path that reads
a texel whole takes part: an interpolated sample (straightening, lens
warps, CA) is a blend that f16 could not hold exactly, so the composer
gives it no key and it reads directly as before.
The cache is written on the second frame with a given key, not the first:
a crop or zoom drag changes the key every frame, and writing then would
add a render-sized write to exactly the gestures that can afford it least.
It is kept only up to 3840x2400, so an export never parks a full-frame
copy on the device, and `release_caches` drops it.
Measured with a scratch probe rendering the synthetic 60 MP frame from
examples/frame_budget.rs, forty frames per run after six warm-up, five
runs of each binary alternated, median of the per-run p50 (GPU idle apart
from the power cap):
scene before after
neutral 2560x1600 fit 10.62 ms 3.88 ms
exposure 2560x1600 fit 10.83 ms 3.87 ms
nr chroma 2560x1600 fit 19.84 ms 12.69 ms
neutral 3840x2160 fit 21.05 ms 7.11 ms
exposure 3840x2160 fit 21.08 ms 6.94 ms
clarity 3840x2160 fit 42.20 ms 27.88 ms
neutral 2560x1600 1:1 3.83 ms 3.84 ms (control: nothing to gain)
The rgba8 output of every scene hashed identically before and after, in
isolated runs and across all 38 scene/size/view combinations of the
probe. New tests walk a pass through direct, write and read frames, a
slider move, a neighbourhood operation and a framing change, and compare
every frame with a fresh pass that can only have read directly.
Rating and flagging in develop (FR-UI-5, amended 2026-09-19) landed with
the keyboard audit: 0-5, P, X and U in develop's key scope, stars and
Pick/Reject in its top bar, and flag and stars on the roll's cells. Two
of the amendment's rules were held by nothing. The keys must judge the
photograph on screen and never a selection left behind in the grid, and
judging must not move on to the next frame, which is culling's
auto-advance and not develop's.
The rating, flag and label callbacks that take a row each spelled the
row-to-image lookup themselves. It is now one function, `image_at_row`,
which answers None for a negative row as well as one past the end: the
roll passes -1 when the open photograph is outside the loaded window,
and the right answer then is to judge nothing. A test says so. The key
bindings live in Slint, where no test can press them, so a second test
reads develop's handler as the gestures gate and the canvas-order test
do, and checks that each judgement key calls the row callback on
`library-roll-current` and that none of them steps the roll or the
cursor.
The writes themselves go through `apply_judgement`, the grid's own path:
one catalog statement, then the sidecar and XMP writes behind it.
Adding a labelled Help button to develop's top bar made the strip about
100 pixels wider than a 1600-pixel window. The strip scrolls, so nothing
became unreachable, but Settings was off the right-hand end until the
bar was dragged.
Help is now a square IconButton with a drawn question mark (a new "help"
icon, drawn rather than typed for the reason icons.slint gives about
Android's fonts), and screen readers still hear it as "Help". That saves
60 pixels, which was not enough alone: the strip had fit with 3 to spare
before. The rest comes from the spacing. Controls that belong together
now sit in groups at gap-sm with gap between groups: Pick and Reject,
Undo and Redo, Copy, Paste and Presets, and "?" and Settings. The empty
export-status caption no longer takes a slot and two spacings while it
has nothing to say. At 1600 wide with the panel open the whole bar now
shows, Settings included.
The grid's header keeps its worded Help button; it has room for it.
The develop "Open this list" gesture now names the "?" button, and the
gesture book is regenerated from it.
The "Controls and shortcuts" sheet was drawn by LibraryGrid, so only the
grid's Help button and its F1 could open it. Develop, where most of the
keys it lists are bound (Ctrl+E, Ctrl+Shift+C, A/D, Z, R, H, [ ]), had no
way to it: a photographer who wanted to look a shortcut up had to leave
the photograph they wanted it for.
The sheet now hangs off the shell beside the export and copy sheets, on a
`help-open` property both views set. The grid's Help button and F1 raise
it through a callback, and its keys stand down through the `sheet-open`
they already honour for the export sheet, so Escape falls through to the
shell, which closes it. Develop gains a Help button beside Settings in
its top bar, as in the library header, and F1 in its key scope; its keys
decline while the sheet is up, as they do for the other two sheets, so
nothing behind it is rated or stepped. The book already begins with the
Develop section, so from develop it opens where the reader wants it.
Tests hold the shape: the sheet is drawn by the shell and not the grid,
and develop's opening guard names all three sheets and its F1 opens this
one. The new GESTURE: block puts the develop route in the book.
Holding D in develop across the edge of the loaded window panicked with
"RefCell already borrowed" at the first step that had to move it. The
`*ctl.offset.borrow()` written inside the `if let` condition is a
temporary that lives to the end of the `if let` block (edition 2021),
and the block calls `load_window`, which borrows the offset mutably.
The unit tests drive the placement arithmetic, not the RefCells, so
they could not see it; stepping 400 frames in the app did.
The offset is copied out before the test. `follow_open` gets the same
treatment for `image_ids`: the lookup's result is bound first, so no
borrow is held while it writes properties back to the window.
In a library's develop view the arrows, space, A and D opened
`library-roll-pick(library-roll-current ± 1)`. `roll-current` is a row
of the loaded window, set when a photograph was opened and never again.
Two things followed on any library larger than one window:
- Holding D stopped dead at the end of the loaded window: the pick of a
row past `ctl.paths` found no path and did nothing, a few screenfuls
into a library of thousands. A stopped at its start the same way.
- Any reload of the window — a background sync, a judgement that drops
the open frame out of a filter — left `roll-current` on a row that now
held another photograph. The roll marked it, develop's rating keys
judged it, and the next step walked on from it.
The keys now call `library-roll-step(delta)`, and Rust steps as
`move_cursor` does in the grid: from the open photograph's ordinal
(`index`, which `report_position` already keeps), clamped to the
library, loading the window around the target only when it lies
outside. The window-bringing half of `place_cursor` is shared for this
as `bring_window_to`, so the grid cursor and the roll move the window
the same way. Catalog reads stay proportional to the window: one
`load_window` per window crossed, none per step within it.
The open photograph is also remembered by image id. Every
`load_window` finds it again in the rows it just read (a scan of one
window, no query), puts `roll-current` and `index` back on it, or takes
the mark off when it is not there. When it is missing but its ordinal
is still inside the window, it has left the grid and its successor
moved up into its place, so the next step forward lands on that ordinal
instead of skipping the successor.
A click on the roll still reports a row; it is right because the cells
it is drawn from and `ctl.paths` are replaced together, and it now goes
through the same open path as a step.
Fixes#64.
Three habits the pass found broken, in the style of the 2026-09-19 notes:
SQL text passed to `execute`/`query_row` in a loop is a prepare per row
(the merge, `persist` and the shard sync all paid it), a case-insensitive
`LIKE` cannot use the `source_ref` key where a range can, and the backfill
inside `Catalog::open` is paid by every worker thread, the develop view's
fetches included. And where the two new benches are and how to run them.
When a scan pull takes in sidecars another device wrote, `apply_judgement`
finds the photographs each one describes with
`source_ref LIKE '{stem}.%'`. SQLite's LIKE folds ASCII case, and nothing
indexes `source_ref` case-insensitively, so every lookup read all 24,000
names of the root through the `(root_id, source_ref)` index: 1.5-2 ms per
sidecar, 520-690 ms for the 342 `.drsc` files the reference catalog has
read. Another device culling a shoot is several hundred of them.
Every name beginning `{stem}.` lies in the half-open range
`[{stem}., {stem}/)` -- `/` is the byte after `.` -- which the unique key
serves as a seek. The rows LIKE matched beyond these differed only in
case, and the check that decides, `sidecar_path(source) == sidecar`, has
always compared exactly and refused them; the escaping of `%` and `_`
goes too, since a range has no wildcards.
persist_bench: 342 lookups 634-691 ms -> 9 ms CPU. Checked against the
reference catalog directly as well: for all 18,430 distinct sidecar
names its images imply, the range and the old LIKE, each filtered by
`sidecar_path`, pick the same photographs. A test pins the neighbours of
the range: a case variant, a longer stem, a subfolder named like the
stem, and a folder whose name holds `%` and `_`.
The XMP reader's LIKE (`xmp_sync::images_for`) is left alone: its check
is case-insensitive, so the range would not be a superset there, and it
only runs when the exact darktable-style name is not found.
`persist` runs after every scan, for every photograph the scan listed. On
a settled library that is the folders whose ETag changed -- a sidecar
written there by a rating is enough -- so one relisted folder of 1,600
images is an ordinary pass, and a first scan is all 24,000.
Per photograph it prepared four statements from their SQL (a folder
lookup, the image upsert, the id read-back, the remote upsert) and then,
after the commit, found the image again by path and enqueued its
thumbnail job as an autocommitting statement of its own -- a commit per
photograph, for rows that were almost all already queued.
Now the statements are prepared once per pass, a folder's id is looked up
once per folder rather than once per photograph in it, and the job is
enqueued inside the transaction with the id already in hand. That also
makes the job atomic with the row it points at, which is what the old
ordering after the commit was trying to guarantee. `jobs::enqueue` uses a
cached statement for the same reason.
persist_bench on a copy of the reference catalog, CPU, best of runs:
largest folder (1,589 images) 102-118 ms -> 10-13 ms
whole library (23,582 images) 1.55-2.19 s -> 188-192 ms
The fingerprint of images, remote, jobs and folders after the run is the
same for both builds.
`adopt_orphan_terms` runs in the backfill on every catalog open. Its
check -- is there a vocabulary row for this word, tombstones included --
cannot use `keyword_terms_name`, which is partial on `deleted = 0`, so the
correlated subquery scanned the vocabulary once for each of the 10,800
assignment rows before `DISTINCT` threw the repeats away: 3.5 ms per open
on the reference library.
The distinct words are taken first and the check runs once per word -- a
few dozen scans of a few dozen rows. Same rows out, since `DISTINCT` over
the assignments is exactly the set of words.
catalog_bench, best of 20: 3.45 ms -> 0.30 ms.
`Catalog::open` runs the backfill every time, and every worker thread
opens its own catalog: the develop view does it to fetch each original and
again for each neighbour it prefetches, and the sync, sweep, burst and
thumbnail workers each do it too. On the reference library (24k images)
an open cost 26 ms of CPU, and most of it was `pair_raw_and_jpeg` reading
all 17,000 RAWs into a map of lowercased stems to find partners for the
1,900 JPEGs that have none -- the same 1,900 on every open.
It now starts from the small side. The unpaired JPEGs are read first, and
it stops there if there are none; otherwise it reads the RAWs in the
folders those JPEGs sit in (plus the unfiled ones when an unfiled JPEG is
waiting), which is 142 on the reference library. A pair is same-folder by
definition, so no pairing is lost; the RAWs are read in id order, so where
two share a stem the later one still wins as it did in the table scan; and
a pass with nothing to pair no longer opens and commits an empty write
transaction.
catalog_bench, best of 20, CPU: `Catalog::open` 26 ms -> 12 ms together
with the next commit (the backfill 24 ms -> 11 ms; this step is ~10 ms of
that). A test covers pairs found among other folders and unfiled images.
Every sync pass exports this device's faces to the shard store and imports
what peers sent, and both walked the whole library asking the store's index
about one image at a time: the export 19,000 `indexed_at` lookups (one per
face marker), the import 23,000 `held_model` lookups (one per image with a
server id), each a statement prepared and run against the index. With
nothing new either way -- the usual pass -- that was all they did.
Measured with catalog_bench against copies of the reference catalog and
face store, best of 5, CPU:
export_to_shards (steady) 119 ms -> 27 ms
import_from_shards (steady) 250 ms -> 87 ms
Each now reads the index in one statement into a map. The import's query
is `held_model`'s, ordered the same way, keeping the first row per file,
and nothing in the loop changes which pipeline a file is held under
(`set_indexed_at` touches only a file already decided; candidates are
distinct files). The export's `put_image_at` does rewrite entries -- but
only its own file's, its generation and the siblings it supersedes -- so a
file already written in this pass is asked of the store again, and every
other answer is the one the lookup would have given. An index that cannot
be read gives an empty map, which is what each failed lookup returned.
The store index and the catalog are identical after the old and new
builds' runs.
A sync pass that brought nothing new cost 450-540 ms of CPU in
`merge_remote_catalog` on the reference library (24k images, 19k faces),
measured by catalog_bench merging a copy of the catalog with itself.
Most of it was the loop over the other device's confirmed faces and the
faces under its ignored groups -- 13,000 rows. For each one it prepared
three statements from scratch (`query_row`/`execute` with a SQL string
compile the statement every call) and then rewrote the `face_person` row
with the values it already held, dirtying a page per face on every pass.
The rejection loop prepared three more per row.
The statements are now `prepare_cached`, the local assignment is read once
per face (whether it is confirmed, and what it holds, come from the same
row), and the upsert is skipped when the row already says exactly that.
`faces_assigned` is still counted for those rows, so the report is the one
the old code gave, and nothing else reads the difference: the row is
byte-for-byte what the upsert would have written.
After: 279 ms (best of 5, CPU), with every catalog table identical after
the run to the old build's.
Two benches for reading side by side before and after a change, against a
copy of a real catalog, in the manner of identity_bench:
- `dr-catalog --example catalog_bench CATALOG [FACES_DIR]` times
`Catalog::open` and the backfill inside it step by step, the upload
snapshot, a merge of the catalog with a copy of itself, and the face
shard export and import in the steady state where nothing is new.
- `persist_bench`, an ignored test in dr-ui's scan module because
`persist` and `apply_judgement` are private to it, replays the
catalog's own rows through `persist` (the largest folder, and the whole
library) and looks up every `.drsc` sidecar the catalog has read. It
works on a scratch copy and prints a fingerprint of what `persist` left,
so two builds can be shown to agree.
Both print best, median and CPU time; the CPU figure is the one to compare
while other builds share the machine.
Every TextRow drew a field and a hint but no name: "Filename template"
and "Destination" on the settings page and in the export sheet, the two
storage budgets, the export size fields. The label was there, painted
behind the entry, with a clipped "ws" of "Thumbnails and previews"
poking out beside the thumbnail field.
The field sits in a Rectangle so the row can dim it and watch it lose
focus, and that Rectangle took its height from `field.preferred-height`.
`Field` sets its own `height` outright and has no layout inside it, so
its preferred height is zero. The box was zero tall, the row around it
took its height from the unit label beside it, and the field, centred
on an empty box, sat half its own height above the row, on top of the
FieldRow. Segmented never showed it because its chips live in a layout
that reports a real height.
Reading `field.height` instead gives the box the height the field
actually draws at, so the row reserves it and the label sits above. The
note on `Field.label` that recorded the fault now records the trap.
"On desktop the develop view draws the compute pass's texture directly"
was the README's way of marking Android as the exception. Since TD-1 was
paid off in 0.15.0 the tablet draws the same texture, so the sentence
now names both.
"Where it stands" still named TD-1 as the one deliberate compromise worth
knowing about: the Android develop view reading its frame back through the
CPU because wgpu's swapchain tore in portrait. 0.15.0 paid that off — the
swapchain is pre-rotated and the frame reaches the compositor as a texture,
as on desktop, and architecture.md §6.1 now reads "No exceptions since
0.15.0". The release commit updated the version line above and left this
paragraph describing a state the release had just ended.
Every scene was recorded again on a build of this branch rebased onto the
keyboard work and TD-1, since nearly every scene depends on files those
changed: the develop top bar now carries a star strip and Pick/Reject, the
roll shows flags and stars, and the grid's selection bar gains Label and
Flag. `--changed` could not be trusted to find them, because the rebase
made each picture's commit newer than the sources it was recorded from.
The develop-zoom caption said the wheel goes on "until the pixels are
blocks". On Xvfb the deepest frames come out smooth even though the app
draws past 1:1 nearest-neighbour on a real display (confirmed by eye on
the desktop), and a GIF shrunk to 960 wide could not show 3-pixel blocks
anyway. The caption now says what the clip shows; the prose above it,
which describes what the app does, stays.
The rebase combined this branch's README additions with master's, and
index.html is generated from the README; manual-check passes on the
regenerated page.
The traceability job now runs `tools/manual/record.sh --check`: every
picture docs/manual/README.md shows must be made by a scene in
tools/manual/scenes.py, and every picture a scene makes must be shown.
It reads the two files and nothing else, so it needs no app, display or
LFS pull.
--changed now dates a scene by the newest commit among its pictures
rather than each picture alone. A scene that also makes a picture which
re-records byte for byte (panorama-aligned beside panorama.gif) no
longer stays listed for ever. A scene all of whose pictures come out
identical (launch) stays listed until one differs, which costs one
harmless re-run.
Four scenes, recorded and looked at frame by frame:
- library-labels: 6, 7, 8 and 9 over four New York frames, 7 again to
take one off, then the Green chip narrowing the grid and back.
- compose-perspective: two towers shot from below, stood upright with
Vertical at about +58, then held against Before.
- crop-orphan: a stroke in the top-left corner, a crop that leaves it
outside, the notice with Undo crop and Keep crop, and Undo crop.
- local-intersect: a linear gradient over the lower half, then
Intersect and two strokes that survive only where it is.
develop-zoom already ends on hard-edged pixels (previous commit). The
text added is a sentence or two under the existing headings, so the
other branch's structure and anchors are left alone.
scenes.py aimed every press at window pixels, and the develop column had
already moved under it: Compose now sits above Adjust, so the old
exposure coordinate lands on a straighten slider. Every scene now names
what it presses by its accessible label through the automation hook,
places points on the photograph relative to the canvas, and opens its
photographs by file name. Each starts from a known place and undoes what
it did, so one can be recorded alone; the few that continue another's
state name it, and running one runs that first into a scratch folder.
Each scene also declares the pictures it makes and the sources they
depend on. `record.sh --check` fails when the manual shows a picture no
scene makes, or a scene makes one it does not show; it reads two files.
`record.sh --changed` re-records the scenes whose sources, or own code,
changed since the commit that last touched their pictures. record.sh
builds with the automation feature, restores the library from
DR_LIBRARY_SNAPSHOT, starts from a fresh profile and pins inference to
the CPU; the launch screen is recorded from an empty profile of its own.
Re-recorded with the ported scenes, and looked at frame by frame. What
differs from the pictures they replace:
- develop, presets, settings, local, compose, film, wb, light: the
current develop column (Compose with Vertical and Horizontal above
Adjust, the Label button), otherwise the same moments.
- library pictures: the filter bar's colour-label chips; no collection
left over from an earlier run in the sidebar; library-selection is the
twelve alpine frames rather than eight of them and four New York ones.
- library-rating rates two frames nobody had rated, so the stars are set
and not cleared.
- develop-zoom goes on past 1:1 with the wheel and ends on the file's
pixels as hard-edged blocks.
- repair covers a real mark on the road, with a size that fits it; film
is shown on the Chinatown frame instead of the road.
- panorama tries Perspective, Spherical and Cylindrical before filling.
- launch, launch-folder and panorama-aligned came out byte-identical.
Every scene in tools/manual aimed at window pixels written in by hand, so
a panel that gained a row moved every slider under it and the recording
went on dragging where the slider used to be. The develop column has
already moved that way (Compose now sits above Adjust), and nothing said.
A build with the `automation` feature listens on the Unix socket named
by DR_AUTOMATION and answers where an element is: by its accessible
label, the name a screen reader reads, or by its markup id for the few
things that are not controls (the canvas, the crop rectangle). It uses
Slint's element queries, which need the compiler's debug tables, so the
feature also turns those on in build.rs. It only answers questions; the
input is still xdotool's real pointer. No default build has the feature,
and one that has it listens only when the variable is set.
drive.py gains click-on, drag-on, hold-on, wait-for, wait-gone, labels
and ids. The grid's cells are now named by their file, each rating star
by its value, the sidebar's + as "New collection", and the Adjust
heading's reset as "Reset all adjustments" - controls a screen reader
could not reach before either.
The pre-rotation patches landed in the previous four commits. This
records how the debt was paid, by a fourth route its own list missed:
patching wgpu-hal and Slint's Skia surface locally rather than waiting
for either upstream. It also updates architecture.md §1 and §6.1, which
still said Android draws with OpenGL behind a readback.
The verification is stated as what it was: the user found the
release-signed build clean on the tablet in portrait. No dumpsys
composition or bufferTransform readings were taken, because adb would
not hold the device that morning, and no frame times were measured.
With Skia drawing pre-rotated on wgpu's Vulkan swapchain, Android no
longer needs to draw with Skia over OpenGL, which was the only reason
the develop view read its frame back through memory (TD-1).
So `unstable-wgpu-29` moves back to the common slint dependency. The
android-activity backend then builds `SkiaRenderer::default_wgpu_29`,
and `shared_gpu` loses its Android arm. The one wgpu device is handed to
Slint through `BackendSelector::require_wgpu_29` on both platforms.
`slint::android::init_with_event_listener` runs before `dr_ui::run`, so
the selector reaches the Android adapter before its window exists.
`renderer-femtovg-wgpu` stays desktop-only, since Android has no FemtoVG.
The two `#[cfg(target_os = "android")]` readbacks in `develop::render`
(the frame through `export_pixels` and the focus overlay through
`read_overlay`) are gone. `read_overlay` stays for the tests that check
what the overlay marks.
Built for arm64 and release-signed. Not yet run on the tablet.
The other half of the wgpu-hal patch: that one lets a caller promise a
pre-rotated swapchain, and this is the caller keeping the promise.
On configure, `WGPUSurface` reads the surface's `currentTransform`,
sizes the swapchain in the panel's orientation (swapped for a quarter
turn), tells wgpu-hal to use that transform, and before each frame
concatenates the matching rotation onto the Skia canvas. Everything
Slint draws goes through that one matrix, so an imported wgpu texture is
rotated with the rest of the window. Input is not rotated, and must not
be, because Android delivers it in window coordinates.
Three details that would each have been a visible bug:
- `resize_event` compared the new size against the swapchain's. The
swapchain is transposed while a quarter turn is in effect, so the
comparison now uses the window's size, kept beside it.
- A half turn, landscape to reverse landscape, changes the transform
without resizing the window, and wgpu-hal hides the SUBOPTIMAL that
would report it. So the transform is re-read before every frame. That
costs one query into the native window.
- The item renderer snapped the origin to the pixel grid only when the
canvas matrix was a pure translation. Under a rotation that is never
true, so portrait would have lost pixel alignment everywhere. The check
now accepts right-angle rotations and flips without scaling.
The direction of each rotation follows the Vulkan spec's reading of
preTransform (the image is drawn already rotated clockwise by the
transform). It has not been confirmed on the device yet.
wgpu-hal creates every swapchain with `preTransform = IDENTITY` (#3345).
On a tablet whose panel is mounted landscape, a portrait window then
hands Android an unrotated buffer: SurfaceFlinger falls back to rotating
it on the GPU (composition CLIENT), and on this device those frames tear.
That is why Android draws with Skia over OpenGL today, and why the
develop view pays a readback (TD-1).
The field cannot just be set to `currentTransform` inside wgpu. It is a
promise that the image is already drawn rotated and sized in the panel's
orientation, and only the renderer above wgpu can keep it. So the patch
is the smallest thing that lets that renderer ask:
`vulkan::Surface::current_transform` reads the surface's transform, and
`set_pre_transform` makes the next swapchain use it. The default stays
IDENTITY, so desktop and any caller that does not opt in behave exactly
as upstream.
The Android develop view reads its frame back through memory (TD-1)
because wgpu's Vulkan swapchain never pre-rotates, and a portrait window
on this tablet's landscape panel then tears. The fix is a small patch to
each of these two crates, and this commit is only the ground it lands on:
both are byte-for-byte the crates.io sources the lockfile already
resolved, so the commits that follow are the patch and nothing else.
third_party/ is excluded from the workspace, or every path dependency
under the root would become a member and `--workspace` would test and
lint upstream code as ours. The README says how to carry the patches
across a Slint or wgpu bump, which matters because a stale version here
does not fail the build — cargo just warns and uses the unpatched crate.
FR-DEV-16's promise that the gesture book cannot describe a binding the
application lacks is now enforced by the gestures gate in both directions,
and every binding it names is bound and tagged, so it is marked met, with
what develop answers beyond the list.
FR-UI-5 is not met in full: its keyboard half and the 2026-09-19 amendment
are, but no develop slider takes the scroll wheel, so its status says that
rather than rounding up.
FR-UI-5's 2026-09-19 amendment asks for the rating and flag wherever they
can be set and on the roll's cells, so that stepping along a set in develop
shows what has been judged. The roll drew thumbnails only, so the keys that
now judge the open photograph left no trace on its neighbours.
Each roll cell carries a small badge with a tick or a cross and a star
count, drawn only when there is something to show, in shapes and a number
rather than colours (NFR-A11Y-3).
F1 opens the help sheet from the grid, and nothing on the keyboard closed
it: Escape fell through to the shell, and every other key went on judging,
labelling and keywording the photographs hidden behind the sheet.
Escape and Back now close the sheet first, ahead of the grid's other
sheets, since it is drawn over all of them. While it is open the grid's
handler declines every other key, so a stray P or Ctrl+K changes nothing
the reader cannot see.
dtourolle