Commit Graph
23 Commits
Author SHA1 Message Date
dtourolleandClaude Opus 5 34ac2f14d3 Sync faces and the catalog before thumbnails, so a fresh device sees its names and collections first
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 4s
Build and test / Desktop (Linux) (push) Failing after 35s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 30s
Traceability / Requirement traces (push) Failing after 48s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The thumbnail stage ran first and, on a device that had just adopted its
peers' shards, spent its time re-uploading hundreds of megabytes under its
own client id while faces, people, collections and dates waited behind it.
Faces go first — the catalog merge assigns identities to faces this device
holds — then the catalog, then thumbnails.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 21:04:10 +02:00
dtourolleandClaude Opus 5 f71d7bacc6 Take the server's shards and dates when the scan completes, not after the sweep
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 36s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 43m11s
Build and test / Windows (x86_64, cross) (push) Failing after 41m4s
The derived sync fired only after the metadata sweep, so a fresh device
re-derived every thumbnail it scrolled past, re-detected faces and re-read
every header for hours before adopting the shards and snapshot that held
all of it. It now fires as soon as the scan completes — the first moment
the rows the merges key on exist — and the sweep starts behind it. In
steady state that pass is one listing.

The catalog merge gains a fourth half: capture metadata (captured_at,
offset, camera, lens, ISO) for images still at metadata_state < 2, matched
by oc:fileid from a remote row at 2. A date is a fact about the file's
bytes, not local state, and the snapshot already carried it. The sweep's
per-chunk query then finds nothing left, and the timeline is whole on a
fresh device without a header fetch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-20 20:37:59 +02:00
dtourolle 7fba28f7d8 Upload a snapshot of a thumbnail shard, never the live file
Every shard is in WAL mode and every put opens its own connection, so
while thumbnails are being generated on several threads — which is when
the first sync pass runs — the log is never checkpointed and the main
file holds whatever the last quiet moment left in it. For a shard created
seconds earlier that is nothing: a zero-byte file with the schema still
in the log. The sync read that file and uploaded it, and every other
device merging it failed with "no such table: thumbs" on every pass.

Copy the shard through SQLite's backup API into scratch first, which
serialises against writers and carries the log, and upload that.
2026-09-20 00:21:14 +02:00
dtourolle 5c00942b84 One completeness job over a registry of repairs, and a re-index button
A library's records are never all complete at once. A face found before
its quality was kept has no quality; one found before the eye models
existed has no reading; one adopted from a peer's shard has no crop; an
image the fast detector examined on a 1024 px proxy has boxes the current
detector would not have drawn; an image the scan stat'ed has no capture
date. On the reference library that is 17,762 faces under the bare
w600k_mbf id with no quality, no reading and no dense landmarks, 4,144 of
them without a crop, beside 12,217 images the fast detector examined and
found nothing in. Every one of those gaps was its own pass — V14's
measuring pass, §17.5's eye pass, the sweep's proxy repair, the sweep's
detector upgrade — with its own work list, its own count and its own idea
of done, and adding a per-face field meant adding a pass. There was no
pass at all for the case the library is actually in: boxes and landmarks
drawn by a weaker detector on a proxy, which every later per-face pass
would have read from.

dr_ui::repairs replaces them with one job over a registry. A Repair names
one thing a record can lack — the predicate that says which images still
owe it, the input its handler needs (a header, the original, or a native
render), the handler, and what to record for an image that can never be
done. The job unions the predicates into one work list, fetches each
image once at the most any claimant asks for, renders it at most once,
and runs every handler whose predicate that image still matches, checked
again before each because a detection writes every field a per-face
handler would fill. The registry today: face-proxy, face-quality,
face-eyes, face-crop, face-detection, face-upgrade, metadata — the last
there to say that this is not a face job. Adding a field is one entry.

A repair's predicate is the only definition of its work: the count the
settings page shows, the list the job fetches and the check before its
handler run are one predicate, so the job converges. That is why the
registry is cut to what the device can do rather than listing what it
skips — an entry is a count and a set of originals to fetch — and why an
eye reading that cannot be cut is not a criterion.

The catalog side is generic to match: record_updates writes whichever
fields a FaceUpdate carries and re-marks the image so the shards export
it; faces_needing and count_needing answer a predicate the caller
supplies, replacing the measuring pass's three special cases.

Two buttons on the settings page run the job and differ in one
predicate. "Index faces" converges on coverage: has anything examined
this image. "Re-index every face" converges on provenance: face-detection
claims every image with no marker under the chosen detector, in either
of its forms (FaceDetector::model_ids, so a desktop in f32 and a tablet
on the Hexagon do not re-index each other's work), and a marker saying a
weaker one looked is not that. An original over the fetch budget is left
exactly as it was under the re-index, where the sweep marks it examined:
a re-detection with nothing found would delete the faces, and "cannot
fetch" is not "no faces".
2026-09-19 18:52:13 +02:00
dtourolle f100db89ca Verify the catalog snapshot before it is sent, and after it lands
Two checks around the upload, both cheap next to what they prevent.

Before: the snapshot is quick_checked before it leaves. It is the copy
every other device merges from, and a damaged one costs each of them a
download, a failed merge and a refusal to push.

After: the staged upload's size on the server is compared to the bytes
sent before it is rotated into place. A chunked upload is assembled
server-side, and an assembly that goes wrong is a file of plausible
size no device can open — caught here, on the device that caused it,
for one listing; otherwise on every other device, after the fact. A
mismatch, or a size the server will not confirm, discards the upload
and leaves the current copy and its generations untouched.
2026-09-13 19:31:58 +02:00
dtourolle 2ce0fcc74a Keep three generations of the catalog on the server
The server held one copy of the catalog, overwritten in place on every
push. When that copy was damaged there were two answers, both bad:
refuse to touch it for ever, which is what every device did for a week,
or overwrite it with ours, which loses whatever another device had
added since — the escape hatch of the previous commit.

A push now uploads to catalog.upload.sqlite, rotates catalog.sqlite to
.1 (and .1 to .2, .2 to .3, dropping the oldest), and moves the upload
into place. Rotation is server-side renames, oldest first so that every
destination is empty when it is written to — move_to refuses to
overwrite, by design — and a failure at any step leaves a gap in the
generations and never a missing current copy. The only transfer is the
upload itself.

A damaged current copy that arrived whole now merges from the newest
readable generation before ours goes over it, which loses nothing, and
is kept as .1 by the ordinary rotation rather than by a separate 40 MB
upload. NFR-R2 asks for the catalog to be backed up; this is the half
of it that lives with the copy other devices read.
2026-09-13 19:31:58 +02:00
dtourolle 693195fa96 Replace a damaged catalog on the server instead of pinning it there
The catalog sync refuses to upload when it cannot read the server's copy,
because the upload is a read-modify-write and writing blind would discard
another device's collections. That is the right rule for a timeout, a
dropped connection or a newer schema — the remote is fine, only our view
of it failed.

A file SQLite calls malformed is not that. No device will ever read it
again, so refusing to write over it preserves nothing — and every client
declines in turn, pinning the damaged file in place for good. Collections
and people then stop crossing between devices on all of them at once,
each logging "catalog not pushed" on every pass. This library did exactly
that from 2026-09-07, on the desktop and on a freshly installed phone
alike, while 32 collections sat undelivered.

Now a copy that arrived whole and still will not open is set aside under
a dated name and replaced by ours. Whole is checked against the size the
server advertises: a truncated download will not open either, and on a
phone that is the far likelier story, so anything short — or any size the
listing cannot confirm — is treated as the transport failure it is and
the server's copy is left alone. A placeholder's size is not trusted for
the comparison, since it means nothing.

The report says when this happened, and the log line calls it "pushed
over a damaged copy" rather than folding it into an ordinary push: it is
the one push that discarded something.
2026-09-13 19:01:00 +02:00
dtourolle 4f31123b0c Let the user choose which SCRFD finds their faces
faces.md §12.3 measured what the cheapest detector costs: the small
faces in every group shot, and a dog embedded a dozen times. Which
trade is right depends on the machine doing the sweep — a desktop left
overnight and a tablet on a battery want different answers — so the
detector is now a per-device setting, Fast / Balanced / Thorough on
the settings page beside the indexing button, persisted with the rest
of the settings file.

A detector is half of a model id. Every face, marker, shard and
calibration is keyed on faces.model_id precisely so that a model change
is a new id and a re-index rather than a silent change under existing
data, and a detector change is a model change: it decides which faces
exist and where the landmarks that align them land. So each choice
names its own pipeline. 500M keeps the bare "w600k_mbf" every existing
library was written under, so an upgrade disturbs nothing; the others
are qualified. Choosing one restarts coverage from zero under the new
id, the sweep re-detects, confirmed names carry across by box overlap,
and the sync shards are keyed by the same id so a peer on another
setting neither adopts nor pollutes them. The library controller
carries the id into the sync the same way it carries the cache budget,
because the sync starts from places that have no settings in reach.

All three shape-fixed exports ship — APK, Arch, Flatpak — since a
tablet has no other way to obtain the one it was not installed with;
the APK grows by twenty megabytes for the choice.
2026-09-11 22:12:53 +02:00
dtourolleandClaude Opus 5 353382c07f Hand the photographer's place between devices
A place recorded on the tablet should be where the desktop opens.

Exchanged through `.darkroom-derived/place.json`, beside the thumbnail
shards and the catalog snapshot. Newest timestamp wins outright: unlike
the catalog this is replaced rather than merged, because two devices
cannot both be where the photographer is and so there is nothing of
theirs inside ours to preserve.

It still refuses to upload over a copy it could not read, for a smaller
version of the reason `sync_catalog` does: a record we have not compared
against may be the newer one, and overwriting it would move the other
device's photographer without ever having seen where they were.

Last in the pass, and its failures are logged rather than reported.
Everything else in that folder is *derived* -- a faster way to learn what
the device could work out for itself -- so losing it costs time. A place
is a fact only the other device knew, and losing it costs a scroll. A
sync that ran out of connectivity should spend what it had on the shards.

The full pass runs after a thumbnail sweep or when Sync is pressed,
neither of which happens on an ordinary launch -- so a handover would
arrive one launch late, which is one too many for a feature whose whole
claim is picking up where you stopped. `spawn_place_fetch` is the small
half: one GET of a few hundred bytes, started beside the scan.

And it can still be refused. A handover is welcome on the way in and
unwelcome once the photographer has started: a grid that jumped
elsewhere mid-scroll because a round trip finally landed would have lost
their place to the feature meant to keep it. Any scroll, scrub, scope
change, filter or opened photograph closes the latch, and a record
arriving after that is written to disk and takes effect next launch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-30 20:40:22 +02:00
dtourolle 21d599b420 Test the guard, since the bug was a branch nobody ran
Build and test / Desktop (Linux) (push) Successful in 2h5m41s
Build and test / Layer separation (push) Successful in 50s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 1m37s
Build and test / Android (aarch64) (push) Successful in 59m40s
The catalog clobber existed because `if let Ok(bytes)` had a failure arm
that was never exercised. Fixing it without covering that arm leaves the
next person free to collapse it back.

Three tests against a backend whose read fails in a chosen way, counting
writes — because what went wrong was not a wrong value but a write that
should not have happened at all:

- a dehydrated snapshot uploads nothing
- an unreadable one uploads nothing either, since "refused" is no more
  "absent" than "not downloaded" is
- and a genuine first sync still uploads, which is the half that keeps
  `NotFound` distinct from `NotMaterialised` rather than merely cautious

Checked against the original shape: the first two fail on it and the
third passes. A guard test that cannot tell the bug from the fix is
decoration.

`async-trait` joins dev-dependencies to stand the double up behind
`dyn RemoteBackend`.
2026-08-29 10:40:38 +02:00
dtourolle 4168d67cfa Do not push a catalog over one we could not read
`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped

    if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {

which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.

The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.

Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.

And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
2026-08-29 10:36:26 +02:00
dtourolle f12aece07e Make storage pluggable, and prove it with a folder backend
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.

A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:

- **Capabilities** — already there, and the reason the engine can drive
  two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
  whatever form its connector addresses, with no server in it. Loads
  every existing config unchanged (`backend` defaults to `nextcloud`,
  `endpoint` is stored under its historical `server` key), and
  `Account::namespace()` reproduces the old catalog directory byte for
  byte, because changing it would abandon a catalog, its thumbnail
  shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
  `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
  names a connector.

`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.

Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.

`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.

docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
2026-08-29 09:57:52 +02:00
dtourolleandClaude Opus 5 2fb3eb5d2d Upload a shard that was sealed after the server saw it
Build and test / Desktop (Linux) (push) Successful in 2h7m52s
Build and test / Layer separation (push) Successful in 57s
🐳 Android image / Build and push (push) Successful in 12s
Build and test / android-image (push) Successful in 13s
Traceability / Requirement traces (push) Successful in 57s
Build and test / Android (aarch64) (push) Successful in 20m58s
The tablet showed 442 of a person's 648 faces and could never catch up. Its
shard ledger said why: it had adopted the laptop's shard 0 at 2,711,552 bytes,
and that shard is 20,402,176 bytes on disk.

The upload skipped it. A sealed shard the server already had under that name was
taken to be "byte-identical by construction", so its mere presence was proof
enough and the size was never consulted. That premise is false, and this library
is the counter-example: an *open* shard is uploaded on every pass as it fills —
that is how a growing shard reaches the other devices — so the server ordinarily
holds a partial copy of a shard that is later completed and sealed. Sealing then
froze that partial copy in place for ever.

Nothing downstream could recover from it. The tablet's `has_adopted` check is
keyed on name *and* size and would have re-downloaded a changed shard gladly; it
was never offered one, because the sending side had stopped looking.

So the comparison is on size, sealed or not. The client id is in the name, so
nobody else can have written the file and size is a sound test. A sealed shard
whose size already matches is still skipped on the first comparison, which is
all the original cheapness was worth.

The thumbnail upload had the identical test and the identical hole, and is fixed
with it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-29 08:32:01 +02:00
dtourolleandClaude Opus 5 d70fe9da8d Decide whether to send a shard before reading it
The upload read every shard off disk and only then asked whether it needed
sending. For a library with 94 MB of face shards already on the server, every
idle sync read 94 MB to conclude it had nothing to do.

The question does not need the bytes. A sealed shard the server already has is
byte-identical by construction, and the client id is in the name, so nobody else
could have written it — the name settles it. The open shard is compared on size,
which `stat` answers.

The progress line moves after the skip for the same reason: announced before it,
an idle pass claimed to be sending five shards and sent none.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 23:08:16 +02:00
dtourolleandClaude Opus 5 a1e8f494b9 Say what the face sync is doing while it does it
The face pass set the status to "checking faces…" once and then said nothing
until it was finished. On a library whose first export after a re-index is 9,849
photographs and 85 MB of shards, that is eight minutes of a progress bar sitting
still — which is indistinguishable from a hang, and was reported as one twice.

Nothing was wrong with the sync. The only fault was that it was silent.

Three places now report, which are the three that take real time:

- **Preparing**, per image with a count, since this is the long one and the only
  one whose length the user cannot guess from anything on screen.
- **Sending**, per shard with its size, because a face shard carries crops and
  runs to tens of megabytes — one of them is a visible wait on any connection.
  Announced before the upload rather than after, since the wait *is* the upload.
- **Taking in** a peer's shard, which is a download and then a row-by-row merge.

The export reports every 25 images rather than every one, so the channel behind
it stays lost in the write it accompanies. `export_to_shards` keeps its old
signature and delegates, so the callers that do not want progress do not grow a
parameter for it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-28 21:56:16 +02:00
dtourolleandClaude Opus 5 242374fd0f Let the interface hold a backend without knowing whose it is
`dr-sync` defines `RemoteBackend` and a capability model the engine adapts
to, so a second backend can be added without touching the code that uses
one. That boundary was documentation. Seven files in `dr-ui` constructed a
`NextcloudBackend` directly, ten functions took one by concrete type, and
exactly two call sites in the tree — both inside `dr-sync` itself — ever held
the trait object. A WebDAV or local-folder backend would have had a
well-written trait to implement and nowhere to go afterwards.

The change is smaller than the finding suggests, because the trait was
already right. Every method the UI has ever called on a backend — `get`,
`put`, `list`, `delete`, `create_dir`, `move_to` — was already on it, so
nothing had to be added and no behaviour moved. Ten signatures widened to
`&dyn RemoteBackend`, sixteen constructions became `remote::connect`, and
`remote.rs` is now the only file in the interface that names a connector.

`connect` returns `Result<Box<dyn RemoteBackend>, RemoteError>`. The error
type is `dr-sync`'s rather than the connector's, which is why every call site
kept its shape — the `match`, the `let Ok(..) else`, and
`.map_err(ScanFailure::local)?` all still read as they did.

One wrinkle worth recording: `&Box<dyn Trait>` does not reach `&dyn Trait` on
its own. The compiler reaches for unsizing, which wants
`Box<dyn RemoteBackend>: RemoteBackend`, and reports a confusing missing impl
rather than suggesting a deref. Twelve call sites therefore say `&*backend`,
and two say `let backend: &dyn RemoteBackend = &*backend` where a borrow is
shared across lanes.

What this does *not* do is abstract credentials. `AppCredentials` is an app
password from Login Flow v2 — a Nextcloud protocol, not a general notion of
authenticating to a remote — and seven files still name it. An OAuth token, a
bucket key pair and an app password have no useful common shape, so deciding
what an account is across backends before a second one exists would be a
confident guess. code-health.md CH-2 now records that as the remaining half,
and it should wait for the backend that forces it.

Verified: fmt clean, clippy clean at -D warnings, 2041 tests pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-27 19:51:23 +02:00
dtourolleandClaude Opus 5 96d07da15f Sync face data as sealed shards, so a second device does not re-index
Indexing 23,500 images is about two hours of CPU, and the result is
byte-identical on every device: the same model over the same proxy
produces the same embedding. Paying for it once per account rather than
once per device is the point.

Shards rather than the catalog snapshot, because the snapshot goes up
whole on every sync and a fully indexed library carries roughly 30 MB of
embeddings. That is exactly the cost the thumbnail store's 25 MB cap
exists to bound, so face shards use the same cap -- imported from
dr_thumbs rather than restated, since the number is a statement about
sync cost and the two must not drift apart.

The split follows the one already there: bulk immutable data in sealed
shards, small mutable data in the catalog snapshot. Faces, landmarks,
embeddings and run markers shard; people, names and assignments ride the
catalog and merge by uuid.

Keyed on oc:fileid throughout, never on image_id, because a row id means
nothing on another device.

The run marker travels with the faces it describes. Without it a
receiving device cannot tell an image with no faces from one never
examined, and would re-detect every landscape it had just adopted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 22:41:50 +02:00
dtourolleandClaude Opus 5 11a63fc7a8 Refresh the collection tree when a sync brings membership
New-York gained 127 photographs from the tablet and the sidebar went on showing
no number beside it.

Two reasons, and the second is why the first was never noticed. The sync's
completion handler reloads the grid and never rebuilds the tree — the scan
already does, and the sync merges the same tables. And the condition it reloads
under was `collections_gained > 0`, which counts collections, not members: a
sync that files 127 photographs into a collection both devices already had
gains no collection at all, so the count was zero and nothing refreshed.

`SyncReport` now carries `members_gained` from the merge report, and either one
rebuilds the tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 22:10:40 +02:00
dtourolleandClaude Opus 5 9e47133304 Run the formatter over the shard-naming change
Build and test / android-image (push) Canceled after 1m38s
Build and test / Android (aarch64) (push) Canceled after 0s
Build and test / Desktop (Linux) (push) Canceled after 1m34s
Build and test / Layer separation (push) Canceled after 0s
🐳 Android image / Build and push (push) Canceled after 1m38s
Traceability / Requirement traces (push) Successful in 1m2s
`cargo fmt --all -- --check` is a CI gate and 9a51cc8 landed two files past
it, so the build has been red on master since regardless of what came after.
Whitespace only — a wrapped `Ok(...)` and two `assert_eq!`s split across
lines. No logic is touched and the tests are unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 22:29:28 +02:00
dtourolleandClaude Opus 5 9a51cc88d6 Give a shard's remote name the client that wrote it
Build and test / Desktop (Linux) (push) Failing after 50s
Build and test / Layer separation (push) Successful in 24s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Failing after 59s
Build and test / Android (aarch64) (push) Failing after 9m39s
Shard ids are per store: every client fills its own numbering from 0, so
"shard 3" names different thumbnails on every device. The derived sync
published them into a flat shard-NNNN.sqlite namespace anyway, which left
two clients writing one name.

Both failures that follow were live. On upload, a client's open shard
overwrote a peer's file of the same id — content the peer still believed
was published and would never restore, because its own copy was sealed and
the name existed. On download, the loop skipped any remote id it already
held locally, which is the only safe reading of a name that says nothing
about who wrote it, so a client holding shards 0..5 never fetched the
peer's 0..5 at all. Between them, two populated clients exchanged almost
nothing: only shards numbered above the other's highest. A fresh device
worked, having no local shards to collide with, which is why this went
unnoticed — it is exactly the case the feature was written for.

The name is now shard-<client>-NNNN.sqlite. The client id is minted per
store in index.sqlite, beside the numbering it qualifies rather than in
settings: a store deleted and rebuilt restarts at shard 0 and must not
claim the remote names its predecessor wrote. Since our own ids now say
nothing about what we have taken from others, index.sqlite also keeps a
ledger of adopted remote names and the size each had when merged. A size
rather than a flag, because a peer's sealed shard never returns but its
open one grows, and re-merging the grown copy is how the thumbnails it
gained since arrive.

Flat names already on servers still parse, reporting no owner, so each
client adopts them once, and nothing is written under that form again. One
whose id and byte size match a local shard is that client's own earlier
upload by the same identity argument the upload path already makes for
sealed shards, so the rename does not cost every client a re-download of
its whole store. Older builds ignore the new names and stop receiving
shards until updated; their own uploads are still adopted, so nothing is
lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 21:15:12 +02:00
dtourolleandClaude Opus 5 03326242a1 Make the CI checks say what they mean, and format the workspace
Build and test / Desktop (Linux) (push) Successful in 1h23m26s
Build and test / Android (aarch64) (push) Failing after 2s
Build and test / Layer separation (push) Successful in 50s
Traceability / Requirement traces (push) Failing after 1m9s
The Android job's "Verify minimum API level" step has never verified the
minimum API level. It took the first `*.so` anywhere under the target
directory, which is a host proc-macro from debug/deps — an x86-64 object
built by the runner's gcc, whose .comment section cannot mention Android
and so can never contradict the expected value. It now reads the artifact
under the target triple, compares against MIN_API parsed from the
Dockerfile rather than a second copy of the number, and fails on a
mismatch. Both sides are checked non-empty first: two failed parses would
otherwise compare equal and pass, which is the same silent success in a
new costume.

The Android image installs one SDK package per layer and keeps the
output. sdkmanager is a JVM program that aborts when it cannot get memory,
and the single `> /dev/null` step reported that as a bare "exit code 134"
while a retry re-downloaded everything that had already succeeded.

tools/ci-local.sh runs all four jobs — desktop, android, layering,
traceability — against the host toolchain, which is pinned to the same
1.92.0 CI installs. Its matrix check compares regeneration against the
working tree rather than against HEAD: CI starts from a clean checkout, so
git's answer is the right one there and reports every local run stale here.

The rest is rustfmt across the workspace, and the clippy findings that
surfaced once it did: manual_contains in dr-thumbs and collections_ui, a
map iterated as pairs for its keys, an index loop over a slice, and two
runtime assertions on a constant now made at compile time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-16 12:02:51 +02:00
dtourolle 4d78041d1d Many imorovments
Build and test / Desktop (Linux) (push) Failing after 1m8s
Build and test / Android (aarch64) (push) Failing after 2s
Build and test / Layer separation (push) Canceled after 23s
Traceability / Requirement traces (push) Failing after 59s
2026-08-12 22:16:15 +02:00
dtourolle 8ad5c86ff9 Add the library, collections, and trash views; theme from style.yaml
The UI gains the views the catalog work was building toward: a windowed
library grid with ratings and flags, the collection tree with drag-to-add,
and trash with restore. derived_sync pushes thumbnail shards and the catalog
snapshot to the server's derived folder.

Tokens now have one source of truth. build.rs reads style.yaml and generates
theme.slint into OUT_DIR, which answers every existing
`import { Theme } from "theme.slint"` unchanged, because Slint resolves
imports against the importing file's directory first and the include paths
after. Generating into OUT_DIR rather than beside the hand-written Slint is
the point: a generated file sitting in ui/ looks exactly like the files
around it that are meant to be edited, and an edit to it would survive until
the next touch of style.yaml — a bug that hides for weeks. build.rs fails
loudly if a stale ui/theme.slint exists, which would otherwise shadow the
generated one silently and make every palette change vanish with no error.

The palette moves to near-neutral dark with achromatic signalling, so the
accent means "modified" or "active" rather than "heading". Shared components
land in widgets.slint: a token that binds several values into one concept is
a component, not a row in a YAML file.

Adds an optional live-style feature that makes the tokens in-out so they can
be written at startup — a feature rather than the default because it stops
the properties being constant-folded.

serde_norway is the YAML crate: serde_yaml and serde_yml are both deprecated,
and its mappings preserve insertion order, which is what lets the generated
Slint keep the token ordering the author chose.

Assisted-by: LLM
2026-08-09 21:11:38 +02:00