3b783d7bd45759770236aca600d3cf6aa1b53410
24
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
b1d1c47261 |
Start every worker thread through the executors module
Thirty-nine spawn sites in dr-ui, and one in the Android entry point, called std::thread::spawn or a Builder of their own, and most of the threads they started were <unnamed> in a panic message or a profiler. Each now calls executors::spawn with its executor and a role, so the thread is named <executor>:<role> — net:sync, decode:thumbs, io:catalog-open — and knows which executor it is on. The three that already set a name (automation, import, prefetch) keep their name as the role. Behaviour is unchanged: each job still gets a thread of its own when it starts, and spawn panics where std::thread::spawn did. The module's documentation now says how a job is assigned: by what it spends its time on, so a sweep that fetches bytes and then decodes them is Decode, and a sidecar write that touches the catalog is Network. Left as they were: the segmentation and refine workers in masks_ui.rs, which another change is reworking, and test-only threads. |
||
|
|
34ac2f14d3 |
Sync faces and the catalog before thumbnails, so a fresh device sees its names and collections first
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m24s
Benchmarks / Frame budget (on demand) (push) Skipped
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 4s
Build and test / Desktop (Linux) (push) Failing after 35s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Layer separation (push) Successful in 30s
Traceability / Requirement traces (push) Failing after 48s
Build and test / Android (aarch64) (push) Failing after 0s
Build and test / Windows (x86_64, cross) (push) Failing after 0s
The thumbnail stage ran first and, on a device that had just adopted its peers' shards, spent its time re-uploading hundreds of megabytes under its own client id while faces, people, collections and dates waited behind it. Faces go first — the catalog merge assigns identities to faces this device holds — then the catalog, then thumbnails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
f71d7bacc6 |
Take the server's shards and dates when the scan completes, not after the sweep
Benchmarks / CPU and I/O (per commit) (push) Successful in 8m28s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 36s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Successful in 43m11s
Build and test / Windows (x86_64, cross) (push) Failing after 41m4s
The derived sync fired only after the metadata sweep, so a fresh device re-derived every thumbnail it scrolled past, re-detected faces and re-read every header for hours before adopting the shards and snapshot that held all of it. It now fires as soon as the scan completes — the first moment the rows the merges key on exist — and the sweep starts behind it. In steady state that pass is one listing. The catalog merge gains a fourth half: capture metadata (captured_at, offset, camera, lens, ISO) for images still at metadata_state < 2, matched by oc:fileid from a remote row at 2. A date is a fact about the file's bytes, not local state, and the snapshot already carried it. The sweep's per-chunk query then finds nothing left, and the timeline is whole on a fresh device without a header fetch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
7fba28f7d8 |
Upload a snapshot of a thumbnail shard, never the live file
Every shard is in WAL mode and every put opens its own connection, so while thumbnails are being generated on several threads — which is when the first sync pass runs — the log is never checkpointed and the main file holds whatever the last quiet moment left in it. For a shard created seconds earlier that is nothing: a zero-byte file with the schema still in the log. The sync read that file and uploaded it, and every other device merging it failed with "no such table: thumbs" on every pass. Copy the shard through SQLite's backup API into scratch first, which serialises against writers and carries the log, and upload that. |
||
|
|
5c00942b84 |
One completeness job over a registry of repairs, and a re-index button
A library's records are never all complete at once. A face found before its quality was kept has no quality; one found before the eye models existed has no reading; one adopted from a peer's shard has no crop; an image the fast detector examined on a 1024 px proxy has boxes the current detector would not have drawn; an image the scan stat'ed has no capture date. On the reference library that is 17,762 faces under the bare w600k_mbf id with no quality, no reading and no dense landmarks, 4,144 of them without a crop, beside 12,217 images the fast detector examined and found nothing in. Every one of those gaps was its own pass — V14's measuring pass, §17.5's eye pass, the sweep's proxy repair, the sweep's detector upgrade — with its own work list, its own count and its own idea of done, and adding a per-face field meant adding a pass. There was no pass at all for the case the library is actually in: boxes and landmarks drawn by a weaker detector on a proxy, which every later per-face pass would have read from. dr_ui::repairs replaces them with one job over a registry. A Repair names one thing a record can lack — the predicate that says which images still owe it, the input its handler needs (a header, the original, or a native render), the handler, and what to record for an image that can never be done. The job unions the predicates into one work list, fetches each image once at the most any claimant asks for, renders it at most once, and runs every handler whose predicate that image still matches, checked again before each because a detection writes every field a per-face handler would fill. The registry today: face-proxy, face-quality, face-eyes, face-crop, face-detection, face-upgrade, metadata — the last there to say that this is not a face job. Adding a field is one entry. A repair's predicate is the only definition of its work: the count the settings page shows, the list the job fetches and the check before its handler run are one predicate, so the job converges. That is why the registry is cut to what the device can do rather than listing what it skips — an entry is a count and a set of originals to fetch — and why an eye reading that cannot be cut is not a criterion. The catalog side is generic to match: record_updates writes whichever fields a FaceUpdate carries and re-marks the image so the shards export it; faces_needing and count_needing answer a predicate the caller supplies, replacing the measuring pass's three special cases. Two buttons on the settings page run the job and differ in one predicate. "Index faces" converges on coverage: has anything examined this image. "Re-index every face" converges on provenance: face-detection claims every image with no marker under the chosen detector, in either of its forms (FaceDetector::model_ids, so a desktop in f32 and a tablet on the Hexagon do not re-index each other's work), and a marker saying a weaker one looked is not that. An original over the fetch budget is left exactly as it was under the re-index, where the sweep marks it examined: a re-detection with nothing found would delete the faces, and "cannot fetch" is not "no faces". |
||
|
|
f100db89ca |
Verify the catalog snapshot before it is sent, and after it lands
Two checks around the upload, both cheap next to what they prevent. Before: the snapshot is quick_checked before it leaves. It is the copy every other device merges from, and a damaged one costs each of them a download, a failed merge and a refusal to push. After: the staged upload's size on the server is compared to the bytes sent before it is rotated into place. A chunked upload is assembled server-side, and an assembly that goes wrong is a file of plausible size no device can open — caught here, on the device that caused it, for one listing; otherwise on every other device, after the fact. A mismatch, or a size the server will not confirm, discards the upload and leaves the current copy and its generations untouched. |
||
|
|
2ce0fcc74a |
Keep three generations of the catalog on the server
The server held one copy of the catalog, overwritten in place on every push. When that copy was damaged there were two answers, both bad: refuse to touch it for ever, which is what every device did for a week, or overwrite it with ours, which loses whatever another device had added since — the escape hatch of the previous commit. A push now uploads to catalog.upload.sqlite, rotates catalog.sqlite to .1 (and .1 to .2, .2 to .3, dropping the oldest), and moves the upload into place. Rotation is server-side renames, oldest first so that every destination is empty when it is written to — move_to refuses to overwrite, by design — and a failure at any step leaves a gap in the generations and never a missing current copy. The only transfer is the upload itself. A damaged current copy that arrived whole now merges from the newest readable generation before ours goes over it, which loses nothing, and is kept as .1 by the ordinary rotation rather than by a separate 40 MB upload. NFR-R2 asks for the catalog to be backed up; this is the half of it that lives with the copy other devices read. |
||
|
|
693195fa96 |
Replace a damaged catalog on the server instead of pinning it there
The catalog sync refuses to upload when it cannot read the server's copy, because the upload is a read-modify-write and writing blind would discard another device's collections. That is the right rule for a timeout, a dropped connection or a newer schema — the remote is fine, only our view of it failed. A file SQLite calls malformed is not that. No device will ever read it again, so refusing to write over it preserves nothing — and every client declines in turn, pinning the damaged file in place for good. Collections and people then stop crossing between devices on all of them at once, each logging "catalog not pushed" on every pass. This library did exactly that from 2026-09-07, on the desktop and on a freshly installed phone alike, while 32 collections sat undelivered. Now a copy that arrived whole and still will not open is set aside under a dated name and replaced by ours. Whole is checked against the size the server advertises: a truncated download will not open either, and on a phone that is the far likelier story, so anything short — or any size the listing cannot confirm — is treated as the transport failure it is and the server's copy is left alone. A placeholder's size is not trusted for the comparison, since it means nothing. The report says when this happened, and the log line calls it "pushed over a damaged copy" rather than folding it into an ordinary push: it is the one push that discarded something. |
||
|
|
4f31123b0c |
Let the user choose which SCRFD finds their faces
faces.md §12.3 measured what the cheapest detector costs: the small faces in every group shot, and a dog embedded a dozen times. Which trade is right depends on the machine doing the sweep — a desktop left overnight and a tablet on a battery want different answers — so the detector is now a per-device setting, Fast / Balanced / Thorough on the settings page beside the indexing button, persisted with the rest of the settings file. A detector is half of a model id. Every face, marker, shard and calibration is keyed on faces.model_id precisely so that a model change is a new id and a re-index rather than a silent change under existing data, and a detector change is a model change: it decides which faces exist and where the landmarks that align them land. So each choice names its own pipeline. 500M keeps the bare "w600k_mbf" every existing library was written under, so an upgrade disturbs nothing; the others are qualified. Choosing one restarts coverage from zero under the new id, the sweep re-detects, confirmed names carry across by box overlap, and the sync shards are keyed by the same id so a peer on another setting neither adopts nor pollutes them. The library controller carries the id into the sync the same way it carries the cache budget, because the sync starts from places that have no settings in reach. All three shape-fixed exports ship — APK, Arch, Flatpak — since a tablet has no other way to obtain the one it was not installed with; the APK grows by twenty megabytes for the choice. |
||
|
|
353382c07f |
Hand the photographer's place between devices
A place recorded on the tablet should be where the desktop opens. Exchanged through `.darkroom-derived/place.json`, beside the thumbnail shards and the catalog snapshot. Newest timestamp wins outright: unlike the catalog this is replaced rather than merged, because two devices cannot both be where the photographer is and so there is nothing of theirs inside ours to preserve. It still refuses to upload over a copy it could not read, for a smaller version of the reason `sync_catalog` does: a record we have not compared against may be the newer one, and overwriting it would move the other device's photographer without ever having seen where they were. Last in the pass, and its failures are logged rather than reported. Everything else in that folder is *derived* -- a faster way to learn what the device could work out for itself -- so losing it costs time. A place is a fact only the other device knew, and losing it costs a scroll. A sync that ran out of connectivity should spend what it had on the shards. The full pass runs after a thumbnail sweep or when Sync is pressed, neither of which happens on an ordinary launch -- so a handover would arrive one launch late, which is one too many for a feature whose whole claim is picking up where you stopped. `spawn_place_fetch` is the small half: one GET of a few hundred bytes, started beside the scan. And it can still be refused. A handover is welcome on the way in and unwelcome once the photographer has started: a grid that jumped elsewhere mid-scroll because a round trip finally landed would have lost their place to the feature meant to keep it. Any scroll, scrub, scope change, filter or opened photograph closes the latch, and a record arriving after that is written to disk and takes effect next launch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
21d599b420 |
Test the guard, since the bug was a branch nobody ran
Build and test / Desktop (Linux) (push) Successful in 2h5m41s
Build and test / Layer separation (push) Successful in 50s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 1m37s
Build and test / Android (aarch64) (push) Successful in 59m40s
The catalog clobber existed because `if let Ok(bytes)` had a failure arm that was never exercised. Fixing it without covering that arm leaves the next person free to collapse it back. Three tests against a backend whose read fails in a chosen way, counting writes — because what went wrong was not a wrong value but a write that should not have happened at all: - a dehydrated snapshot uploads nothing - an unreadable one uploads nothing either, since "refused" is no more "absent" than "not downloaded" is - and a genuine first sync still uploads, which is the half that keeps `NotFound` distinct from `NotMaterialised` rather than merely cautious Checked against the original shape: the first two fail on it and the third passes. A guard test that cannot tell the bug from the fix is decoration. `async-trait` joins dev-dependencies to stand the double up behind `dyn RemoteBackend`. |
||
|
|
4168d67cfa |
Do not push a catalog over one we could not read
`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped
if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {
which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.
The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.
Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.
And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
|
||
|
|
f12aece07e |
Make storage pluggable, and prove it with a folder backend
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.
A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:
- **Capabilities** — already there, and the reason the engine can drive
two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
whatever form its connector addresses, with no server in it. Loads
every existing config unchanged (`backend` defaults to `nextcloud`,
`endpoint` is stored under its historical `server` key), and
`Account::namespace()` reproduces the old catalog directory byte for
byte, because changing it would abandon a catalog, its thumbnail
shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
`ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
names a connector.
`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.
Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.
`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.
docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
|
||
|
|
2fb3eb5d2d |
Upload a shard that was sealed after the server saw it
Build and test / Desktop (Linux) (push) Successful in 2h7m52s
Build and test / Layer separation (push) Successful in 57s
🐳 Android image / Build and push (push) Successful in 12s
Build and test / android-image (push) Successful in 13s
Traceability / Requirement traces (push) Successful in 57s
Build and test / Android (aarch64) (push) Successful in 20m58s
The tablet showed 442 of a person's 648 faces and could never catch up. Its shard ledger said why: it had adopted the laptop's shard 0 at 2,711,552 bytes, and that shard is 20,402,176 bytes on disk. The upload skipped it. A sealed shard the server already had under that name was taken to be "byte-identical by construction", so its mere presence was proof enough and the size was never consulted. That premise is false, and this library is the counter-example: an *open* shard is uploaded on every pass as it fills — that is how a growing shard reaches the other devices — so the server ordinarily holds a partial copy of a shard that is later completed and sealed. Sealing then froze that partial copy in place for ever. Nothing downstream could recover from it. The tablet's `has_adopted` check is keyed on name *and* size and would have re-downloaded a changed shard gladly; it was never offered one, because the sending side had stopped looking. So the comparison is on size, sealed or not. The client id is in the name, so nobody else can have written the file and size is a sound test. A sealed shard whose size already matches is still skipped on the first comparison, which is all the original cheapness was worth. The thumbnail upload had the identical test and the identical hole, and is fixed with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
d70fe9da8d |
Decide whether to send a shard before reading it
The upload read every shard off disk and only then asked whether it needed sending. For a library with 94 MB of face shards already on the server, every idle sync read 94 MB to conclude it had nothing to do. The question does not need the bytes. A sealed shard the server already has is byte-identical by construction, and the client id is in the name, so nobody else could have written it — the name settles it. The open shard is compared on size, which `stat` answers. The progress line moves after the skip for the same reason: announced before it, an idle pass claimed to be sending five shards and sent none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
a1e8f494b9 |
Say what the face sync is doing while it does it
The face pass set the status to "checking faces…" once and then said nothing until it was finished. On a library whose first export after a re-index is 9,849 photographs and 85 MB of shards, that is eight minutes of a progress bar sitting still — which is indistinguishable from a hang, and was reported as one twice. Nothing was wrong with the sync. The only fault was that it was silent. Three places now report, which are the three that take real time: - **Preparing**, per image with a count, since this is the long one and the only one whose length the user cannot guess from anything on screen. - **Sending**, per shard with its size, because a face shard carries crops and runs to tens of megabytes — one of them is a visible wait on any connection. Announced before the upload rather than after, since the wait *is* the upload. - **Taking in** a peer's shard, which is a download and then a row-by-row merge. The export reports every 25 images rather than every one, so the channel behind it stays lost in the write it accompanies. `export_to_shards` keeps its old signature and delegates, so the callers that do not want progress do not grow a parameter for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
242374fd0f |
Let the interface hold a backend without knowing whose it is
`dr-sync` defines `RemoteBackend` and a capability model the engine adapts to, so a second backend can be added without touching the code that uses one. That boundary was documentation. Seven files in `dr-ui` constructed a `NextcloudBackend` directly, ten functions took one by concrete type, and exactly two call sites in the tree — both inside `dr-sync` itself — ever held the trait object. A WebDAV or local-folder backend would have had a well-written trait to implement and nowhere to go afterwards. The change is smaller than the finding suggests, because the trait was already right. Every method the UI has ever called on a backend — `get`, `put`, `list`, `delete`, `create_dir`, `move_to` — was already on it, so nothing had to be added and no behaviour moved. Ten signatures widened to `&dyn RemoteBackend`, sixteen constructions became `remote::connect`, and `remote.rs` is now the only file in the interface that names a connector. `connect` returns `Result<Box<dyn RemoteBackend>, RemoteError>`. The error type is `dr-sync`'s rather than the connector's, which is why every call site kept its shape — the `match`, the `let Ok(..) else`, and `.map_err(ScanFailure::local)?` all still read as they did. One wrinkle worth recording: `&Box<dyn Trait>` does not reach `&dyn Trait` on its own. The compiler reaches for unsizing, which wants `Box<dyn RemoteBackend>: RemoteBackend`, and reports a confusing missing impl rather than suggesting a deref. Twelve call sites therefore say `&*backend`, and two say `let backend: &dyn RemoteBackend = &*backend` where a borrow is shared across lanes. What this does *not* do is abstract credentials. `AppCredentials` is an app password from Login Flow v2 — a Nextcloud protocol, not a general notion of authenticating to a remote — and seven files still name it. An OAuth token, a bucket key pair and an app password have no useful common shape, so deciding what an account is across backends before a second one exists would be a confident guess. code-health.md CH-2 now records that as the remaining half, and it should wait for the backend that forces it. Verified: fmt clean, clippy clean at -D warnings, 2041 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
96d07da15f |
Sync face data as sealed shards, so a second device does not re-index
Indexing 23,500 images is about two hours of CPU, and the result is byte-identical on every device: the same model over the same proxy produces the same embedding. Paying for it once per account rather than once per device is the point. Shards rather than the catalog snapshot, because the snapshot goes up whole on every sync and a fully indexed library carries roughly 30 MB of embeddings. That is exactly the cost the thumbnail store's 25 MB cap exists to bound, so face shards use the same cap -- imported from dr_thumbs rather than restated, since the number is a statement about sync cost and the two must not drift apart. The split follows the one already there: bulk immutable data in sealed shards, small mutable data in the catalog snapshot. Faces, landmarks, embeddings and run markers shard; people, names and assignments ride the catalog and merge by uuid. Keyed on oc:fileid throughout, never on image_id, because a row id means nothing on another device. The run marker travels with the faces it describes. Without it a receiving device cannot tell an image with no faces from one never examined, and would re-detect every landscape it had just adopted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
11a63fc7a8 |
Refresh the collection tree when a sync brings membership
New-York gained 127 photographs from the tablet and the sidebar went on showing no number beside it. Two reasons, and the second is why the first was never noticed. The sync's completion handler reloads the grid and never rebuilds the tree — the scan already does, and the sync merges the same tables. And the condition it reloads under was `collections_gained > 0`, which counts collections, not members: a sync that files 127 photographs into a collection both devices already had gains no collection at all, so the count was zero and nothing refreshed. `SyncReport` now carries `members_gained` from the merge report, and either one rebuilds the tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
|
9e47133304 |
Run the formatter over the shard-naming change
Build and test / android-image (push) Canceled after 1m38s
Build and test / Android (aarch64) (push) Canceled after 0s
Build and test / Desktop (Linux) (push) Canceled after 1m34s
Build and test / Layer separation (push) Canceled after 0s
🐳 Android image / Build and push (push) Canceled after 1m38s
Traceability / Requirement traces (push) Successful in 1m2s
`cargo fmt --all -- --check` is a CI gate and
|
||
|
|
9a51cc88d6 |
Give a shard's remote name the client that wrote it
Build and test / Desktop (Linux) (push) Failing after 50s
Build and test / Layer separation (push) Successful in 24s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Failing after 59s
Build and test / Android (aarch64) (push) Failing after 9m39s
Shard ids are per store: every client fills its own numbering from 0, so "shard 3" names different thumbnails on every device. The derived sync published them into a flat shard-NNNN.sqlite namespace anyway, which left two clients writing one name. Both failures that follow were live. On upload, a client's open shard overwrote a peer's file of the same id — content the peer still believed was published and would never restore, because its own copy was sealed and the name existed. On download, the loop skipped any remote id it already held locally, which is the only safe reading of a name that says nothing about who wrote it, so a client holding shards 0..5 never fetched the peer's 0..5 at all. Between them, two populated clients exchanged almost nothing: only shards numbered above the other's highest. A fresh device worked, having no local shards to collide with, which is why this went unnoticed — it is exactly the case the feature was written for. The name is now shard-<client>-NNNN.sqlite. The client id is minted per store in index.sqlite, beside the numbering it qualifies rather than in settings: a store deleted and rebuilt restarts at shard 0 and must not claim the remote names its predecessor wrote. Since our own ids now say nothing about what we have taken from others, index.sqlite also keeps a ledger of adopted remote names and the size each had when merged. A size rather than a flag, because a peer's sealed shard never returns but its open one grows, and re-merging the grown copy is how the thumbnails it gained since arrive. Flat names already on servers still parse, reporting no owner, so each client adopts them once, and nothing is written under that form again. One whose id and byte size match a local shard is that client's own earlier upload by the same identity argument the upload path already makes for sealed shards, so the rename does not cost every client a re-download of its whole store. Older builds ignore the new names and stop receiving shards until updated; their own uploads are still adopted, so nothing is lost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
03326242a1 |
Make the CI checks say what they mean, and format the workspace
The Android job's "Verify minimum API level" step has never verified the minimum API level. It took the first `*.so` anywhere under the target directory, which is a host proc-macro from debug/deps — an x86-64 object built by the runner's gcc, whose .comment section cannot mention Android and so can never contradict the expected value. It now reads the artifact under the target triple, compares against MIN_API parsed from the Dockerfile rather than a second copy of the number, and fails on a mismatch. Both sides are checked non-empty first: two failed parses would otherwise compare equal and pass, which is the same silent success in a new costume. The Android image installs one SDK package per layer and keeps the output. sdkmanager is a JVM program that aborts when it cannot get memory, and the single `> /dev/null` step reported that as a bare "exit code 134" while a retry re-downloaded everything that had already succeeded. tools/ci-local.sh runs all four jobs — desktop, android, layering, traceability — against the host toolchain, which is pinned to the same 1.92.0 CI installs. Its matrix check compares regeneration against the working tree rather than against HEAD: CI starts from a clean checkout, so git's answer is the right one there and reports every local run stale here. The rest is rustfmt across the workspace, and the clippy findings that surfaced once it did: manual_contains in dr-thumbs and collections_ui, a map iterated as pairs for its keys, an index loop over a slice, and two runtime assertions on a constant now made at compile time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
4d78041d1d | Many imorovments | ||
|
|
8ad5c86ff9 |
Add the library, collections, and trash views; theme from style.yaml
The UI gains the views the catalog work was building toward: a windowed
library grid with ratings and flags, the collection tree with drag-to-add,
and trash with restore. derived_sync pushes thumbnail shards and the catalog
snapshot to the server's derived folder.
Tokens now have one source of truth. build.rs reads style.yaml and generates
theme.slint into OUT_DIR, which answers every existing
`import { Theme } from "theme.slint"` unchanged, because Slint resolves
imports against the importing file's directory first and the include paths
after. Generating into OUT_DIR rather than beside the hand-written Slint is
the point: a generated file sitting in ui/ looks exactly like the files
around it that are meant to be edited, and an edit to it would survive until
the next touch of style.yaml — a bug that hides for weeks. build.rs fails
loudly if a stale ui/theme.slint exists, which would otherwise shadow the
generated one silently and make every palette change vanish with no error.
The palette moves to near-neutral dark with achromatic signalling, so the
accent means "modified" or "active" rather than "heading". Shared components
land in widgets.slint: a token that binds several values into one concept is
a component, not a row in a YAML file.
Adds an optional live-style feature that makes the tokens in-out so they can
be written at startup — a feature rather than the default because it stops
the properties being constant-folded.
serde_norway is the YAML crate: serde_yaml and serde_yml are both deprecated,
and its mappings preserve insertion order, which is what lets the generated
Slint keep the token ordering the author chose.
Assisted-by: LLM
|