fea5b1b1ee49aab05ce0b464dacce4b24ef69ddd
55
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  dtourolleandClaude Opus 5
|
1ad35e2b87 |
Read the library's sidecars, so a cull done elsewhere arrives
Judgements only ever travelled outward. A rating went to the catalog and to the photograph's sidecar, the sidecar reached the server, and there it stopped: the scan indexes files, `derived_sync` exchanges thumbnails, face shards and collections, `dr_catalog::merge` reconciles everything in a catalog except `versions.rating` and `versions.flag`, and the one sidecar reader that existed ran when a single photograph was opened in develop and handed its answer to the develop graph. `JobKind::ReadSidecar` was declared for exactly this when the job queue was written and was never enqueued or handled anywhere. The grid draws `versions.rating`. So a day of culling on the tablet could not reach the laptop by any path the application had, and the laptop's catalog says so plainly: 23,568 images, one of them judged. `pull_sidecars` closes it, off the back of work the scan already does. `dr_sync::scan` reports the `.drsc` files it meets in listings it was making anyway — no extra request, and a directory whose ETag is unchanged is still pruned before it is listed at all. A new `sidecars` table records the ETag of each one this device has taken in, so the fetch is one GET per sidecar that genuinely changed rather than one per photograph. A library nobody has edited costs nothing. The judgement is taken rather than maximised. The sidecar is the authoritative store and the fuse has already settled any contest between devices on `revision`, so lowering a rating from four to one on the tablet lowers it here — taking the larger would have refused every demotion the photographer ever made, which is most of what a second pass over a shoot is. A zero is the exception: it means *never judged*, not "judged zero", so a sidecar carrying none cannot erase a star this device holds. That is `merge_judgement`'s asymmetry and it carries the same known cost — clearing a rating does not propagate. A sidecar names a stem, so both halves of a RAW-and-JPEG pair are judged: they are one photograph (FR-CAT-11) sharing one document, and judging only one of them would leave the grid disagreeing with itself over which it drew. The `LIKE` that finds them is a filter, not the decision — `sidecar_path` is applied to every candidate, because a folder is entitled to contain a `%` and a rating landing on the wrong frame would be silent and permanent. Failing to read one is not a failure to scan: the ETag goes unrecorded, the ratings already here stay where they are, and the next scan tries again. The count is reported to the status line as well as the log, because a grid that silently gains three hundred stars is indistinguishable from one that has gone wrong — and because while this number was structurally zero there was nothing to tell the photographer their cull had not arrived. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
601c984894 |
Fold a photograph's rival default versions back into one
Picking the newer of two default versions stopped the wrong edit being shown, but it did not close the split: the losing version stayed in the file, and a device holding disjoint work — a crop made here, an exposure change made there — still contributed only one of the two. Worse, the next write made it larger. `amend` looks its version up by uuid, neither of the two was ours, so the miss minted a *third* `default = 1` block and the file grew one rival per device per photograph. `Sidecar::fuse_default_versions` folds them down. The version with the highest `(revision, modified)` is the accumulator and every other default is merged into it as the remote, which is what makes the fold order-independent — `Version::merge` raises its own revision to `max + 1` as it goes, so merging a chain in ascending order stops being ascending after the first step and a third device would be dropped. Contested values resolve to the winner, disjoint keys survive from both sides because the merge is key-wise, and ratings come across under `merge_judgement`, so a device that never judged the frame cannot erase one that did. The result is a function of the file's bytes alone, so two devices that fuse independently reach the same document and converge instead of overwriting each other. Called wherever a sidecar is parsed: - `amend`, with the write's own uuid, so the fold lands on the identity this device is about to use and the lookup below it hits instead of missing. - `spawn_sidecar_fetch`, so opening a photograph shows everything done to it rather than whichever half won. - `drain_one`, because `merge_into` reconciles by uuid and would otherwise publish the split rather than resolve it. - `presets::load_local` and `save_local` — a local sidecar's folder may be synced by something else entirely, and gets the same split. A file with one default under the expected uuid comes back byte-identical, so this costs nothing on the ordinary write and no sidecar is uploaded merely for having been read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
710fcbc1bd |
Format the face work with the workspace's own rustfmt
Not authored in this session. `cargo fmt --all` reformats every crate, so running it while working on `dr-segment` picked up five files from the recent face and library work that had been committed unformatted. Committed on its own rather than swept into the change that happened to produce it: the diff is pure whitespace, and mixed into a commit that alters an algorithm it would be noise in exactly the place someone is trying to read carefully. `cargo fmt --all -- --check` is a CI gate (tools/ci-local.sh), so this had to land somewhere regardless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
6acc98baad |
Carry the photograph's header into an export made from develop
The same file exported from the library grid kept its camera, its lens,
its capture date and its rights statement. Exported from the develop
button it kept none of them, and `{date}` in a filename template
resolved to nothing at all. Two buttons, one photograph, two different
files -- and the develop one was the version the photographer had just
finished working on.
A session now remembers the header it was opened from, and
`open_session` takes that header rather than the orientation read out of
it, so a photograph cannot be opened for editing without saying which
file it came from. `render_open_frame` clones it onto
`Source::Rendered`; both arms of `export_one` -- the worker's own decode
and the frame handed over already rendered -- turn a header into a
`{date}` and a `SourceMetadata` through the same function, so the two
paths cannot come to different readings of one file. What of it actually
reaches the exported bytes is still decided inside `dr-export` from the
settings, which is what keeps the location-stripping option working here
rather than giving it a second implementation to disagree with.
The alternative was to hang the metadata on `Source::Rendered` alone and
keep it beside the session in the interface. That touches less, but it
makes the header and the pixels two cells to hold in step across the six
places an image is opened, replaced or fails to open, and the failure
mode of getting that pairing wrong is not a missing tag: it is one
photograph exported under another's byline and coordinates, silently.
Kept on the session, the two travel together or not at all.
The header is stored decoded rather than transcribed at open time,
deliberately. `dr-export` argues that source metadata is a parameter and
not a field on `Frame`, because two exports of one frame may legitimately
disclose different amounts; by the same reasoning a session may remember
where its pixels came from without that being a decision about what to
publish, and the allowlist that decides remains the single function in
`export.rs`.
A file with no header is left with none -- an empty `{date}` and nothing
for the encoder to copy -- rather than today's date standing in for a
capture time nobody recorded.
|
||
|
dtourolleandClaude Opus 5
|
d44bffa4a8 |
Remember where the photographer was
Opening the application was always a fresh arrival at the beginning of the library, whatever you had been doing when you closed it. What is written down is the view, the scope, the rating filter and the photograph on screen -- the open one in develop, the first visible one in the grid. Not just a scroll position: a position without the filter that produced it names a row of a list that no longer exists. Restoring them has an order for the same reason -- scope, then filter, then position, then the view -- because each step changes what an ordinal *means*. Addressed by remote path and collection UUID, never by an ordinal or a row id. `images.id` and `collections.id` are local to one catalog, and a grid ordinal is local to one ordering; a record naming either would land somewhere arbitrary on a second device and after any filter change on this one. Where the ordinal is needed, `library::ordinal_of_path` computes it through the grid's own `ORDER BY`, taken verbatim by a window function rather than spelled a second time as an inequality -- which is the mistake `grid_order_for` already warns about, and which a manually ordered collection would make unreadable. Every failure degrades rather than reports. A collection this device has not merged leaves the scope at the whole library; a photograph that has since been deleted falls back to when it was taken, which puts the grid in the right week; a torn file yields no place and the library opens at the top. Reopening develop is the one thing that requires an exact match, because a canvas on a path that no longer resolves is a filename over an empty frame. The record lives in `dr-types` beside `Settings` and the store lives here beside `SettingsStore`, for the reason `dr-types`' manifest gives: a JSON serialiser in `core/` would be paid for by every crate there. Two files and two lifetimes, though -- resetting preferences must not forget where you were. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
4af3b93dfa |
Index faces from the native render, not from a preview of it
Implements the FR-CULL-8 written two commits ago. The sweep fetched the JPEG preview embedded in each RAW and used that one buffer for both detection and the crop; it now fetches the original, renders it through the same path export uses, reduces that for the detector, and warps the crop back out of the native frame. Three pieces, and each exists for a reason worth stating. dr_face::Pixels lets the warp sample 8-bit RGBA directly. A 24 MP native frame is 96 MB as RGBA and 288 MB converted to the f32 RGB align.rs was written against, and the warp reads about forty thousand pixels out of it. Converting the whole frame to sample 0.2% of it is NFR-RES-2's budget spent on a copy, per image, for a whole library. The variant costs one branch per sample and a test asserts both layouts produce identical crops. The detector gets a box-filtered reduction to 1600px, not the native frame and not a point-sampled one. Averaging rather than sampling because the detector's job is finding small faces and decimation is precisely the operation that removes them: at 4x, fifteen of every sixteen pixels are discarded and a 40px face survives or not depending on where it falls relative to the sample grid. 1600 rather than 640 leaves the letterbox a mild 2.5x rather than a 9x, and bounds the f32 buffer at 20 MB. Landmarks come back in the reduction's coordinates and are scaled to native in one place before any crop pixel is read. This is the failure mode that would not announce itself -- unscaled landmarks put every crop near the top-left corner, which yields faces of something else, cleanly embedded and confidently clustered. The sweep fetches SWEEP_LANES-wide and renders sequentially. Not a placeholder for a parallel version: there is one GPU, so concurrent renders queue on it regardless, and each materialises a native frame. Overlapping them would multiply the one allocation that threatens the memory budget while buying parallelism that does not exist. The chunk drops from 96 to 6 for the same reason -- 96 held 8 MB previews, this holds whole RAWs. The stored edit is deliberately not applied, which is where this departs from export::render_from_library. Face geometry is normalised to the frame, so indexing a cropped render would record boxes against a frame that changes whenever the user changes their mind, and every stored box would quietly become wrong. Orientation is applied: that is a fact about the file rather than an edit. examples/face_native.rs renders one file and indexes it both ways, so the claim behind all of this can be checked against photographs rather than re-read out of the catalog it came from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
e7b526c550 |
Specify face indexing at native resolution, and say what the proxy cost
FR-CULL-8 said detection runs against the thumbnail or proxy tier and never a full decode, and faces.md §5 said the aligned crop is sampled from that same proxy. Both are wrong in the same place: they treat detection and cropping as one resolution problem when they are two, with opposite answers. Detection does not care. §4.1 fixes the graph's input at 640x640 and letterboxes whatever arrives, so a face filling 2% of the frame reaches the model at 12px whether the buffer handed over is 1024px or 6000px. Every pixel above the detector's own input is discarded before inference. The crop cares about nothing else. §5's warp produces the fixed 112x112 ArcFace sees, so source resolution converts directly into whether those 112 pixels were photographed or interpolated. Reading crop_px across the 18,671 faces the proxy-tier implementation stored: 47.3% were upsampled to reach the embedder, 314 of them by more than 2x, the smallest from 34 source pixels. An upsampled crop does not fail loudly -- it yields a confident embedding of detail that was never there, and the damage appears three stages later as clusters that will not separate. So FR-CULL-8 now specifies four stages with the resolutions named separately: render native through FR-EXP-9's pipeline, downscale for the detector, map boxes and landmarks back to native, crop and align from the native render. The affordability the old rule bought is met instead by when the pass runs -- background, preempted, resumable -- and the requirement says plainly what it now costs on a remote library: the original rather than FR-NC-3's byte range, 412 GB across the reference library's 19,107 images, so a whole-library pass is a transfer under FR-NC-6 rather than something that may start on its own. MIN_CROP_EDGE replaces the MIN_DETECT_EDGE this branch briefly had. Same number, guarding the quantity that turned out to matter. faces.md §7b records both measurements, and marks the second as unexplained rather than dressing it as a finding. Grouped by the buffer detection ran against, faces per image was 0.078 at 1024 or below and 1.82 at 2048 or better, controlled for file type and size. That gap is real and reproducible and I cannot account for it, because the letterbox above says detector input should not matter. M4 is where it gets settled. The crop measurement does not depend on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
144d2e4e84 |
Revert the detection floor: it guards the wrong resolution
Reverts 53f7cdf and e92d22d. The floor those added sat on Detector::detect, refusing any buffer under 1025px on the reasoning that a small buffer finds no faces. That reasoning does not survive §4.1: the detector letterboxes every input to 640x640, so a face occupying 2% of the frame presents at 12px to the model whether it is handed a 1024px buffer or a 6000px one. Detector input is precisely the quantity that does not matter. Worse than merely useless, it blocks the design FR-CULL-8 now specifies, where the detector is deliberately fed a downscale and the crop is taken from the native render. A guard on detect() rejects exactly that call. What the measurement actually supports is a floor on the *crop* source, which is where resolution converts into embedding quality, and which faces.crop_px already records: 47% of the reference library's faces were upsampled to reach 112x112. That floor is a separate change against the native-resolution path and does not belong on the detector. The 23x faces-per-image gap by source_edge that motivated the original commit is kept in faces.md §7b, restated as the unexplained observation it is rather than the causal claim it was written as. V12 stands: those runs cropped at 1024 whatever detection did, and that is reason enough to look at them again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
6783d0c723 |
Settle the images whose best preview is under the floor
The floor introduces a state the sweep had no arm for. An image whose largest embedded preview is genuinely smaller than 1025px now comes back from index_preview as an error, lands in the generic Err arm, and is counted as failed -- which means no face_index row, which means it is still outstanding, which means the next sweep fetches exactly the same bytes and refuses them again. For ever, on every run, at one range request each. The previous behaviour was wrong but at least terminated; this would not. So ProxyTooSmall gets its own arm, and it records a marker at the true edge rather than nothing. That is the difference between "we looked and found nothing" -- which would be a lie, since nothing was looked at -- and "this was examined at 900px, which is the best this file has". The first is unrecoverable; the second is a fact source_edge was added to carry, and a later floor or a bigger proxy can select on it deliberately the way V12 just did. Counted separately from failures all the way up, because they are not failures and reading them as such would misdescribe a library of small scans as a broken network. The summary line says how many and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
1c5849ebe8 |
Unpack the bundled models on a worker, not on the way to the first frame
Launching v0.10.0 on the tablet produces an ANR: "Waited 5000ms for MotionEvent", 5,827 ms on one input sequence, over a window that has never painted. The app recovers and sits at 0% afterwards, so it is a startup cost rather than a hang. The structural fact behind it is that `android_main` runs with the activity's input channel unserviced. Nothing drains it until Slint reaches `poll_events`, and Slint does not reach `poll_events` until `dr_ui::run` calls `window.run()` on its last line. Every millisecond before that is a millisecond the input dispatcher waits on, so five thousand of them is an ANR whatever the work happens to be. The largest single piece of that work was here. `install_bundled_models` copies 41 MB on the first launch after an install — 24.9 MB of scene model, 13.6 MB of embedder, 2.5 MB of detector — each read whole out of the APK into a `Vec` and written to `/data`, in a loop, on that thread. v0.10.0 is the release that added the scene model, which is 60% of that total, and it is the release the ANR appeared in. The 8,010 minor faults in the report are about what 41 MB of freshly touched pages costs. So it moves to a detached thread and the function returns as soon as the thread is running. Nothing on the launch path wanted the result: the only two things that read these files are the People screen and the scene tab, both of which are reached by hand, minutes later, from workers of their own. What that costs is a window in which a model looks absent. `library::face_models` and `library::scene_model` decide availability on `is_file()`, so during the copy both report their feature unavailable — which is the same answer they give a build shipping no weights at all, the ordinary case both were written around. Briefly pessimistic rather than wrong, and the temporary-name-then-rename that was already there is what keeps it from being worse than that: a lookup never sees a half-written file, only an absent one. Both call sites now say so. A completion line reports the bytes copied and the milliseconds taken, including when it is zero, so the second launch after an install can be told from the first in a log rather than by inference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
7312aceded |
Get the scene model onto the devices that need it
The decoder can load from a path; nothing yet put a file at one. Four packaging routes, and one lookup that finds the result. ## Not `include_bytes!`, unlike the instance model The instance model is 11 MB and compiled in, which was the right call for it: Android hands the app no filesystem path (ARCH §6.9) and 11 MB is tolerable. The scene model is 24 MB, and 35 MB of constants in the binary is paid by every install whether or not the tab is ever opened. So it follows `models/face/` instead — carried as an APK asset, unpacked once at first launch into the shared directory a desktop install already uses, after which every lookup finds it where it finds a desktop user's. Assets are stored rather than deflated in the APK, so unpacking is a copy rather than an inflate. `embedded-scene-model` exists for the desktop build with nowhere else to read from, and for tests wanting the real graph. Off by default, which is the asymmetry with `embedded-model` and the reason for a separate feature. ## Three files, all or none `scene_model` insists on the graph, its vocabulary and the category descriptor together, for the reason `face_models` insists on its pair: a graph alone decodes to 150 anonymous channels. Reporting the set missing beats starting and failing at the first inference. ## The two model sets are not the same kind of thing `install_bundled_models` now carries both, and the distinction is worth keeping in view. Face weights are absent from the repository *by design* — the InsightFace grant is research-only (docs/faces.md §2) — so a build carrying none is ordinary. The scene model is committed, so a build carrying none means a checkout without `git lfs pull`. Neither is fatal. A photo editor that refuses to start over a missing grading feature is worse than one that starts without it, so both report themselves unavailable exactly as face indexing already did. The LFS-pointer guards apply to the `.onnx` only. The vocabulary and the descriptor are legitimately a few kilobytes, and a size check that fails on them would be a guard against the wrong thing. `scene_model` is exported ahead of the tab that will consume it so the packaging added here has something to be verified against — assets written where no lookup looks would be a silent mistake for as long as the tab took to arrive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
7418b040da |
Refuse a scan whose root has gone, instead of reporting it empty
FR-PLAT-AND-2, and a silent failure on both platforms. `dr_sync::scan` stepped over a NotFound or PermissionDenied the way it does for a child that vanished mid-walk -- correct for a child, wrong for the root, where it ended the walk, returned Ok with nothing in it, and reported a successful scan of a library that was no longer there. A lost root is now its own error. The images under it are marked Availability::Offline per FR-CAT-9 and no catalog row is deleted; `library::persist` clears the mark per file as each one is listed again, so a root that comes back needs no repair step. Partly satisfied rather than closed, and the gap is worth stating. The recovery half is real and reachable on Android today, because `map_status` turns Nextcloud's 403 and 404 into it and Nextcloud is how a phone actually gets a library in this build. The causes the requirement names -- revocation, reinstall, a removed card -- are properties of a persisted tree permission, and there is none: SAF does not exist here, `SourceRef::Document` is constructed only in test modules, and `LocalStorage` rejects the variant outright. When SAF lands it becomes a third producer of this error and nothing above it changes, which is why the discovery belongs in the connector. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
55cb9b5b30 |
Keep the test scene's own arithmetic from overflowing a u64
The first compile this branch ever had. `cargo fmt` reflowed four files and clippy passed at -D warnings untouched, but one test panicked: `the_signature_does_not_change_with_scale`, on "attempt to multiply with overflow". It is the fixture, not the feature. `scene()`'s little LCG multiplied the block's y by the golden-ratio constant with a plain `*` while the term beside it already used `wrapping_mul`, so any scene taller than about 104 pixels overflowed in debug. Only the scale test builds one that large, which is why 345 of 346 passed around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
e5db849f04 |
Mark a burst in the grid, and let it be folded away
The counterpart to the grouping: where the signatures come from, and how a group reaches a cell. Signatures are computed from the 256px thumbnails dr-thumbs already holds -- vastly more resolution than a 9x8 reduction can use -- so a library that has been browsed, or that has synced somebody else's shards, has already paid for them and no RAW is decoded for this. The consequence is stated rather than hidden: an image with no thumbnail gets no signature and never joins a burst. That is self-correcting, and it is why the pass runs when the thumbnail sweep finishes rather than on a timer. Nothing happens at import and nothing happens at query time. The mark is drawn as a child of the cell's TouchArea, for the same reason the star strip is: a click on it must not also reach `cell-clicked` and throw the user into develop, and children are hit-tested before the element they sit in. It is never hidden on hover the way the stars are -- a collapsed burst stands in for frames that are not on screen, and something has to say so whether or not a pointer is nearby. Folding changes what the grid's *query* returns rather than what its cells draw, because the grid is a window over an ordered query and the frames a fold hides are mostly not loaded. So the predicate joins VISIBLE in every query that lists or counts cells -- the window, the header's count, the run a shift-click resolves, and the ordinal a scrub lands on -- under the discipline VISIBLE's own comment sets out: present in four places of five is worse than absent, because the counts disagree with the cells and neither looks wrong on its own. There is a test for exactly that. `the_window_read_walks_the_ordering_index` now includes the burst clause. It asserts on the query plan while holding its own copy of the query, so left alone it would have gone on reporting green against a query the grid no longer runs. If the clause costs `images_grid_order` and puts the sort back, that fails here rather than becoming jitter someone measures in six months. The pass keeps its own drain timer in a thread-local instead of taking fields on the library controller, so everything the feature needs to run lives in one file and the screen that starts it holds nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
f5edd49b6b |
Let a manual collection be put in the order it is meant to be seen in
`collection_members.position` and `Sort::CollectionPosition` have been in the catalog since collections were, and nothing above dr-catalog has ever written or read either: `collections::set_order` had no callers, and the grid ordered everything by capture time whatever it was scoped to — dr-ui does not construct a `Query` at all, it has its own `GRID_ORDER` constant. So a manual collection was a set with an order nobody could see or change. Three pieces, because it could not be fewer: `grid_order_for` decides the ordering from the scope, and both readers take it from there. That is the load-bearing part. An ordinal only names a photograph relative to an ordering, so the window read and the span read have to agree — a shift-click resolved through a different ORDER BY than the cells were drawn with selects a different run than the one on screen, and the user finds out when the export runs. `read_ids_span` already stated that invariant about `GRID_ORDER`; this widens it to an ordering that depends on the scope. Only a single manual collection has one. A set draws its descendants' images too, and two children's positions are unrelated integers that interleave arbitrarily; a smart collection has no member rows to carry a position at all. Both fall back to capture time and refuse the drop rather than pretending. The drop is on the cell, on whichever half of it the finger landed — the trailing edge is the only way to name the last place in a collection, since there is no cell beyond the last one to drop in front of. `reordered` is pure and the membership is rewritten whole. `set_order` sets the positions it is given and leaves the rest, so a partial write would interleave the moved run with rows nobody touched; and it is read unfiltered, so what the filter is hiding keeps its place relative to what the user can see. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
5768100816 |
Borrow the library to index it, and give it back
The passes that need every photograph's bytes — thumbnails, face indexing — now borrow each one and release it at the end. On a placeholder library that is the difference between peak disk being the working set and being the whole library. Including on cancellation, which was nearly missed: the face sweep returns mid-loop when the user presses Stop, and without releasing there the disk is spent and nothing is delivered for it. `materialise` now answers whether *it* fetched the content. The pool used to work that out by listing a file's parent directory — one listing per file across a library — when the backend already had to `stat` it to decide whether to ask. One syscall instead of a directory walk, and it removes the bug class the tests found earlier: a file at the library root has no `parent()`, so every one of them read as already-downloaded. **Pinning is the retention control**, and it drives the model the catalog already had rather than a second one. `tier_desired` is what the user asked to keep hydrated, `pending_pins` is the resumable work list, and a pinned collection is never dehydrated for the same reason it was never evicted. It was in fact *broken* here before: `get` on a stub failed, and the pin worker logged "one unreadable file must not abandon the whole pin" and silently did nothing. Pinned originals on such a library are recorded with `path = NULL` (`Cache::record_in_place`) rather than copied under `originals/`. Two reasons, and the second is the important one. A copy would hold every pinned photograph twice, with the budget able to evict the half that was not costing the disk. And `release` deletes the file a row names — so a row that names none cannot delete anything, which puts the one catastrophic operation out of reach by construction rather than by remembering not to call it. Deleting a materialised file inside a synced tree removes the photograph from the server and every other device. Handing disk back is `spawn_dehydrate`, which asks the client. Two gaps written down rather than papered over (docs/storage.md §7): a hydrating pass cannot yet quote its cost, because a stub reports no size; and the two sweeps hold separate pools, so a library indexed for both fetches twice. |
||
|
dtourolle
|
c102ba9df2 |
Treat a placeholder as the photograph, not as a one-byte file
The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none. |
||
|
dtourolle
|
f12aece07e |
Make storage pluggable, and prove it with a folder backend
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.
A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:
- **Capabilities** — already there, and the reason the engine can drive
two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
whatever form its connector addresses, with no server in it. Loads
every existing config unchanged (`backend` defaults to `nextcloud`,
`endpoint` is stored under its historical `server` key), and
`Account::namespace()` reproduces the old catalog directory byte for
byte, because changing it would abandon a catalog, its thumbnail
shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
`ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
names a connector.
`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.
Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.
`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.
docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
|
||
|
dtourolleandClaude Opus 5
|
af5a13b3f7 |
Narrow the grid to several people at once, either way
"Show photos" could only ever mean one person. The two questions a photographer actually asks are "every picture of Anna or Bob" and "the pictures they are both in", and the second is not reachable by any sequence of single-person filters — no amount of switching between one person and another finds the frame they share. So the filter holds a *set* of people and a mode. `RatingFilter` was already the right home, as its own doc says: every query path threads it, so the count in the header and the cells in the grid are narrowed by the same thing, and this composes with stars, flags and the date range for free. The union is `EXISTS ... person_id IN (...)`. The intersection counts **distinct** people per image and compares against the size of the selection — one subquery rather than one per person, and it does not grow the statement with the selection. `DISTINCT` is what makes it correct: three faces of Anna in one frame must not satisfy a filter asking for Anna and Bob, and there is a test that says so. Any rather than All is the default. With one person the modes are the same filter, and adding a second to a union can only ever show more — so a user who has not noticed the toggle never ends up staring at an empty grid wondering what they broke. The toggle only appears at two people, because a control that demonstrably does nothing is a control that teaches the user to ignore it. Building the set needs no picker of its own: the Identity screen gains "And also…" beside "Show photos", offered only once the grid is already narrowed to somebody. Each person is a chip on the filter bar and each chip removes just that person, so a selection of three can be taken apart one at a time rather than only cleared wholesale. `RatingFilter` stops being `Copy`, since it now holds a `Vec`. Every query path already took it by reference; the casualties were two struct updates and one `Cell` that becomes a `RefCell`. 484 dr-ui tests pass, including the union, the intersection, that one person reads the same in both modes, and the repeated-faces trap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
d6380fecc8 |
Make the People screen a place work can be done
Five faults, all on one screen, and the Slint and Rust halves of each have to land together. **Regroup froze the window.** It ran inside the Slint callback, on the UI thread. It is much faster now, but fast is not bounded — the work grows with the library, and the one thing that must not grow with the library is how long the window stops answering. It runs on a worker thread with an mpsc channel and a 250ms poll, like every other long pass in this module, and the button says what it is doing instead of the window going quiet. Cancellation is dropping the receiver. Reclustering also prunes the empty groups the previous pass left, so pressing the button twice no longer fills the rail with "Unnamed (0 faces)". **The faces were a single row running off the screen.** The comment on the layout claimed to be a wrapping row; Slint has no flow layout and a HorizontalLayout does not wrap, so a person with forty faces was a person whose faces could not be reviewed past the fifth. It is now laid out the way the library grid lays out thumbnails, with the same arithmetic: choose how many columns of roughly the requested size fit, then divide the width between them so the cells fill the row exactly and nothing overhangs. **The header did not fit a phone.** A 240px name field beside five buttons is wider than an Android screen — and worse than not fitting, a layout cannot be narrower than its children's minimums, so the row reported that oversized minimum upwards and inflated the whole screen. The faces grid is its sibling, so it would have been measured against a width that was never on the display. The header is now two rows, the actions sit in a Flickable that scrolls rather than overflowing, and the rail narrows to 132px on the compact class. **Strangers crowded out the people who matter.** Most clusters in a real library are passers-by and other people's guests. "Not interested" sets a group aside; the rail hides it and says how many are hidden, with one button to bring them back. Reversible, and never a deletion — see the catalog commit for why. **A face was a dead end.** Identifying someone and then having no way to see their photographs is a filing cabinet with no drawer handles. "Show photos" narrows the library grid to that person and leaves a chip on the filter bar saying so, which is also how it is cleared. It is a term on `RatingFilter` rather than a grid scope of its own, exactly as that struct's own doc says new narrowing terms should be — so the count and the cells are narrowed by the same thing, and it composes with the others for free. Suggested faces count, not only confirmed ones, or a freshly grouped person would show an empty grid. Crops are read from where they are now stored, falling back to cutting one out of the proxy for faces indexed before that existed. 480 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
7dfbe3184a |
Merge master into the face branch
Build and test / Desktop (Linux) (push) Successful in 21m37s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Successful in 25s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 44m27s
Master moved 24 commits while this branch was building the face sweep, and one of them changed how the interface reaches a server: `remote.rs` is now the only file that names a connector, and everything else takes `&dyn RemoteBackend`. The face sweep was written before that landed and built a `NextcloudBackend` directly. Git merged the text without complaint and the result did not compile, which is the useful kind of conflict — it goes through `remote::connect` now, like every other pass. Two documentation conflicts, both resolved toward master. `code-health.md` was an add/add: master's copy carries the CH resolution for the backend seam and a better provenance note, so it wins outright, with its measured figures re-taken against the merged tree rather than either side's — `run()` is 1,855 lines now, 2,042 tests, 793 traceability tags. `traceability.md` is generated, so it was regenerated rather than hand-merged. The seam grades in code-health.md are unchanged by this merge. That is worth noticing rather than glossing: the face work went into the seams that already existed — `run()`, `library.rs`, `AppWindow` — which is exactly the pressure CH-1 describes rather than evidence against it. All four CI jobs pass: desktop (fmt, clippy, test, build), layering, traceability, and the Android cross-build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
28c046130c | Merge branch 'master' into android-bundled-face-models | ||
|
dtourolleandClaude Opus 5
|
371038614f |
Fetch the repairs first, or they never happen
The repair added in |
||
|
dtourolleandClaude Opus 5
|
242374fd0f |
Let the interface hold a backend without knowing whose it is
`dr-sync` defines `RemoteBackend` and a capability model the engine adapts to, so a second backend can be added without touching the code that uses one. That boundary was documentation. Seven files in `dr-ui` constructed a `NextcloudBackend` directly, ten functions took one by concrete type, and exactly two call sites in the tree — both inside `dr-sync` itself — ever held the trait object. A WebDAV or local-folder backend would have had a well-written trait to implement and nowhere to go afterwards. The change is smaller than the finding suggests, because the trait was already right. Every method the UI has ever called on a backend — `get`, `put`, `list`, `delete`, `create_dir`, `move_to` — was already on it, so nothing had to be added and no behaviour moved. Ten signatures widened to `&dyn RemoteBackend`, sixteen constructions became `remote::connect`, and `remote.rs` is now the only file in the interface that names a connector. `connect` returns `Result<Box<dyn RemoteBackend>, RemoteError>`. The error type is `dr-sync`'s rather than the connector's, which is why every call site kept its shape — the `match`, the `let Ok(..) else`, and `.map_err(ScanFailure::local)?` all still read as they did. One wrinkle worth recording: `&Box<dyn Trait>` does not reach `&dyn Trait` on its own. The compiler reaches for unsizing, which wants `Box<dyn RemoteBackend>: RemoteBackend`, and reports a confusing missing impl rather than suggesting a deref. Twelve call sites therefore say `&*backend`, and two say `let backend: &dyn RemoteBackend = &*backend` where a borrow is shared across lanes. What this does *not* do is abstract credentials. `AppCredentials` is an app password from Login Flow v2 — a Nextcloud protocol, not a general notion of authenticating to a remote — and seven files still name it. An OAuth token, a bucket key pair and an app password have no useful common shape, so deciding what an account is across backends before a second one exists would be a confident guess. code-health.md CH-2 now records that as the remaining half, and it should wait for the backend that forces it. Verified: fmt clean, clippy clean at -D warnings, 2041 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
051447bda6 |
Keep the proxy a found face will be cropped from
Every cell on the People screen read "no preview" while the sweep was happily reporting 893 faces found. Both were true. Faces were being detected and stored correctly; there was simply nothing left to draw them from. A face is stored normalised and drawn by cropping the proxy it was found on — `identity::decode_proxy` reads `FACE_TIER` out of the thumbnail store. The fetching sweep fetched a preview, detected on it, wrote the faces and dropped the pixels. So every face it found pointed at a proxy that had never been stored, and the grid had nothing to cut. Worse, that state could not repair itself: the image has its `face_index` row, so it is not outstanding work and no later pass would look at it again. Two fixes, and the first is nearly free. The sweep now keeps the proxy — it has already paid the round trip and the decode, and the crop needs those same pixels the moment the user opens the person. Kept **only where a face was found**: two thirds of a personal library is landscapes and documents (docs/faces.md §7a), those will never be cropped, and skipping them keeps this well clear of the whole-library cost `SWEEP_THUMB_SIZE` deliberately avoids. The downscale to the large class happens after detection, which is the last use of the full buffer. Second, the sweep now picks up images that have faces with no proxy, whatever put them in that state — this bug, or an ordinary cache eviction, which would have produced exactly the same empty grid. Re-running detection repairs it and loses nothing: `record_detections` replaces rather than appends and carries the user's confirmations across the replacement. That makes the screen self-healing rather than dependent on nobody ever evicting a thumbnail. The proxy is stored *before* the detections. A kill between the two then leaves a proxy with no faces — which the next pass simply re-indexes — rather than faces with no proxy, which is the state that cannot recover. Note for the library already part way through a sweep: the 986 images indexed before this will be picked up by the repair route on the next run. 470 tests pass, including one that a face whose proxy is gone becomes work again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
c8c6368542 |
Index the whole library by fetching what it has not seen
"Index faces in the whole library" could not. Its work list was intersected
with the thumbnail store at `ThumbSize::Large`, and nothing fills that class for
a whole library — `SWEEP_THUMB_SIZE` is deliberately `Grid`, because the large
class is ~860 MB of shards against ~200 MB and every syncing device pays it. So
the only images with a large proxy were the ones the user had personally zoomed
into or opened in the loupe. On this library that was 220 of 23,529.
The comment defending it misread the requirement:
// Requesting one here would put face indexing on the network path,
// which FR-CULL-8 explicitly keeps it off.
FR-CULL-8 keeps indexing off the **full decode**, not the network, and then says
the opposite in the same paragraph: "where no proxy exists, the job requests one
at background priority rather than decoding inline". faces.md §7 repeats it.
Neither was implemented.
So the pass fetches. Same two-stage route the thumbnail sweep uses — the header,
then the located preview's own byte range (FR-NC-3) — so no whole file is pulled
and no RAW is decoded, because an embedded preview is a JPEG. The work list is
now every visible image with no `face_index` row for the model: 23,308 here,
against nearly none before.
**It indexes at the resolution the preview actually has**, not the 1024 the old
tier would have given. `locate_preview` already picks the largest embedded
preview, and the thumbnail sweep was decoding it and throwing the detail away at
`downscale_to(256)`. A face 2% across the frame is 5 px on a grid thumbnail and
~61 px at the cap here — and 112 is what the embedder samples, so this is the
difference between an upsampled crop and a real one. `crop_px` records which,
per face, as §7 intended.
Capped at 3072 rather than truly full: `index_proxy` needs packed `f32` RGB at
12 bytes a pixel, so a 24 MP frame is ~288 MB and the fetch lanes hold one each.
The constant is named and sits next to the reason.
Orientation is applied **before** detection, not after downscaling. That costs a
permutation of a larger buffer — ~15 ms against a ~150 ms decode — and buys the
entire class of bug this codebase keeps having: detection then runs on the
photograph rather than the sensor, so every box and landmark is already in the
space the catalog stores and the overlay draws, with no second mapping to get
backwards.
One detector and one embedder serve every lane. The lanes are concurrent futures
on a single thread, not threads, and inference contains no await, so a `RefCell`
borrow never overlaps another — a pair per lane would duplicate ~16 MB of
weights for no parallelism.
Images with no face in them are recorded too. `face_index` records that
detection *ran*, and zero is its most valuable value: without the row every
landscape and document scan returns on every pass, for ever, and in a personal
library that is most of it (§7a).
The old store-only pass survives as `spawn_store_face_sweep` for
`examples/face_index.rs`, which indexes a local store with no network. The
settings copy no longer claims indexing reads "the photographs already
thumbnailed above", and the audit line says "to fetch" rather than "awaiting a
proxy", which had become a blocker that no longer blocks.
Verified against the real catalog: the new work list returns 23,308 where the
old one returned effectively nothing. 469 tests pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
dtourolleandClaude Opus 5
|
2d95807542 |
Package the models on every platform, not just the phone
The Android bundling landed the weights under that platform's asset directory, which was the wrong home the moment a second packager wanted them. `makepkg -si` produced a desktop install with no model at all — the same "no face model is installed" the phone used to show, for the same reason: nothing put the files anywhere the app looks. So `models/face/` at the root is the one copy, and both packagers read it: assemble-apk.sh bundles it as APK assets, and the PKGBUILD installs it to /usr/share/darkroom/models. Both refuse an LFS pointer rather than shipping a 130-byte file that fails inside the graph loader on a user's machine. `face_models` now searches three places, most specific first: the account's own directory, the shared user directory, then $XDG_DATA_DIRS. So a packaged pair is found automatically and a pair the user placed by hand still outranks it — which is what keeps a deliberate choice of weights from being overridden by an upgrade. $XDG_DATA_DIRS rather than a hard-coded /usr/share: that is the variable a distribution, a prefix install or a Nix-style store already sets to say where its data went, and its documented default is exactly the two paths that would otherwise have been hard-coded. Empty on Android, which has no such directories — there the APK's copy is unpacked into the shared user directory instead, because an asset inside a package is not a path anything can read from. Verified: the APK still carries both models at assets/models/, the PKGBUILD parses and installs from the new path, 467 tests pass. Includes the pkgver 0.6.0 → 0.7.0 bump that was already sitting uncommitted in the working tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
eaafacc3fb |
Give the phone the model it had no way to obtain
Face indexing was compiled into the APK all along — dr-ui takes dr-face with `inference` on every target, so SCRFD, alignment, MBF, calibration and clustering were all in there. What was missing was the weights, and on Android there was no way to supply them. Route C (docs/faces.md §2.2) says the user obtains the model and the app loads it. On a desktop that is a real gesture: drop two files in ~/.local/share/darkroom/models/ and indexing starts working. On Android it is not a gesture at all. `internal_data_path` is app-private, `run-as` needs a debuggable build, and the in-app fetch route C specifies was never built — so the settings page reported "no face model is installed" on every launch with nothing behind the message. Not "off until you supply weights"; off. So the shape-fixed pair goes into LFS under the APK's assets, assemble-apk.sh copies it into the package, and `android_main` unpacks it to the shared models directory before anything asks whether a model is present. Three things that are not incidental: The models directory is now shared across accounts rather than per-account. Weights are identified by `faces.model_id`, not by who is signed in, so two accounts had no reason to hold two copies — and the unpack runs before any session exists to key a per-account path off. `face_models` still prefers a per-account directory when one is populated, so anyone mid-migration keeps the ability to pin one library to its own pair. The unpack writes under a temporary name and renames. `face_models` decides availability on `is_file()` alone, so a copy truncated by the process being killed would leave a file that passes that test and fails inside tract — reported to the user as a broken model rather than a missing one. assemble-apk.sh refuses an LFS pointer. At ~130 bytes it looks exactly like a model to `cp`, and unchecked it reaches the device and fails in the graph loader instead of telling someone to run `git lfs pull` — the same guard dr-segment's build script applies to yolo26n-seg.onnx. The licensing half is unchanged and recorded in §2.2a: the InsightFace grant is research-only, this is a private repository and a self-installed build, and these files come back out before anything is published. The weights are still not a cargo build input — dr-face has no `models/` directory and no `embedded-model` feature, and nothing in the build reads them. The APK assembly step copies two files and is the only thing in the tree that knows they exist. Verified on device: both models unpack on first launch (2524817 and 13616095 bytes) and the APK carries them at assets/models/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
26a1eb7e28 |
Record that face detection has run, not just what it found
An image with no faces in it was indistinguishable from one that had never been looked at, so every landscape, still life and document scan in the library was re-detected on every pass, for ever. In a real library that is most of it: on the 23,527-image test library, 64 of the first 110 images indexed contain no face at all. Schema v9 adds face_index, a run marker per (image, model) carrying the face count and the proxy edge it read. Keyed on the model, so a model change puts every image back in the queue by itself. That makes a coverage figure possible, which is the thing a user actually wants to see. The audit also splits the outstanding set by whether a proxy exists, because 23,417 awaiting a proxy and 110 ready to index are different problems, and telling the user to run indexing again would not fix the first. The Identity screen gains Index faces, Stop, and the coverage line. examples/face_index.rs is the same check and sweep without a window, which is the right shape for an overnight pass. Measured on the real library in release: 3.5 images/second, 110 images and 125 faces in 30 seconds, and a second run correctly finds nothing left to do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
3b5952769b |
Emit floats an f32 can hold, and drop the format! that formats nothing
CI runs cargo fmt --check and clippy -D warnings, and this branch had never been through either. Both would have failed it. The bulk was the generated colour tables: eight significant figures where an f32 carries about 7.2, so the eighth is noise that rounds away at compile time and clippy's excessive_precision says so 109 times over. Fixed in the generator rather than only in the file, so it stays fixed -- and the file is trimmed in place rather than re-derived, because regenerating it needs a colour-science stack that has nothing to do with the defect. The format! in the composer is mine too, from extracting the rendering tail: the braces in it were escaped because the text used to live inside a larger template, and once extracted the escapes are noise and the call formats nothing. Also here, and clearly not mine: an unused import and a shadowed binding in dr-gpu, and an unused import in a test. They are pre-existing -- clippy has been failing on master before this branch existed, on lints like is_multiple_of that arrived with a toolchain rather than with anyone's code. Fixed because CI cannot go green around them, and called out because a merge commit is a bad place to quietly edit someone else's crate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolle
|
6925aa2a86 |
Merge master into film-simulation
🐳 Android image / Build and push (push) Successful in 0s
Build and test / android-image (push) Successful in 1s
Build and test / Desktop (Linux) (push) Failing after 59s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Successful in 1m1s
Build and test / Android (aarch64) (push) Failing after 22m44s
Master gained the library's index paging while this branch was building the film simulation, and the two met in library_ui.rs. Only the generated traceability matrix conflicted; it is regenerated here rather than hand-resolved, which is what it is for. |
||
|
dtourolleandClaude Opus 5
|
6d6ef8d34b |
Page the grid along an index instead of sorting the library each time
Scrolling jittered, and this was the largest single reason. Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and neither half of that was being answered the cheap way. **The sort.** `GRID_ORDER` leads with `captured_at IS NULL`, so undated frames fall to the end. No ordinary index answers that — the leading term is an expression, not a column — so SQLite sorted the whole library into a temp b-tree on every window read, then threw away the first `k` rows of it. Schema V7 indexes the expression exactly as the query writes it, partial on the same `shadowed_by IS NULL AND trashed_at IS NULL` the grid filters by, so the read becomes a walk along the index. **The join.** `LEFT JOIN remote` was paged *after* it was joined, so reading 280 cells at offset 20,000 first seeked into `remote` for all 24,000 rows and then discarded 23,720 of them. The file ids are now fetched for the 280 rows that survived — the shape the badge and rating reads already use, one query for the window rather than one per cell. Measured together on 24,000 images at offset 20,000: **15.2 ms → 0.36 ms**, inside a scroll handler that has 16.7 ms to draw a frame. The test asserts on the query plan rather than on a duration, because there is no other symptom. A `GRID_ORDER` edited out of step with the index, or a column added back that drags `remote` in again, both still return exactly the right cells — just after sorting the library — and the jitter would come back with nothing to point at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
baa8957e80 |
Let a photographer choose the film, and remember which one
The stock model rendered correctly and nothing could ask for it. This is the picker, and the sidecar key that makes the choice outlive the session. How the choice persists was the open question, and the answer was already written down twice in sidecar.rs: `rating` is a top-level key "because a rating is not an edit", and `masks` are one "because a layer is not a scalar". A stock is that kind of thing -- a choice of material, not a number a slider moves -- so it is a top-level key too. It stores the **id**, not an index. Stocks are files that users add, so an index would mean installing a profile silently changed which film every existing photograph had been developed on. A name this build has no profile for still round-trips untouched, because the alternative is that syncing to an older phone quietly un-develops the picture. Only the names travel. Turning one back into tables needs the profile database, which dr-pipeline deliberately does not link, so `Version::apply` clears the film and the session re-bakes -- after the parameters, because the bake reads the film's own exposure sliders and the print balance is solved against them. That is also why moving those sliders rebuilds the lookup where no other control in the panel does: an enlarger's filtration depends on how the negative was exposed. The panel keeps its rule. It still names no operation and still generates every control from a declared parameter kind; the stock gets a bespoke control beside those, exactly as the mask stack does, and for the same reason. The film's exposure and print exposure arrive as ordinary generated sliders. Two defaults worth stating. Picking a colour negative prints it, because an unprinted one is an orange strip and offering that as the first thing somebody sees after choosing Portra reads as a bug rather than as a choice -- the toggle is there for anyone who wants the scan. And a paste carries no film: a preset is a parameter map, and a stock is not a parameter, so pasting one would paste a choice the clipboard never took. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
54f9cb54fb |
Take the whole run a shift-click names, not the part that happens to be loaded
Shift-clicking two photographs selected only the cells between them that were in the loaded window. The grid is a window of a hundred or so over a library of twenty thousand, and `apply_press` resolved the range against that window — `ids[lo_row..=hi_row]`, clamped to what was there. Everything else in the run had no id anywhere in the UI, so it was silently dropped. The user cannot see that: the selection count is off screen along with the photographs, and the gesture only announces itself when the drop files a dozen images instead of two hundred. The two ends are *ordinals*, and only the catalog knows what lies between them. `read_ids_span` asks it, through the same predicates, the same rating filter and the same ordering the window itself is read with — an ordinal names a photograph only relative to an ordering, so a run taken through any other one is a run through a different library. That ordering is now a constant, `GRID_ORDER`, shared by the window, the trash's own order beside it, and the run: capture time first, with the file name breaking ties and nothing more. A card written by two cameras interleaves names that have nothing to do with each other, and what "everything between these two" means to a photographer is a stretch of an afternoon. The query is reached through a closure handed to `CollectionsController` at wiring time rather than a catalog handle, because the scope and the filter that bound the run belong to the grid's controller. `apply_press` stays a pure function of what it is given, which is what keeps the selection rules testable with no library open — and the tests pass a run that reads a plain slice. Where there is nothing to ask, the loaded window is still used: a poorer answer than the catalog's and a far better one than a gesture that appears to do nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
fa4ad6e2d6 |
Drag the date range on the axis it is chosen from
The range could be turned on with a finger and not aimed with one. Its two ends were typed as `YYYY-MM-DD` into 108px fields behind a soft keyboard, to name days already drawn on the axis a thumb away; and the chip that seeds them takes its span from the timeline's zoom and pan, which are a wheel and a middle button. A touch screen has neither, so on Android the filter was a switch with no aim. The band is now on the timeline. Two ends with grips, dragged along the bars, released to filter — the histogram was already how a period is found, and this makes it how a period is stated. Both ends snap to whole days, which is what the typed fields mean, what `show_range` reads back out, and a floor under a range dragged shut. The fields stay for what dragging cannot do: name an exact day, and say in words what the range is. For that to work the axis had to stop following the range. Redrawn to the band, it moved the ground under the very handles doing the narrowing, and there was nothing outside the range left to widen back into. While there: a fixed number of equal bins instead of calendar buckets. Between one calendar unit and the next the bar count is free to wander by a factor of twelve, so zooming in halved it two steps out of three — the same picture drawn wider until it jumped back to fine. Equal bins also include the empty ones, so a bar's position on the track and the date under it are finally the same quantity; before, a library with gaps drew a February six months wide and the marker, the band and a click all pointed somewhere else. The count is a setting, 32 or 64, because the right answer is a question about the screen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
fbf286d504 |
Keep the mask when the photograph is closed
A local adjustment survived until the session ended and then did not exist. Nothing reported it, because nothing had failed. The sidecar format has carried masks since they were added, `Version::apply` restores them into the graph, and `Version::update` captures them — that work landed complete and was never called. The autosave writes `copy_settings()`, which is a `Preset`: a map from (operation, parameter) to a number. A mask is not a parameter. It is a rule about *where*, with a chain of its own, so it fell outside the only thing being written, and the read path had nothing to read. The save now carries the stack beside the preset, on both the local and the remote path. Deliberately not merged but replaced wholesale: this is the stack as it stands, so a layer the user deleted has to leave the file too. Merging two devices' stacks is `Sidecar::merge`'s job and belongs to sync (FR-NC-9). A paste still carries no masks, and the `Option` is how that is said. Settings travel between photographs; a mask does not, because it is drawn against one frame and describes nothing on another — and `Scope` cannot express that, since it filters parameters and a mask is not one. The test fails against the old save path with "the mask must come back with the photograph", which is the whole of the defect: not a crash, not an error, just an edit that was not there in the morning. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
d91f1ec277 |
Thumbnail the whole library on request, and send the shards up
The grid fetches a preview only for cells that are actually browsed, which is the right posture over a link that must not be saturated to show one screen (FR-NC-3). The cost is that the thumbnail store ends up holding the fraction of the library someone happened to scroll past — and that store is the one derived thing worth syncing, since a second device that downloads the shards gets a full grid without touching a single RAW. So the complete set is worth an hour of range fetches paid once, deliberately, on a machine that can afford it. That is what this adds: a pass over every visible image with an `oc:fileid`, launched from the settings page and reported in the activity register like every other background job. The work list is what the store lacks rather than a flag in the catalog, so it is resumable by construction and safe to press twice. Lane-parallel like the metadata sweep, but it does what that sweep declined to. The store is `&mut` and cannot cross lanes, which is why dating the library skips thumbnails entirely; here the lanes fetch, decode and *encode*, and only the ~20 KB result crosses back to the one thread that owns the store and writes the chunk. The parallelism is real and the single-writer rule is not bent. Grid class only. The large class is ~860 MB of shards against ~200 MB on the reference library, paid by every device that syncs them; a photograph looked at closely still gets its large thumbnail from the interactive path. Dates come free — the header a preview needs is the header EXIF lives in — and the pass ends by pushing the shards to the server. A filled store that never leaves this device would be most of the cost for none of the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
1526c957cf | wip: ingest | ||
|
dtourolle
|
735683b849 | wip: ingest | ||
|
dtourolleandClaude Opus 5
|
edcaf42ded |
Show only the photographs taken in the period you are looking at
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
Build and test / Desktop (Linux) (push) Failing after 57m33s
Build and test / Layer separation (push) Successful in 36s
Traceability / Requirement traces (push) Failing after 29s
Build and test / Android (aarch64) (push) Failing after 9m28s
The library could be narrowed by rating, flag and availability, but not by when a photograph was taken — so finding a fortnight meant scrolling to it and holding position. The range rides on `RatingFilter` for the reason `local_only` already does: every query path threads that one struct, so the count in the header cannot claim a total the grid does not draw. Undated images are excluded whenever either end is set — they cannot be inside or outside a span, and drawing them made the range look as though it had not applied. Taken from the timeline rather than typed into two date fields. Finding the period is what the histogram is for, and having found it the user should not have to read the dates off the axis and key them back in. The histogram keeps drawing the full extent while the range is on, or there would be nowhere to widen back out from. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
02d629922f |
Draw the date histogram over the collection you are looking at
Build and test / Desktop (Linux) (push) Failing after 57m25s
Build and test / Layer separation (push) Successful in 33s
Traceability / Requirement traces (push) Failing after 27s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Build and test / Android (aarch64) (push) Failing after 9m45s
The timeline counted the whole library whatever the grid was showing, so opening a collection left a fortnight in Arosa as one column of a fifteen-year axis — an axis describing photographs that were not on screen. Scope the buckets and the span to the same collection and rating filter the grid uses. `timeline_range` counts `images` alone and cannot express the membership join, so the scoped query lives beside the other scoped readers in the UI and shares their descendants-of-scope rule. `catalog_span` now delegates to the same scoped reader. Zoom and scrub measured the full library while the bars were scoped, so a scrub could land on an instant the collection did not contain and send the view somewhere the user had not asked to go. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
09f6cf8c0f |
Name the sidecar the drain could not deliver
The write path learned to say which file the server refused; the drain path —
the one that runs for work queued while offline — still reported only a count.
That is the path that matters most, because everything it carries was made
with no connection and exists on one device.
With it named, the failure on the tablet is:
draining PhotosRaw/2026/2026-08-03/_MG_9221.drsc:
permission denied
on a credential that pushes the catalog to the same library root in the same
pass. So this is not the account and not the app password: one path is
writable and another is not, which points at the server — a read-only share
over that folder, or a file access control rule on the extension — rather than
at anything this side can retry its way out of.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
dtourolleandClaude Opus 5
|
6a7ed37aed |
Say which sidecar the server refused, and where
A queued sidecar failing to upload was reported as a count and a reason — "1 queued sidecar(s) still undelivered: permission denied" — with the path only at debug level, which the app filters out by default. That is unsynced user work: a rating or an edit that exists on one device and nowhere else. Which photograph it belongs to, and which path the server refused, is the whole of what makes the failure actionable, and without it a 403 on a single file reads the same as a whole library failing to sync. Observed on the tablet, where writes to the derived folder succeed on the same credential — the catalog pushes fine — while one sidecar beside its image is refused. That combination says the path matters and the account does not, so the path is the thing worth printing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
c9c5c43aa1 |
Carry the library across when its home moves
The previous commit moved durable data out of Android's cache directory, and on its own that would have been an upgrade that quietly discarded work. The app looks in the new location, finds nothing, and rescans a library of tens of thousands of images over the network — while the old copy, including every offline rating and edit that had not yet synced, sits in a directory the system is free to delete. So the account's directory is moved once at startup, before anything opens a store. A rename rather than a copy: both are inside the app's own data on one filesystem, so it is atomic and cannot half-finish. An existing destination wins and the move is skipped — that covers a second run and a fresh install, and in neither case may this overwrite live data. A failed move is logged, not fatal. The cost is a rescan, which is recoverable; refusing to start is not. Two tests, against ordinary directories rather than the platform's idea of a cache: one puts an unsynced sidecar in the old location and asserts it is readable in the new one afterwards, the other pins that live data is never overwritten. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
b8e9793908 |
Keep offline work out of a directory Android empties
The catalog, the sidecar cache and the export outbox were all landing in the app's cache directory on Android, where the system deletes them without asking under storage pressure. `catalog_path` derived its base from `XDG_DATA_HOME` or `HOME`, and neither is set on Android, so it fell through to `temp_dir()` — which resolves there to `/data/user/0/<pkg>/cache`. Confirmed on the tablet: the app logs its catalog under `cache/darkroom/...`. What sits beside that catalog is not disposable. `sidecars/` is the commit point for every rating and edit made with no connection — the whole mechanism that makes offline culling safe — and `outbox/` holds exports the user has already been told succeeded. A day of sorting on a train, evicted by an OS housekeeping pass before it ever reached the server, is the worst failure this application can have, and it would leave no error and no trace. The fallback is now `SessionStore::data_dir()`, the persistent per-app directory the Android entry point establishes before any store opens — the same one credentials and sessions already use. Desktop is untouched: the XDG data location is still preferred, so nobody's catalog moves. Two tests. One asserts no durable path contains `/cache/` or `/tmp/`; the other pins the outbox to the catalog's parent, because three call sites derive their location that way and a change here moves all of them at once. Found while verifying that offline browsing works on the tablet with wifi disabled — which it does: the app cold-starts with no network, reports the scan failure without crashing, and serves its grid from the local store. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
40d4e439a9 |
Drop the parameter the offline sidecar writer never used
`write_one_sidecar` took an `Option<&NextcloudBackend>` it ignored. It was a leftover from the shape the write path had before online and offline separated into two functions: the offline one records to the cache and queues, and has no server to talk to by definition. A parameter that is always `None` and always unused says the opposite — that there is a case where it is `Some` — and the next reader has to check. The closure it was threaded through is renamed to say what it does rather than how it is called. `run(None)` needed the reader to know what `None` meant; `queue_all()` is the sentence. Also regenerates the traceability matrix, which now records FR-CAT-9's queue. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
cb1d2be240 |
Choose the export folder by walking the server, not by typing it
The destination for a Nextcloud export was a text field. Nobody recalls the exact spelling of a path three levels down, and getting it wrong does not fail — `create_dir` makes whatever was typed, so a misremembered folder becomes a new one at the root and the exports are somewhere nobody looks. So it is picked the way the library root is picked, using the same `FolderBrowser` model the launch screen drives: up, into, and "use this folder", confirming the folder currently *shown* rather than one selected in the list. Same rule in both places, so the phrase means one thing. The model is shared; the worker is not. `settings_ui::spawn_folder_list` is a near-twin of the launch screen's, because that one reaches into the `LaunchController` for its session and reports onto the launch screen's error line, while this one is handed credentials and writes to the settings page. Factoring them together needs a function taking both controllers or a trait implemented twice to abstract two call sites — more machinery than the twenty lines it saves. What matters is shared already: navigation behaves identically because both drive the same model. The callbacks are wired in `lib.rs` rather than in `settings_ui::wire`, because listing a remote folder needs credentials and the settings page holds no session on purpose — it is reachable before a library is opened and must not depend on one existing. With no account the picker says to sign in first, rather than showing an empty list that reads as a server with no folders. Details that are decisions rather than accidents: the picker opens at the library root rather than at whatever half-typed path is in the field, which would list nothing and look broken. The listing area is a fixed 180px, since a folder with sixty children would otherwise push the rest of the settings page off the bottom. "Up" is disabled at the root rather than hidden, so the row does not jump as the user navigates. A failed listing leaves the picker open on the folder it was showing — where the user had got to is not something to discard over a dropped request. And the chosen folder saves immediately like every other setting on a page that has no Save button. The poll timer lives on the controller for the reason `LaunchController` keeps its own there: a `slint::Timer` stops when dropped, so one local to the function that starts it would be collected before the listing arrived. Carries in-flight work from a parallel session — a segmentation pass in dr-gpu, a sidecar cache, and the develop panel's continuing changes. 1020 tests pass, fmt clean. One clippy warning remains and is not mine: `sidecar_cache::dir` is unused while that work is in progress. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
e00c99b864 |
Let a photograph leave: an export button, and a cache to leave from
dr-export could turn a frame into bytes and nothing could ask it to. This is the button, and the place the bytes go. **Everything is staged first.** An export bound for the server is written to a local outbox and uploaded afterwards; offline is not a special case, it is the same path with a drain that finds the server absent. Doing it the other way — upload directly, stage only on failure — makes the failure path the one that is rarely exercised and always broken, and a network drop mid-batch leaves some exports existing and some not with nothing recording which. Staged first, an export is finished the moment it is written and the upload is a promise kept later. The outbox sits beside the catalog rather than under the cache. dr_catalog's cache already draws that line: passive entries are a convenience and go under LRU, pinned ones are a promise and never do. An export awaiting upload is a promise — the user was told it succeeded — and sweeping it for disk would destroy the only copy. Bytes are written before the destination record, so a kill between the two leaves an orphan the drain ignores rather than a record pointing at nothing. The status line says "Queued for Exports/2026", never "Exported to Nextcloud", until it has actually landed. There is a test asserting that wording, because the tempting shorter sentence is a claim the app cannot keep. The drain runs on the sync pass, before the shards: a thumbnail shard can be rebuilt from the originals and the catalog is an index, but a queued export exists nowhere else. `DevelopSession::render_for_export` renders the framed size rather than reusing the frame on screen, which is deliberately viewport-sized (FR-DSP-1) — encoding that would hand the user a soft, screen-sized file with nothing to say anything had been lost (FR-EXP-9). One compromise, recorded rather than hidden: the export runs synchronously on the UI thread, so the window is unresponsive for the few hundred milliseconds a full-resolution render and encode takes. Moving a DevelopSession and its GPU pass to a worker is a larger change than one button earns, and it is batch export that makes the wait intolerable rather than merely noticeable. Still missing: the Nextcloud folder *picker*. The destination is typed into Settings for now. `FolderBrowser` in launch.rs is already the reusable model for it — it browses a remote tree and nothing about it is specific to choosing a library root — but wiring it into the settings page needs a listing worker and browser UI there, which is its own piece of work. Carries in-flight work from a parallel session — presets, the develop copy and paste, and the node schema's `presentation` and `enum` support. One misplaced callback in settings_ui.rs is moved from `render` to `wire`: registered in `render` it borrowed a `&SettingsController` into a 'static closure and would not compile, and that file's own docs say render pushes properties while wire connects callbacks. 992 tests pass, clippy and fmt clean. Traceability 48.3% -> 51.0%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
489465faf0 |
Show the photograph the way it was taken
Nothing read EXIF orientation, so every frame from a body held sideways lay on its side — in the grid, in develop, and in the read-only preview. The tag is honoured as part of *reading the file*, at the same standing as a RAW's masked-photosite crop, never as an edit. It lives as a baseline on Framing rather than as a starting value for quarter_turns, which is what keeps four things true: a sideways file opens unmodified, reset returns it to upright rather than to the sensor's scan order, its sidecar stays empty, and the rotate button still moves the image 90° whatever the file underneath it says. Framing::effective composes the baseline with the user's own turns through the group law rather than by adding turns and OR-ing flags. The naive version gets one case wrong — an odd baseline turn plus a user mirror — and gets it wrong quietly, because the result is still a plausible orientation. The composition collapses to a single permutation, so obeying the tag costs nothing per pixel. dr_decode::orientation is a header-only IFD walk, separate from metadata() for the reason the entry points are separate at all: the grid asks once per cell and must not build a rawler decoder to get one tag. CR3 and RAF fall back to the full read, being neither TIFF nor JPEG. Written down as FR-DEV-3h. Known gap: thumbnails cached before this stay sideways. The store is keyed by file and size, and its shards sync — invalidating them would have every client re-download 25 MB a shard, which is not this commit's call to make. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
03326242a1 |
Make the CI checks say what they mean, and format the workspace
The Android job's "Verify minimum API level" step has never verified the minimum API level. It took the first `*.so` anywhere under the target directory, which is a host proc-macro from debug/deps — an x86-64 object built by the runner's gcc, whose .comment section cannot mention Android and so can never contradict the expected value. It now reads the artifact under the target triple, compares against MIN_API parsed from the Dockerfile rather than a second copy of the number, and fails on a mismatch. Both sides are checked non-empty first: two failed parses would otherwise compare equal and pass, which is the same silent success in a new costume. The Android image installs one SDK package per layer and keeps the output. sdkmanager is a JVM program that aborts when it cannot get memory, and the single `> /dev/null` step reported that as a bare "exit code 134" while a retry re-downloaded everything that had already succeeded. tools/ci-local.sh runs all four jobs — desktop, android, layering, traceability — against the host toolchain, which is pinned to the same 1.92.0 CI installs. Its matrix check compares regeneration against the working tree rather than against HEAD: CI starts from a clean checkout, so git's answer is the right one there and reports every local run stale here. The rest is rustfmt across the workspace, and the clippy findings that surfaced once it did: manual_contains in dr-thumbs and collections_ui, a map iterated as pairs for its keys, an index loop over a slice, and two runtime assertions on a constant now made at compile time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |