67 Commits

Include renames

Author	SHA1	Message	Date
dtourolle	e6ac31d39d	Give the face sweep a size budget, so a panorama is never fetched ... The sweep fetches the whole original before it can learn anything about it, and the one file in the reference library the decoder refuses on sight is a 521 MB stitched panorama — so every pass on the tablet spent half a gigabyte of Wi-Fi to find that out again. The catalog already knows the byte count, and that is enough to decide before the fetch: originals over 256 MB are marked examined with nothing found and a zero edge, counted as failed, and named in the log. Below the line is every camera RAW the library holds; above it, four files, all panoramas. A budget and not a verdict on panoramas. The right treatment for one is a tiled pass — read it in strips, detect in each, stitch the boxes back — and the zero edge is what that pass would select on. Until it exists, this is what keeps a background sweep on a phone from paying for the decision the decoder cannot make.	2026-09-19 12:14:05 +02:00
dtourolle	327decfab1	Fuse every detector's faces into one population per embedder ... Choosing "Thorough" made the library look empty. The detector setting writes under its own faces.model_id, and every reader of "the faces" keyed on that exact id: the clustering pass, the coverage figure, the sweep's work list, the shard export and import, and the sync merge's face matching. On the reference library that restarted coverage at 1,834 of 19,140, drew a People rail of 36 faces for a person with 520, queued a ~400 GB re-fetch on each device, and stranded the desktop's 3,583 confirmations under the old id: the tablet held the same faces under the new one and the merge refused to match them. Same photograph, same box, same embedder, two ids — that is one face, not two libraries. The embedder half of the id is now the key. embedder_of and embedder_sql give it to every query; writes keep the full id, so which detector drew a box stays on record. record_detections is unchanged and is where the generations meet: an image holds one pipeline's faces at a time, and a re-detection carries confirmations across by box overlap. The merge's match_faces applies the same rule within an embedder. The calibration is keyed on the embedder too, since the similarity space did not change. Shards travel every generation, each under its own id, and a peer adopts whichever it is sent — including a stronger detector's pass over an image it indexed itself with a weaker one, which is the re-detection its own sweep would otherwise queue, already done. Never downwards: a tablet on Fast keeps the desktop's Thorough faces. The sweep gains the same tail — images a weaker detector indexed, after the ones nothing has — driven by FaceDetector::supersedes, so choosing a stronger detector still improves the library over time without first making it disappear.	2026-09-19 10:49:28 +02:00
dtourolle	f8addbee53	Mark a file the decoder cannot open, so the sweep stops fetching it ... A decode failure in the face sweep was counted, logged at debug where nobody saw it, and left unmarked — so the next pass fetched the same file and failed the same way. For the 521 MB panorama behind rawler's panic that was half a gigabyte per sweep, on a tablet. It is now marked examined with nothing found and a zero edge, which is what a later "try again with a better decoder" pass would select on, and the warning names the file. The failure count is unchanged: it did fail.	2026-09-19 10:45:08 +02:00
dtourolle	78cb00634e	Fetch the photographs around the open one ahead of the step to them ... Walking the photo roll was one download per frame: every step showed "Downloading…" over an empty canvas while tens of megabytes came down, and moving between a pair of near-identical frames paid that a dozen times. Now, once the opened photograph has landed, the ones around it are fetched into the originals cache while it is being looked at, so the next step is a disk read. A single worker serves the latest wish only, closest first and working outwards — next, previous, next-but-one, previous-but-one… — one file at a time. Each open replaces the wish, so a fast walk never leaves a trail of stale downloads competing with the one being waited on. A process-wide in-flight registry makes a click on a photograph that is still being fetched ahead wait for that transfer and read it from disk, rather than start a second download of the same file. How far each side is a setting under STORAGE — Off, 2, 5, 10 or 20, defaulting to 5 — and it is moot while "keep originals after opening" is off, since a fetch the cache would discard on arrival is transfer for nothing. Nothing is fetched ahead while offline. The transfers show in the activity list while they run and are removed when they end.	2026-09-19 10:36:50 +02:00
dtourolle	c1e0f09be7	Say where the face models were looked for when they are not found ... "The chosen detector is not installed" was the whole of what a user saw, on a machine where the files were three directories away from where the lookup went. The search order was in a doc comment and nowhere a user could read it. Now a missing pair logs the detector file it wanted and every directory it tried, which is what the first Windows install needed and what the next misplaced download will.	2026-09-13 19:20:30 +02:00
dtourolle	ef1154af94	Resolve every base directory in one place, and on Windows ... Five sites each read XDG_*_HOME and fell back to $HOME/.local/… on their own, which is fine on Linux and wrong everywhere else: Windows sets neither variable, so every one of them degraded to a path relative to the working directory — for a Start Menu launch, C:\Windows\System32. The models lookup walked XDG_DATA_DIRS the same way. dr_plat::dirs now holds the rule per platform: XDG on Unix, the known folders on Windows — %APPDATA% for config, which roams, and %LOCALAPPDATA% for data and state, which do not — and the executable's own directory as the system data dir, which is where the installer puts the models. The Android overrides stay where they were; only the fallback behind them moved. Both rule sets are unit-tested on either host, and the Windows one was confirmed by running the application under Wine: its log landed in AppData\Local\darkroom\state and nothing was written anywhere else.	2026-09-12 07:34:05 +02:00
dtourolle	896188a489	Read the sidecars other editors write, and write them back on request ... FR-CAT-13 asked for standard XMP and `core/dr-xmp` answered the file: it has read and written `dc:subject`, `xmp:Rating`, `xmp:Label` and the IPTC core since 5fa4c07, under an ownership rule that leaves everything else in the document untouched. What nothing did was call it. No scan found an `.xmp` beside a raw, no catalog row was filled from one, no judgement wrote one back, and the "external modification detected, reload offered" clause had no mechanism. A library imported from Lightroom came in and could not go back out. The scan collects `.xmp` beside `.drsc` from the listings it was already paying for, and the pull reads each one whose ETag has moved. Both namings resolve: darktable's `IMG_0001.CR3.xmp` names its file exactly, Lightroom's `IMG_0001.xmp` names the stem, and under the stem the JPEG beside a RAW is the same photograph and takes the same document, as DarkRoom's own sidecar already does. Each is reconciled with the catalog winning — keywords union, a rating or label taken only where the catalog has none — because a standard XMP carries nothing that could say whether its value is newer. A genuine disagreement is not resolved; it is written to a table, and the settings page offers the sidecars' values against it. That button is the reload the requirement asks to be offered, and the ETag that moved is the detection it asks for: an `.xmp` edited elsewhere is exactly a file the pull's ordinary incrementality re-reads. Writing goes the other way behind a setting that starts off, since NFR-R4 makes writes beside somebody's originals theirs to switch on. With it on, a judgement or a keyword rewrites the sidecar of whichever spelling exists, or creates Lightroom's. The record is read from the catalog whole at that moment rather than carried from the gesture, so a rating and a keyword a second apart are two writes of one file that agree. And the file's own title, caption, copyright and hierarchy come through the rewrite: the catalog has no columns for them, `rewrite` replaces the owned set wholesale, and a record that said nothing about them would have deleted them from a Lightroom sidecar on every star. The rating's two axes cross the format's one field both ways: a rejection is Adobe's `-1` and stars are stars, and stars arriving on a rejected frame lift the rejection, since the file said it was worth a number. An unrated file says nothing and clears nothing, on the rule the `.drsc` merge keeps. `versions.label` finally has a reader and a writer, with the code table moved out of the query so the two cannot drift.	2026-09-12 01:08:11 +02:00
dtourolle	d3b6127db6	Let a photographer name the state they liked, and go back to it or look at it ... FR-DEV-5 asked for named snapshots of an edit state and FR-DEV-7 for a comparison against a chosen one, and neither existed. The history stack is per sitting and forgotten with it, on purpose — the gap that mattered was an automatically saved mis-drag with no way back, and that was closed first. What was left was the other half: a state the photographer wants to keep *because* it is worth keeping, which is a different thing from a step and is not served by making the steps last longer. A snapshot is an edit state, and an edit state is exactly what a sidecar version stores, so it is stored as one: a `[version]` block carrying `snapshot-of = <uuid>`. The parameters, the masks and their parts, the repairs and the film all arrive through the blocks that already carry them, a merge keys on the uuid as it does for any version, and a build that predates the key reads the block as a named version and keeps it — the right failure. Only the pointer is new. The one reader that has to know is `default_version`, which must never answer with a snapshot: a file whose edit is missing is not a file whose edit is one of its saved moments. The snapshots of an edit are listed by that pointer, oldest first, the same on every device. Writing them back removes what this sitting deleted and puts in what it holds, and leaves standing whatever it never saw — a snapshot the other device took since the photograph was opened here is not this device's to remove by not knowing about it. That is the rule the version merge already keeps, applied one level down, and it is why the save carries the deleted ids rather than replacing the list wholesale as the masks are. Each is re-pointed at the uuid the save settled on, because the default may have been fused onto its canonical identity since the snapshot was taken. Restoring is one history step, so undo takes it back whole, as a paste is. Taking and deleting are not steps: they change nothing about the photograph, and an undo that removed a snapshot would be undoing a decision to remember. Holding the eye beside one renders the snapshot and hands the edit straight back — the same suspension "Before" uses, against a point the photographer chose rather than the file. Two sessions on the same photograph get ids that cannot collide, stamped with the second and a random word, because the merge folds equal ids into one.	2026-09-12 01:08:10 +02:00
dtourolle	4f31123b0c	Let the user choose which SCRFD finds their faces ... faces.md §12.3 measured what the cheapest detector costs: the small faces in every group shot, and a dog embedded a dozen times. Which trade is right depends on the machine doing the sweep — a desktop left overnight and a tablet on a battery want different answers — so the detector is now a per-device setting, Fast / Balanced / Thorough on the settings page beside the indexing button, persisted with the rest of the settings file. A detector is half of a model id. Every face, marker, shard and calibration is keyed on faces.model_id precisely so that a model change is a new id and a re-index rather than a silent change under existing data, and a detector change is a model change: it decides which faces exist and where the landmarks that align them land. So each choice names its own pipeline. 500M keeps the bare "w600k_mbf" every existing library was written under, so an upgrade disturbs nothing; the others are qualified. Choosing one restarts coverage from zero under the new id, the sweep re-detects, confirmed names carry across by box overlap, and the sync shards are keyed by the same id so a peer on another setting neither adopts nor pollutes them. The library controller carries the id into the sync the same way it carries the cache budget, because the sync starts from places that have no settings in reach. All three shape-fixed exports ship — APK, Arch, Flatpak — since a tablet has no other way to obtain the one it was not installed with; the APK grows by twenty megabytes for the choice.	2026-09-11 22:12:53 +02:00
dtourolle	3d6d69ec90	Wrap the face-sweep repair match the way rustfmt wants it ... CI's Desktop job failed at the Format step on 16f3fb4: rustfmt puts the `match faces_without_proxy(...)` on its own line under the `let` and re-indents its arms, and the commit was written without running it. No code changes; only the layout of that one match in library.rs.	2026-09-11 22:00:38 +02:00
dtourolle	16f3fb41a3	Measure the faces already found rather than finding them again ... Every face stored before its quality was kept holds a unit vector, and V14 forgot the run marker of each image holding one so that the next sweep would look again. Looking again meant detecting again: a whole re-detection per image, with every suggestion on it thrown away and the confirmations carried across by box overlap, to recover one number. The sweep now has a measuring pass between the proxy repair and the un-indexed images. It lists every image holding an unmeasured face, fetches the original once, warps each stored face from the landmarks it already has, embeds it, and writes the raw vector and its length over the old row. Ids, boxes and identities are untouched; the marker is re-written fresh so the sync exports the measured vectors. A face whose landmarks no longer make a warp is dropped, as detection would have refused to store it. `faces_unindexed` leaves those images to the measuring pass, so the V14 deletion no longer costs a second detection.	2026-09-11 21:50:12 +02:00
dtourolle	8b3abdb787	Keep each face's quality, and never compare against a poor one ... The embedder's raw output has a length, and the length is a reading of how recognisable the crop was: a blur, an occlusion or a hard profile comes out short. Normalising threw it away. A short vector sits near the middle of the sphere and matches a little of everyone, which is how one bad crop bridges two people in a grouping pass. So the length is kept — the store now holds the raw vector, re-normalised on load, with the length beside it as `faces.quality` — and a face under MIN_GALLERY_QUALITY (14) is a probe: measured against the gallery and placed where it fits, but never what another face is measured against. Two probes are never paired, and a probe is nobody's evidence for a confidence. The People screen shows the number as "Quality 17.3", dimmed below the floor. Faces indexed before this stored unit vectors and have no reading; they are admitted to the gallery, and schema V14 forgets the run marker of every image holding one so the next indexing pass measures them. A peer's unmeasured shard faces are not adopted, or a sync would write that marker back.	2026-09-11 21:50:12 +02:00
dtourolleandClaude Opus 5	1ad35e2b87	Read the library's sidecars, so a cull done elsewhere arrives ... Judgements only ever travelled outward. A rating went to the catalog and to the photograph's sidecar, the sidecar reached the server, and there it stopped: the scan indexes files, `derived_sync` exchanges thumbnails, face shards and collections, `dr_catalog::merge` reconciles everything in a catalog except `versions.rating` and `versions.flag`, and the one sidecar reader that existed ran when a single photograph was opened in develop and handed its answer to the develop graph. `JobKind::ReadSidecar` was declared for exactly this when the job queue was written and was never enqueued or handled anywhere. The grid draws `versions.rating`. So a day of culling on the tablet could not reach the laptop by any path the application had, and the laptop's catalog says so plainly: 23,568 images, one of them judged. `pull_sidecars` closes it, off the back of work the scan already does. `dr_sync::scan` reports the `.drsc` files it meets in listings it was making anyway — no extra request, and a directory whose ETag is unchanged is still pruned before it is listed at all. A new `sidecars` table records the ETag of each one this device has taken in, so the fetch is one GET per sidecar that genuinely changed rather than one per photograph. A library nobody has edited costs nothing. The judgement is taken rather than maximised. The sidecar is the authoritative store and the fuse has already settled any contest between devices on `revision`, so lowering a rating from four to one on the tablet lowers it here — taking the larger would have refused every demotion the photographer ever made, which is most of what a second pass over a shoot is. A zero is the exception: it means *never judged*, not "judged zero", so a sidecar carrying none cannot erase a star this device holds. That is `merge_judgement`'s asymmetry and it carries the same known cost — clearing a rating does not propagate. A sidecar names a stem, so both halves of a RAW-and-JPEG pair are judged: they are one photograph (FR-CAT-11) sharing one document, and judging only one of them would leave the grid disagreeing with itself over which it drew. The `LIKE` that finds them is a filter, not the decision — `sidecar_path` is applied to every candidate, because a folder is entitled to contain a `%` and a rating landing on the wrong frame would be silent and permanent. Failing to read one is not a failure to scan: the ETag goes unrecorded, the ratings already here stay where they are, and the next scan tries again. The count is reported to the status line as well as the log, because a grid that silently gains three hundred stars is indistinguishable from one that has gone wrong — and because while this number was structurally zero there was nothing to tell the photographer their cull had not arrived. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-05 16:14:03 +02:00
dtourolleandClaude Opus 5	601c984894	Fold a photograph's rival default versions back into one ... Picking the newer of two default versions stopped the wrong edit being shown, but it did not close the split: the losing version stayed in the file, and a device holding disjoint work — a crop made here, an exposure change made there — still contributed only one of the two. Worse, the next write made it larger. `amend` looks its version up by uuid, neither of the two was ours, so the miss minted a *third* `default = 1` block and the file grew one rival per device per photograph. `Sidecar::fuse_default_versions` folds them down. The version with the highest `(revision, modified)` is the accumulator and every other default is merged into it as the remote, which is what makes the fold order-independent — `Version::merge` raises its own revision to `max + 1` as it goes, so merging a chain in ascending order stops being ascending after the first step and a third device would be dropped. Contested values resolve to the winner, disjoint keys survive from both sides because the merge is key-wise, and ratings come across under `merge_judgement`, so a device that never judged the frame cannot erase one that did. The result is a function of the file's bytes alone, so two devices that fuse independently reach the same document and converge instead of overwriting each other. Called wherever a sidecar is parsed: - `amend`, with the write's own uuid, so the fold lands on the identity this device is about to use and the lookup below it hits instead of missing. - `spawn_sidecar_fetch`, so opening a photograph shows everything done to it rather than whichever half won. - `drain_one`, because `merge_into` reconciles by uuid and would otherwise publish the split rather than resolve it. - `presets::load_local` and `save_local` — a local sidecar's folder may be synced by something else entirely, and gets the same split. A file with one default under the expected uuid comes back byte-identical, so this costs nothing on the ordinary write and no sidecar is uploaded merely for having been read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-05 16:13:53 +02:00
dtourolleandClaude Opus 5	710fcbc1bd	Format the face work with the workspace's own rustfmt ... Not authored in this session. `cargo fmt --all` reformats every crate, so running it while working on `dr-segment` picked up five files from the recent face and library work that had been committed unformatted. Committed on its own rather than swept into the change that happened to produce it: the diff is pure whitespace, and mixed into a commit that alters an algorithm it would be noise in exactly the place someone is trying to read carefully. `cargo fmt --all -- --check` is a CI gate (tools/ci-local.sh), so this had to land somewhere regardless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 21:29:12 +02:00
dtourolle	6acc98baad	Carry the photograph's header into an export made from develop ... The same file exported from the library grid kept its camera, its lens, its capture date and its rights statement. Exported from the develop button it kept none of them, and `{date}` in a filename template resolved to nothing at all. Two buttons, one photograph, two different files -- and the develop one was the version the photographer had just finished working on. A session now remembers the header it was opened from, and `open_session` takes that header rather than the orientation read out of it, so a photograph cannot be opened for editing without saying which file it came from. `render_open_frame` clones it onto `Source::Rendered`; both arms of `export_one` -- the worker's own decode and the frame handed over already rendered -- turn a header into a `{date}` and a `SourceMetadata` through the same function, so the two paths cannot come to different readings of one file. What of it actually reaches the exported bytes is still decided inside `dr-export` from the settings, which is what keeps the location-stripping option working here rather than giving it a second implementation to disagree with. The alternative was to hang the metadata on `Source::Rendered` alone and keep it beside the session in the interface. That touches less, but it makes the header and the pixels two cells to hold in step across the six places an image is opened, replaced or fails to open, and the failure mode of getting that pairing wrong is not a missing tag: it is one photograph exported under another's byline and coordinates, silently. Kept on the session, the two travel together or not at all. The header is stored decoded rather than transcribed at open time, deliberately. `dr-export` argues that source metadata is a parameter and not a field on `Frame`, because two exports of one frame may legitimately disclose different amounts; by the same reasoning a session may remember where its pixels came from without that being a decision about what to publish, and the allowlist that decides remains the single function in `export.rs`. A file with no header is left with none -- an empty `{date}` and nothing for the encoder to copy -- rather than today's date standing in for a capture time nobody recorded.	2026-08-30 21:28:31 +02:00
dtourolleandClaude Opus 5	d44bffa4a8	Remember where the photographer was ... Opening the application was always a fresh arrival at the beginning of the library, whatever you had been doing when you closed it. What is written down is the view, the scope, the rating filter and the photograph on screen -- the open one in develop, the first visible one in the grid. Not just a scroll position: a position without the filter that produced it names a row of a list that no longer exists. Restoring them has an order for the same reason -- scope, then filter, then position, then the view -- because each step changes what an ordinal *means*. Addressed by remote path and collection UUID, never by an ordinal or a row id. `images.id` and `collections.id` are local to one catalog, and a grid ordinal is local to one ordering; a record naming either would land somewhere arbitrary on a second device and after any filter change on this one. Where the ordinal is needed, `library::ordinal_of_path` computes it through the grid's own `ORDER BY`, taken verbatim by a window function rather than spelled a second time as an inequality -- which is the mistake `grid_order_for` already warns about, and which a manually ordered collection would make unreadable. Every failure degrades rather than reports. A collection this device has not merged leaves the scope at the whole library; a photograph that has since been deleted falls back to when it was taken, which puts the grid in the right week; a torn file yields no place and the library opens at the top. Reopening develop is the one thing that requires an exact match, because a canvas on a path that no longer resolves is a filename over an empty frame. The record lives in `dr-types` beside `Settings` and the store lives here beside `SettingsStore`, for the reason `dr-types`' manifest gives: a JSON serialiser in `core/` would be paid for by every crate there. Two files and two lifetimes, though -- resetting preferences must not forget where you were. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 20:39:50 +02:00
dtourolleandClaude Opus 5	4af3b93dfa	Index faces from the native render, not from a preview of it ... Implements the FR-CULL-8 written two commits ago. The sweep fetched the JPEG preview embedded in each RAW and used that one buffer for both detection and the crop; it now fetches the original, renders it through the same path export uses, reduces that for the detector, and warps the crop back out of the native frame. Three pieces, and each exists for a reason worth stating. dr_face::Pixels lets the warp sample 8-bit RGBA directly. A 24 MP native frame is 96 MB as RGBA and 288 MB converted to the f32 RGB align.rs was written against, and the warp reads about forty thousand pixels out of it. Converting the whole frame to sample 0.2% of it is NFR-RES-2's budget spent on a copy, per image, for a whole library. The variant costs one branch per sample and a test asserts both layouts produce identical crops. The detector gets a box-filtered reduction to 1600px, not the native frame and not a point-sampled one. Averaging rather than sampling because the detector's job is finding small faces and decimation is precisely the operation that removes them: at 4x, fifteen of every sixteen pixels are discarded and a 40px face survives or not depending on where it falls relative to the sample grid. 1600 rather than 640 leaves the letterbox a mild 2.5x rather than a 9x, and bounds the f32 buffer at 20 MB. Landmarks come back in the reduction's coordinates and are scaled to native in one place before any crop pixel is read. This is the failure mode that would not announce itself -- unscaled landmarks put every crop near the top-left corner, which yields faces of something else, cleanly embedded and confidently clustered. The sweep fetches SWEEP_LANES-wide and renders sequentially. Not a placeholder for a parallel version: there is one GPU, so concurrent renders queue on it regardless, and each materialises a native frame. Overlapping them would multiply the one allocation that threatens the memory budget while buying parallelism that does not exist. The chunk drops from 96 to 6 for the same reason -- 96 held 8 MB previews, this holds whole RAWs. The stored edit is deliberately not applied, which is where this departs from export::render_from_library. Face geometry is normalised to the frame, so indexing a cropped render would record boxes against a frame that changes whenever the user changes their mind, and every stored box would quietly become wrong. Orientation is applied: that is a fact about the file rather than an edit. examples/face_native.rs renders one file and indexes it both ways, so the claim behind all of this can be checked against photographs rather than re-read out of the catalog it came from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 19:41:30 +02:00
dtourolleandClaude Opus 5	e7b526c550	Specify face indexing at native resolution, and say what the proxy cost ... FR-CULL-8 said detection runs against the thumbnail or proxy tier and never a full decode, and faces.md §5 said the aligned crop is sampled from that same proxy. Both are wrong in the same place: they treat detection and cropping as one resolution problem when they are two, with opposite answers. Detection does not care. §4.1 fixes the graph's input at 640x640 and letterboxes whatever arrives, so a face filling 2% of the frame reaches the model at 12px whether the buffer handed over is 1024px or 6000px. Every pixel above the detector's own input is discarded before inference. The crop cares about nothing else. §5's warp produces the fixed 112x112 ArcFace sees, so source resolution converts directly into whether those 112 pixels were photographed or interpolated. Reading crop_px across the 18,671 faces the proxy-tier implementation stored: 47.3% were upsampled to reach the embedder, 314 of them by more than 2x, the smallest from 34 source pixels. An upsampled crop does not fail loudly -- it yields a confident embedding of detail that was never there, and the damage appears three stages later as clusters that will not separate. So FR-CULL-8 now specifies four stages with the resolutions named separately: render native through FR-EXP-9's pipeline, downscale for the detector, map boxes and landmarks back to native, crop and align from the native render. The affordability the old rule bought is met instead by when the pass runs -- background, preempted, resumable -- and the requirement says plainly what it now costs on a remote library: the original rather than FR-NC-3's byte range, 412 GB across the reference library's 19,107 images, so a whole-library pass is a transfer under FR-NC-6 rather than something that may start on its own. MIN_CROP_EDGE replaces the MIN_DETECT_EDGE this branch briefly had. Same number, guarding the quantity that turned out to matter. faces.md §7b records both measurements, and marks the second as unexplained rather than dressing it as a finding. Grouped by the buffer detection ran against, faces per image was 0.078 at 1024 or below and 1.82 at 2048 or better, controlled for file type and size. That gap is real and reproducible and I cannot account for it, because the letterbox above says detector input should not matter. M4 is where it gets settled. The crop measurement does not depend on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 19:41:20 +02:00
dtourolleandClaude Opus 5	144d2e4e84	Revert the detection floor: it guards the wrong resolution ... Reverts 53f7cdf and e92d22d. The floor those added sat on Detector::detect, refusing any buffer under 1025px on the reasoning that a small buffer finds no faces. That reasoning does not survive §4.1: the detector letterboxes every input to 640x640, so a face occupying 2% of the frame presents at 12px to the model whether it is handed a 1024px buffer or a 6000px one. Detector input is precisely the quantity that does not matter. Worse than merely useless, it blocks the design FR-CULL-8 now specifies, where the detector is deliberately fed a downscale and the crop is taken from the native render. A guard on detect() rejects exactly that call. What the measurement actually supports is a floor on the *crop* source, which is where resolution converts into embedding quality, and which faces.crop_px already records: 47% of the reference library's faces were upsampled to reach 112x112. That floor is a separate change against the native-resolution path and does not belong on the detector. The 23x faces-per-image gap by source_edge that motivated the original commit is kept in faces.md §7b, restated as the unexplained observation it is rather than the causal claim it was written as. V12 stands: those runs cropped at 1024 whatever detection did, and that is reason enough to look at them again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 19:41:12 +02:00
dtourolleandClaude Opus 5	6783d0c723	Settle the images whose best preview is under the floor ... The floor introduces a state the sweep had no arm for. An image whose largest embedded preview is genuinely smaller than 1025px now comes back from index_preview as an error, lands in the generic Err arm, and is counted as failed -- which means no face_index row, which means it is still outstanding, which means the next sweep fetches exactly the same bytes and refuses them again. For ever, on every run, at one range request each. The previous behaviour was wrong but at least terminated; this would not. So ProxyTooSmall gets its own arm, and it records a marker at the true edge rather than nothing. That is the difference between "we looked and found nothing" -- which would be a lie, since nothing was looked at -- and "this was examined at 900px, which is the best this file has". The first is unrecoverable; the second is a fact source_edge was added to carry, and a later floor or a bigger proxy can select on it deliberately the way V12 just did. Counted separately from failures all the way up, because they are not failures and reading them as such would misdescribe a library of small scans as a broken network. The summary line says how many and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 19:41:04 +02:00
dtourolleandClaude Opus 5	1c5849ebe8	Unpack the bundled models on a worker, not on the way to the first frame ... Launching v0.10.0 on the tablet produces an ANR: "Waited 5000ms for MotionEvent", 5,827 ms on one input sequence, over a window that has never painted. The app recovers and sits at 0% afterwards, so it is a startup cost rather than a hang. The structural fact behind it is that `android_main` runs with the activity's input channel unserviced. Nothing drains it until Slint reaches `poll_events`, and Slint does not reach `poll_events` until `dr_ui::run` calls `window.run()` on its last line. Every millisecond before that is a millisecond the input dispatcher waits on, so five thousand of them is an ANR whatever the work happens to be. The largest single piece of that work was here. `install_bundled_models` copies 41 MB on the first launch after an install — 24.9 MB of scene model, 13.6 MB of embedder, 2.5 MB of detector — each read whole out of the APK into a `Vec` and written to `/data`, in a loop, on that thread. v0.10.0 is the release that added the scene model, which is 60% of that total, and it is the release the ANR appeared in. The 8,010 minor faults in the report are about what 41 MB of freshly touched pages costs. So it moves to a detached thread and the function returns as soon as the thread is running. Nothing on the launch path wanted the result: the only two things that read these files are the People screen and the scene tab, both of which are reached by hand, minutes later, from workers of their own. What that costs is a window in which a model looks absent. `library::face_models` and `library::scene_model` decide availability on `is_file()`, so during the copy both report their feature unavailable — which is the same answer they give a build shipping no weights at all, the ordinary case both were written around. Briefly pessimistic rather than wrong, and the temporary-name-then-rename that was already there is what keeps it from being worse than that: a lookup never sees a half-written file, only an absent one. Both call sites now say so. A completion line reports the bytes copied and the milliseconds taken, including when it is zero, so the second launch after an install can be told from the first in a log rather than by inference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 18:41:34 +02:00
dtourolleandClaude Opus 5	7312aceded	Get the scene model onto the devices that need it ... The decoder can load from a path; nothing yet put a file at one. Four packaging routes, and one lookup that finds the result. ## Not `include_bytes!`, unlike the instance model The instance model is 11 MB and compiled in, which was the right call for it: Android hands the app no filesystem path (ARCH §6.9) and 11 MB is tolerable. The scene model is 24 MB, and 35 MB of constants in the binary is paid by every install whether or not the tab is ever opened. So it follows `models/face/` instead — carried as an APK asset, unpacked once at first launch into the shared directory a desktop install already uses, after which every lookup finds it where it finds a desktop user's. Assets are stored rather than deflated in the APK, so unpacking is a copy rather than an inflate. `embedded-scene-model` exists for the desktop build with nowhere else to read from, and for tests wanting the real graph. Off by default, which is the asymmetry with `embedded-model` and the reason for a separate feature. ## Three files, all or none `scene_model` insists on the graph, its vocabulary and the category descriptor together, for the reason `face_models` insists on its pair: a graph alone decodes to 150 anonymous channels. Reporting the set missing beats starting and failing at the first inference. ## The two model sets are not the same kind of thing `install_bundled_models` now carries both, and the distinction is worth keeping in view. Face weights are absent from the repository *by design* — the InsightFace grant is research-only (docs/faces.md §2) — so a build carrying none is ordinary. The scene model is committed, so a build carrying none means a checkout without `git lfs pull`. Neither is fatal. A photo editor that refuses to start over a missing grading feature is worse than one that starts without it, so both report themselves unavailable exactly as face indexing already did. The LFS-pointer guards apply to the `.onnx` only. The vocabulary and the descriptor are legitimately a few kilobytes, and a size check that fails on them would be a guard against the wrong thing. `scene_model` is exported ahead of the tab that will consume it so the packaging added here has something to be verified against — assets written where no lookup looks would be a silent mistake for as long as the tab took to arrive. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-30 10:48:54 +02:00
dtourolleandClaude Opus 5	7418b040da	Refuse a scan whose root has gone, instead of reporting it empty ... FR-PLAT-AND-2, and a silent failure on both platforms. `dr_sync::scan` stepped over a NotFound or PermissionDenied the way it does for a child that vanished mid-walk -- correct for a child, wrong for the root, where it ended the walk, returned Ok with nothing in it, and reported a successful scan of a library that was no longer there. A lost root is now its own error. The images under it are marked Availability::Offline per FR-CAT-9 and no catalog row is deleted; `library::persist` clears the mark per file as each one is listed again, so a root that comes back needs no repair step. Partly satisfied rather than closed, and the gap is worth stating. The recovery half is real and reachable on Android today, because `map_status` turns Nextcloud's 403 and 404 into it and Nextcloud is how a phone actually gets a library in this build. The causes the requirement names -- revocation, reinstall, a removed card -- are properties of a persisted tree permission, and there is none: SAF does not exist here, `SourceRef::Document` is constructed only in test modules, and `LocalStorage` rejects the variant outright. When SAF lands it becomes a third producer of this error and nothing above it changes, which is why the discovery belongs in the connector. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-29 23:20:57 +02:00
dtourolleandClaude Opus 5	55cb9b5b30	Keep the test scene's own arithmetic from overflowing a u64 ... The first compile this branch ever had. `cargo fmt` reflowed four files and clippy passed at -D warnings untouched, but one test panicked: `the_signature_does_not_change_with_scale`, on "attempt to multiply with overflow". It is the fixture, not the feature. `scene()`'s little LCG multiplied the block's y by the golden-ratio constant with a plain `*` while the term beside it already used `wrapping_mul`, so any scene taller than about 104 pixels overflowed in debug. Only the scale test builds one that large, which is why 345 of 346 passed around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-29 23:20:26 +02:00
dtourolleandClaude Opus 5	e5db849f04	Mark a burst in the grid, and let it be folded away ... The counterpart to the grouping: where the signatures come from, and how a group reaches a cell. Signatures are computed from the 256px thumbnails dr-thumbs already holds -- vastly more resolution than a 9x8 reduction can use -- so a library that has been browsed, or that has synced somebody else's shards, has already paid for them and no RAW is decoded for this. The consequence is stated rather than hidden: an image with no thumbnail gets no signature and never joins a burst. That is self-correcting, and it is why the pass runs when the thumbnail sweep finishes rather than on a timer. Nothing happens at import and nothing happens at query time. The mark is drawn as a child of the cell's TouchArea, for the same reason the star strip is: a click on it must not also reach `cell-clicked` and throw the user into develop, and children are hit-tested before the element they sit in. It is never hidden on hover the way the stars are -- a collapsed burst stands in for frames that are not on screen, and something has to say so whether or not a pointer is nearby. Folding changes what the grid's *query* returns rather than what its cells draw, because the grid is a window over an ordered query and the frames a fold hides are mostly not loaded. So the predicate joins VISIBLE in every query that lists or counts cells -- the window, the header's count, the run a shift-click resolves, and the ordinal a scrub lands on -- under the discipline VISIBLE's own comment sets out: present in four places of five is worse than absent, because the counts disagree with the cells and neither looks wrong on its own. There is a test for exactly that. `the_window_read_walks_the_ordering_index` now includes the burst clause. It asserts on the query plan while holding its own copy of the query, so left alone it would have gone on reporting green against a query the grid no longer runs. If the clause costs `images_grid_order` and puts the sort back, that fails here rather than becoming jitter someone measures in six months. The pass keeps its own drain timer in a thread-local instead of taking fields on the library controller, so everything the feature needs to run lives in one file and the screen that starts it holds nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-29 23:20:26 +02:00
dtourolleandClaude Opus 5	f5edd49b6b	Let a manual collection be put in the order it is meant to be seen in ... `collection_members.position` and `Sort::CollectionPosition` have been in the catalog since collections were, and nothing above dr-catalog has ever written or read either: `collections::set_order` had no callers, and the grid ordered everything by capture time whatever it was scoped to — dr-ui does not construct a `Query` at all, it has its own `GRID_ORDER` constant. So a manual collection was a set with an order nobody could see or change. Three pieces, because it could not be fewer: `grid_order_for` decides the ordering from the scope, and both readers take it from there. That is the load-bearing part. An ordinal only names a photograph relative to an ordering, so the window read and the span read have to agree — a shift-click resolved through a different ORDER BY than the cells were drawn with selects a different run than the one on screen, and the user finds out when the export runs. `read_ids_span` already stated that invariant about `GRID_ORDER`; this widens it to an ordering that depends on the scope. Only a single manual collection has one. A set draws its descendants' images too, and two children's positions are unrelated integers that interleave arbitrarily; a smart collection has no member rows to carry a position at all. Both fall back to capture time and refuse the drop rather than pretending. The drop is on the cell, on whichever half of it the finger landed — the trailing edge is the only way to name the last place in a collection, since there is no cell beyond the last one to drop in front of. `reordered` is pure and the membership is rewritten whole. `set_order` sets the positions it is given and leaves the rest, so a partial write would interleave the moved run with rows nobody touched; and it is read unfiltered, so what the filter is hiding keeps its place relative to what the user can see. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-29 23:18:06 +02:00
dtourolle	5768100816	Borrow the library to index it, and give it back ... The passes that need every photograph's bytes — thumbnails, face indexing — now borrow each one and release it at the end. On a placeholder library that is the difference between peak disk being the working set and being the whole library. Including on cancellation, which was nearly missed: the face sweep returns mid-loop when the user presses Stop, and without releasing there the disk is spent and nothing is delivered for it. `materialise` now answers whether *it* fetched the content. The pool used to work that out by listing a file's parent directory — one listing per file across a library — when the backend already had to `stat` it to decide whether to ask. One syscall instead of a directory walk, and it removes the bug class the tests found earlier: a file at the library root has no `parent()`, so every one of them read as already-downloaded. **Pinning is the retention control**, and it drives the model the catalog already had rather than a second one. `tier_desired` is what the user asked to keep hydrated, `pending_pins` is the resumable work list, and a pinned collection is never dehydrated for the same reason it was never evicted. It was in fact *broken* here before: `get` on a stub failed, and the pin worker logged "one unreadable file must not abandon the whole pin" and silently did nothing. Pinned originals on such a library are recorded with `path = NULL` (`Cache::record_in_place`) rather than copied under `originals/`. Two reasons, and the second is the important one. A copy would hold every pinned photograph twice, with the budget able to evict the half that was not costing the disk. And `release` deletes the file a row names — so a row that names none cannot delete anything, which puts the one catastrophic operation out of reach by construction rather than by remembering not to call it. Deleting a materialised file inside a synced tree removes the photograph from the server and every other device. Handing disk back is `spawn_dehydrate`, which asks the client. Two gaps written down rather than papered over (docs/storage.md §7): a hydrating pass cannot yet quote its cost, because a stub reports no size; and the two sweeps hold separate pools, so a library indexed for both fetches twice.	2026-08-29 09:57:53 +02:00
dtourolle	c102ba9df2	Treat a placeholder as the photograph, not as a one-byte file ... The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none.	2026-08-29 09:57:52 +02:00
dtourolle	f12aece07e	Make storage pluggable, and prove it with a folder backend ... `RemoteBackend` existed from the first release and bought nothing it was designed for. Seven files in `dr-ui` constructed a `NextcloudBackend` directly, an account *was* a server URL beside a DAV user id, the local cache directory was named after a hostname, and the launch screen knew that signing in meant a browser handshake. The trait was real; the seam was documentation. A trait over operations is only a quarter of it. Pluggable storage needs four things, and this adds the other three: - **Capabilities** — already there, and the reason the engine can drive two backends at the speed each actually runs at. - **Configuration** — `dr_sync::Account`: where a library lives, in whatever form its connector addresses, with no server in it. Loads every existing config unchanged (`backend` defaults to `nextcloud`, `endpoint` is stored under its historical `server` key), and `Account::namespace()` reproduces the old catalog directory byte for byte, because changing it would abandon a catalog, its thumbnail shards, and the sidecars holding unsynced offline work. - **Registration** — `BackendProvider` and `BackendRegistry`. `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that names a connector. `Connection` (an account plus an optional `Secret`) replaces the credentials-and-user-id pair that was threaded through fifteen signatures in an order that could be swapped. `Secret`'s inner string is reachable only through `expose()` and its `Debug` prints `Secret(***)`, so the indirect leak — a `{:?}` on anything holding one — no longer compiles into a leak. Nextcloud is unchanged and keeps every peculiarity: propagating ETags, chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused PUT, the 423 retry classification, Login Flow v2. Those are what the capability model exists to serve, not something to hide. `dr-sync-folder` is the second connector: a local disk, a network mount, an external drive, or a folder a Nextcloud client already syncs. No account, no credential — the route that works where no secrets daemon does. It declares `LocalEtags` rather than claiming propagation a POSIX directory cannot provide, which costs nothing because 50k `stat` calls are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode survives a rename but differs between devices and is reused after a delete, so two machines would disagree about which photograph a thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the wrong one is a bug. docs/storage.md is the contract — the traits, the four steps to add a backend, and what each connector declares. ARCH §8.0 and §8.4a, and FR-NC-13, say why.	2026-08-29 09:57:52 +02:00
dtourolleandClaude Opus 5	af5a13b3f7	Narrow the grid to several people at once, either way ... "Show photos" could only ever mean one person. The two questions a photographer actually asks are "every picture of Anna or Bob" and "the pictures they are both in", and the second is not reachable by any sequence of single-person filters — no amount of switching between one person and another finds the frame they share. So the filter holds a *set* of people and a mode. `RatingFilter` was already the right home, as its own doc says: every query path threads it, so the count in the header and the cells in the grid are narrowed by the same thing, and this composes with stars, flags and the date range for free. The union is `EXISTS ... person_id IN (...)`. The intersection counts **distinct** people per image and compares against the size of the selection — one subquery rather than one per person, and it does not grow the statement with the selection. `DISTINCT` is what makes it correct: three faces of Anna in one frame must not satisfy a filter asking for Anna and Bob, and there is a test that says so. Any rather than All is the default. With one person the modes are the same filter, and adding a second to a union can only ever show more — so a user who has not noticed the toggle never ends up staring at an empty grid wondering what they broke. The toggle only appears at two people, because a control that demonstrably does nothing is a control that teaches the user to ignore it. Building the set needs no picker of its own: the Identity screen gains "And also…" beside "Show photos", offered only once the grid is already narrowed to somebody. Each person is a chip on the filter bar and each chip removes just that person, so a selection of three can be taken apart one at a time rather than only cleared wholesale. `RatingFilter` stops being `Copy`, since it now holds a `Vec`. Every query path already took it by reference; the casualties were two struct updates and one `Cell` that becomes a `RefCell`. 484 dr-ui tests pass, including the union, the intersection, that one person reads the same in both modes, and the repeated-faces trap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 22:17:59 +02:00
dtourolleandClaude Opus 5	d6380fecc8	Make the People screen a place work can be done ... Five faults, all on one screen, and the Slint and Rust halves of each have to land together. **Regroup froze the window.** It ran inside the Slint callback, on the UI thread. It is much faster now, but fast is not bounded — the work grows with the library, and the one thing that must not grow with the library is how long the window stops answering. It runs on a worker thread with an mpsc channel and a 250ms poll, like every other long pass in this module, and the button says what it is doing instead of the window going quiet. Cancellation is dropping the receiver. Reclustering also prunes the empty groups the previous pass left, so pressing the button twice no longer fills the rail with "Unnamed (0 faces)". **The faces were a single row running off the screen.** The comment on the layout claimed to be a wrapping row; Slint has no flow layout and a HorizontalLayout does not wrap, so a person with forty faces was a person whose faces could not be reviewed past the fifth. It is now laid out the way the library grid lays out thumbnails, with the same arithmetic: choose how many columns of roughly the requested size fit, then divide the width between them so the cells fill the row exactly and nothing overhangs. **The header did not fit a phone.** A 240px name field beside five buttons is wider than an Android screen — and worse than not fitting, a layout cannot be narrower than its children's minimums, so the row reported that oversized minimum upwards and inflated the whole screen. The faces grid is its sibling, so it would have been measured against a width that was never on the display. The header is now two rows, the actions sit in a Flickable that scrolls rather than overflowing, and the rail narrows to 132px on the compact class. **Strangers crowded out the people who matter.** Most clusters in a real library are passers-by and other people's guests. "Not interested" sets a group aside; the rail hides it and says how many are hidden, with one button to bring them back. Reversible, and never a deletion — see the catalog commit for why. **A face was a dead end.** Identifying someone and then having no way to see their photographs is a filing cabinet with no drawer handles. "Show photos" narrows the library grid to that person and leaves a chip on the filter bar saying so, which is also how it is cleared. It is a term on `RatingFilter` rather than a grid scope of its own, exactly as that struct's own doc says new narrowing terms should be — so the count and the cells are narrowed by the same thing, and it composes with the others for free. Suggested faces count, not only confirmed ones, or a freshly grouped person would show an empty grid. Crops are read from where they are now stored, falling back to cutting one out of the proxy for faces indexed before that existed. 480 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 21:21:31 +02:00
dtourolleandClaude Opus 5	7dfbe3184a	Merge master into the face branch ... Master moved 24 commits while this branch was building the face sweep, and one of them changed how the interface reaches a server: `remote.rs` is now the only file that names a connector, and everything else takes `&dyn RemoteBackend`. The face sweep was written before that landed and built a `NextcloudBackend` directly. Git merged the text without complaint and the result did not compile, which is the useful kind of conflict — it goes through `remote::connect` now, like every other pass. Two documentation conflicts, both resolved toward master. `code-health.md` was an add/add: master's copy carries the CH resolution for the backend seam and a better provenance note, so it wins outright, with its measured figures re-taken against the merged tree rather than either side's — `run()` is 1,855 lines now, 2,042 tests, 793 traceability tags. `traceability.md` is generated, so it was regenerated rather than hand-merged. The seam grades in code-health.md are unchanged by this merge. That is worth noticing rather than glossing: the face work went into the seams that already existed — `run()`, `library.rs`, `AppWindow` — which is exactly the pressure CH-1 describes rather than evidence against it. All four CI jobs pass: desktop (fmt, clippy, test, build), layering, traceability, and the Android cross-build. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 20:27:45 +02:00
dtourolle	28c046130c	Merge branch 'master' into android-bundled-face-models	2026-08-27 20:22:42 +02:00
dtourolleandClaude Opus 5	371038614f	Fetch the repairs first, or they never happen ... The repair added in 051447b was appended to the work list: wanted.extend(repair); Behind every un-indexed image in the library. On this library that is position 23,000-odd — roughly two hours of fetching before the first repair is reached, which inside one session is indistinguishable from the feature not existing. The log said it had found them and the screen stayed empty, which is the worst combination of the two. They go first now. There are a few hundred of them against tens of thousands of un-indexed images, and they are precisely the images the People screen is failing to draw at this moment — so the ordering costs nothing and is the difference between the grid filling in within a minute and not filling in at all. A failure to build the un-indexed half no longer discards the repairs either: the pass runs with whatever it has rather than returning empty. Verified on the live catalog: 455 images hold faces, 251 already have a proxy from the fixed sweep, and the remaining 204 are what now sits at the front of the queue. The stored proxies check out — a valid JPEG at the large class, 49 KB — so the storing half of 051447b was already working. 471 tests pass, including one that the orphan is ordered ahead of the library. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 19:58:32 +02:00
dtourolleandClaude Opus 5	242374fd0f	Let the interface hold a backend without knowing whose it is ... `dr-sync` defines `RemoteBackend` and a capability model the engine adapts to, so a second backend can be added without touching the code that uses one. That boundary was documentation. Seven files in `dr-ui` constructed a `NextcloudBackend` directly, ten functions took one by concrete type, and exactly two call sites in the tree — both inside `dr-sync` itself — ever held the trait object. A WebDAV or local-folder backend would have had a well-written trait to implement and nowhere to go afterwards. The change is smaller than the finding suggests, because the trait was already right. Every method the UI has ever called on a backend — `get`, `put`, `list`, `delete`, `create_dir`, `move_to` — was already on it, so nothing had to be added and no behaviour moved. Ten signatures widened to `&dyn RemoteBackend`, sixteen constructions became `remote::connect`, and `remote.rs` is now the only file in the interface that names a connector. `connect` returns `Result<Box<dyn RemoteBackend>, RemoteError>`. The error type is `dr-sync`'s rather than the connector's, which is why every call site kept its shape — the `match`, the `let Ok(..) else`, and `.map_err(ScanFailure::local)?` all still read as they did. One wrinkle worth recording: `&Box<dyn Trait>` does not reach `&dyn Trait` on its own. The compiler reaches for unsizing, which wants `Box<dyn RemoteBackend>: RemoteBackend`, and reports a confusing missing impl rather than suggesting a deref. Twelve call sites therefore say `&*backend`, and two say `let backend: &dyn RemoteBackend = &*backend` where a borrow is shared across lanes. What this does *not* do is abstract credentials. `AppCredentials` is an app password from Login Flow v2 — a Nextcloud protocol, not a general notion of authenticating to a remote — and seven files still name it. An OAuth token, a bucket key pair and an app password have no useful common shape, so deciding what an account is across backends before a second one exists would be a confident guess. code-health.md CH-2 now records that as the remaining half, and it should wait for the backend that forces it. Verified: fmt clean, clippy clean at -D warnings, 2041 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 19:51:23 +02:00
dtourolleandClaude Opus 5	051447bda6	Keep the proxy a found face will be cropped from ... Every cell on the People screen read "no preview" while the sweep was happily reporting 893 faces found. Both were true. Faces were being detected and stored correctly; there was simply nothing left to draw them from. A face is stored normalised and drawn by cropping the proxy it was found on — `identity::decode_proxy` reads `FACE_TIER` out of the thumbnail store. The fetching sweep fetched a preview, detected on it, wrote the faces and dropped the pixels. So every face it found pointed at a proxy that had never been stored, and the grid had nothing to cut. Worse, that state could not repair itself: the image has its `face_index` row, so it is not outstanding work and no later pass would look at it again. Two fixes, and the first is nearly free. The sweep now keeps the proxy — it has already paid the round trip and the decode, and the crop needs those same pixels the moment the user opens the person. Kept **only where a face was found**: two thirds of a personal library is landscapes and documents (docs/faces.md §7a), those will never be cropped, and skipping them keeps this well clear of the whole-library cost `SWEEP_THUMB_SIZE` deliberately avoids. The downscale to the large class happens after detection, which is the last use of the full buffer. Second, the sweep now picks up images that have faces with no proxy, whatever put them in that state — this bug, or an ordinary cache eviction, which would have produced exactly the same empty grid. Re-running detection repairs it and loses nothing: `record_detections` replaces rather than appends and carries the user's confirmations across the replacement. That makes the screen self-healing rather than dependent on nobody ever evicting a thumbnail. The proxy is stored *before* the detections. A kill between the two then leaves a proxy with no faces — which the next pass simply re-indexes — rather than faces with no proxy, which is the state that cannot recover. Note for the library already part way through a sweep: the 986 images indexed before this will be picked up by the repair route on the next run. 470 tests pass, including one that a face whose proxy is gone becomes work again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 19:35:12 +02:00
dtourolleandClaude Opus 5	c8c6368542	Index the whole library by fetching what it has not seen ... "Index faces in the whole library" could not. Its work list was intersected with the thumbnail store at `ThumbSize::Large`, and nothing fills that class for a whole library — `SWEEP_THUMB_SIZE` is deliberately `Grid`, because the large class is ~860 MB of shards against ~200 MB and every syncing device pays it. So the only images with a large proxy were the ones the user had personally zoomed into or opened in the loupe. On this library that was 220 of 23,529. The comment defending it misread the requirement: // Requesting one here would put face indexing on the network path, // which FR-CULL-8 explicitly keeps it off. FR-CULL-8 keeps indexing off the **full decode**, not the network, and then says the opposite in the same paragraph: "where no proxy exists, the job requests one at background priority rather than decoding inline". faces.md §7 repeats it. Neither was implemented. So the pass fetches. Same two-stage route the thumbnail sweep uses — the header, then the located preview's own byte range (FR-NC-3) — so no whole file is pulled and no RAW is decoded, because an embedded preview is a JPEG. The work list is now every visible image with no `face_index` row for the model: 23,308 here, against nearly none before. **It indexes at the resolution the preview actually has**, not the 1024 the old tier would have given. `locate_preview` already picks the largest embedded preview, and the thumbnail sweep was decoding it and throwing the detail away at `downscale_to(256)`. A face 2% across the frame is 5 px on a grid thumbnail and ~61 px at the cap here — and 112 is what the embedder samples, so this is the difference between an upsampled crop and a real one. `crop_px` records which, per face, as §7 intended. Capped at 3072 rather than truly full: `index_proxy` needs packed `f32` RGB at 12 bytes a pixel, so a 24 MP frame is ~288 MB and the fetch lanes hold one each. The constant is named and sits next to the reason. Orientation is applied **before** detection, not after downscaling. That costs a permutation of a larger buffer — ~15 ms against a ~150 ms decode — and buys the entire class of bug this codebase keeps having: detection then runs on the photograph rather than the sensor, so every box and landmark is already in the space the catalog stores and the overlay draws, with no second mapping to get backwards. One detector and one embedder serve every lane. The lanes are concurrent futures on a single thread, not threads, and inference contains no await, so a `RefCell` borrow never overlaps another — a pair per lane would duplicate ~16 MB of weights for no parallelism. Images with no face in them are recorded too. `face_index` records that detection *ran*, and zero is its most valuable value: without the row every landscape and document scan returns on every pass, for ever, and in a personal library that is most of it (§7a). The old store-only pass survives as `spawn_store_face_sweep` for `examples/face_index.rs`, which indexes a local store with no network. The settings copy no longer claims indexing reads "the photographs already thumbnailed above", and the audit line says "to fetch" rather than "awaiting a proxy", which had become a blocker that no longer blocks. Verified against the real catalog: the new work list returns 23,308 where the old one returned effectively nothing. 469 tests pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 18:51:53 +02:00
dtourolleandClaude Opus 5	2d95807542	Package the models on every platform, not just the phone ... The Android bundling landed the weights under that platform's asset directory, which was the wrong home the moment a second packager wanted them. `makepkg -si` produced a desktop install with no model at all — the same "no face model is installed" the phone used to show, for the same reason: nothing put the files anywhere the app looks. So `models/face/` at the root is the one copy, and both packagers read it: assemble-apk.sh bundles it as APK assets, and the PKGBUILD installs it to /usr/share/darkroom/models. Both refuse an LFS pointer rather than shipping a 130-byte file that fails inside the graph loader on a user's machine. `face_models` now searches three places, most specific first: the account's own directory, the shared user directory, then $XDG_DATA_DIRS. So a packaged pair is found automatically and a pair the user placed by hand still outranks it — which is what keeps a deliberate choice of weights from being overridden by an upgrade. $XDG_DATA_DIRS rather than a hard-coded /usr/share: that is the variable a distribution, a prefix install or a Nix-style store already sets to say where its data went, and its documented default is exactly the two paths that would otherwise have been hard-coded. Empty on Android, which has no such directories — there the APK's copy is unpacked into the shared user directory instead, because an asset inside a package is not a path anything can read from. Verified: the APK still carries both models at assets/models/, the PKGBUILD parses and installs from the new path, 467 tests pass. Includes the pkgver 0.6.0 → 0.7.0 bump that was already sitting uncommitted in the working tree. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 18:05:23 +02:00
dtourolleandClaude Opus 5	eaafacc3fb	Give the phone the model it had no way to obtain ... Face indexing was compiled into the APK all along — dr-ui takes dr-face with `inference` on every target, so SCRFD, alignment, MBF, calibration and clustering were all in there. What was missing was the weights, and on Android there was no way to supply them. Route C (docs/faces.md §2.2) says the user obtains the model and the app loads it. On a desktop that is a real gesture: drop two files in ~/.local/share/darkroom/models/ and indexing starts working. On Android it is not a gesture at all. `internal_data_path` is app-private, `run-as` needs a debuggable build, and the in-app fetch route C specifies was never built — so the settings page reported "no face model is installed" on every launch with nothing behind the message. Not "off until you supply weights"; off. So the shape-fixed pair goes into LFS under the APK's assets, assemble-apk.sh copies it into the package, and `android_main` unpacks it to the shared models directory before anything asks whether a model is present. Three things that are not incidental: The models directory is now shared across accounts rather than per-account. Weights are identified by `faces.model_id`, not by who is signed in, so two accounts had no reason to hold two copies — and the unpack runs before any session exists to key a per-account path off. `face_models` still prefers a per-account directory when one is populated, so anyone mid-migration keeps the ability to pin one library to its own pair. The unpack writes under a temporary name and renames. `face_models` decides availability on `is_file()` alone, so a copy truncated by the process being killed would leave a file that passes that test and fails inside tract — reported to the user as a broken model rather than a missing one. assemble-apk.sh refuses an LFS pointer. At ~130 bytes it looks exactly like a model to `cp`, and unchecked it reaches the device and fails in the graph loader instead of telling someone to run `git lfs pull` — the same guard dr-segment's build script applies to yolo26n-seg.onnx. The licensing half is unchanged and recorded in §2.2a: the InsightFace grant is research-only, this is a private repository and a self-installed build, and these files come back out before anything is published. The weights are still not a cargo build input — dr-face has no `models/` directory and no `embedded-model` feature, and nothing in the build reads them. The APK assembly step copies two files and is the only thing in the tree that knows they exist. Verified on device: both models unpack on first launch (2524817 and 13616095 bytes) and the APK carries them at assets/models/. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-27 13:18:07 +02:00
dtourolleandClaude Opus 5	26a1eb7e28	Record that face detection has run, not just what it found ... An image with no faces in it was indistinguishable from one that had never been looked at, so every landscape, still life and document scan in the library was re-detected on every pass, for ever. In a real library that is most of it: on the 23,527-image test library, 64 of the first 110 images indexed contain no face at all. Schema v9 adds face_index, a run marker per (image, model) carrying the face count and the proxy edge it read. Keyed on the model, so a model change puts every image back in the queue by itself. That makes a coverage figure possible, which is the thing a user actually wants to see. The audit also splits the outstanding set by whether a proxy exists, because 23,417 awaiting a proxy and 110 ready to index are different problems, and telling the user to run indexing again would not fix the first. The Identity screen gains Index faces, Stop, and the coverage line. examples/face_index.rs is the same check and sweep without a window, which is the right shape for an overnight pass. Measured on the real library in release: 3.5 images/second, 110 images and 125 faces in 30 seconds, and a second run correctly finds nothing left to do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-26 22:13:41 +02:00
dtourolleandClaude Opus 5	3b5952769b	Emit floats an f32 can hold, and drop the format! that formats nothing ... CI runs cargo fmt --check and clippy -D warnings, and this branch had never been through either. Both would have failed it. The bulk was the generated colour tables: eight significant figures where an f32 carries about 7.2, so the eighth is noise that rounds away at compile time and clippy's excessive_precision says so 109 times over. Fixed in the generator rather than only in the file, so it stays fixed -- and the file is trimmed in place rather than re-derived, because regenerating it needs a colour-science stack that has nothing to do with the defect. The format! in the composer is mine too, from extracting the rendering tail: the braces in it were escaped because the text used to live inside a larger template, and once extracted the escapes are noise and the call formats nothing. Also here, and clearly not mine: an unused import and a shadowed binding in dr-gpu, and an unused import in a test. They are pre-existing -- clippy has been failing on master before this branch existed, on lints like is_multiple_of that arrived with a toolchain rather than with anyone's code. Fixed because CI cannot go green around them, and called out because a merge commit is a bad place to quietly edit someone else's crate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-25 22:28:14 +02:00
dtourolle	6925aa2a86	Merge master into film-simulation ... Master gained the library's index paging while this branch was building the film simulation, and the two met in library_ui.rs. Only the generated traceability matrix conflicted; it is regenerated here rather than hand-resolved, which is what it is for.	2026-08-25 20:34:28 +02:00
dtourolleandClaude Opus 5	6d6ef8d34b	Page the grid along an index instead of sorting the library each time ... Scrolling jittered, and this was the largest single reason. Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and neither half of that was being answered the cheap way. **The sort.** `GRID_ORDER` leads with `captured_at IS NULL`, so undated frames fall to the end. No ordinary index answers that — the leading term is an expression, not a column — so SQLite sorted the whole library into a temp b-tree on every window read, then threw away the first `k` rows of it. Schema V7 indexes the expression exactly as the query writes it, partial on the same `shadowed_by IS NULL AND trashed_at IS NULL` the grid filters by, so the read becomes a walk along the index. **The join.** `LEFT JOIN remote` was paged *after* it was joined, so reading 280 cells at offset 20,000 first seeked into `remote` for all 24,000 rows and then discarded 23,720 of them. The file ids are now fetched for the 280 rows that survived — the shape the badge and rating reads already use, one query for the window rather than one per cell. Measured together on 24,000 images at offset 20,000: **15.2 ms → 0.36 ms**, inside a scroll handler that has 16.7 ms to draw a frame. The test asserts on the query plan rather than on a duration, because there is no other symptom. A `GRID_ORDER` edited out of step with the index, or a column added back that drags `remote` in again, both still return exactly the right cells — just after sorting the library — and the jitter would come back with nothing to point at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-25 20:01:04 +02:00
dtourolleandClaude Opus 5	baa8957e80	Let a photographer choose the film, and remember which one ... The stock model rendered correctly and nothing could ask for it. This is the picker, and the sidecar key that makes the choice outlive the session. How the choice persists was the open question, and the answer was already written down twice in sidecar.rs: `rating` is a top-level key "because a rating is not an edit", and `masks` are one "because a layer is not a scalar". A stock is that kind of thing -- a choice of material, not a number a slider moves -- so it is a top-level key too. It stores the **id**, not an index. Stocks are files that users add, so an index would mean installing a profile silently changed which film every existing photograph had been developed on. A name this build has no profile for still round-trips untouched, because the alternative is that syncing to an older phone quietly un-develops the picture. Only the names travel. Turning one back into tables needs the profile database, which dr-pipeline deliberately does not link, so `Version::apply` clears the film and the session re-bakes -- after the parameters, because the bake reads the film's own exposure sliders and the print balance is solved against them. That is also why moving those sliders rebuilds the lookup where no other control in the panel does: an enlarger's filtration depends on how the negative was exposed. The panel keeps its rule. It still names no operation and still generates every control from a declared parameter kind; the stock gets a bespoke control beside those, exactly as the mask stack does, and for the same reason. The film's exposure and print exposure arrive as ordinary generated sliders. Two defaults worth stating. Picking a colour negative prints it, because an unprinted one is an orange strip and offering that as the first thing somebody sees after choosing Portra reads as a bug rather than as a choice -- the toggle is there for anyone who wants the scan. And a paste carries no film: a preset is a parameter map, and a stock is not a parameter, so pasting one would paste a choice the clipboard never took. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>	2026-08-25 19:57:41 +02:00
dtourolleandClaude Opus 5	54f9cb54fb	Take the whole run a shift-click names, not the part that happens to be loaded ... Shift-clicking two photographs selected only the cells between them that were in the loaded window. The grid is a window of a hundred or so over a library of twenty thousand, and `apply_press` resolved the range against that window — `ids[lo_row..=hi_row]`, clamped to what was there. Everything else in the run had no id anywhere in the UI, so it was silently dropped. The user cannot see that: the selection count is off screen along with the photographs, and the gesture only announces itself when the drop files a dozen images instead of two hundred. The two ends are *ordinals*, and only the catalog knows what lies between them. `read_ids_span` asks it, through the same predicates, the same rating filter and the same ordering the window itself is read with — an ordinal names a photograph only relative to an ordering, so a run taken through any other one is a run through a different library. That ordering is now a constant, `GRID_ORDER`, shared by the window, the trash's own order beside it, and the run: capture time first, with the file name breaking ties and nothing more. A card written by two cameras interleaves names that have nothing to do with each other, and what "everything between these two" means to a photographer is a stretch of an afternoon. The query is reached through a closure handed to `CollectionsController` at wiring time rather than a catalog handle, because the scope and the filter that bound the run belong to the grid's controller. `apply_press` stays a pure function of what it is given, which is what keeps the selection rules testable with no library open — and the tests pass a run that reads a plain slice. Where there is nothing to ask, the loaded window is still used: a poorer answer than the catalog's and a far better one than a gesture that appears to do nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-24 19:50:21 +02:00
dtourolleandClaude Opus 5	fa4ad6e2d6	Drag the date range on the axis it is chosen from ... The range could be turned on with a finger and not aimed with one. Its two ends were typed as `YYYY-MM-DD` into 108px fields behind a soft keyboard, to name days already drawn on the axis a thumb away; and the chip that seeds them takes its span from the timeline's zoom and pan, which are a wheel and a middle button. A touch screen has neither, so on Android the filter was a switch with no aim. The band is now on the timeline. Two ends with grips, dragged along the bars, released to filter — the histogram was already how a period is found, and this makes it how a period is stated. Both ends snap to whole days, which is what the typed fields mean, what `show_range` reads back out, and a floor under a range dragged shut. The fields stay for what dragging cannot do: name an exact day, and say in words what the range is. For that to work the axis had to stop following the range. Redrawn to the band, it moved the ground under the very handles doing the narrowing, and there was nothing outside the range left to widen back into. While there: a fixed number of equal bins instead of calendar buckets. Between one calendar unit and the next the bar count is free to wander by a factor of twelve, so zooming in halved it two steps out of three — the same picture drawn wider until it jumped back to fine. Equal bins also include the empty ones, so a bar's position on the track and the date under it are finally the same quantity; before, a library with gaps drew a February six months wide and the marker, the band and a click all pointed somewhere else. The count is a setting, 32 or 64, because the right answer is a question about the screen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 18:42:42 +02:00
dtourolleandClaude Opus 5	fbf286d504	Keep the mask when the photograph is closed ... A local adjustment survived until the session ended and then did not exist. Nothing reported it, because nothing had failed. The sidecar format has carried masks since they were added, `Version::apply` restores them into the graph, and `Version::update` captures them — that work landed complete and was never called. The autosave writes `copy_settings()`, which is a `Preset`: a map from (operation, parameter) to a number. A mask is not a parameter. It is a rule about *where*, with a chain of its own, so it fell outside the only thing being written, and the read path had nothing to read. The save now carries the stack beside the preset, on both the local and the remote path. Deliberately not merged but replaced wholesale: this is the stack as it stands, so a layer the user deleted has to leave the file too. Merging two devices' stacks is `Sidecar::merge`'s job and belongs to sync (FR-NC-9). A paste still carries no masks, and the `Option` is how that is said. Settings travel between photographs; a mask does not, because it is drawn against one frame and describes nothing on another — and `Scope` cannot express that, since it filters parameters and a mask is not one. The test fails against the old save path with "the mask must come back with the photograph", which is the whole of the defect: not a crash, not an error, just an edit that was not there in the morning. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-23 12:29:41 +02:00
dtourolleandClaude Opus 5	d91f1ec277	Thumbnail the whole library on request, and send the shards up ... The grid fetches a preview only for cells that are actually browsed, which is the right posture over a link that must not be saturated to show one screen (FR-NC-3). The cost is that the thumbnail store ends up holding the fraction of the library someone happened to scroll past — and that store is the one derived thing worth syncing, since a second device that downloads the shards gets a full grid without touching a single RAW. So the complete set is worth an hour of range fetches paid once, deliberately, on a machine that can afford it. That is what this adds: a pass over every visible image with an `oc:fileid`, launched from the settings page and reported in the activity register like every other background job. The work list is what the store lacks rather than a flag in the catalog, so it is resumable by construction and safe to press twice. Lane-parallel like the metadata sweep, but it does what that sweep declined to. The store is `&mut` and cannot cross lanes, which is why dating the library skips thumbnails entirely; here the lanes fetch, decode and *encode*, and only the ~20 KB result crosses back to the one thread that owns the store and writes the chunk. The parallelism is real and the single-writer rule is not bent. Grid class only. The large class is ~860 MB of shards against ~200 MB on the reference library, paid by every device that syncs them; a photograph looked at closely still gets its large thumbnail from the interactive path. Dates come free — the header a preview needs is the header EXIF lives in — and the pass ends by pushing the shards to the server. A filled store that never leaves this device would be most of the cost for none of the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-22 19:34:59 +02:00
dtourolle	1526c957cf	wip: ingest	2026-08-22 15:34:43 +02:00