a92ae4576f65fcc7ad8e29d95630f8a19c537ece
714
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  dtourolleandClaude Opus 5
|
dbaf5358d1 |
Measure a crop drag against the frame, not against the rect it is moving
Dragging the crop rectangle tracked the pointer at half speed: the rect slid out from under the cursor, and the handle being held stopped being the one under the finger. On a photograph where the whole point is to place an edge by eye, that made the tool close to unusable. The cause is that a `TouchArea` reports `mouse-x` relative to itself, and every area in the overlay is positioned *by the very rect the drag is editing*. Rust echoes the applied rect back on each step, so the area moves under the pointer and the reported position falls by exactly the amount it had just risen. `mouse-x - pressed-x` therefore subtracts the drag from itself, and the fixed point of that feedback is a rect that moves half as far as the pointer does — which is why it looked like sluggish tracking rather than like a coordinate bug. Both terms are now taken in the frame's own coordinates: `parent.x + mouse-x`, where `parent.x` tracks precisely the movement `mouse-x` lost, with the press captured in those same coordinates on the way down. The two movements cancel and the rect follows the pointer exactly. `GradientHandles` in masks.slint was already written this way, and for the same reason — its handles are placed by the mask they drag. The header note here now says why the shape matters, since the wrong version compiles, looks plausible, and is only wrong once the rect starts moving. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
480c46c41a |
Quote the percentile the measurement can actually support
The before/after published a p99 ratio of 6.0x at 4K. It is not supported and the correction is worth more than the number was. The baseline run's `fit` rows spread 2.3x between median and 99th percentile while every row of the after run spreads about 1.1x. A stage costing `radius x pixels` has no reason to be bimodal, and `fit` is the memory-bound configuration — it walks the whole 482 MB source on a stride where `1:1` reads a contiguous window. Something else had the machine. The merge brings in the cross-check that settles it: "Try every GPU, not only the fastest one" measured the same baseline code on the same card and reports 4.67 ms p99 for M3 clarity fit at 2560x1600, against 10.40 ms here. Two measurements of one thing differing by 2.2x mean the noisier one is wrong. So both percentiles are now published and the p50 column is the claim: 2.8x at 4K rather than 6.0x. The p99 improvement is real and larger; this run cannot say by how much, and says so. What the caveat does not touch: every after figure is inside the 16 ms budget with a p99 within 26% of its median at every size and both views, and clarity against all-four is a within-run comparison. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
30162e80df |
Merge branch 'master' into clarity-reduced-base
# Conflicts: # docs/traceability.md |
||
|
dtourolleandClaude Opus 5
|
8b251b84a0 |
Record what the reduced base actually bought
TD-4 asked for the measurement as well as the change, and this is it: 25.05 ms to 4.17 ms at 3840 x 2160, six times faster, with clarity no longer dominating the neighbourhood stage it used to be 97% of. Measured before and after on the same machine and the same adapter minutes apart, baseline at the branch's merge-base, so the only variable is the change. That adapter is not the RTX 3050 the rest of this document was measured on, so the new table says to read it on its own rather than against the ones above — the before/after is comparable, the absolute figures are not, and quietly replacing the existing tables would have changed the instrument. Also recorded: the declared halo is now quantised to multiples of the output scale, because the 2-sigma truncation rounds on the reduced grid. 29 px becomes 28 at 1920x1200 and 38 becomes 40 at 2560x1600. It is inside what the cross-form test holds — 0.03 stops of peak, 2% of reach — but it is a change in reach and not only in cost, and a tile scheduler would be handed it. Better written down now than found later as a seam. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
bff95e25ad |
Let clarity's base be computed where it is still fully determined
Clarity's Gaussian sigma is 1.2% of the frame's shorter edge, so its radius is a property of the viewport: 52 render pixels at 4K, two separable passes of 105 taps each over 8.3 M pixels. That measured 33.9 ms — seven times the entire fused point chain, for one slider — and is docs/technical-debt.md TD-4. A detail pass may now declare `output_scale`, and clarity's base is computed on a grid a quarter the size on each axis. The pass that combines needs the blur *and* the full-resolution colour, and a colour that has been through a quarter-scale target is no longer full resolution. So a scaled pass cannot simply join the ping-pong: there are two chains now. The full-resolution one carries the colour and no scaled pass touches it; the reduced one carries the base and reaches the combining pass through a second binding as `reduced_at()`. The reduce is a dispatch of its own rather than something the first blur half does on the way past, and that is the whole difference between this and the strided kernel the module documentation rules out. A stride samples an image that is not band-limited and aliases high-frequency content down into the base, which is then subtracted, and arrives in the output as mottling across smooth gradients. This band-limits first and samples after. What is discarded is content the base could not represent at any resolution, because a Gaussian at sigma = 26 px holds nothing above one cycle per 26 px and the quarter-scale grid carries one per 8 — so the reduced base is not an approximation of the full-resolution one, it is the same function sampled where it is still determined. Which is also why the scale belongs to the band rather than to the stage. Texture's sigma is a decade finer, so the reduce pass's own box would be wider than the Gaussian it was prefiltering; texture never reduces. And clarity steps 4 -> 2 -> 1 as sigma falls, because a quarter of a small sigma is not a Gaussian either — the case that gives up is the one that was already cheap. `radius` stays in each pass's own pixels and `ComposedDetail::radius` multiplies it back up, so 13 reduced pixels at scale 4 still report the 52 render pixels a tile would have to be grown by. The halo a scheduler sees does not move. The halo tests pass unchanged, which was TD-4's stated bar; they render at 1024 px and so exercise the reduced path rather than stepping around it. Added `crossing_the_reduction_threshold_does_not_change_the_picture`, because nothing yet compared the reduced form against a *less* reduced one — every other test measures one form against itself. It renders the same edit either side of the 4 -> 2 step-down and holds the peak excursion to 0.03 stops and the reach to 2% of the frame. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
3b5d564495 |
Try every GPU, not only the fastest one
Build and test / Desktop (Linux) (push) Successful in 2h7m41s
Build and test / Layer separation (push) Successful in 46s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Successful in 41s
Build and test / Android (aarch64) (push) Successful in 21m1s
`request_adapter` with `HighPerformance` returns one adapter and no second chance. That is right on a healthy machine and wrong on one with a sick GPU, which is not rare: observed 2026-08-29 on a laptop whose discrete card had hit an NVRM assertion failure and a fullchip reset. The driver still advertised it, wgpu dutifully picked it as the highest performing, and the process died on it — while a working integrated GPU and a working external card sat unused in the same enumeration. A photo editor that will not start because the *fastest* GPU is broken, on a machine holding two that are not, is worse than a slow one. So: enumerate, order by preference, take the first that yields a device. The ordering reproduces what `HighPerformance` meant, so a healthy machine picks what it always picked and pays one enumeration for it. A CPU adapter sorts last rather than being excluded — software rendering is a poor experience and a working one. Which GPU to prefer is now a policy rather than an assumption, because the fastest is not obviously the right one. A 24 MP frame is ~96 MB of RGBA and every upload and export readback crosses PCIe on a discrete card, where an integrated GPU shares memory and crosses nothing — and does not empty a battery. Measured before choosing a default, on this machine's Iris Xe against its RX 5700 XT. The fused colour pass is within 1.5x, which is the shape shared memory suits. The neighbourhood stage is 5-8x slower, and that decides it: clarity at 1920x1200 costs 20 ms on the iGPU, over the budget on its own at the smallest size tested. So `Performance` stays the default and `Efficiency` is offered rather than chosen (`DARKROOM_GPU=integrated`). docs/frame-budget.md carries the table, and says what it does *not* show: the harness renders from a resident texture and never uploads or reads back, so the transfer cost an iGPU avoids appears in none of it. Import, export and the thumbnail sweeps may well go the other way. What this cannot fix: a GPU sick enough to accept `request_device` and segfault afterwards, which arrives as a driver crash rather than an error. It moves the boundary from "the preferred adapter is unusable" to "unusable and dishonest about it". |
||
|
dtourolleandClaude Opus 5
|
0407fb8d2d |
Format the two new examples
They were written after the last fmt run and CI gates on --check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
3944e17aa5 |
Merge: face confidences that mean something, and a regroup three times faster
Two things the People screen was getting wrong, and then the arithmetic underneath it. It refused to show a confidence at all until the library had fitted its own calibration, which needs 200 confirmed positive pairs — made on the screen that was withholding the number used to rank them. The default curve is the reference implementation's fitted one, a published operating point rather than an invention, so the percentage is shown and the screen says which curve it came from. The number itself was the mean similarity between a face and every other member of its group, which punished well-photographed people and never asked who else the face might be. It is now the mean of the best ten matches into the identity, times that identity's share of the evidence against every other identity the user has ruled on. Leave-one-out over this library's 2,702 confirmations across 54 named people: 99.33% placed on the right person, and the number shown for the right person moves from a median of 90.4% to 99.3%. Both of those were measured rather than argued, on a copy of a real 18,143-face library, and the instrument is committed with them. Measuring is also what found the rest. A regroup went from 10.0s to 3.1s: the similarity scan was walking the whole embedding array once per row and not vectorising at all, and the agglomeration was spending 3.06s having every component scan the entire pair list for its own pairs. The scan now picks its kernel per machine — AVX2 where the CPU has it, NEON on the tablet, where the whole suite and a full regroup have both now been run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
39c34d4e44 |
Say what actually counts as a rival, now that a name anchors too
assign's denominator is the identities the user has ruled on, and it reads Cluster::person to find them. Master's "Let a name hold a group together" widened what sets that field: a confirmation, a name, or an ignore, where before it was a confirmation alone. The behaviour is right either way — a named person is exactly the identity a suggestion should be discounted against — but the module note and faces.md §9.1 both said "a confirmation", which is now too narrow. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
da20d42d33 |
Merge master: pluggable storage, and a name that anchors
Conflicts were docs/traceability.md alone, and it is generated — so it was regenerated rather than hand-merged. dr-face was untouched on the other side; ui/dr-ui/src/faces.rs and identity_ui.rs auto-merged, the first around recluster's anchoring and the second around load_faces. Worth recording because the two branches met on the same problem from different ends. Master's "Let a name hold a group together" is the fix for the sixteen Catherines — fourteen of them empty — that this branch found while measuring the library and reported without fixing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
b2250cc460 |
Measure a regroup on the tablet, not just on the desktop
The GPU question needed a number nobody had: how a regroup divides on the hardware whose CPU is weakest. dr-face carries no weights and touches no display, and dr-catalog's example needs only a catalog file, so both run under adb shell against a copy of a real library. On the same 18,143 faces — desktop against the tablet — scan 0.96s / 2.61s, agglomerate 1.69s / 2.16s, score 0.26s / 0.40s. The scan is half the pass on the tablet and under a third on the desktop, because twenty cores of AVX2 pull ahead of NEON much further than the merge engine's single-threaded hashing does. So a GPU GEMM is worth roughly 2× a regroup on the tablet and 1.5× here, and it is the tablet that should decide whether it is built. The two architectures agree exactly: the same 1,531,969 evidence pairs, the same 2,518 groups holding the same 16,246 faces, the same reliability table. That is a better check on the NEON kernel than the unit test can be. Two instruments, both read-only: the example now prints its phases, and dr-face gains scan_bench, which needs no library at all and so can answer "how fast is this machine" on a device with nothing on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
f4395bd17c |
Run the face tests on the tablet, where the NEON kernel actually runs
The similarity scan picks its dot product per machine, and the NEON one is the kernel that ships to the phone and the tablet — and the one a desktop cargo test never executes. A wrong lane index or a mishandled tail there is a silent wrong answer on exactly the devices nobody runs the suite on, which is a poor place for the only untested code path. dr-face carries no weights and touches no display, so its tests are a plain ARM64 binary that runs under adb shell with nothing installed. The script builds it against the SDK's newest NDK, pushes it, runs it and cleans up. It checks for the device first, so a tablet that is not plugged in costs a second rather than the two minutes it takes to compile for it. Not wired into CI, which has no device attached. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
8f596262b8 |
Record where a regroup's time actually goes
The §9 note said the scan was half a regroup and implied the agglomeration was an irreducible sequential walk. Both halves of that are now wrong, and the numbers are the point of the section. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
a67402961d |
Let the merge engine's dot product use the machine's kernel too
Engine::cross is the one place a dot product is computed during agglomeration — when two groups become adjacent through a third and their sub-threshold pairs, never summed because they were never interesting, have to be accounted for. It was calling the portable loop while the scan beside it had AVX2 or NEON, which on the reference library was 1,753,514 dot products taking 0.54s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
b4d39ba33a |
File each pair under its component once, not once per component
Seeding the merge heaps was 3.06s of a 5.93s regroup on the reference 18,143-face library — more than half the pass, spent before a single merge was considered. Every component scanned the whole pair list looking for the pairs that were its own: 475 components against 804,499 pairs, 382 million set lookups to place 804,499 of them. A pair can only ever join two faces of one component, since that is what a component is, so the union-find that finds the components can file the pairs at the same time and hand each agglomeration the list it needs. The membership set inside agglomerate goes with it — it existed only to run that filter — and the heap can be sized up front now that the pair count is known. Ordering is preserved deliberately: pairs are filed in the order they arrive, which is the global (i, j) order, so the seeded heap breaks its ties exactly as before and the merge order is unchanged. Same 2,518 groups holding the same 16,246 faces on the reference library, at 3.5s rather than 5.9s. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
e596eb0657 |
Give the similarity scan the machine's SIMD, and its cache
The scan is O(n²) dot products and nothing else, so its speed is the face subsystem's speed — and it was running at 0.7 flops per cycle. Two separate faults, both measured over the reference 18,143-face library on twenty cores. It walked the whole embedding array once per row, ~336 GB of traffic, where a column tile that fits in L2 is read once per tile of rows: 4.64s → 2.81s. And the workspace builds for baseline x86-64 — SSE2, no FMA — into which the portable loop was not being vectorised at all: 2.81s → 0.86s, 195 GFLOP/s. So the dot product is now chosen per machine. AVX2 + FMA where is_x86_feature_detected! finds it; NEON unconditionally on aarch64, since Advanced SIMD is in that baseline and every Android device the app builds for has it — with the explicit vfmaq, because LLVM will not fuse a multiply and an add without being told to. The portable loop stays as the definition the others are tested against, and the_fastest_kernel_agrees_with_the_portable_one is the only check the NEON path gets on a machine that is not aarch64. Faces::embeddings is one flat buffer rather than a Vec per face: the pointer chase defeated both the prefetcher and the tiling, and it is also the layout a GPU pass would want. Behaviour is unchanged and that is checked rather than asserted — the same 1,531,969 pairs from all three kernels, and on the real library the same 2,518 groups holding the same 16,246 faces with the same confidence distribution. A full regroup there goes from 10.0s to 5.9s; the rest is the agglomeration, which is a sequential heap walk and is where the next look should go. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
ebb7d3cf5c |
Score a suggestion against the people the user has named
The number beside a suggestion was the mean calibrated probability between the face and the rest of its group, which measures the wrong thing twice. It punishes coverage: a person with two hundred faces over fifteen years is *meant* to have members a given photograph is orthogonal to, so a correct suggestion onto a well-photographed person scored low for being well photographed. And it never asked who else the face might be — a face matching Anna at 0.95 and nobody else, and one matching Anna at 0.95 and her sister at 0.93, came out identical, when the second is the only one worth the user's attention. dr_face::assign answers both, and multiplies them: the mean of the best ten calibrated matches into the identity (the old mean, capped, which is what stops coverage counting against it), times that identity's share of the evidence against every *named* rival. Only named people compete, and per person rather than per group. Both halves of that had to be measured on a real 18,000-face library rather than reasoned about. Normalising across every group made the number useless — median suggestion 21%, four in five under half — because clustering leaves one person spread over many groups, so a face competed against itself; and keying rivals by group left Catherine competing with Catherine, median 39%. Per named person: median 99.5%. Rivals are gathered below the merge threshold, down to even odds: a named person matching at 0.6 will never be merged into but is exactly the competition to discount for. That would be a second similarity scan, the expensive half of regrouping a library, so cluster_scored scans once at the looser floor and hands the merge engine the subset at or above the threshold — pair for pair what it would have scanned for itself, held to that by a test. Leave-one-out over that library's 2,702 confirmations across 54 named people: 99.33% of faces placed on the right person against the old mean's 99.15%, and the number shown for the right person moves from a median of 90.4% to 99.3%. It errs low — 100% correct wherever it states 80% or more — which is the safe direction, and docs/faces.md §9.1 says plainly that the low bands are not calibrated. The example that measures it comes too: this is a claim about a library's numbers, and nobody should have to take it on faith. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolle
|
21d599b420 |
Test the guard, since the bug was a branch nobody ran
Build and test / Desktop (Linux) (push) Successful in 2h5m41s
Build and test / Layer separation (push) Successful in 50s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 1m37s
Build and test / Android (aarch64) (push) Successful in 59m40s
The catalog clobber existed because `if let Ok(bytes)` had a failure arm that was never exercised. Fixing it without covering that arm leaves the next person free to collapse it back. Three tests against a backend whose read fails in a chosen way, counting writes — because what went wrong was not a wrong value but a write that should not have happened at all: - a dehydrated snapshot uploads nothing - an unreadable one uploads nothing either, since "refused" is no more "absent" than "not downloaded" is - and a genuine first sync still uploads, which is the half that keeps `NotFound` distinct from `NotMaterialised` rather than merely cautious Checked against the original shape: the first two fail on it and the third passes. A guard test that cannot tell the bug from the fix is decoration. `async-trait` joins dev-dependencies to stand the double up behind `dyn RemoteBackend`. |
||
|
dtourolle
|
4168d67cfa |
Do not push a catalog over one we could not read
`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped
if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {
which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.
The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.
Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.
And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
|
||
|
dtourolle
|
702d83c218 |
Measure the borrow, rather than asserting it bounds the disk
The claim that hydration-as-a-borrow makes peak disk the working set rather than the library was so far an argument. `--example vfs_cycle` runs it: 100 photographs of 25 MB, 90 dehydrated, a pass over all of them through the real engine. Peak 275 MB — the resting set plus one photograph — against 2,500 MB had the pass simply fetched everything. Back to 250 MB afterwards, and all ten files the user already kept still there, which is the half of the contract that matters more. Recorded in docs/storage.md §6.3 and ARCH §9.0a, because a bound argued from a number nobody measured is one that gets quietly lost. |
||
|
dtourolle
|
5768100816 |
Borrow the library to index it, and give it back
The passes that need every photograph's bytes — thumbnails, face indexing — now borrow each one and release it at the end. On a placeholder library that is the difference between peak disk being the working set and being the whole library. Including on cancellation, which was nearly missed: the face sweep returns mid-loop when the user presses Stop, and without releasing there the disk is spent and nothing is delivered for it. `materialise` now answers whether *it* fetched the content. The pool used to work that out by listing a file's parent directory — one listing per file across a library — when the backend already had to `stat` it to decide whether to ask. One syscall instead of a directory walk, and it removes the bug class the tests found earlier: a file at the library root has no `parent()`, so every one of them read as already-downloaded. **Pinning is the retention control**, and it drives the model the catalog already had rather than a second one. `tier_desired` is what the user asked to keep hydrated, `pending_pins` is the resumable work list, and a pinned collection is never dehydrated for the same reason it was never evicted. It was in fact *broken* here before: `get` on a stub failed, and the pin worker logged "one unreadable file must not abandon the whole pin" and silently did nothing. Pinned originals on such a library are recorded with `path = NULL` (`Cache::record_in_place`) rather than copied under `originals/`. Two reasons, and the second is the important one. A copy would hold every pinned photograph twice, with the budget able to evict the half that was not costing the disk. And `release` deletes the file a row names — so a row that names none cannot delete anything, which puts the one catastrophic operation out of reach by construction rather than by remembering not to call it. Deleting a materialised file inside a synced tree removes the photograph from the server and every other device. Handing disk back is `spawn_dehydrate`, which asks the client. Two gaps written down rather than papered over (docs/storage.md §7): a hydrating pass cannot yet quote its cost, because a stub reports no size; and the two sweeps hold separate pools, so a library indexed for both fetches twice. |
||
|
dtourolle
|
c102ba9df2 |
Treat a placeholder as the photograph, not as a one-byte file
The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none. |
||
|
dtourolle
|
6c363cee97 |
Let the launch screen scroll, now that it offers three routes
The signed-out screen was a `VerticalLayout { alignment: center }` with
no scroll. That was already tight with two routes; the folder option
made the column taller than a 900x560 window, and a centred layout that
overflows clips at *both* ends — so the masthead and the last route
disappear together, with nothing on screen to suggest either existed.
A Flickable whose viewport follows the content, and a top padding that
centres the column only while it fits. Both cases checked against the
running app: centred at 1000x1800, top-aligned and scrollable at
900x560.
|
||
|
dtourolle
|
64462e9fd3 |
Test the folder sign-in instead of trusting it
The whole of a folder library's sign-in lived inside a Slint callback, which cannot run without a display server — so the one path that decides whether a mistyped folder becomes a *stored* account had no test at all. That failure is quiet and lasting: an account for a directory that is not there skips the launch screen on the next start and reads as a library that has lost its photographs. `open_folder_library` is that logic, lifted out whole. Four tests: it stores an account with no credential and reaches the keyring for nothing, a typo is refused before anything is written, the messages read as instructions because they go straight to the screen's error line, and a file is not a library. |
||
|
dtourolle
|
cbe5c4fcde |
Measure the folder walk against a real tree, not a claim
The folder connector declares `LocalEtags`, which means the engine walks the whole library on every scan with no pruning. That is the honest capability, and the argument for it being affordable was so far an assertion about `stat` versus `PROPFIND`. `--example scan` runs the real path — `dr_sync::scan` over the connector, then a ranged read of the kind the thumbnail worker makes. Read-only; it never writes into the folder it is pointed at. 2,299 images across 233 directories in 137 ms, and 380 across 13 in 29 ms. Against 34.1 s for 17,185 RAWs over WebDAV *with* pruning available. Recorded in docs/storage.md §5.2 and ARCH §8.4a, because a capability trade-off argued from a number nobody measured is the kind that gets quietly reversed later. |
||
|
dtourolle
|
f12aece07e |
Make storage pluggable, and prove it with a folder backend
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.
A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:
- **Capabilities** — already there, and the reason the engine can drive
two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
whatever form its connector addresses, with no server in it. Loads
every existing config unchanged (`backend` defaults to `nextcloud`,
`endpoint` is stored under its historical `server` key), and
`Account::namespace()` reproduces the old catalog directory byte for
byte, because changing it would abandon a catalog, its thumbnail
shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
`ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
names a connector.
`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.
Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.
`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.
docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
|
||
|
dtourolleandClaude Opus 5
|
c8e05831f4 |
Show the confidence, and say which curve it came from
FR-CULL-9 was read as "no fit, no number", so every library without 200 confirmed positive pairs showed "Confidence unavailable" on every suggestion — which is every library, until enough confirmations exist to fit one. The confirmations are made on this screen, ranked by the number it was withholding, so the degraded state was also the permanent one. There has always been a curve: Calibration::default is the reference implementation's fitted MBF sigmoid, which is what clustering already operates at. It is a published operating point, not an invention, and what the requirement forbids is presenting it *as though it were measured on this library*. So the percentage is shown, and the screen says once, above the grid, where the curve came from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
1b8b7998a2 |
Let a name hold a group together, the way a confirmation does
Build and test / Desktop (Linux) (push) Successful in 2h6m43s
Build and test / Layer separation (push) Successful in 46s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 29s
Build and test / Android (aarch64) (push) Successful in 21m27s
Sixteen people called Catherine, fourteen of them holding no faces at all, and her actual photographs split across the two that did. That is not a sync fault; it is this function, and it has been quietly doing it on every Regroup. Naming a cluster does not confirm its faces. They stay *suggestions* — and only confirmed faces anchored here, so the next pass cut them loose, regrouped them into a brand new person, and left the named one holding nothing. `prune_empty_unnamed` will not clean that up, because it has a name. Name the new group the same thing, and it happens again. Repeat over a few sessions and you have sixteen of her. A name is a judgement about *this group*, of exactly the kind FR-CULL-12 says travels and inference does not — the same argument that already anchors a group the user set aside. So all three kinds of ruling anchor now: confirmed, ignored, and named. It also fixes the quieter half of the same fault. A face indexed later that matches a named person now merges *into* them, rather than arriving as a rival group the user has to name all over again. Two tests, and the first fails without the change — it reports "Catherine" finishing the pass with zero faces while a fresh unnamed person holds the two she was named for. This does not retro-fit an existing library: the fourteen empty Catherines stay until they are merged by hand, and the two holding faces are separate identities that only the user can say are one person. What it stops is making more. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
2fb3eb5d2d |
Upload a shard that was sealed after the server saw it
Build and test / Desktop (Linux) (push) Successful in 2h7m52s
Build and test / Layer separation (push) Successful in 57s
🐳 Android image / Build and push (push) Successful in 12s
Build and test / android-image (push) Successful in 13s
Traceability / Requirement traces (push) Successful in 57s
Build and test / Android (aarch64) (push) Successful in 20m58s
The tablet showed 442 of a person's 648 faces and could never catch up. Its shard ledger said why: it had adopted the laptop's shard 0 at 2,711,552 bytes, and that shard is 20,402,176 bytes on disk. The upload skipped it. A sealed shard the server already had under that name was taken to be "byte-identical by construction", so its mere presence was proof enough and the size was never consulted. That premise is false, and this library is the counter-example: an *open* shard is uploaded on every pass as it fills — that is how a growing shard reaches the other devices — so the server ordinarily holds a partial copy of a shard that is later completed and sealed. Sealing then froze that partial copy in place for ever. Nothing downstream could recover from it. The tablet's `has_adopted` check is keyed on name *and* size and would have re-downloaded a changed shard gladly; it was never offered one, because the sending side had stopped looking. So the comparison is on size, sealed or not. The client id is in the name, so nobody else can have written the file and size is a sound test. A sealed shard whose size already matches is still skipped on the first comparison, which is all the original cheapness was worth. The thumbnail upload had the identical test and the identical hole, and is fixed with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>v0.8.0 |
||
|
dtourolleandClaude Opus 5
|
f9e331eed2 |
Let a test build be opened up, when asked
`adb shell run-as` refuses on a release build — "package not debuggable" — and the app's private storage is then unreachable from the host. That storage is where the face shards, the thumbnail store and the catalog live, so when a device disagrees with the desktop about what it has synced there is no way to find out which of them is right. An evening was spent guessing at exactly that. `DARKROOM_DEBUGGABLE=1 ./docker/android/package.sh --install` now sets `android:debuggable` through aapt2's `--debug-mode`, and nothing else changes. Set through aapt2 rather than written into `AndroidManifest.xml` on purpose: the flag then exists only for the build that asked for it, and a release build cannot inherit it because somebody forgot to take it out again. A debuggable APK lets any process on the device read this app's files, so it belongs on a test tablet and nowhere else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
f41b3f6e8e |
Answer "what would the other device end up with" without the other device
Build and test / Desktop (Linux) (push) Successful in 2h7m52s
Build and test / Layer separation (push) Successful in 59s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 4s
Traceability / Requirement traces (push) Successful in 37s
Build and test / Android (aarch64) (push) Successful in 22m22s
A tablet showed 200 of a person's 611 faces after syncing, and the obvious
suspects — that suggestions deliberately do not travel, that the cross-device
face match was too strict — were both wrong. Finding that out meant reading a
catalog on a release-signed Android build, which cannot be done.
So this stands the second device up locally: an empty catalog, given the images
a scan would have found, the shards adopted into it exactly as a sync does, and
the real catalog merged in as the remote. Then it counts, per person, against
what the source holds.
person source here
Catherine 611 611
Me 242 242
Ian 219 219
Which settled it: the merge carries everything, and the shortfall was transfer —
shards that never finished arriving. Worth keeping, because "did the sync lose
this or has it not got here yet" is a question that will come up again, and
guessing at it cost most of an evening.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
dtourolleandClaude Opus 5
|
bb3b512b81 |
Merge: a face sync that can actually complete
Three faults between a laptop that had indexed a library and a tablet that only ever received part of it. The face store was the one part of the catalog still on the rollback journal at synchronous=FULL — 21.3 ms a commit against the 0.05 ms everything else pays, four commits per photograph, on the order of fourteen minutes of pure fsync for ten thousand images. It now writes the way the catalog and the thumbnail store do, with a checkpoint before upload so the file that goes to the server is complete without its write-ahead log. Every idle pass read 94 MB of shards to decide it had nothing to send; the name and the size answer that. And the 60-second total request timeout was a floor on link speed rather than a hang detector: a 25 MB shard needed a sustained 425 KB/s or it failed, and then retried and failed again indefinitely. It now times out on inactivity. The merge itself was never at fault — a probe that stands up an empty catalog, adopts the shards and merges the real one in gets all 611 of a person's faces, not 200. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
d2af6a3981 |
Time out on a stalled transfer, not on a slow one
The HTTP client had a 60-second *total* request timeout. That is not a hang detector; it is a floor on link speed. A face shard runs to 25 MB, so it demanded a sustained 425 KB/s or the transfer failed — and having failed it was retried on the next pass and failed again, for ever. A tablet on ordinary wifi could therefore never finish taking in a library's faces, and nothing said why: each attempt looked like a network blip rather than an arithmetic impossibility. The catalog snapshot is 36 MB and has the same problem. `read_timeout` fires when no bytes arrive for the period, which is the condition actually worth failing on. A slow transfer that is still moving now finishes, however long it takes; a connection that has genuinely died is still caught in a minute. The connect timeout is unchanged. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
d70fe9da8d |
Decide whether to send a shard before reading it
The upload read every shard off disk and only then asked whether it needed sending. For a library with 94 MB of face shards already on the server, every idle sync read 94 MB to conclude it had nothing to do. The question does not need the bytes. A sealed shard the server already has is byte-identical by construction, and the client id is in the name, so nobody else could have written it — the name settles it. The open shard is compared on size, which `stat` answers. The progress line moves after the skip for the same reason: announced before it, an idle pass claimed to be sending five shards and sent none. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
2c84aa1224 |
Write face shards the way the rest of the catalog writes
The face store was the one part of the catalog still on SQLite's default rollback journal at `synchronous = FULL`. The catalog itself runs WAL at `NORMAL` (`schema::configure`) and so does the thumbnail store; nothing decided this one should differ, it was simply never set. Measured on this project's own filesystem, that is **21.3 ms per commit against 0.05 ms** — four hundred times. And an export commits four times per photograph: the shard's transaction, then three separate autocommitting writes to the index. Ten thousand images is on the order of fourteen minutes spent doing nothing but waiting for fsync, before a byte goes to the server. That is the "checking faces…" that appeared to hang. So: WAL and `synchronous = NORMAL`, matching the rest, and the three index writes fold into one transaction. `NORMAL` is the same trade the catalog makes — a shard is derived data, and losing the last commit to a power cut costs one image re-exported. WAL brings an obligation with it, because **a shard is uploaded by reading its file**: the newest commits live in a `-wal` sidecar that no upload sends, so without a checkpoint the server would receive a database missing exactly the faces just written, and a peer would adopt it and see nothing wrong. `checkpoint` folds the logs back in, with `TRUNCATE` rather than the default passive mode, which gives up when a reader holds the log and would leave the same gap while reporting success. Two tests: that the store is in WAL like everything else, and — the one that matters — that a checkpointed shard copied *without* its `-wal` still holds every face. That second one fails without the checkpoint, which is how it was confirmed to be testing something. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
0bba882fb1 |
Read the Android API level out of the ELF, not out of file(1)
Build and test / Desktop (Linux) (push) Successful in 2h8m13s
Build and test / Layer separation (push) Successful in 1m3s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
Traceability / Requirement traces (push) Successful in 41s
Build and test / Android (aarch64) (push) Successful in 1h0m55s
Every push has failed the Android job for weeks — back through fourteen runs —
with "FAIL: linked for Android 'unknown', expected 28", on a .so that was linked
perfectly correctly at API 28 the whole time.
The check ran `file` on the linked object and pulled the level out of its
description with a regex. `file` only prints "for Android 28" when its magic
database is new enough to decode `.note.android.ident`, and this image's is not
— it stops at "dynamically linked, not stripped". The regex matched nothing, and
`${API:-unknown}` turned that silence into a confident-looking failure about the
artefact rather than about the tool inspecting it.
The API level is the first word of that note, little-endian, so it is read
straight out of the ELF with `readelf` — which is in the image via
build-essential, and cannot go out of date the way a magic database can. An
absent note is now its own message rather than being folded into the mismatch
case, since "nothing states an API level" and "states the wrong one" are
different faults.
`file` stays in the image and in the log: it names the NDK that built the
object, which is worth having when this does go wrong. Nothing depends on it.
Verified inside the real container against the linked .so: the note reads
1c 00 00 00, and the step prints "OK: linked for Android 28".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
dtourolleandClaude Opus 5
|
e997be8c72 |
Say which version this is: 0.8.0
Build and test / Desktop (Linux) (push) Successful in 2h14m44s
Build and test / Layer separation (push) Successful in 54s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Successful in 1m45s
Build and test / Android (aarch64) (push) Failing after 53m50s
A hundred commits since 0.7.0, and the face subsystem in them is not a set of fixes — it is the difference between a feature that was shipped and one that works. Clustering finished at all for the first time: the old agglomeration rescanned every live pair and recomputed average link from scratch after every merge, and on a real library it did not return. A sparse above-threshold graph, connected components and Lance-Williams sums put 1,813 faces at 0.28s, and the merge threshold moved to 0.80 because the old 0.90 was measured — not guessed — to leave a third of the library ungrouped. Faces are no longer indexed at any size or any sharpness. Both floors were measured over the real library with `face_index --quality`: 32 source pixels across the aligned crop, and a contrast-invariant sharpness that catches the large-but-blurred face whose confident, wrong embedding used to weld two people together. The crop is cut once and kept, so the People screen is no longer a derivative of a thumbnail cache entitled to evict anything at any moment. The screen itself became usable: the faces wrap into a grid instead of running off the edge, the header fits a phone, a group can be set aside, and a person's photographs are a button away — as a union or an intersection of several people. And face sync now reaches the other device. Re-indexed images re-export, shards written before the crop and index-time columns are repaired rather than failing every insert, people and the user's judgements about them cross the wire at all, and the pass says what it is doing while it does it. The Android versionCode follows without being restated — package.sh packs MAJOR*10000 + MINOR*100 + PATCH, so 0.8.0 is 800, above the 700 already on devices and therefore an upgrade rather than a refusal. `pkgrel` returns to 1, since this is a new version rather than a rebuild of the last one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
a8ce1ff19c |
Merge: a face sync that says what it is doing
Build and test / Desktop (Linux) (push) Successful in 39m22s
Build and test / Layer separation (push) Successful in 52s
🐳 Android image / Build and push (push) Successful in 4s
Build and test / android-image (push) Successful in 5s
Traceability / Requirement traces (push) Successful in 53s
Build and test / Android (aarch64) (push) Failing after 53m19s
The face pass announced itself once and then went quiet for the whole export, upload and adopt. On a first export after a re-index that is eight minutes of a motionless progress bar, which reads as a hang. It now reports the images it is preparing, the shards it is sending and their size, and the shards it is taking in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
a1e8f494b9 |
Say what the face sync is doing while it does it
The face pass set the status to "checking faces…" once and then said nothing until it was finished. On a library whose first export after a re-index is 9,849 photographs and 85 MB of shards, that is eight minutes of a progress bar sitting still — which is indistinguishable from a hang, and was reported as one twice. Nothing was wrong with the sync. The only fault was that it was silent. Three places now report, which are the three that take real time: - **Preparing**, per image with a count, since this is the long one and the only one whose length the user cannot guess from anything on screen. - **Sending**, per shard with its size, because a face shard carries crops and runs to tens of megabytes — one of them is a visible wait on any connection. Announced before the upload rather than after, since the wait *is* the upload. - **Taking in** a peer's shard, which is a download and then a row-by-row merge. The export reports every 25 images rather than every one, so the channel behind it stays lost in the write it accompanies. `export_to_shards` keeps its old signature and delegates, so the callers that do not want progress do not grow a parameter for it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
35febf31dd |
Merge: a face sync that finishes
Build and test / Desktop (Linux) (push) Successful in 35m30s
Build and test / Layer separation (push) Successful in 53s
Traceability / Requirement traces (push) Successful in 38s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 58m37s
The export asked whether each image was already in the shard store at its current index time, and answered by opening the shard database — schema batch, pragma probes and all — once per photograph. 9,849 opens per sync pass for this library, before any face was written, which showed up as a sync stuck on "checking faces…" and never coming back. The index time moves to the store's own index, which is already open, and the write handle is held across a run instead of reopened per image. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
0729dfa359 |
Stop the face sync opening a database per photograph
"Checking faces…" never finished. `export_to_shards` asks, for every indexed image in the library, whether the shard store already holds that image at that index time — and `indexed_at` answered by opening the shard database, running its six-statement schema batch and two `pragma_table_info` queries, then querying. Once per image. 9,849 times for this library, on every sync pass, before a single face had been written. The index time now lives in the store's `index.sqlite` alongside the shard number, so the question is one indexed lookup on a connection that is already open. It stays in the shard as well — that copy is the one that travels — but nothing reads it from there on the hot path. The write side had the same shape: `put_image` opened the shard afresh for each image, which mattered little when exports were a handful of new photographs and matters a great deal now that a re-index sends thousands. The handle is kept and reused, invalidated by shard id so sealing a full one and moving to the next drops it without anything having to remember to. `INDEX_SCHEMA` is `CREATE ... IF NOT EXISTS` like the shard schema, so the new column is added on open for an index already on disk — the same trap, caught the same way. Two tests: that the index time survives reopening the store, and that an index written before the column can still be opened and written to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
9052b1a1de |
Merge: repair shards that predate the crop and index-time columns
Build and test / Desktop (Linux) (push) Successful in 33m10s
Build and test / Layer separation (push) Successful in 34s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / android-image (push) Successful in 3s
Traceability / Requirement traces (push) Successful in 38s
Build and test / Android (aarch64) (push) Failing after 52m58s
The schema batch is all CREATE IF NOT EXISTS, so two columns added to it never reached any shard already on disk — and every export into one failed on "no such column", quietly, logged at warn while the sync reported success. That, rather than anything in the export logic, is why this library's shard stayed at 1,807 faces while the catalog reached 15,194. Shards are now upgraded when opened, and a peer's read-only shard is read as it stands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
76eece8500 |
Add the columns an existing shard never got
`SHARD_SCHEMA` is entirely `CREATE ... IF NOT EXISTS`, which does exactly nothing to a table that already exists. So `crop` and `indexed_at`, both added to that batch, never appeared in any shard that had been written before — and the `INSERT` naming them failed with "no such column". Which took face export down completely, on every library that had ever synced a face. Silently: `export_to_shards` returns the error, `sync_face_shards` logs it at warn, and the sync goes on looking successful while the catalog fills with faces no other device will ever see. This library's shard sat frozen at 1,807 faces with 15,194 in the catalog, and the reason was this rather than anything in the export logic. Shards are upgraded on open now: both columns are additive and nullable, so catching up is one `ALTER` each. There is deliberately no version counter — "does this column exist" is the question actually being asked, and asking it directly cannot fall out of step the way a counter can. A peer's shard is opened read-only and cannot be repaired, so one written before crops is read as it stands, with a `NULL` standing in for the column. An adopted face simply has no crop, which is the truth about it. Four tests, built against the pre-crop schema written out in full rather than derived from the current one — the point being that it is *not* the current schema and must not track it. Verified against the real 1,807-face shard on this machine: the ALTERs apply, writes succeed, and nothing already in it is lost. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
43097f5033 |
Merge: make face sync actually reach the other device
Build and test / Desktop (Linux) (push) Successful in 31m3s
Build and test / Layer separation (push) Successful in 36s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
Traceability / Requirement traces (push) Successful in 30s
Build and test / Android (aarch64) (push) Failing after 50m38s
Two independent faults, both of which had to be fixed before a tablet could show what a laptop had indexed. An image was exported to the face shards exactly once, so a re-index updated the catalog and nothing else — this library went from 1,807 faces to 15,194 and kept syncing the original 1,807. And people never crossed a device boundary at all: the catalog merge handled collections and keywords only, so the far end received every face and no groups, and drew an empty People screen over a full catalog. Names, confirmations, rejections and set-aside groups now merge by uuid, with faces matched across devices by photograph and box overlap. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
4ed10f7b23 |
Let a person cross from one device to another
The face shards carry boxes, landmarks and embeddings. What they deliberately do not carry is who anybody **is** — the person rows, their names, and the assignments joining the two. Those travel in the catalog snapshot, which is a whole-file copy and does contain them. But the snapshot is *merged*, not adopted, and this merge only ever looked at collections and keywords. `face_shard`'s own module note says people travel in the snapshot; nothing implemented it. So a second device received every face and no people at all, and drew an empty People screen over a full catalog. Exactly what a tablet showed after syncing thousands of faces from a laptop. What travels is what the user decided, following the rule the rest of this module already follows — judgements travel, inference is rebuilt: - **People**, by uuid on `revision`, exactly as a collection is: the name, and whether the group was set aside. - **Confirmations**, and **rejections** — "this is not her" is a fact too, and is why re-clustering does not put it back. - **The suggestions inside an ignored group**, which are otherwise ordinary inference but are what anchors the ignore. Without them a group set aside on one device reappears on the other, the same fault that made "Not interested" not stick locally. Ordinary suggestions are not carried. Both devices hold the same embeddings and clustering is deterministic, so each recomputes them and arrives at the same answer; shipping them would double the merge for no new information. **A face has no cross-device identity**, and unlike a collection there is no uuid to give it one. Both devices do agree on `oc:fileid` and roughly on the box, so a remote face is matched to the local face on the same photograph whose box overlaps it most, above 0.5 IoU. That is not a new rule — it is the one `record_detections` already uses to carry a confirmation across a re-index, and it is loose on purpose: the question is "the same face in the frame", not "the same rectangle". A local confirmation is never overwritten. Two devices confirming one face as different people is a real disagreement and an assignment carries no revision to settle it with; taking the remote's answer would let a sync undo what the user just did on the device in their hands. The remote's schema is probed rather than assumed: `remote_is_mergeable` admits any catalog at or below this version, so one written before faces existed, or before V10 added `ignored`, is ordinary. An absent table skips this half instead of aborting a merge that would otherwise have succeeded. Nine tests, including that the name lands on the overlapping face and not its neighbour in the same frame, that a set-aside group stays set aside, that an ordinary suggestion does not travel, and that merging twice changes nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
cf614efa61 |
Send a re-indexed image's faces to the other devices
`export_to_shards` asked `store.contains(file_id)` and skipped anything the shard store had already heard of. So an image was exported exactly once, and re-indexing it updated the catalog and nothing else — every other device kept the first answer for ever. That is not hypothetical. This library was re-indexed after the detection floors changed and crops were added, going from 1,807 faces to 15,194; the shard store still held the original 1,807, written before any of it. Nothing the re-index produced could reach another device. The shard's `indexed` table now carries the catalog's own `indexed_at`, and the export compares against it. A re-indexed image goes again; an unchanged one still costs nothing. Copied from the catalog rather than stamped when the shard is written, because a shard-local write time advances even when nothing changed and could not answer the question. The column is nullable so a shard written before it still reads: absent means "cannot vouch for it", which forces one re-export and then settles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
b60f9d10d9 |
Merge: 32-pixel faces, a header that does not overlap, and set-aside that sticks
Build and test / Desktop (Linux) (push) Successful in 30m54s
Build and test / Layer separation (push) Successful in 44s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
Traceability / Requirement traces (push) Successful in 43s
Build and test / Android (aarch64) (push) Failing after 52m25s
The size floor comes down to 32 source pixels with the blur floor moved to match — they are coupled, since an upsampled face scores low on sharpness whatever its original quality, and dropping one without the other would have undone itself. The Identity header stops pinning its rows shorter than the controls in them, so the name field no longer draws through the buttons below it. And a group set aside now survives Regroup: its faces anchor the way confirmations do, so they stay where the user put them instead of regrouping into a fresh person with no ignore flag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
41daa4cca7 |
Keep a group set aside actually set aside
"Not interested" hid a group, and the next Regroup brought it straight back. Reclustering anchors the faces the user has ruled on so a pass cannot move them. It took *confirmations* as the only kind of ruling — but setting a group aside is a ruling too, and the faces it covers are only ever suggestions. So an ignored group's faces entered clustering loose, regrouped into a fresh person that carried no ignore flag, and reappeared in the rail. The original group was left behind holding nothing, hidden and empty. Anchoring them on `ignored` as well as on `confirmed` fixes it, and does one better: a face indexed later that matches a group which was set aside now merges *into* it, so a stranger photographed again stays set aside instead of arriving as somebody new. That is the case that would otherwise have made the feature feel like it only half worked. Three tests, and the first fails without the change — it reports the group coming back with its two faces while the original sits ignored and empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
c81d6865a7 |
Stop the name field running through the buttons under it
The Identity header was two rows pinned to 32px and 36px. Neither number was big enough for what the row held: a `Field` is `Theme.touch-target` — 44px — and a `Button` is `Theme.control-height`. Slint honours a child's own height and lets it overflow the box the layout gave it, so the name field drew 44px from the top of a 32px row while the button strip began at 38px. The overlap was 6px of text box sitting on top of "Confirm all". Neither row states a height any more. The first takes the height of what is in it, and the strip takes the height of a control — read from the theme rather than from the row inside it, since `actions` sizes itself from the Flickable's viewport and measuring it back would be a binding loop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
1d4c3348af |
Let a 32-pixel face count, and move the blur floor with it
64 source pixels was too strict: it threw away 70% of everything the detector
finds, and plenty of what it took were faces a person could name.
Lowering it is not a one-line change, because the two floors are coupled. A face
under 112 pixels is *upsampled* to reach the embedder and upsampling invents no
edges, so a small face scores low on sharpness however crisp the original was.
Re-measured over the reference library with `face_index --quality`:
min crop min sharp size cut blur cut kept
32 0.000 44% 0% 56%
32 0.002 44% 3% 53%
32 0.005 44% 8% 48%
32 0.010 44% 16% 40%
32 0.020 44% 27% 30%
64 0.020 70% 7% 23%
Holding the blur floor at 0.020 while dropping the size floor to 32 would have
rejected a further 27% — for being small rather than for being blurred — and
kept only 30%, barely more than the 23% the strict pair kept. Most of the point
of lowering the size floor would have gone straight back out through the other
gate.
0.005 removes 8% of what the size floor leaves, which is the same job 0.020 was
doing at 64 (7%): the large-but-soft face this gate exists for. Together they
now keep 48% of what the detector finds, against 23% before.
The box pre-filter follows down to 24, staying below what the real floor accepts
so it cannot reject a face that would have cleared 32.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|