2d958075424349b1e2d1af499fddc4f6b9f51aa8
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  dtourolleandClaude Opus 5
|
26a1eb7e28 |
Record that face detection has run, not just what it found
An image with no faces in it was indistinguishable from one that had never been looked at, so every landscape, still life and document scan in the library was re-detected on every pass, for ever. In a real library that is most of it: on the 23,527-image test library, 64 of the first 110 images indexed contain no face at all. Schema v9 adds face_index, a run marker per (image, model) carrying the face count and the proxy edge it read. Keyed on the model, so a model change puts every image back in the queue by itself. That makes a coverage figure possible, which is the thing a user actually wants to see. The audit also splits the outstanding set by whether a proxy exists, because 23,417 awaiting a proxy and 110 ready to index are different problems, and telling the user to run indexing again would not fix the first. The Identity screen gains Index faces, Stop, and the coverage line. examples/face_index.rs is the same check and sweep without a window, which is the right shape for an overnight pass. Measured on the real library in release: 3.5 images/second, 110 images and 125 faces in 30 seconds, and a second run correctly finds nothing left to do. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
aac3136407 |
Store faces and the people they belong to
Schema v8: people, faces, face_person, face_person_rejected, and the per-library calibration. Follows catalog.md 10.1 with two additions the spec work turned up. crop_px, because at the 1024px proxy tier a group shot reaches the embedder at ~50 source pixels upsampled to 112 and a portrait at 340. FR-CULL-9 names face size as an axis along which an uncalibrated similarity misbehaves, so it is a stored feature rather than a UI hint. face_person_rejected, because rejection is not the absence of an assignment. Without it the next clustering pass re-suggests exactly the face the user just pushed away, and the tool feels broken. record_detections replaces rather than appends, since DetectFaces is coalesced per image -- and carries confirmations across the replacement by box overlap, so re-indexing with a better model cannot discard the user's own labelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
6d6ef8d34b |
Page the grid along an index instead of sorting the library each time
Scrolling jittered, and this was the largest single reason. Every window the grid loads is `ORDER BY ... LIMIT n OFFSET k`, and neither half of that was being answered the cheap way. **The sort.** `GRID_ORDER` leads with `captured_at IS NULL`, so undated frames fall to the end. No ordinary index answers that — the leading term is an expression, not a column — so SQLite sorted the whole library into a temp b-tree on every window read, then threw away the first `k` rows of it. Schema V7 indexes the expression exactly as the query writes it, partial on the same `shadowed_by IS NULL AND trashed_at IS NULL` the grid filters by, so the read becomes a walk along the index. **The join.** `LEFT JOIN remote` was paged *after* it was joined, so reading 280 cells at offset 20,000 first seeked into `remote` for all 24,000 rows and then discarded 23,720 of them. The file ids are now fetched for the 280 rows that survived — the shape the badge and rating reads already use, one query for the window rather than one per cell. Measured together on 24,000 images at offset 20,000: **15.2 ms → 0.36 ms**, inside a scroll handler that has 16.7 ms to draw a frame. The test asserts on the query plan rather than on a duration, because there is no other symptom. A `GRID_ORDER` edited out of step with the index, or a column added back that drags `remote` in again, both still return exactly the right cells — just after sorting the library — and the jitter would come back with nothing to point at. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
2147eaa6a5 |
Put a keyword on a photograph, not only search for one
The catalog has been able to *find* by keyword since v1 — query.rs joins the keywords table, matches it exactly, and substring-matches it for free text — and nothing anywhere could ever put a word there. A user could filter to a keyword they had no way to apply. This is the missing half: create, rename, delete, list, assign, unassign, and the two reads a panel needs. Bulk-only for assignment, because keywording a selection is the common case rather than the exception — the photographer picks out the frames with the puffin in them and applies "puffin" once, in one transaction. Schema v6 adds `keyword_terms`, and deliberately does *not* touch the v1 join. The assignment keeps the word as text because the catalog is a rebuildable index and the durable copies of that fact — the sidecar, XMP dc:subject — both carry a string; a foreign key would mean a catalog rebuilt from sidecars had to invent identity rows before it could record anything, and would break the query path that already works. So the text is the fact, and the new table is only the identity a rename and a deletion can be keyed on. `keyword_terms.name` carries no unique index, which looks like an oversight and is not: two devices that each type "Iceland" are both right until they meet, and a constraint would abort the merge at that moment. Uniqueness is converged upon instead — create resolves an existing name, fuse_duplicates collapses a cross-device pair onto the smaller uuid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
03326242a1 |
Make the CI checks say what they mean, and format the workspace
The Android job's "Verify minimum API level" step has never verified the minimum API level. It took the first `*.so` anywhere under the target directory, which is a host proc-macro from debug/deps — an x86-64 object built by the runner's gcc, whose .comment section cannot mention Android and so can never contradict the expected value. It now reads the artifact under the target triple, compares against MIN_API parsed from the Dockerfile rather than a second copy of the number, and fails on a mismatch. Both sides are checked non-empty first: two failed parses would otherwise compare equal and pass, which is the same silent success in a new costume. The Android image installs one SDK package per layer and keeps the output. sdkmanager is a JVM program that aborts when it cannot get memory, and the single `> /dev/null` step reported that as a bare "exit code 134" while a retry re-downloaded everything that had already succeeded. tools/ci-local.sh runs all four jobs — desktop, android, layering, traceability — against the host toolchain, which is pinned to the same 1.92.0 CI installs. Its matrix check compares regeneration against the working tree rather than against HEAD: CI starts from a clean checkout, so git's answer is the right one there and reports every local run stale here. The rest is rustfmt across the workspace, and the clippy findings that surfaced once it did: manual_contains in dr-thumbs and collections_ui, a map iterated as pairs for its keys, an index loop over a slice, and two runtime assertions on a constant now made at compile time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolleandClaude Opus 5
|
fa12afed18 |
Keep originals on this device, by pin and by use
Fills in `image_cache`, which the previous commit's "On this device" filter read but nothing wrote. Also carries in-flight work that shared these files: the Android TLS root store, the settings page, and a regenerated traceability report. # Two populations, deliberately separate An original is kept here for one of two reasons, and conflating them produces the exact failure the feature exists to prevent. **Pinned** originals were asked for. Pinning a collection before a trip is a promise, so pinned rows are never evicted and never counted against the budget — a cap that could silently delete a pinned trip would make pinning worthless, because it could not be relied on without checking. **Passively cached** originals are a side effect of working: develop already downloads the whole file, so keeping it costs no bandwidth and saves the entire transfer next time. This population is what the budget bounds, evicted least-recently-used, because it otherwise grows until a day of culling fills a disk. Sharing one budget would let a large pin starve the passive cache, or let browsing evict a pin. They are separate. # What was built `dr_catalog::cache` owns the bookkeeping — held tier, size, last use, pinned — and writes the bytes; deciding to download stays with the caller, which is what keeps a crate with no network out of the network's business. Files are written to a temporary and renamed, so a dropped connection cannot leave a truncated file recorded as a complete original. They are named by image id, not filename: `Photos/IMG_0001.CR2` and `Trips/IMG_0001.CR2` are different photographs, and a flat cache keyed on the name would serve one for the other. `spawn_full_fetch` became read-through. A hit is a disk read; a miss stores what it downloads and enforces the budget. A cache that cannot be opened is a miss, not a failure to open the photograph. Pinning writes intent — `tier_desired` — without downloading, so the button responds immediately, and `spawn_pin_fetch` fills it in sequentially afterwards. Sequential because these are tens of megabytes each: the lanes that make the thumbnail sweep fast buy little against one connection's bandwidth and cost a great deal of memory. A pin interrupted by a lost connection resumes from where it stopped. Schema v5 adds `pinned` and `path`. `pinned` is a column rather than something inferred from `pinned_by_rule`, which is ON DELETE SET NULL and so cannot answer for an image whose rule was deleted. A v4 catalog migrates in place; existing rows default to unpinned, the safe direction. The budget and "keep opened originals" come from the settings page rather than a constant, and are applied at startup rather than only on change — a cache capped at 2 GB last session would otherwise spend this one filling to the default. Turning off keeping leaves what is already cached readable: those bytes are paid for, and refusing them would re-download images sitting right there, including pinned ones. Also removes a doubled `#[test]` introduced in the previous commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
dtourolle
|
d5b1f6bff5 |
Add collections, ratings, and soft delete to the catalog
Three features over a shared schema migration. Collections: a tree of manual collections plus smart collections whose membership *is* their stored selector. Dropping images onto a smart collection is refused rather than silently discarded, so the UI can say why the drop did nothing — member rows there would be a second source of truth that nothing reads. Ratings: the star and pick/reject axes, kept independent. Trash: soft delete to a folder, then permanent delete. Catalog::open now backfills after migrating. A migration adds a column but cannot know what the value should be for rows that already existed; backfilling on open is what stops those rows being silently partial. Timeline queries exclude shadowed JPEGs, which would otherwise double every paired shot in the histogram, and gain a range-bounded variant so zooming in returns finer buckets rather than the same coarse ones with the ends cropped. Assisted-by: LLM |
||
|
dtourolle
|
c8bb08e661 |
Add folder scan with format selection; validate A3 on a real library
Library setup as the user described it: pick a folder, choose which RAW
types to look for, scan recursively.
dr-types::FormatFilter the tick-box selection, seeing through VFS
placeholder suffixes so a dehydrated CR2 still
matches as a CR2
dr-sync::scan recursive walk, Depth:1 per directory, pruning
unchanged subtrees where the backend propagates
directory ETags
Verified against nextcloud.tourolle.paris (34.0.2) on a real library:
browse root 32 entries, 98ms
scan PhotosRaw 17,185 RAW files in 334 directories, 34.1s
(7,836 CR2 + 9,349 DNG)
range read 262KB of a 21.5MB DNG in 119ms — 1.22% of the file,
and enough to read "Canon EOS 6D | ISO 100"
That last line is assumption A3 validated on real data. Cataloguing this
library by whole-file fetch would move roughly 370GB; the range path
moves a few MB.
Pruning is capability-gated rather than assumed: with per-entry ETags a
probe costs a request and proves nothing about children, so it is skipped
entirely. A test asserts zero probes in that case.
Still unresolved: /core/preview returns 400 for every parameter
combination tried, including on a JPEG the server reports as having a
preview. Not a request-shape bug — it fails identically bare. Recorded
rather than worked around; ARCH §6.7 already treats server previews as
opportunistic, so nothing depends on it.
|