Document trash, collections, thumbnails, and the UI direction

Specs for the work that follows: soft delete via a MOVE that preserves the remote id, the collection tree and smart collections, the thumbnail store, and the derived-state folder. Adds two design documents. ui-refinement.md names the structural gaps between the v0.1 UI and something that feels like a photo editor. view-composition.md proposes a view controller for the display layer, against the 500-line run() that has become one by accretion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 20:36:23 +02:00
parent 5365123d92
commit 02b66ddfcf
6 changed files with 1335 additions and 61 deletions
@@ -372,12 +372,19 @@ pub enum Selector {
    IsoRange { min: u32, max: u32 },      // ⊕
    Availability(Availability),           // ⊕ "what can I edit right now"
    Text(String),                         // ⊕ filename/keyword substring
+    Person { id: PersonId, include_suggested: bool },  // ⊕ §10 (FR-CULL-11)
    All_(Vec<Selector>),
    Any(Vec<Selector>),
    Not(Box<Selector>),
 }
 ```

+`Person` carries `include_suggested` rather than defaulting silently. A saved collection built from
+confirmed faces must not quietly change membership because a later indexing pass guessed at another
+face; the user chose "photos of Anna", not "photos the model currently believes contain Anna". The
+default is `false`, and the interactive filter offers the looser form explicitly as a way to *find*
+faces to confirm.
+
 `Availability` as a selector earns its place: on a tablet the most useful filter is often "what do I
 actually have here", and it is also the natural thing to *pin* — "keep everything I've flagged that
 isn't already local".
@@ -402,6 +409,7 @@ pub enum JobKind {
    ContentHash,       // on demand only
    FetchPreview,      // remote range-extract (FR-NC-3)
    FetchOriginal,     // pinned or explicitly requested
+    DetectFaces,       // §10, on the proxy tier (FR-CULL-8)
 }
 ```

@@ -482,16 +490,117 @@ Server previews (`/core/preview`) are tried only where PROPFIND reported `nc:has
 verified stock Nextcloud ships no RAW preview provider, so for RAW this is nearly always absent — it
 is an opportunistic saving, never the mechanism.

-### 7.3 Storage
+### 7.3 Storage: sharded, shared, synced

-Thumbnails are content-addressed by `(content_hash | source_ref, size_class)` and stored as files
-under the platform cache directory, with the `cache` table holding the index. Files, not BLOBs:
-SQLite handles small blobs well but a 50k-image thumbnail cache is gigabytes, and mixing it into the
-catalog would bloat the file the app must open in under two seconds.
+Decided 2026-08-09, implemented in `dr-thumbs`. Thumbnails live in **sharded SQLite databases that
+sync to Nextcloud**, so a second device gets a full grid without re-fetching a byte of RAW.

-Two size classes at v1 — grid (256px) and filmstrip/loupe (1024px) — both long-edge, both JPEG. The
-cache is LRU-capped per NFR-RES-4, and thumbnails evict before proxies and long after sidecars,
-which never evict at all (FR-NC-6b).
+```text
+thumbs/
+  index.sqlite        fileid → shard, plus size accounting
+  shard-0000.sqlite   ≤ 25 MB, sealed
+  shard-0001.sqlite   ≤ 25 MB, active
+```
+
+**Why a thumbnail is worth syncing when the catalog mostly is not.** It is expensive to produce — a
+range fetch plus a decode, per image — and byte-identical for every client looking at the same file.
+This does not make it authoritative: losing the store costs regeneration and nothing else, so §6.12
+is untouched.
+
+**Why shards, and why small.** The 25 MB cap is about *sync granularity*, not SQLite's limits. One
+growing database means every client re-downloads all of it whenever a single thumbnail is added.
+With sequential fill only the newest shard is ever dirty, so an up-to-date client transfers one small
+file. Sealed shards are immutable, which makes them safe to cache forever and cheap to skip.
+
+At ~20 KB per 256px JPEG a shard holds roughly 1,200 thumbnails, so the 17,185-image reference
+library lands in ~14 shards.
+
+**Keyed on `oc:fileid`** — stable across server-side rename and move (FR-NC-5), and already in hand
+from PROPFIND. Accepted consequence: shards are account-scoped, so the same photograph on two
+servers is thumbnailed twice.
+
+**Stored as JPEG, not raw pixels.** A 256×170 RGBA buffer is ~174 KB against ~15 KB encoded. Since
+shards sync, that 11× is transfer cost paid by every client, not just disk.
+
+Three invariants, each tested:
+
+| Invariant | Why it matters |
+|---|---|
+| A sealed shard never reopens | Reopening one forces every client that holds it to re-download |
+| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
+| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |
+
+**Not yet built:** the transfer itself. `ThumbStore::shards()` reports which shards are sealed —
+the input a sync pass needs — and `merge_shard` adopts a downloaded one, but nothing uploads or
+downloads them yet. Until that lands the store is a local cache that happens to have the right shape.
+
+Two size classes remain planned — grid (256px) and filmstrip/loupe (1024px). Only the grid class is
+implemented. The cache is LRU-capped per NFR-RES-4, and thumbnails evict before proxies and long
+after sidecars, which never evict at all (FR-NC-6b).
+
+---
+
+## 7a. Editing collections
+
+Decided 2026-08-09. The schema for collections landed with §2 and the cross-device merge rules with
+§8; this is the layer between them — the operations a user actually performs, in
+`dr_catalog::collections`.
+
+### 7a.1 Hierarchy and membership are independent
+
+Two structures, deliberately not entangled:
+
+| | Mechanism | Meaning |
+|---|---|---|
+| Hierarchy | `collections.parent_id` | A collection inside a collection (Lightroom's "collection set"). A parent is an ordinary collection, not a separate kind, so a set can hold images of its own |
+| Membership | `collection_members` | An image is in as many collections as the user likes. Nothing moves on disk; no collection owns an image |
+
+**Adding an image to a child does not write a row for the parent.** A parent's contents are the union
+of its own members and its descendants', computed on read. Materialising it instead would make one
+add touch every ancestor, and a reparent rewrite membership — both of which §8's row-level merge
+would then have to reconcile. The read path pays a bounded tree walk instead, which at sidebar scale
+is nothing.
+
+The consequence the UI depends on: dragging images onto a collection is **additive**. It does not
+remove them from anywhere, which is why the gesture's default action is `copy` and not `move`.
+
+### 7a.2 Rules that exist to prevent silent damage
+
+| Rule | Why |
+|---|---|
+| Every mutation bumps `revision` | §8.4 resolves conflicts by revision. An edit that updates `modified` alone is invisible to the merge, so the *other* device silently wins and the user's work vanishes |
+| A no-op add does **not** bump it | Otherwise an idle device that re-dropped the same images outranks one that did real work |
+| Deleting a parent **promotes** its children | The schema's `ON DELETE CASCADE` would take the whole subtree. Losing a nested collection because its container was tidied away is not recoverable |
+| Deletion leaves a tombstone | Without it, merging with a device that still holds the collection resurrects it (§8.4) |
+| Cycles are refused at the write | Both kinds — parenting under a descendant, and a smart collection whose selector reaches itself. A cycle is unbounded recursion in the tree walk, so it must not be *representable*, not merely handled when drawn |
+| Tree walks are depth-guarded anyway | A merge can deliver a row this device never validated. The read path must terminate, so it truncates and logs rather than hanging the UI thread |
+| A drop onto a smart collection is refused | Its membership *is* its selector; member rows would be a second source of truth that nothing reads |
+| Deep counts are `count(DISTINCT image_id)` | An image in both a parent and a child is one photograph. A count that disagrees with the number of cells drawn makes both untrustworthy |
+
+`collections.uuid` is generated from the OS CSPRNG. A collision fuses two unrelated collections at
+the next merge, so the fallback path (used only if `/dev/urandom` cannot be read) logs loudly rather
+than degrading identity quality in silence.
+
+### 7a.3 Drag and drop is Slint's, not ours
+
+The first implementation hand-rolled the gesture on `TouchArea` — tracking the press, measuring
+travel to distinguish a click from a drag, and deciding the drop target from the last row hovered.
+**It did not work**, for a reason worth recording: an interactive `Flickable` claims any drag
+beginning inside it for scrolling and *cancels* the child `TouchArea`'s press, so the gesture could
+never leave the grid. It also had a correctness hole — a tree rebuilt mid-drag could redirect the
+drop, since a captured pointer is invisible to every other element.
+
+Slint 1.17's `DragArea`/`DropArea` own all of it: capture, the click-versus-drag threshold,
+arbitration against the `Flickable`, the image under the cursor, and hit-testing the release. What
+remains in `collections_ui` is only what Slint cannot know — the payload (which images, read from the
+selection when the drag starts) and the **spring**: a dwell timer that opens a collapsed collection
+so a nested child can be reached mid-drag, and closes again whatever the drag merely passed over.
+
+One hazard survives the change and is easy to reintroduce. Every consequence of a drop — rebuilding
+the tree, refreshing the badges, rereading the grid — *replaces a Slint model*, and doing that inside
+the `dropped` handler destroys the elements Slint is still using to deliver that event. So the drop
+records its target and `drag-finished` acts on it. This is the same hazard `sync_rows` in `lib.rs`
+documents for the adjust panel, and it presents as a control that works once and then goes dead.

 ---

@@ -578,7 +687,104 @@ because if the SQLite path proves troublesome, this is the fallback with a known

 ---

-## 9. Requirements touched
+## 10. People and faces
+
+Specified by FR-CULL-8 … FR-CULL-12, NFR-SEC-5, [architecture.md §6.4](architecture.md). Gated on
+spike S14 and decision D13 — the runtime and the model licences are unresolved, so this is the shape
+of the subsystem, not a build order.
+
+### 10.1 Schema (a v5 migration)
+
+```sql
+CREATE TABLE people (
+    id       INTEGER PRIMARY KEY,
+    uuid     TEXT NOT NULL UNIQUE,   -- merge identity, not the name (ARCH §6.3)
+    name     TEXT NOT NULL,
+    -- Tombstone-by-redirect. A merged person must outlive its merge, or a
+    -- device that still has it resurrects it — same hazard collections have.
+    merged_into INTEGER REFERENCES people(id) ON DELETE SET NULL,
+    created  INTEGER NOT NULL,
+    revision INTEGER NOT NULL DEFAULT 1,
+    modified INTEGER NOT NULL
+);
+
+CREATE TABLE faces (
+    id         INTEGER PRIMARY KEY,
+    image_id   INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
+    -- Normalised to the image's long edge, so a face survives the proxy it was
+    -- found on being regenerated at another resolution.
+    x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL,
+    landmarks  BLOB,                 -- 5 × (x, y) f32, the alignment input
+    detector_confidence REAL NOT NULL,
+    embedding  BLOB NOT NULL,        -- 512 × f16, L2-normalised
+    -- Which model produced this. An embedding is only comparable to others
+    -- from the same model; mixing them silently yields nonsense similarities.
+    model_id   TEXT NOT NULL,
+    detected_at INTEGER NOT NULL
+);
+CREATE INDEX faces_image ON faces(image_id);
+
+CREATE TABLE face_person (
+    face_id     INTEGER PRIMARY KEY REFERENCES faces(id) ON DELETE CASCADE,
+    person_id   INTEGER NOT NULL REFERENCES people(id) ON DELETE CASCADE,
+    -- Calibrated P(this face is this person), never a raw cosine (FR-CULL-9).
+    probability REAL NOT NULL,
+    -- The user said so. Never overwritten by a later inference pass.
+    confirmed   INTEGER NOT NULL DEFAULT 0
+);
+CREATE INDEX face_person_person ON face_person(person_id, confirmed);
+```
+
+Three things in that schema are load-bearing:
+
+**`model_id` on every face.** Embeddings from different models are not comparable — this is the one
+mistake that produces plausible-looking garbage rather than an error. Storing the model with the
+embedding means a model change is detectable and re-indexable, instead of quietly poisoning every
+similarity in the library.
+
+**Normalised bounding boxes.** Detection runs on whichever proxy exists (FR-CULL-8). Storing pixel
+coordinates would bind a face to a resolution that the cache is entitled to evict and regenerate
+differently.
+
+**`confirmed` as a column, not a probability of 1.0.** A confirmation is a different kind of fact
+from a confident guess, and collapsing them loses the ability to recompute suggestions without
+touching user data.
+
+### 10.2 Why clustering is not a job kind
+
+Detection is per-image and parallel, so it is a job (`DetectFaces`, coalesced per image like any
+other). Clustering is a *whole-library* operation over the embeddings detection produced — it has no
+natural `subject_id`, and running it per-image would rebuild the world on every photograph.
+
+It therefore runs as a debounced library-level pass, triggered when detection has been idle and the
+face count has moved materially since the last clustering. The same reasoning as sidecar writes: the
+work is cheap to defer, expensive to repeat, and nobody is waiting on it.
+
+### 10.3 The calibration lives with the library
+
+FR-CULL-9 requires similarity to be a calibrated probability, fitted from this library's own faces.
+That fit is a property of the catalog and its model, so it is stored alongside — a small table
+holding the fit parameters, its validity flag, and a hash of the face set it was derived from, so a
+materially changed library recomputes rather than trusting a stale fit.
+
+When the fit is not valid — a library with too few faces to have positive pairs — the UI says the
+confidence is unavailable. It does not fall back to an untuned default dressed up as a measurement.
+
+### 10.4 What this does not settle
+
+- **Which model, and which runtime.** D13. Everything above holds regardless of the answer, which is
+  why it is specified in terms of "a 512-d embedding from a stated model" rather than a named one.
+- **The clustering algorithm.** Density-based over the calibrated distance is the obvious starting
+  point, but the parameters are an S14 question, not a design-time one.
+- **Whether embeddings sync.** NFR-SEC-5 permits it, opt-in. The shard mechanism in §7.3 is the
+  obvious carrier if they do, but nothing here depends on that decision.
+- **Faces in trashed images.** FR-CAT-15's trash moves files; whether their faces stay indexed and
+  keep contributing to clusters is unspecified. Probably they should be excluded from suggestions but
+  not deleted, so a restore does not re-index.
+
+---
+
+## 11. Requirements touched

 | ID | How this document addresses it |
 |---|---|
@@ -587,7 +793,7 @@ because if the SQLite path proves troublesome, this is the fallback with a known
 | FR-CAT-4 | §4.1 windowed queries, memory independent of catalog size |
 | FR-CAT-5 | §3.5 two-pass metadata |
 | FR-CAT-6 | §4.3 indexed filter compilation, §5 selectors |
-| FR-CAT-7 | §2 collections schema, §5 manual and smart |
+| FR-CAT-7 | §2 collections schema, §5 manual and smart, §7a hierarchy, membership and editing |
 | FR-CAT-9 | §3.4 the offline/deleted distinction and the sweep guard |
 | FR-CAT-11 | §3.5 lazy content hashing |
 | FR-NC-3 | §7.2 range-extract for remote thumbnails |
@@ -598,3 +804,9 @@ because if the SQLite path proves troublesome, this is the fallback with a known
 | NFR-ARCH-2 | §6.3 priority classes shared with the GPU scheduler |
 | NFR-ARCH-3 | §4.3 query cancellation, §6 job cancellation |
 | NFR-RES-4 | §7.3 LRU cap, eviction order |
+| FR-CULL-8 | §10.1 `faces` schema, §6.1 `DetectFaces` job kind on the proxy tier |
+| FR-CULL-9 | §10.3 per-library calibration, stored with its validity and source hash |
+| FR-CULL-10 | §10.1 `people` and `face_person`, merge-by-redirect, §10.2 clustering as a library pass |
+| FR-CULL-11 | §5 `Selector::Person`, confirmed-only by default |
+| FR-CULL-12 | §10.1 derived data in the catalog; names to the sidecar, UUID as merge identity |
+| NFR-SEC-5 | §10.4 sync left undecided and off; nothing in §10 emits an embedding |