# DarkRoom — Catalog, library view, and background work

**Status:** Draft v0.1 · 2026-08-09
**Companion to:** [requirements.md](requirements.md), [architecture.md](architecture.md)

Specifies `dr-catalog`: the index the library view queries, how it stays current without rescanning
everything, and how thumbnails get made. [architecture.md §6.2](architecture.md) sketches the schema
in eight lines; this expands it to the point of implementability and fills the two gaps that sketch
leaves open — **incremental local scan** and **the job queue**.

Sync's remote side is already designed ([architecture.md §8](architecture.md)): ETag pruning turns a
no-op sync of 50k images into one request. Nothing equivalent existed for a local root, which is the
central problem this document solves.

---

## 1. What this must not do

Stated first because every design choice below follows from it.

| Must not | Why |
|---|---|
| Stat 50k files to open the catalog | NFR-P1: catalog open < 2 s desktop, < 4 s Android. SAF `DocumentsContract` queries are far slower than `stat` (spike S10). |
| Re-derive thumbnails for unchanged images | NFR-P3 throughput is for *new* work; redoing it on every connect makes first paint unbounded. |
| Fetch previews for remote images nobody looks at | A 50k remote library at 1–3 MB per range-extract is 50–150 GB. FR-NC-6 forbids bulk transfer by default. |
| Evaluate cache rules per grid cell | ARCH §9.5 already answers this: `tier_desired` is materialised. |
| Block the UI executor on any of it | NFR-P9, NFR-ARCH-1. |

The unifying principle: **work is proportional to what changed, or to what the user is looking at —
never to library size.**

---

## 2. Schema

Extends [architecture.md §6.2](architecture.md). Additions beyond that sketch are marked ⊕.

```sql
-- Roots -----------------------------------------------------------------
roots(
  id INTEGER PRIMARY KEY,
  kind TEXT,                    -- 'local' | 'saf' | 'remote'
  grant_blob BLOB,              -- SAF persisted permission; NULL on Linux
  label TEXT,
  last_seen INTEGER,
  scan_generation INTEGER       -- ⊕ bumped per completed scan; see §3.4
);

-- Folders: the unit of change detection, local and remote alike ---------
folders(
  id INTEGER PRIMARY KEY,
  root_id INTEGER NOT NULL REFERENCES roots(id),
  parent_id INTEGER REFERENCES folders(id),
  path TEXT NOT NULL,
  etag TEXT,                    -- remote: propagating ETag (ARCH §8.4)
  mtime INTEGER,                -- ⊕ local: directory mtime
  entry_count INTEGER,          -- ⊕ local: direct children, mtime's blind spot
  scanned_generation INTEGER,   -- ⊕ deletion sweep; see §3.4
  UNIQUE(root_id, path)
);

-- Images ----------------------------------------------------------------
images(
  id INTEGER PRIMARY KEY,
  root_id INTEGER NOT NULL REFERENCES roots(id),
  folder_id INTEGER REFERENCES folders(id),     -- ⊕ folder filter without LIKE
  source_ref TEXT NOT NULL,
  content_hash TEXT,            -- NULL until hashed; see §3.5
  format TEXT,
  w INTEGER, h INTEGER,
  captured_at INTEGER,          -- UTC seconds; NULL if EXIF absent
  captured_offset INTEGER,      -- ⊕ minutes east of UTC; see §4.2
  camera TEXT, lens TEXT,
  iso INTEGER, aperture REAL, shutter REAL,
  availability INTEGER,
  file_size INTEGER,            -- ⊕ cheap change signal alongside mtime
  file_mtime INTEGER,           -- ⊕
  metadata_state INTEGER,       -- ⊕ 0=none 1=stat-only 2=full EXIF; §3.5
  sidecar_mtime INTEGER,
  UNIQUE(root_id, source_ref)
);

-- Versions, keywords, remote, cache: per ARCH §6.2, unchanged -----------

-- Collections ⊕ ---------------------------------------------------------
collections(
  id INTEGER PRIMARY KEY,
  name TEXT NOT NULL,
  parent_id INTEGER REFERENCES collections(id),  -- collection sets
  kind INTEGER NOT NULL,        -- 0 = manual, 1 = smart
  selector_json TEXT,           -- smart only; the §5 Selector
  created INTEGER
);

collection_members(
  collection_id INTEGER NOT NULL REFERENCES collections(id) ON DELETE CASCADE,
  image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
  position INTEGER,             -- manual ordering; NULL = by capture time
  PRIMARY KEY(collection_id, image_id)
);

-- Jobs ⊕ ----------------------------------------------------------------
jobs(
  id INTEGER PRIMARY KEY,
  kind INTEGER NOT NULL,
  subject_id INTEGER,           -- image or folder, per kind
  priority INTEGER NOT NULL,
  state INTEGER NOT NULL,       -- 0=pending 1=running 2=failed
  attempts INTEGER NOT NULL DEFAULT 0,
  not_before INTEGER,           -- retry backoff
  payload TEXT,
  UNIQUE(kind, subject_id)      -- coalescing; see §6.2
);
```

Indices that exist for a stated query, not speculatively:

```sql
CREATE INDEX images_captured   ON images(captured_at);        -- §4 timeline
CREATE INDEX images_folder     ON images(folder_id);
CREATE INDEX images_hash       ON images(content_hash) WHERE content_hash IS NOT NULL;
CREATE INDEX folders_parent    ON folders(parent_id);
CREATE INDEX jobs_ready        ON jobs(state, priority DESC, not_before);
CREATE INDEX versions_image    ON versions(image_id);
CREATE INDEX members_image     ON collection_members(image_id);
```

`content_hash` is indexed *partially*. It is NULL for most rows most of the time (§3.5), and a
partial index over the non-NULL subset is both smaller and what FR-CAT-9's reconnection-by-hash
and FR-CAT-11's duplicate detection actually query.

---

## 3. Incremental scan

### 3.1 The local analogue of ETag pruning

Nextcloud propagates ETags up the tree, so one request proves a whole library unchanged
([architecture.md §8.4](architecture.md)). A filesystem offers no such guarantee — a directory's
mtime changes when its *direct* entries change, and not when a grandchild does. There is no
cheap "did anything below here change" probe.

So local scan prunes at each level rather than at the root:

```
scan(folder):
    (mtime, count) = stat(folder)
    if (mtime, count) == stored:
        # This directory's own entries are unchanged. Its files need no
        # examination at all — but subdirectories may still have changed
        # internally, so recurse into known children without listing.
        for child in stored_children(folder):
            scan(child)
    else:
        entries = list(folder)                  # the expensive call
        reconcile(folder, entries)              # §3.3
        for child in entries.dirs: scan(child)
    mark scanned(folder, current_generation)
```

Cost is **one `stat` per directory** when nothing changed, versus one per *file*. A 50k-image
library in ~2k folders costs 2k stats — a few milliseconds locally, and the difference between
meeting and missing NFR-P1 on SAF.

The recursion into unchanged directories is not redundant: it is what makes a change to one deep
file detectable at all, given no upward propagation. What it avoids is the *listing* — on SAF a
`DocumentsContract` query returning 200 rows costs far more than a metadata probe on the directory
itself.

### 3.2 Why entry-count as well as mtime

Directory mtime alone misses a real case: delete one file and create another within the same
timestamp granularity, and mtime can be unchanged while contents differ. Some filesystems and most
SAF providers report coarse timestamps, which widens the window.

Storing `(mtime, entry_count)` closes the common form of this — a paired add and remove changes
neither, but that is rarer than a bare add or remove, and both of those move the count. It is a
cheap narrowing, not a proof.

**Where correctness must not depend on it,** the user gets an explicit *Rescan folder* action
(FR-CAT-1), and reconnection matches by content hash (FR-CAT-9). Sync's remote path is unaffected —
ETags are authoritative there.

### 3.3 Reconciling a changed directory

For each entry in a listing:

| Situation | Action |
|---|---|
| Not in catalog | Insert with `metadata_state = 1`; enqueue `ExtractMetadata` |
| In catalog, `(size, mtime)` match | Nothing — the common case |
| In catalog, `(size, mtime)` differ | Re-enqueue `ExtractMetadata` and `Thumbnail`; clear `content_hash` |
| In catalog, absent from listing | Deletion candidate — §3.4 |
| Placeholder (`*.nextcloud`) | Catalogue as the image it stands for; `Availability::Offline` (ARCH §9.0) |

Sidecars are examined in the same pass: a `.drsc` whose mtime exceeds `images.sidecar_mtime` enqueues
a `ReadSidecar` job. This is how an edit made on another device — landed by the Nextcloud client,
not by us — reaches the catalog.

### 3.4 Deletion without a full sweep

A file removed outside the app appears only as an *absence*, which a pruned scan cannot see: the
folder it vanished from has a changed mtime and is listed, but a folder never visited is never
compared.

Generation counting handles this without a full pass. Each scan bumps `roots.scan_generation`, and
every folder reached — whether listed or skipped — records it. After the walk:

```sql
-- Folders never reached: their parent no longer lists them.
DELETE FROM folders
 WHERE root_id = ?1 AND scanned_generation < ?2;
```

Images under a deleted folder cascade. Images missing from a *listed* folder are caught directly in
§3.3. Together these cover deletion with no additional traversal.

Deletion here means **removing the catalog row for a source proven absent**, which FR-CAT-9 sharply
distinguishes from a source merely unreachable. A root that fails to open at all — unplugged drive,
revoked SAF grant — aborts the scan and marks the root offline. It never runs the sweep, because
every folder would look unreached and the sweep would delete the entire library.

That guard is the single most dangerous line in this design, and it is stated as an invariant:
**the deletion sweep runs only after a scan that completed without a root-level access error.**

### 3.5 Metadata in two passes

Full EXIF extraction requires opening and parsing each file. At 50k images that is minutes, and it
must not stand between the user and a usable grid.

`metadata_state` records how far each image has got:

| State | Holds | Cost |
|---|---|---|
| 0 — none | Row exists, nothing read | — |
| 1 — stat-only | Name, size, mtime, format from extension | Free, from the listing |
| 2 — full | EXIF: capture time, camera, lens, exposure, dimensions | One open + parse |

The grid is usable at state 1: it can show filenames, sort by filename or file mtime, and display
placeholder cells. Promotion to state 2 runs as background jobs, prioritised by what is on screen
(§6.3), so visible images get real capture times within a frame or two of being scrolled to.

**Capture-time filtering (§4) needs state 2**, so a freshly scanned library's timeline is incomplete
until the pass finishes. The UI states this plainly — a progress affordance on the timeline, not a
silently wrong filter. Which is the FR-NC-6c principle applied to metadata rather than pixels: say
what you actually have.

`content_hash` is a *third*, still lazier tier. It requires reading the whole file, so it is computed
only when something needs it: import duplicate detection (FR-CAT-11), or reconnecting a moved source
(FR-CAT-9). Never during a routine scan.

---

## 4. The library view

### 4.1 Query model

The UI never assembles SQL. It hands the catalog a `Query` and receives a stable, windowable result:

```rust
pub struct Query {
    pub filter: Selector,        // §5 — same type cache rules use
    pub sort: Sort,
    pub descending: bool,
}

pub enum Sort {
    CapturedAt,
    Added,
    FileName,
    Rating,
    /// Manual order within a collection; falls back to CapturedAt elsewhere.
    CollectionPosition,
}
```

Results are fetched by window, never wholesale — FR-CAT-4 requires memory bounded independently of
catalog size:

```rust
impl Catalog {
    fn count(&self, q: &Query) -> Result<usize, CatalogError>;
    fn window(&self, q: &Query, range: Range<usize>) -> Result<Vec<GridRow>, CatalogError>;
}
```

`GridRow` carries exactly what a cell draws — id, thumbnail key, availability, rating, flag, capture
time — and nothing that would require a join per cell. Availability badges read `tier_desired`
directly (ARCH §9.5), so no rule evaluation happens on the render path.

A `LIMIT/OFFSET` window degrades at high offsets, since SQLite must walk the skipped rows. Scrolling
is overwhelmingly *sequential*, so the catalog keeps a keyset cursor for forward and backward paging
and falls back to OFFSET only for a scrollbar jump. Jumps are rare and single; scrolling is
continuous.

### 4.2 Time

Capture time is the spine of a photo library, and it has one persistent trap: **a photograph's
timestamp is local to where it was taken.** Store UTC alone and a shoot that ran 09:00–17:00 in
Tokyo displays as spanning two days in Paris. Store local time alone and ordering across a timezone
change is wrong.

So both: `captured_at` in UTC for ordering, `captured_offset` in minutes for display and for
day-bucketing. EXIF `OffsetTimeOriginal` supplies it where present; where absent — common on older
bodies — the offset is NULL and the catalog falls back to the library's configured display timezone,
flagged so the UI can show it as inferred.

Day, month, and year buckets are computed against **local** time. "Everything from 3 August" means
the photographer's 3 August.

The timeline affordance is a histogram of counts per bucket, which the grid uses for scrubbing:

```rust
pub enum Granularity { Year, Month, Day, Hour }

pub struct TimeBucket {
    pub start: i64,       // UTC seconds, bucket start
    pub count: u32,
}

fn timeline(&self, q: &Query, g: Granularity) -> Result<Vec<TimeBucket>, CatalogError>;
```

This is one grouped aggregate over the `images_captured` index, not 50k rows into the UI. It is what
makes "drag across two years to find the trip" work, and it is the cheapest useful thing a library
view can offer over a flat grid.

### 4.3 Filtering interactively

FR-CAT-6 requires filter results to update interactively on 50k images. Three things make that hold:

1. **Filters compile to indexed predicates.** A `Selector` becomes a WHERE clause over indexed
   columns. Keyword and collection membership become `EXISTS` subqueries against their own indices.
2. **Count and first window are one round trip.** The grid needs a row count to size its scrollbar
   and the first screenful to paint; the catalog returns both together.
3. **A filter change cancels the one in flight.** Typing in a search box issues a query per
   keystroke; each supersedes the last (NFR-ARCH-3). Without this the UI queues work it will discard.

---

## 5. Selectors: one type, three uses

[architecture.md §9.2](architecture.md) defines `Selector` for cache rules. The same type expresses
library filters and smart collections. This is deliberate and worth stating as a design decision,
because three near-identical predicate languages is a classic way for a catalog to rot.

| Use | Meaning |
|---|---|
| Library filter | What the grid shows now |
| Smart collection | A saved, named filter (FR-CAT-7) |
| Cache rule | What is kept locally, at which tier (FR-NC-6a) |

One consequence is directly useful: any filter the user has narrowed to can be saved as a smart
collection, and any collection can be pinned offline, with no conversion step. "Show me 5-star images
from the last 90 days" → save as a collection → pin it for the trip. Three features, one mechanism.

`Selector` moves to `dr-types` so `dr-catalog` and `dr-sync` share it without either depending on the
other. It gains variants the cache-rule sketch did not need:

```rust
pub enum Selector {
    All,                                  // ⊕ the empty filter
    Collection(CollectionId),
    Folder { root: RootId, path: String, recursive: bool },
    DateRange(DateSelector),
    Rating { min: u8 },
    Label(ColourLabel),
    Flag(FlagState),
    Keyword(String),
    Camera(String),                       // ⊕ FR-CAT-6 indexed field
    Lens(String),                         // ⊕
    IsoRange { min: u32, max: u32 },      // ⊕
    Availability(Availability),           // ⊕ "what can I edit right now"
    Text(String),                         // ⊕ filename/keyword substring
    Person { id: PersonId, include_suggested: bool },  // ⊕ §10 (FR-CULL-11)
    All_(Vec<Selector>),
    Any(Vec<Selector>),
    Not(Box<Selector>),
}
```

`Person` carries `include_suggested` rather than defaulting silently. A saved collection built from
confirmed faces must not quietly change membership because a later indexing pass guessed at another
face; the user chose "photos of Anna", not "photos the model currently believes contain Anna". The
default is `false`, and the interactive filter offers the looser form explicitly as a way to *find*
faces to confirm.

`Availability` as a selector earns its place: on a tablet the most useful filter is often "what do I
actually have here", and it is also the natural thing to *pin* — "keep everything I've flagged that
isn't already local".

Compilation is a straightforward recursive walk producing SQL with bound parameters. **Nothing
user-supplied is ever interpolated into SQL text.** `Text` becomes a bound `LIKE` pattern with `%`,
`_`, and the escape character escaped.

---

## 6. Background work

### 6.1 Job kinds

```rust
pub enum JobKind {
    ScanFolder,        // §3, recursive from a folder
    ExtractMetadata,   // state 1 → 2
    Thumbnail,         // §7
    ReadSidecar,       // external sidecar change detected
    WriteSidecar,      // local edit → disk, debounced (ARCH §6.1)
    ContentHash,       // on demand only
    FetchPreview,      // remote range-extract (FR-NC-3)
    FetchOriginal,     // pinned or explicitly requested
    DetectFaces,       // §10, on the proxy tier (FR-CULL-8)
}
```

### 6.2 Coalescing is the point

`UNIQUE(kind, subject_id)` on `jobs` means enqueueing is idempotent: an image touched five times
during a scan has one thumbnail job, not five. Enqueue is
`INSERT … ON CONFLICT DO UPDATE SET priority = max(priority, excluded.priority)`, so a re-request at
higher priority promotes the existing row rather than duplicating it.

This is what makes "regenerate on update" safe to call liberally. Every code path that notices a
change can just enqueue; the table absorbs the redundancy.

### 6.3 Priority

Reuses the existing GPU scheduler classes ([architecture.md §5.3](architecture.md)) so one notion of
priority governs the whole app:

| Class | Jobs | Preempts |
|---|---|---|
| `Interactive` | Metadata and thumbnails for visible cells; preview for the open image | everything |
| `Prefetch` | The scroll margin; next image in culling | Background |
| `Background` | Bulk metadata, rule-driven fetches, hashing | — |

Visible-cell work is enqueued by the grid as it scrolls, at `Interactive`. The effect is that a
freshly scanned library fills in *where the user is looking* first, and grinds through the rest
behind them.

### 6.4 Durability and failure

Jobs live in the catalog, so they survive process death — which on Android is routine, not
exceptional (FR-PLAT-AND-3). On startup, rows in state `running` revert to `pending`: the process
that owned them is gone.

Failures increment `attempts` and set `not_before` to an exponential backoff. After a bounded retry
count the job is marked failed and attached to its image as a typed error (NFR-ARCH-4) — one
corrupt file does not stall the queue, and the user can see which files failed and why.

**A job runner never touches the UI executor**, and `Interactive` work runs on the decode pool with
the I/O pool behind it (ARCH §7.1).

---

## 7. Thumbnails

### 7.1 When

Not "on first connect" as a bulk operation. Thumbnails are generated:

- **On demand**, for cells entering the viewport plus the prefetch margin — at `Interactive`
- **On change**, when §3.3 sees a differing `(size, mtime)`
- **On rule**, for images a cache rule pins at `Preview` or above — at `Background`
- **Never** for a remote image nobody has looked at and no rule covers

For a local library this converges on "everything, eventually", because scrolling reaches everything
and the background pass has nothing else to do. For a remote library it converges on "what you
actually browsed".

**Measured on a real 17,185-RAW library, 2026-08-09:** cataloguing it by whole-file fetch would move
roughly **370 GB**; the range-extract path moves a few MB for the images actually viewed. This is
the single largest cost difference in the design, and it is why §7.1 is a list of narrow triggers
rather than "generate them all on connect".

### 7.2 How, by availability

| Availability | Source | Cost |
|---|---|---|
| `Original`, local | Embedded JPEG via `dr-decode` preview path | ~200 KB read, no demosaic |
| `Original`, no embedded preview | Full decode, downscale | Expensive — `Background` only |
| Remote | Range-extract embedded JPEG (FR-NC-3) | 1–3 MB vs 25–100 MB — **measured: 262 KB of a 21.5 MB DNG, 119 ms, 1.22% of the file** |
| Placeholder / `Offline` | None — render the offline affordance | 0 |

The remote path deliberately does **not** ask the Nextcloud client to hydrate the file. ARCH §9.0
established hydration is whole-file, so it costs ~100× what the range extract does. Hydration stays
reserved for the original tier, where the user has asked for the actual image.

Server previews (`/core/preview`) are tried only where PROPFIND reported `nc:has-preview`. ARCH §6.7
verified stock Nextcloud ships no RAW preview provider, so for RAW this is nearly always absent — it
is an opportunistic saving, never the mechanism.

### 7.3 Storage: sharded, shared, synced

Decided 2026-08-09, implemented in `dr-thumbs`. Thumbnails live in **sharded SQLite databases that
sync to Nextcloud**, so a second device gets a full grid without re-fetching a byte of RAW.

```text
thumbs/
  index.sqlite        fileid → shard, size accounting, client id, adoption ledger
  shard-0000.sqlite   ≤ 25 MB, sealed
  shard-0001.sqlite   ≤ 25 MB, active
```

**Why a thumbnail is worth syncing when the catalog mostly is not.** It is expensive to produce — a
range fetch plus a decode, per image — and byte-identical for every client looking at the same file.
This does not make it authoritative: losing the store costs regeneration and nothing else, so §6.12
is untouched.

**Why shards, and why small.** The 25 MB cap is about *sync granularity*, not SQLite's limits. One
growing database means every client re-downloads all of it whenever a single thumbnail is added.
With sequential fill only the newest shard is ever dirty, so an up-to-date client transfers one small
file. Sealed shards are immutable, which makes them safe to cache forever and cheap to skip.

At ~20 KB per 256px JPEG a shard holds roughly 1,200 thumbnails, so the 17,185-image reference
library lands in ~14 shards.

**Keyed on `oc:fileid`** — stable across server-side rename and move (FR-NC-5), and already in hand
from PROPFIND. Accepted consequence: shards are account-scoped, so the same photograph on two
servers is thumbnailed twice.

**Stored as JPEG, not raw pixels.** A 256×170 RGBA buffer is ~174 KB against ~15 KB encoded. Since
shards sync, that 11× is transfer cost paid by every client, not just disk.

Three invariants, each tested:

| Invariant | Why it matters |
|---|---|
| A sealed shard never reopens | Reopening one forces every client that holds it to re-download |
| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |

**The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the
library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass
is one listing plus whichever shard is still open.

**Why a remote name carries a client id.** Corrected 2026-08-16. Shard ids are *per store* — every
client fills its own numbering from 0 — so the flat `shard-NNNN.sqlite` namespace the transfer first
used had two clients writing one name. Two failures followed from it, and both were live: the second
client's upload **overwrote** content the first still believed was published, and no client could
distinguish a peer's shard 3 from its own, so the only safe reading of "I already hold 3" was to skip
it. Between them, two populated clients exchanged almost nothing — only shards numbered above the
other's highest. A fresh device worked, which is why it went unnoticed: with no local shards there is
nothing to collide with.

The name is now `shard-<client>-NNNN.sqlite`, where `<client>` is minted per store in `index.sqlite`
beside the numbering it qualifies — a store deleted and rebuilt restarts at shard 0 and must not
claim its predecessor's names. Since a client's own ids no longer say anything about what it has
taken from others, `index.sqlite` also keeps an **adoption ledger** of merged remote names and the
size each had. Size, not a flag: a peer's sealed shard never returns, but its open one grows, and
re-merging the grown copy is how the thumbnails it gained arrive.

Flat names left on servers by earlier builds are still read — they report no owner, so each client
adopts them once — and nothing is written under that form again. A flat name whose id and byte size
match a local shard is that client's own earlier upload by the same identity argument used for
sealed shards, so the rename does not cost every client a re-download of its whole store. Older
builds ignore the new names, so they stop receiving shards until updated; nothing is lost, since
their own uploads are still adopted.

Two size classes remain planned — grid (256px) and filmstrip/loupe (1024px). Only the grid class is
implemented. The cache is LRU-capped per NFR-RES-4, and thumbnails evict before proxies and long
after sidecars, which never evict at all (FR-NC-6b).

---

## 7a. Editing collections

Decided 2026-08-09. The schema for collections landed with §2 and the cross-device merge rules with
§8; this is the layer between them — the operations a user actually performs, in
`dr_catalog::collections`.

### 7a.1 Hierarchy and membership are independent

Two structures, deliberately not entangled:

| | Mechanism | Meaning |
|---|---|---|
| Hierarchy | `collections.parent_id` | A collection inside a collection (Lightroom's "collection set"). A parent is an ordinary collection, not a separate kind, so a set can hold images of its own |
| Membership | `collection_members` | An image is in as many collections as the user likes. Nothing moves on disk; no collection owns an image |

**Adding an image to a child does not write a row for the parent.** A parent's contents are the union
of its own members and its descendants', computed on read. Materialising it instead would make one
add touch every ancestor, and a reparent rewrite membership — both of which §8's row-level merge
would then have to reconcile. The read path pays a bounded tree walk instead, which at sidebar scale
is nothing.

The consequence the UI depends on: dragging images onto a collection is **additive**. It does not
remove them from anywhere, which is why the gesture's default action is `copy` and not `move`.

### 7a.2 Rules that exist to prevent silent damage

| Rule | Why |
|---|---|
| Every mutation bumps `revision` | §8.4 resolves conflicts by revision. An edit that updates `modified` alone is invisible to the merge, so the *other* device silently wins and the user's work vanishes |
| A no-op add does **not** bump it | Otherwise an idle device that re-dropped the same images outranks one that did real work |
| Deleting a parent **promotes** its children | The schema's `ON DELETE CASCADE` would take the whole subtree. Losing a nested collection because its container was tidied away is not recoverable |
| Deletion leaves a tombstone | Without it, merging with a device that still holds the collection resurrects it (§8.4) |
| Cycles are refused at the write | Both kinds — parenting under a descendant, and a smart collection whose selector reaches itself. A cycle is unbounded recursion in the tree walk, so it must not be *representable*, not merely handled when drawn |
| Tree walks are depth-guarded anyway | A merge can deliver a row this device never validated. The read path must terminate, so it truncates and logs rather than hanging the UI thread |
| A drop onto a smart collection is refused | Its membership *is* its selector; member rows would be a second source of truth that nothing reads |
| Deep counts are `count(DISTINCT image_id)` | An image in both a parent and a child is one photograph. A count that disagrees with the number of cells drawn makes both untrustworthy |

`collections.uuid` is generated from the OS CSPRNG. A collision fuses two unrelated collections at
the next merge, so the fallback path (used only if `/dev/urandom` cannot be read) logs loudly rather
than degrading identity quality in silence.

### 7a.3 Drag and drop is Slint's, not ours

The first implementation hand-rolled the gesture on `TouchArea` — tracking the press, measuring
travel to distinguish a click from a drag, and deciding the drop target from the last row hovered.
**It did not work**, for a reason worth recording: an interactive `Flickable` claims any drag
beginning inside it for scrolling and *cancels* the child `TouchArea`'s press, so the gesture could
never leave the grid. It also had a correctness hole — a tree rebuilt mid-drag could redirect the
drop, since a captured pointer is invisible to every other element.

Slint 1.17's `DragArea`/`DropArea` own all of it: capture, the click-versus-drag threshold,
arbitration against the `Flickable`, the image under the cursor, and hit-testing the release. What
remains in `collections_ui` is only what Slint cannot know — the payload (which images, read from the
selection when the drag starts) and the **spring**: a dwell timer that opens a collapsed collection
so a nested child can be reached mid-drag, and closes again whatever the drag merely passed over.

One hazard survives the change and is easy to reintroduce. Every consequence of a drop — rebuilding
the tree, refreshing the badges, rereading the grid — *replaces a Slint model*, and doing that inside
the `dropped` handler destroys the elements Slint is still using to deliver that event. So the drop
records its target and `drag-finished` acts on it. This is the same hazard `sync_rows` in `lib.rs`
documents for the adjust panel, and it presents as a control that works once and then goes dead.

---

## 8. Syncing the catalog file

Decided 2026-08-09. **This qualifies [architecture.md §6.12](architecture.md)** — the catalog
remains a rebuildable index, but the file itself now travels to Nextcloud. The qualification is
worth stating precisely, because the sidecar-authoritative model is load-bearing and this is the
one place it bends.

### 8.1 Why collections forced this

Every other thing the catalog holds has authoritative backing outside it. Ratings, labels,
keywords, and edit graphs live in sidecars next to the images, so a rebuild recovers them.
**Collections do not.** A manual collection is a set of images the user assembled by hand; nothing
in the filesystem records it. Losing the catalog loses them, and no rescan brings them back.

So collections need to be durable across devices somehow. Syncing the catalog file is the chosen
mechanism.

### 8.2 What the file sync does and does not carry

Only **collections and their membership** merge. The rest of a catalog describes *local* state —
folder mtimes, cache file paths, job rows, `tier_actual` — and importing another device's version
of those would be actively wrong. The downloaded remote is read for its collections and discarded.

This is what keeps §6.12 substantially intact: nothing here makes the local database authoritative
for anything a rebuild could not recover. The catalog is still deletable. What syncs is one table
pair that had no other home.

### 8.3 Two hazards the implementation must handle

**A WAL database is not one file.** Committed transactions can sit in `catalog.sqlite-wal` with the
main file lagging, so copying `catalog.sqlite` alone uploads a torn snapshot — internally consistent
as of some older point, silently missing everything since. Upload therefore runs a `TRUNCATE`
checkpoint and then SQLite's backup API, which serialises against concurrent writers rather than
racing them. It never copies the live file.

**Integer primary keys are not identities.** Two devices each allocate `collections.id = 1` for
different collections, so a row-level merge keyed on the integer id would collide them. Collections
therefore carry a **UUID**, and membership maps across devices by **image content hash**. The
integer ids stay local and are never compared across catalogs.

### 8.4 Merge rules

| Concern | Rule | Why |
|---|---|---|
| Which collection wins | Higher `revision` — a counter bumped per local edit. `modified` only breaks an exact tie | A device with a skewed clock cannot silently overwrite real work. The same reason FR-NC-9 avoids mtime for sidecars |
| Membership | **Set union**, not last-writer-wins | Two devices adding different images to one collection keep both. The exception — a removal racing an addition — resolves toward the addition, which is recoverable by removing it again. A lost addition is not |
| Deletion | Tombstone (`deleted = 1`) carrying a revision | Without it, merging against a device that still holds the collection resurrects it. With a revision, deletion competes on equal footing with a rename |
| An image the remote has and we do not | Skip the membership row | It joins on a later merge, once a scan has catalogued the file. Not an error |
| A remote from a newer schema | Decline before attaching | Attempting it would fail mid-transaction rather than declining cleanly |

Merging is idempotent: running it twice reports no changes the second time. That property is tested,
because a merge that oscillates would upload on every sync forever.

### 8.5 What was rejected

**Replace-if-newer.** The literal reading of "sync the file and take the newer one". Rejected
because it is not a merge: whichever device syncs second loses every collection the first did not
have. Binary SQLite files do not merge, so "newer wins" means "older is destroyed".

**A `collections.drsc` sidecar at the library root.** The alternative that would have kept §6.12
untouched, merging as text the way edit sidecars do. Viable, and cheaper in machinery, but it means
a second serialisation format and a second merge implementation for the same data. Recorded here
because if the SQLite path proves troublesome, this is the fallback with a known shape.

---

## 9. What this document does not settle

- **FTS.** `Selector::Text` is a `LIKE` scan over filename and keywords. Adequate at 50k; if free
  text over description and title becomes a real workflow, an FTS5 table is the answer, and it is
  additive.
- **Smart collection materialisation.** Currently evaluated on read. If a smart collection's
  membership needs to be *stable* — for manual ordering, or for a pinned set that must not shift
  under the user — it needs materialising with an invalidation rule. Deferred until there is a
  concrete need.
- **Multi-root capture-time collisions.** FR-CAT-11 detects duplicates on import; the same image
  catalogued under two roots is a related but distinct case, not yet specified.
- **Timeline granularity selection.** Which bucket size the UI picks for a given zoom is a UI
  concern, but the catalog should probably suggest one from the query's date span rather than have
  the UI guess.

---

## 10. People and faces

Specified by FR-CULL-8 … FR-CULL-12, NFR-SEC-5, [architecture.md §6.4](architecture.md). Gated on
spike S14 and decision D13 — the runtime and the model licences are unresolved, so this is the shape
of the subsystem, not a build order.

### 10.1 Schema (a v5 migration)

```sql
CREATE TABLE people (
    id       INTEGER PRIMARY KEY,
    uuid     TEXT NOT NULL UNIQUE,   -- merge identity, not the name (ARCH §6.3)
    name     TEXT NOT NULL,
    -- Tombstone-by-redirect. A merged person must outlive its merge, or a
    -- device that still has it resurrects it — same hazard collections have.
    merged_into INTEGER REFERENCES people(id) ON DELETE SET NULL,
    created  INTEGER NOT NULL,
    revision INTEGER NOT NULL DEFAULT 1,
    modified INTEGER NOT NULL
);

CREATE TABLE faces (
    id         INTEGER PRIMARY KEY,
    image_id   INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
    -- Normalised to the image's long edge, so a face survives the proxy it was
    -- found on being regenerated at another resolution.
    x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL,
    landmarks  BLOB,                 -- 5 × (x, y) f32, the alignment input
    detector_confidence REAL NOT NULL,
    embedding  BLOB NOT NULL,        -- 512 × f16, L2-normalised
    -- Which model produced this. An embedding is only comparable to others
    -- from the same model; mixing them silently yields nonsense similarities.
    model_id   TEXT NOT NULL,
    detected_at INTEGER NOT NULL
);
CREATE INDEX faces_image ON faces(image_id);

CREATE TABLE face_person (
    face_id     INTEGER PRIMARY KEY REFERENCES faces(id) ON DELETE CASCADE,
    person_id   INTEGER NOT NULL REFERENCES people(id) ON DELETE CASCADE,
    -- Calibrated P(this face is this person), never a raw cosine (FR-CULL-9).
    probability REAL NOT NULL,
    -- The user said so. Never overwritten by a later inference pass.
    confirmed   INTEGER NOT NULL DEFAULT 0
);
CREATE INDEX face_person_person ON face_person(person_id, confirmed);
```

Three things in that schema are load-bearing:

**`model_id` on every face.** Embeddings from different models are not comparable — this is the one
mistake that produces plausible-looking garbage rather than an error. Storing the model with the
embedding means a model change is detectable and re-indexable, instead of quietly poisoning every
similarity in the library.

**Normalised bounding boxes.** Detection runs on whichever proxy exists (FR-CULL-8). Storing pixel
coordinates would bind a face to a resolution that the cache is entitled to evict and regenerate
differently.

**`confirmed` as a column, not a probability of 1.0.** A confirmation is a different kind of fact
from a confident guess, and collapsing them loses the ability to recompute suggestions without
touching user data.

### 10.2 Why clustering is not a job kind

Detection is per-image and parallel, so it is a job (`DetectFaces`, coalesced per image like any
other). Clustering is a *whole-library* operation over the embeddings detection produced — it has no
natural `subject_id`, and running it per-image would rebuild the world on every photograph.

It therefore runs as a debounced library-level pass, triggered when detection has been idle and the
face count has moved materially since the last clustering. The same reasoning as sidecar writes: the
work is cheap to defer, expensive to repeat, and nobody is waiting on it.

### 10.3 The calibration lives with the library

FR-CULL-9 requires similarity to be a calibrated probability, fitted from this library's own faces.
That fit is a property of the catalog and its model, so it is stored alongside — a small table
holding the fit parameters, its validity flag, and a hash of the face set it was derived from, so a
materially changed library recomputes rather than trusting a stale fit.

When the fit is not valid — a library with too few faces to have positive pairs — the UI says the
confidence is unavailable. It does not fall back to an untuned default dressed up as a measurement.

### 10.4 What this does not settle

- **Which model, and which runtime.** D13. Everything above holds regardless of the answer, which is
  why it is specified in terms of "a 512-d embedding from a stated model" rather than a named one.
- **The clustering algorithm.** Density-based over the calibrated distance is the obvious starting
  point, but the parameters are an S14 question, not a design-time one.
- **Whether embeddings sync.** NFR-SEC-5 permits it, opt-in. The shard mechanism in §7.3 is the
  obvious carrier if they do, but nothing here depends on that decision.
- **Faces in trashed images.** FR-CAT-15's trash moves files; whether their faces stay indexed and
  keep contributing to clusters is unspecified. Probably they should be excluded from suggestions but
  not deleted, so a restore does not re-index.

---

## 11. Requirements touched

| ID | How this document addresses it |
|---|---|
| FR-CAT-1 | §3 incremental scan, cancellable and resumable via §6 jobs |
| FR-CAT-3 | §7 thumbnail pyramid, two size classes, embedded-preview fast path |
| FR-CAT-4 | §4.1 windowed queries, memory independent of catalog size |
| FR-CAT-5 | §3.5 two-pass metadata |
| FR-CAT-6 | §4.3 indexed filter compilation, §5 selectors |
| FR-CAT-7 | §2 collections schema, §5 manual and smart, §7a hierarchy, membership and editing |
| FR-CAT-9 | §3.4 the offline/deleted distinction and the sweep guard |
| FR-CAT-11 | §3.5 lazy content hashing |
| FR-NC-3 | §7.2 range-extract for remote thumbnails |
| FR-NC-6a | §5 shared selector type |
| FR-NC-6c | §3.5 metadata honesty, §7.2 availability-driven sourcing |
| NFR-P1 | §3.1 one stat per directory, not per file |
| NFR-P3 | §7.1 on-demand generation |
| NFR-ARCH-2 | §6.3 priority classes shared with the GPU scheduler |
| NFR-ARCH-3 | §4.3 query cancellation, §6 job cancellation |
| NFR-RES-4 | §7.3 LRU cap, eviction order |
| FR-CULL-8 | §10.1 `faces` schema, §6.1 `DetectFaces` job kind on the proxy tier |
| FR-CULL-9 | §10.3 per-library calibration, stored with its validity and source hash |
| FR-CULL-10 | §10.1 `people` and `face_person`, merge-by-redirect, §10.2 clustering as a library pass |
| FR-CULL-11 | §5 `Selector::Person`, confirmed-only by default |
| FR-CULL-12 | §10.1 derived data in the catalog; names to the sidecar, UUID as merge identity |
| NFR-SEC-5 | §10.4 sync left undecided and off; nothing in §10 emits an embedding |