dtourolle
b2f3936a53
catalog.md §8.2 still said confirmed and rejected faces were matched to local faces "by box", and that the merge reads only a remote face's box and model; faces.md said `match_faces` "still matches by overlap alone across devices". Since #77 the match falls back to embeddings, on the photographs where a box leaves a remote face over, and since #78 `dedup_people` folds people of one name whose faces agree after every sync. Both now say so, with the thresholds and the reason a less decisive pair stays unmatched, taken from the code's own documentation.
1031 lines
56 KiB
Markdown
1031 lines
56 KiB
Markdown
# DarkRoom — Catalog, library view, and background work
|
||
|
||
**Status:** Draft v0.1 · 2026-08-09
|
||
**Companion to:** [requirements.md](requirements.md), [architecture.md](architecture.md)
|
||
|
||
Specifies `dr-catalog`: the index the library view queries, how it stays current without rescanning
|
||
everything, and how thumbnails get made. [architecture.md §6.2](architecture.md) sketches the schema
|
||
in eight lines; this expands it to the point of implementability and fills the two gaps that sketch
|
||
leaves open — **incremental local scan** and **the job queue**.
|
||
|
||
Sync's remote side is already designed ([architecture.md §8](architecture.md)): ETag pruning turns a
|
||
no-op sync of 50k images into one request. Nothing equivalent existed for a local root, which is the
|
||
central problem this document solves.
|
||
|
||
---
|
||
|
||
## 1. What this must not do
|
||
|
||
Stated first because every design choice below follows from it.
|
||
|
||
| Must not | Why |
|
||
|---|---|
|
||
| Stat 50k files to open the catalog | NFR-P1: catalog open < 2 s desktop, < 4 s Android. SAF `DocumentsContract` queries are far slower than `stat` (spike S10). |
|
||
| Re-derive thumbnails for unchanged images | NFR-P3 throughput is for *new* work; redoing it on every connect makes first paint unbounded. |
|
||
| Fetch previews for remote images nobody looks at | A 50k remote library at 1–3 MB per range-extract is 50–150 GB. FR-NC-6 forbids bulk transfer by default. |
|
||
| Evaluate cache rules per grid cell | ARCH §9.5 already answers this: `tier_desired` is materialised. |
|
||
| Block the UI executor on any of it | NFR-P9, NFR-ARCH-1. |
|
||
|
||
The unifying principle: **work is proportional to what changed, or to what the user is looking at —
|
||
never to library size.**
|
||
|
||
---
|
||
|
||
## 2. Schema
|
||
|
||
Extends [architecture.md §6.2](architecture.md). Additions beyond that sketch are marked ⊕.
|
||
|
||
```sql
|
||
-- Roots -----------------------------------------------------------------
|
||
roots(
|
||
id INTEGER PRIMARY KEY,
|
||
kind TEXT, -- 'local' | 'saf' | 'remote'
|
||
grant_blob BLOB, -- SAF persisted permission; NULL on Linux
|
||
label TEXT,
|
||
last_seen INTEGER,
|
||
scan_generation INTEGER -- ⊕ bumped per completed scan; see §3.4
|
||
);
|
||
|
||
-- Folders: the unit of change detection, local and remote alike ---------
|
||
folders(
|
||
id INTEGER PRIMARY KEY,
|
||
root_id INTEGER NOT NULL REFERENCES roots(id),
|
||
parent_id INTEGER REFERENCES folders(id),
|
||
path TEXT NOT NULL,
|
||
etag TEXT, -- remote: propagating ETag (ARCH §8.4)
|
||
mtime INTEGER, -- ⊕ local: directory mtime
|
||
entry_count INTEGER, -- ⊕ local: direct children, mtime's blind spot
|
||
scanned_generation INTEGER, -- ⊕ deletion sweep; see §3.4
|
||
UNIQUE(root_id, path)
|
||
);
|
||
|
||
-- Images ----------------------------------------------------------------
|
||
images(
|
||
id INTEGER PRIMARY KEY,
|
||
root_id INTEGER NOT NULL REFERENCES roots(id),
|
||
folder_id INTEGER REFERENCES folders(id), -- ⊕ folder filter without LIKE
|
||
source_ref TEXT NOT NULL,
|
||
content_hash TEXT, -- NULL until hashed; see §3.5
|
||
format TEXT,
|
||
w INTEGER, h INTEGER,
|
||
captured_at INTEGER, -- UTC seconds; NULL if EXIF absent
|
||
captured_offset INTEGER, -- ⊕ minutes east of UTC; see §4.2
|
||
camera TEXT, lens TEXT,
|
||
iso INTEGER, aperture REAL, shutter REAL,
|
||
availability INTEGER,
|
||
file_size INTEGER, -- ⊕ cheap change signal alongside mtime
|
||
file_mtime INTEGER, -- ⊕
|
||
metadata_state INTEGER, -- ⊕ 0=none 1=stat-only 2=full EXIF; §3.5
|
||
sidecar_mtime INTEGER,
|
||
UNIQUE(root_id, source_ref)
|
||
);
|
||
|
||
-- Versions, keywords, remote, cache: per ARCH §6.2, unchanged -----------
|
||
|
||
-- Collections ⊕ ---------------------------------------------------------
|
||
collections(
|
||
id INTEGER PRIMARY KEY,
|
||
name TEXT NOT NULL,
|
||
parent_id INTEGER REFERENCES collections(id), -- collection sets
|
||
kind INTEGER NOT NULL, -- 0 = manual, 1 = smart
|
||
selector_json TEXT, -- smart only; the §5 Selector
|
||
created INTEGER
|
||
);
|
||
|
||
collection_members(
|
||
collection_id INTEGER NOT NULL REFERENCES collections(id) ON DELETE CASCADE,
|
||
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
|
||
position INTEGER, -- manual ordering; NULL = by capture time
|
||
PRIMARY KEY(collection_id, image_id)
|
||
);
|
||
|
||
-- Jobs ⊕ ----------------------------------------------------------------
|
||
jobs(
|
||
id INTEGER PRIMARY KEY,
|
||
kind INTEGER NOT NULL,
|
||
subject_id INTEGER, -- image or folder, per kind
|
||
priority INTEGER NOT NULL,
|
||
state INTEGER NOT NULL, -- 0=pending 1=running 2=failed
|
||
attempts INTEGER NOT NULL DEFAULT 0,
|
||
not_before INTEGER, -- retry backoff
|
||
payload TEXT,
|
||
UNIQUE(kind, subject_id) -- coalescing; see §6.2
|
||
);
|
||
```
|
||
|
||
Indices that exist for a stated query, not speculatively:
|
||
|
||
```sql
|
||
CREATE INDEX images_captured ON images(captured_at); -- §4 timeline
|
||
CREATE INDEX images_folder ON images(folder_id);
|
||
CREATE INDEX images_hash ON images(content_hash) WHERE content_hash IS NOT NULL;
|
||
CREATE INDEX folders_parent ON folders(parent_id);
|
||
CREATE INDEX jobs_ready ON jobs(state, priority DESC, not_before);
|
||
CREATE INDEX versions_image ON versions(image_id);
|
||
CREATE INDEX members_image ON collection_members(image_id);
|
||
```
|
||
|
||
`content_hash` is indexed *partially*. It is NULL for most rows most of the time (§3.5), and a
|
||
partial index over the non-NULL subset is both smaller and what FR-CAT-9's reconnection-by-hash
|
||
and FR-CAT-11's duplicate detection actually query.
|
||
|
||
**Made on first use, not by a migration.** A new `user_version` makes every older build refuse this
|
||
catalog's snapshot at sync (`sync::remote_is_mergeable` compares it and nothing else), and a tablet
|
||
a release behind would stop merging collections, keywords and people for a feature it does not
|
||
have. So what later releases added without needing old rows rewritten is created with `IF NOT
|
||
EXISTS` where it is first used, and an older build that meets it ignores it:
|
||
|
||
- `dedup_probes` (FR-CAT-11a, §3.5);
|
||
- the albums (FR-EXP-10, `core/dr-catalog/src/albums.rs`): `albums`, `album_exports` — one row per
|
||
file written into an album, keyed on the file name, since two crops of one photograph are two
|
||
files — and `album_folders`, this device's folder for each (§8.2);
|
||
- `keywords_term_version (keyword, version_id)`, made by `keywords::list`, which counts each word's
|
||
photographs on every selection change and without it read a `keywords` row per assignment to
|
||
learn its version;
|
||
- `faces_box (image_id, model_id, x, y, w, h)`, made by the merge's `match_faces`, which reads every
|
||
local face's box and model and without it opened each ~8 KB `faces` row to do so.
|
||
|
||
A failure to make one of the indexes — a read-only or busy catalog — is logged and the query runs
|
||
without it, as it did before.
|
||
|
||
**Opening does not repeat the backfill.** `schema::backfill` repairs what a write left owing — an
|
||
image a scan inserted without its default version, the RAW/JPEG pairing, a merged version's uuid, a
|
||
keyword assignment whose word has no term — and it used to run in every `Catalog::open`. Every
|
||
worker opens its own connection, so a develop landing paid it five times (~80 ms of CPU on the
|
||
reference catalog) to learn that nothing had changed. `core/dr-catalog/src/backfilled.rs` now
|
||
records, per path and per process, a stamp read before each backfill: the schema version, the
|
||
file's device and inode, and the newest image, version and keyword assignment by content as well as
|
||
id — because none of those tables is `AUTOINCREMENT`, a freed newest id is handed out again, and the
|
||
row that takes it is exactly one the backfill is owed. An open whose stamp matches skips it (~1 ms);
|
||
the first open in a process, a migration, a pull and a replaced file always run it. Kept in memory
|
||
rather than in the catalog, so nothing about it travels in the sync snapshot.
|
||
|
||
---
|
||
|
||
## 3. Incremental scan
|
||
|
||
### 3.1 The local analogue of ETag pruning
|
||
|
||
Nextcloud propagates ETags up the tree, so one request proves a whole library unchanged
|
||
([architecture.md §8.4](architecture.md)). A filesystem offers no such guarantee — a directory's
|
||
mtime changes when its *direct* entries change, and not when a grandchild does. There is no
|
||
cheap "did anything below here change" probe.
|
||
|
||
So local scan prunes at each level rather than at the root:
|
||
|
||
```
|
||
scan(folder):
|
||
(mtime, count) = stat(folder)
|
||
if (mtime, count) == stored:
|
||
# This directory's own entries are unchanged. Its files need no
|
||
# examination at all — but subdirectories may still have changed
|
||
# internally, so recurse into known children without listing.
|
||
for child in stored_children(folder):
|
||
scan(child)
|
||
else:
|
||
entries = list(folder) # the expensive call
|
||
reconcile(folder, entries) # §3.3
|
||
for child in entries.dirs: scan(child)
|
||
mark scanned(folder, current_generation)
|
||
```
|
||
|
||
Cost is **one `stat` per directory** when nothing changed, versus one per *file*. A 50k-image
|
||
library in ~2k folders costs 2k stats — a few milliseconds locally, and the difference between
|
||
meeting and missing NFR-P1 on SAF.
|
||
|
||
The recursion into unchanged directories is not redundant: it is what makes a change to one deep
|
||
file detectable at all, given no upward propagation. What it avoids is the *listing* — on SAF a
|
||
`DocumentsContract` query returning 200 rows costs far more than a metadata probe on the directory
|
||
itself.
|
||
|
||
### 3.2 Why entry-count as well as mtime
|
||
|
||
Directory mtime alone misses a real case: delete one file and create another within the same
|
||
timestamp granularity, and mtime can be unchanged while contents differ. Some filesystems and most
|
||
SAF providers report coarse timestamps, which widens the window.
|
||
|
||
Storing `(mtime, entry_count)` closes the common form of this — a paired add and remove changes
|
||
neither, but that is rarer than a bare add or remove, and both of those move the count. It is a
|
||
cheap narrowing, not a proof.
|
||
|
||
**Where correctness must not depend on it,** the user gets an explicit *Rescan folder* action
|
||
(FR-CAT-1), and reconnection matches by content hash (FR-CAT-9). Sync's remote path is unaffected —
|
||
ETags are authoritative there.
|
||
|
||
### 3.3 Reconciling a changed directory
|
||
|
||
For each entry in a listing:
|
||
|
||
| Situation | Action |
|
||
|---|---|
|
||
| Not in catalog | Insert with `metadata_state = 1`; enqueue `ExtractMetadata` |
|
||
| In catalog, `(size, mtime)` match | Nothing — the common case |
|
||
| In catalog, `(size, mtime)` differ | Re-enqueue `ExtractMetadata` and `Thumbnail`; clear `content_hash` |
|
||
| In catalog, absent from listing | Deletion candidate — §3.4 |
|
||
| Placeholder (`*.nextcloud`) | Catalogue as the image it stands for; `Availability::Offline` (ARCH §9.0) |
|
||
|
||
Sidecars are examined in the same pass: a `.drsc` whose mtime exceeds `images.sidecar_mtime` enqueues
|
||
a `ReadSidecar` job. This is how an edit made on another device — landed by the Nextcloud client,
|
||
not by us — reaches the catalog.
|
||
|
||
### 3.4 Deletion without a full sweep
|
||
|
||
A file removed outside the app appears only as an *absence*, which a pruned scan cannot see: the
|
||
folder it vanished from has a changed mtime and is listed, but a folder never visited is never
|
||
compared.
|
||
|
||
Generation counting handles this without a full pass. Each scan bumps `roots.scan_generation`, and
|
||
every folder reached — whether listed or skipped — records it. After the walk:
|
||
|
||
```sql
|
||
-- Folders never reached: their parent no longer lists them.
|
||
DELETE FROM folders
|
||
WHERE root_id = ?1 AND scanned_generation < ?2;
|
||
```
|
||
|
||
Images under a deleted folder cascade. Images missing from a *listed* folder are caught directly in
|
||
§3.3. Together these cover deletion with no additional traversal.
|
||
|
||
Deletion here means **removing the catalog row for a source proven absent**, which FR-CAT-9 sharply
|
||
distinguishes from a source merely unreachable. A root that fails to open at all — unplugged drive,
|
||
revoked SAF grant — aborts the scan and marks the root offline. It never runs the sweep, because
|
||
every folder would look unreached and the sweep would delete the entire library.
|
||
|
||
That guard is the single most dangerous line in this design, and it is stated as an invariant:
|
||
**the deletion sweep runs only after a scan that completed without a root-level access error.**
|
||
|
||
### 3.5 Metadata in two passes
|
||
|
||
Full EXIF extraction requires opening and parsing each file. At 50k images that is minutes, and it
|
||
must not stand between the user and a usable grid.
|
||
|
||
`metadata_state` records how far each image has got:
|
||
|
||
| State | Holds | Cost |
|
||
|---|---|---|
|
||
| 0 — none | Row exists, nothing read | — |
|
||
| 1 — stat-only | Name, size, mtime, format from extension | Free, from the listing |
|
||
| 2 — full | EXIF: capture time, camera, lens, exposure, dimensions | One open + parse |
|
||
|
||
The grid is usable at state 1: it can show filenames, sort by filename or file mtime, and display
|
||
placeholder cells. Promotion to state 2 runs as background jobs, prioritised by what is on screen
|
||
(§6.3), so visible images get real capture times within a frame or two of being scrolled to.
|
||
|
||
**Capture-time filtering (§4) needs state 2**, so a freshly scanned library's timeline is incomplete
|
||
until the pass finishes. The UI states this plainly — a progress affordance on the timeline, not a
|
||
silently wrong filter. Which is the FR-NC-6c principle applied to metadata rather than pixels: say
|
||
what you actually have.
|
||
|
||
`content_hash` is a *third*, still lazier tier. It requires reading the whole file, so it is computed
|
||
only when something needs it: import duplicate detection (FR-CAT-11), or reconnecting a moved source
|
||
(FR-CAT-9). Never during a routine scan.
|
||
|
||
Consolidating the duplicates a library already holds (FR-CAT-11a, 0.16.0) does not wait for it. It
|
||
proves a group the same by `content_hash` where every copy has one, and otherwise by a digest of
|
||
each copy's first and last megabyte, read by range through the backend and kept in `dedup_probes`
|
||
keyed on the size and mtime it was taken at, so a second review reads nothing.
|
||
`dedup_probes` is created on first use rather than by a migration, because a schema bump would make
|
||
an older build refuse this catalog's snapshot at sync (`core/dr-catalog/src/duplicates.rs`).
|
||
|
||
---
|
||
|
||
## 4. The library view
|
||
|
||
### 4.1 Query model
|
||
|
||
The UI never assembles SQL. It hands the catalog a `Query` and receives a stable, windowable result:
|
||
|
||
```rust
|
||
pub struct Query {
|
||
pub filter: Selector, // §5 — same type cache rules use
|
||
pub sort: Sort,
|
||
pub descending: bool,
|
||
}
|
||
|
||
pub enum Sort {
|
||
CapturedAt,
|
||
Added,
|
||
FileName,
|
||
Rating,
|
||
/// Manual order within a collection; falls back to CapturedAt elsewhere.
|
||
CollectionPosition,
|
||
}
|
||
```
|
||
|
||
Results are fetched by window, never wholesale — FR-CAT-4 requires memory bounded independently of
|
||
catalog size:
|
||
|
||
```rust
|
||
impl Catalog {
|
||
fn count(&self, q: &Query) -> Result<usize, CatalogError>;
|
||
fn window(&self, q: &Query, range: Range<usize>) -> Result<Vec<GridRow>, CatalogError>;
|
||
}
|
||
```
|
||
|
||
`GridRow` carries exactly what a cell draws — id, thumbnail key, availability, rating, flag, capture
|
||
time — and nothing that would require a join per cell. Availability badges read `tier_desired`
|
||
directly (ARCH §9.5), so no rule evaluation happens on the render path.
|
||
|
||
A `LIMIT/OFFSET` window degrades at high offsets, since SQLite must walk the skipped rows. Scrolling
|
||
is overwhelmingly *sequential*, so the catalog keeps a keyset cursor for forward and backward paging
|
||
and falls back to OFFSET only for a scrollbar jump. Jumps are rare and single; scrolling is
|
||
continuous.
|
||
|
||
### 4.2 Time
|
||
|
||
Capture time is the spine of a photo library, and it has one persistent trap: **a photograph's
|
||
timestamp is local to where it was taken.** Store UTC alone and a shoot that ran 09:00–17:00 in
|
||
Tokyo displays as spanning two days in Paris. Store local time alone and ordering across a timezone
|
||
change is wrong.
|
||
|
||
So both: `captured_at` in UTC for ordering, `captured_offset` in minutes for display and for
|
||
day-bucketing. EXIF `OffsetTimeOriginal` supplies it where present; where absent — common on older
|
||
bodies — the offset is NULL and the catalog falls back to the library's configured display timezone,
|
||
flagged so the UI can show it as inferred.
|
||
|
||
Day, month, and year buckets are computed against **local** time. "Everything from 3 August" means
|
||
the photographer's 3 August.
|
||
|
||
The timeline affordance is a histogram of counts per bucket, which the grid uses for scrubbing:
|
||
|
||
```rust
|
||
pub enum Granularity { Year, Month, Day, Hour }
|
||
|
||
pub struct TimeBucket {
|
||
pub start: i64, // UTC seconds, bucket start
|
||
pub count: u32,
|
||
}
|
||
|
||
fn timeline(&self, q: &Query, g: Granularity) -> Result<Vec<TimeBucket>, CatalogError>;
|
||
```
|
||
|
||
This is one grouped aggregate over the `images_captured` index, not 50k rows into the UI. It is what
|
||
makes "drag across two years to find the trip" work, and it is the cheapest useful thing a library
|
||
view can offer over a flat grid.
|
||
|
||
### 4.3 Filtering interactively
|
||
|
||
FR-CAT-6 requires filter results to update interactively on 50k images. Three things make that hold:
|
||
|
||
1. **Filters compile to indexed predicates.** A `Selector` becomes a WHERE clause over indexed
|
||
columns. Keyword and collection membership become `EXISTS` subqueries against their own indices.
|
||
2. **Count and first window are one round trip.** The grid needs a row count to size its scrollbar
|
||
and the first screenful to paint; the catalog returns both together.
|
||
3. **A filter change cancels the one in flight.** Typing in a search box issues a query per
|
||
keystroke; each supersedes the last (NFR-ARCH-3). Without this the UI queues work it will discard.
|
||
|
||
---
|
||
|
||
## 5. Selectors: one type, three uses
|
||
|
||
[architecture.md §9.2](architecture.md) defines `Selector` for cache rules. The same type expresses
|
||
library filters and smart collections. This is deliberate and worth stating as a design decision,
|
||
because three near-identical predicate languages is a classic way for a catalog to rot.
|
||
|
||
| Use | Meaning |
|
||
|---|---|
|
||
| Library filter | What the grid shows now |
|
||
| Smart collection | A saved, named filter (FR-CAT-7) |
|
||
| Cache rule | What is kept locally, at which tier (FR-NC-6a) |
|
||
|
||
One consequence is directly useful: any filter the user has narrowed to can be saved as a smart
|
||
collection, and any collection can be pinned offline, with no conversion step. "Show me 5-star images
|
||
from the last 90 days" → save as a collection → pin it for the trip. Three features, one mechanism.
|
||
|
||
`Selector` moves to `dr-types` so `dr-catalog` and `dr-sync` share it without either depending on the
|
||
other. It gains variants the cache-rule sketch did not need:
|
||
|
||
```rust
|
||
pub enum Selector {
|
||
All, // ⊕ the empty filter
|
||
Collection(CollectionId),
|
||
Folder { root: RootId, path: String, recursive: bool },
|
||
DateRange(DateSelector),
|
||
Rating { min: u8 },
|
||
Label(ColourLabel),
|
||
Flag(FlagState),
|
||
Keyword(String),
|
||
Camera(String), // ⊕ FR-CAT-6 indexed field
|
||
Lens(String), // ⊕
|
||
IsoRange { min: u32, max: u32 }, // ⊕
|
||
Availability(Availability), // ⊕ "what can I edit right now"
|
||
Text(String), // ⊕ filename/keyword substring
|
||
Person { id: PersonId, include_suggested: bool }, // ⊕ §10 (FR-CULL-11)
|
||
All_(Vec<Selector>),
|
||
Any(Vec<Selector>),
|
||
Not(Box<Selector>),
|
||
}
|
||
```
|
||
|
||
`Person` carries `include_suggested` rather than defaulting silently. A saved collection built from
|
||
confirmed faces must not quietly change membership because a later indexing pass guessed at another
|
||
face; the user chose "photos of Anna", not "photos the model currently believes contain Anna". The
|
||
default is `false`, and the interactive filter offers the looser form explicitly as a way to *find*
|
||
faces to confirm.
|
||
|
||
`Availability` as a selector earns its place: on a tablet the most useful filter is often "what do I
|
||
actually have here", and it is also the natural thing to *pin* — "keep everything I've flagged that
|
||
isn't already local".
|
||
|
||
Compilation is a straightforward recursive walk producing SQL with bound parameters. **Nothing
|
||
user-supplied is ever interpolated into SQL text.** `Text` becomes a bound `LIKE` pattern with `%`,
|
||
`_`, and the escape character escaped.
|
||
|
||
---
|
||
|
||
## 6. Background work
|
||
|
||
### 6.1 Job kinds
|
||
|
||
```rust
|
||
pub enum JobKind {
|
||
ScanFolder, // §3, recursive from a folder
|
||
ExtractMetadata, // state 1 → 2
|
||
Thumbnail, // §7
|
||
ReadSidecar, // external sidecar change detected
|
||
WriteSidecar, // local edit → disk, debounced (ARCH §6.1)
|
||
ContentHash, // on demand only
|
||
FetchPreview, // remote range-extract (FR-NC-3)
|
||
FetchOriginal, // pinned or explicitly requested
|
||
DetectFaces, // §10, on the proxy tier (FR-CULL-8)
|
||
}
|
||
```
|
||
|
||
**Thumbnails are not queued (decided 2026-09-26, #73).** `Thumbnail` is kept only so its number
|
||
stays taken, and is listed in `JobKind::RETIRED`. Up to 0.16.0 every remote scan enqueued one per
|
||
photograph, and nothing claimed the kind — no `JobHandler` was ever registered for it, on desktop or
|
||
Android (the same `dr-ui`), and the catalog snapshot carries `jobs` but the merge never reads them
|
||
(§8.2). The reference catalog held 23,582 such rows, about 1 MB with its indexes. Thumbnails are
|
||
made another way, and by the right source of truth:
|
||
|
||
- the grid asks a worker for the cells it is drawing, which serves them from the thumbnail store or
|
||
range-fetches the embedded preview (§7.2);
|
||
- the thumbnail sweep's work list is *what the store does not hold* (`thumbnails_outstanding`).
|
||
|
||
The store is shared between devices (§7.3), so it is the only thing that knows another device already
|
||
made a thumbnail; a per-device queue row cannot. A queue row would be a second, staler record of the
|
||
same debt. Metadata is owed the same way — `metadata_state < 2` is the sweep's work list — so the local
|
||
walk no longer enqueues `ExtractMetadata` either.
|
||
|
||
The rows already queued are dropped by `jobs::drop_retired`, from `runner::recover` at every catalog
|
||
open, rather than by a migration: a schema bump would make a device still on an older build refuse the
|
||
synced snapshot, and "every open" rather than "once" because an older build sharing the catalog
|
||
queues them again on its next scan. With the rows gone it is one probe of the `(kind, subject_id)`
|
||
index. `every_queued_kind_has_a_consumer` (dr-catalog) holds the rule: it reads the shipping sources
|
||
and fails if any kind is enqueued that no handler or claim names.
|
||
|
||
### 6.2 Coalescing is the point
|
||
|
||
`UNIQUE(kind, subject_id)` on `jobs` means enqueueing is idempotent: an image touched five times
|
||
has one job of a kind, not five. Enqueue is
|
||
`INSERT … ON CONFLICT DO UPDATE SET priority = max(priority, excluded.priority)`, so a re-request at
|
||
higher priority promotes the existing row rather than duplicating it.
|
||
|
||
This is what makes "regenerate on update" safe to call liberally. Every code path that notices a
|
||
change can just enqueue; the table absorbs the redundancy.
|
||
|
||
### 6.3 Priority
|
||
|
||
Reuses the existing GPU scheduler classes ([architecture.md §5.3](architecture.md)) so one notion of
|
||
priority governs the whole app:
|
||
|
||
| Class | Jobs | Preempts |
|
||
|---|---|---|
|
||
| `Interactive` | Metadata and thumbnails for visible cells; preview for the open image | everything |
|
||
| `Prefetch` | The scroll margin; next image in culling | Background |
|
||
| `Background` | Bulk metadata, rule-driven fetches, hashing | — |
|
||
|
||
Visible-cell work is enqueued by the grid as it scrolls, at `Interactive`. The effect is that a
|
||
freshly scanned library fills in *where the user is looking* first, and grinds through the rest
|
||
behind them. (As built, thumbnails and metadata get this ordering without the queue — the grid
|
||
requests its visible cells directly and the sweeps take what is left; see §6.1.)
|
||
|
||
### 6.4 Durability and failure
|
||
|
||
Jobs live in the catalog, so they survive process death — which on Android is routine, not
|
||
exceptional (FR-PLAT-AND-3). On startup, rows in state `running` revert to `pending`: the process
|
||
that owned them is gone.
|
||
|
||
Failures increment `attempts` and set `not_before` to an exponential backoff. After a bounded retry
|
||
count the job is marked failed and attached to its image as a typed error (NFR-ARCH-4) — one
|
||
corrupt file does not stall the queue, and the user can see which files failed and why.
|
||
|
||
**A job runner never touches the UI executor**, and `Interactive` work runs on the decode pool with
|
||
the I/O pool behind it (ARCH §7.1).
|
||
|
||
---
|
||
|
||
## 7. Thumbnails
|
||
|
||
### 7.1 When
|
||
|
||
Not "on first connect" as a bulk operation. Thumbnails are generated:
|
||
|
||
- **On demand**, for cells entering the viewport plus the prefetch margin — at `Interactive`
|
||
- **On change**, when §3.3 sees a differing `(size, mtime)`
|
||
- **On rule**, for images a cache rule pins at `Preview` or above — at `Background`
|
||
- **Never** for a remote image nobody has looked at and no rule covers
|
||
|
||
For a local library this converges on "everything, eventually", because scrolling reaches everything
|
||
and the background pass has nothing else to do. For a remote library it converges on "what you
|
||
actually browsed".
|
||
|
||
**Measured on a real 17,185-RAW library, 2026-08-09:** cataloguing it by whole-file fetch would move
|
||
roughly **370 GB**; the range-extract path moves a few MB for the images actually viewed. This is
|
||
the single largest cost difference in the design, and it is why §7.1 is a list of narrow triggers
|
||
rather than "generate them all on connect".
|
||
|
||
### 7.2 How, by availability
|
||
|
||
| Availability | Source | Cost |
|
||
|---|---|---|
|
||
| `Original`, local | Embedded JPEG via `dr-decode` preview path | ~200 KB read, no demosaic |
|
||
| `Original`, no embedded preview | Full decode, downscale | Expensive — `Background` only |
|
||
| Remote | Range-extract embedded JPEG (FR-NC-3) | 1–3 MB vs 25–100 MB — **measured: 262 KB of a 21.5 MB DNG, 119 ms, 1.22% of the file** |
|
||
| Placeholder / `Offline` | None — render the offline affordance | 0 |
|
||
|
||
The remote path deliberately does **not** ask the Nextcloud client to hydrate the file. ARCH §9.0
|
||
established hydration is whole-file, so it costs ~100× what the range extract does. Hydration stays
|
||
reserved for the original tier, where the user has asked for the actual image.
|
||
|
||
Server previews (`/core/preview`) are tried only where PROPFIND reported `nc:has-preview`. ARCH §6.7
|
||
verified stock Nextcloud ships no RAW preview provider, so for RAW this is nearly always absent — it
|
||
is an opportunistic saving, never the mechanism.
|
||
|
||
### 7.3 Storage: sharded, shared, synced
|
||
|
||
Decided 2026-08-09, implemented in `dr-thumbs`. Thumbnails live in **sharded SQLite databases that
|
||
sync to Nextcloud**, so a second device gets a full grid without re-fetching a byte of RAW.
|
||
|
||
```text
|
||
thumbs/
|
||
index.sqlite fileid → shard, size accounting, client id, adoption ledger
|
||
shard-0000.sqlite ≤ 25 MB, sealed
|
||
shard-0001.sqlite ≤ 25 MB, active
|
||
```
|
||
|
||
**Why a thumbnail is worth syncing when the catalog mostly is not.** It is expensive to produce — a
|
||
range fetch plus a decode, per image — and byte-identical for every client looking at the same file.
|
||
This does not make it authoritative: losing the store costs regeneration and nothing else, so §6.12
|
||
is untouched.
|
||
|
||
**Why shards, and why small.** The 25 MB cap is about *sync granularity*, not SQLite's limits. One
|
||
growing database means every client re-downloads all of it whenever a single thumbnail is added.
|
||
With sequential fill only the newest shard is ever dirty, so an up-to-date client transfers one small
|
||
file. Sealed shards are immutable, which makes them safe to cache forever and cheap to skip.
|
||
|
||
At ~20 KB per 256px JPEG a shard holds roughly 1,200 thumbnails, so the 17,185-image reference
|
||
library lands in ~14 shards.
|
||
|
||
**Keyed on `oc:fileid`** — stable across server-side rename and move (FR-NC-5), and already in hand
|
||
from PROPFIND. Accepted consequence: shards are account-scoped, so the same photograph on two
|
||
servers is thumbnailed twice.
|
||
|
||
**Stored as JPEG, not raw pixels.** A 256×170 RGBA buffer is ~174 KB against ~15 KB encoded. Since
|
||
shards sync, that 11× is transfer cost paid by every client, not just disk.
|
||
|
||
Three invariants, each tested:
|
||
|
||
| Invariant | Why it matters |
|
||
|---|---|
|
||
| A sealed shard never reopens | Reopening one forces every client that holds it to re-download |
|
||
| Re-storing an existing id updates in place, never migrates | Migrating would rewrite a sealed shard |
|
||
| Merging another client's shard is insert-only and idempotent | Both copies derive from the same bytes by the same code, so neither is better; preferring ours avoids dirtying a shard others have synced |
|
||
|
||
**When the exchange runs.** Corrected 2026-09-20. It fired only after the metadata sweep — hours
|
||
on a large library — so a fresh device re-derived every thumbnail it looked at, re-detected faces
|
||
and re-read every header before adopting the shards and snapshot that held all of it. It now also
|
||
fires the moment the scan completes, which is the first moment the rows the merges key on exist,
|
||
and the sweep starts behind it. In steady state that pass is one listing. The catalog merge also
|
||
takes **capture metadata** (`captured_at`, offset, camera, lens, ISO) for images still at
|
||
`metadata_state < 2`, matched by `oc:fileid` — a date is a fact about the file's bytes, not local
|
||
state, and the snapshot already carried it; the sweep's per-chunk query then finds nothing left.
|
||
|
||
**The transfer**, in `dr-ui`'s `derived_sync`, exchanges shards with `.darkroom-derived/` under the
|
||
library root. `ThumbStore::shards()` reports which are sealed, so an up-to-date client's whole pass
|
||
is one listing plus whichever shard is still open.
|
||
|
||
**Why a remote name carries a client id.** Corrected 2026-08-16. Shard ids are *per store* — every
|
||
client fills its own numbering from 0 — so the flat `shard-NNNN.sqlite` namespace the transfer first
|
||
used had two clients writing one name. Two failures followed from it, and both were live: the second
|
||
client's upload **overwrote** content the first still believed was published, and no client could
|
||
distinguish a peer's shard 3 from its own, so the only safe reading of "I already hold 3" was to skip
|
||
it. Between them, two populated clients exchanged almost nothing — only shards numbered above the
|
||
other's highest. A fresh device worked, which is why it went unnoticed: with no local shards there is
|
||
nothing to collide with.
|
||
|
||
The name is now `shard-<client>-NNNN.sqlite`, where `<client>` is minted per store in `index.sqlite`
|
||
beside the numbering it qualifies — a store deleted and rebuilt restarts at shard 0 and must not
|
||
claim its predecessor's names. Since a client's own ids no longer say anything about what it has
|
||
taken from others, `index.sqlite` also keeps an **adoption ledger** of merged remote names and the
|
||
size each had. Size, not a flag: a peer's sealed shard never returns, but its open one grows, and
|
||
re-merging the grown copy is how the thumbnails it gained arrive.
|
||
|
||
Flat names left on servers by earlier builds are still read — they report no owner, so each client
|
||
adopts them once — and nothing is written under that form again. A flat name whose id and byte size
|
||
match a local shard is that client's own earlier upload by the same identity argument used for
|
||
sealed shards, so the rename does not cost every client a re-download of its whole store. Older
|
||
builds ignore the new names, so they stop receiving shards until updated; nothing is lost, since
|
||
their own uploads are still adopted.
|
||
|
||
Two size classes remain planned — grid (256px) and filmstrip/loupe (1024px). Only the grid class is
|
||
implemented. The cache is LRU-capped per NFR-RES-4, and thumbnails evict before proxies and long
|
||
after sidecars, which never evict at all (FR-NC-6b).
|
||
|
||
---
|
||
|
||
## 7a. Editing collections
|
||
|
||
Decided 2026-08-09. The schema for collections landed with §2 and the cross-device merge rules with
|
||
§8; this is the layer between them — the operations a user actually performs, in
|
||
`dr_catalog::collections`.
|
||
|
||
### 7a.1 Hierarchy and membership are independent
|
||
|
||
Two structures, deliberately not entangled:
|
||
|
||
| | Mechanism | Meaning |
|
||
|---|---|---|
|
||
| Hierarchy | `collections.parent_id` | A collection inside a collection (Lightroom's "collection set"). A parent is an ordinary collection, not a separate kind, so a set can hold images of its own |
|
||
| Membership | `collection_members` | An image is in as many collections as the user likes. Nothing moves on disk; no collection owns an image |
|
||
|
||
**Adding an image to a child does not write a row for the parent.** A parent's contents are the union
|
||
of its own members and its descendants', computed on read. Materialising it instead would make one
|
||
add touch every ancestor, and a reparent rewrite membership — both of which §8's row-level merge
|
||
would then have to reconcile. The read path pays a bounded tree walk instead, which at sidebar scale
|
||
is nothing.
|
||
|
||
The consequence the UI depends on: dragging images onto a collection is **additive**. It does not
|
||
remove them from anywhere, which is why the gesture's default action is `copy` and not `move`.
|
||
|
||
### 7a.2 Rules that exist to prevent silent damage
|
||
|
||
| Rule | Why |
|
||
|---|---|
|
||
| Every mutation bumps `revision` | §8.4 resolves conflicts by revision. An edit that updates `modified` alone is invisible to the merge, so the *other* device silently wins and the user's work vanishes |
|
||
| A no-op add does **not** bump it | Otherwise an idle device that re-dropped the same images outranks one that did real work |
|
||
| Deleting a parent **promotes** its children | The schema's `ON DELETE CASCADE` would take the whole subtree. Losing a nested collection because its container was tidied away is not recoverable |
|
||
| Deletion leaves a tombstone | Without it, merging with a device that still holds the collection resurrects it (§8.4) |
|
||
| Cycles are refused at the write | Both kinds — parenting under a descendant, and a smart collection whose selector reaches itself. A cycle is unbounded recursion in the tree walk, so it must not be *representable*, not merely handled when drawn |
|
||
| Tree walks are depth-guarded anyway | A merge can deliver a row this device never validated. The read path must terminate, so it truncates and logs rather than hanging the UI thread |
|
||
| A drop onto a smart collection is refused | Its membership *is* its selector; member rows would be a second source of truth that nothing reads |
|
||
| Deep counts are `count(DISTINCT image_id)` | An image in both a parent and a child is one photograph. A count that disagrees with the number of cells drawn makes both untrustworthy |
|
||
|
||
`collections.uuid` is generated from the OS CSPRNG. A collision fuses two unrelated collections at
|
||
the next merge, so the fallback path (used only if `/dev/urandom` cannot be read) logs loudly rather
|
||
than degrading identity quality in silence.
|
||
|
||
### 7a.3 Drag and drop is Slint's, not ours
|
||
|
||
The first implementation hand-rolled the gesture on `TouchArea` — tracking the press, measuring
|
||
travel to distinguish a click from a drag, and deciding the drop target from the last row hovered.
|
||
**It did not work**, for a reason worth recording: an interactive `Flickable` claims any drag
|
||
beginning inside it for scrolling and *cancels* the child `TouchArea`'s press, so the gesture could
|
||
never leave the grid. It also had a correctness hole — a tree rebuilt mid-drag could redirect the
|
||
drop, since a captured pointer is invisible to every other element.
|
||
|
||
Slint 1.17's `DragArea`/`DropArea` own all of it: capture, the click-versus-drag threshold,
|
||
arbitration against the `Flickable`, the image under the cursor, and hit-testing the release. What
|
||
remains in `collections_ui` is only what Slint cannot know — the payload (which images, read from the
|
||
selection when the drag starts) and the **spring**: a dwell timer that opens a collapsed collection
|
||
so a nested child can be reached mid-drag, and closes again whatever the drag merely passed over.
|
||
|
||
One hazard survives the change and is easy to reintroduce. Every consequence of a drop — rebuilding
|
||
the tree, refreshing the badges, rereading the grid — *replaces a Slint model*, and doing that inside
|
||
the `dropped` handler destroys the elements Slint is still using to deliver that event. So the drop
|
||
records its target and `drag-finished` acts on it. This is the same hazard `sync_rows` in `lib.rs`
|
||
documents for the adjust panel, and it presents as a control that works once and then goes dead.
|
||
|
||
---
|
||
|
||
## 8. Syncing the catalog file
|
||
|
||
Decided 2026-08-09. **This qualifies [architecture.md §6.12](architecture.md)** — the catalog
|
||
remains a rebuildable index, but the file itself now travels to Nextcloud. The qualification is
|
||
worth stating precisely, because the sidecar-authoritative model is load-bearing and this is the
|
||
one place it bends.
|
||
|
||
### 8.1 Why collections forced this
|
||
|
||
Every other thing the catalog holds has authoritative backing outside it. Ratings, labels,
|
||
keywords, and edit graphs live in sidecars next to the images, so a rebuild recovers them.
|
||
**Collections do not.** A manual collection is a set of images the user assembled by hand; nothing
|
||
in the filesystem records it. Losing the catalog loses them, and no rescan brings them back.
|
||
|
||
So collections need to be durable across devices somehow. Syncing the catalog file is the chosen
|
||
mechanism.
|
||
|
||
### 8.2 What the file sync does and does not carry
|
||
|
||
Collections were the first thing merged, and the rules in §8.4 were written for them. What else
|
||
merges reuses those rules or keys on the same identities, and each has no other home:
|
||
|
||
- **Collections and their membership** — by uuid and revision, membership as a set union.
|
||
- **Keywords** — the vocabulary by the same verdict, the assignments as a union.
|
||
- **People and identity judgements** — people by uuid and revision, and the confirmed and rejected
|
||
face assignments matched to local faces (`merge::match_faces`): by box first, and — since 0.18.0,
|
||
only on the photographs where a remote face is left over — by embedding, a pair being accepted
|
||
at cosine ≥ 0.7 when each is the other's best by a lead of ≥ 0.2 (#77; [faces.md §18.2](faces.md)). After every merge,
|
||
`dedup_people` folds people of one name whose confirmed faces agree, and a face held twice in
|
||
one photograph, through the ordinary `merged_into` redirect, which older builds already honour
|
||
(#78; [faces.md §19](faces.md)).
|
||
- **Albums** (FR-EXP-10, 0.17.0) — by uuid and revision with tombstones, and what went into each as
|
||
a set union keyed on the server's file id (a content hash on a folder library). An album's
|
||
server folder is a column of its row and travels with it; a folder on *this device* is in
|
||
`album_folders`, which the merge never reads and the upload snapshot drops (§8.3), because a
|
||
path or a SAF grant on one device means nothing on another.
|
||
- **Capture metadata** — the one exception inside `images`: a date, a camera, a lens and an ISO are
|
||
facts about the file's bytes, so a row this device has not yet read takes them from a peer that
|
||
has (`merge_metadata`), matched by `oc:fileid`.
|
||
|
||
The rest of a catalog describes *local* state — folder mtimes, cache file paths, job rows,
|
||
`tier_actual` — and importing another device's version of those would be actively wrong; the
|
||
downloaded remote is read for the tables above and discarded.
|
||
|
||
This is what keeps §6.12 substantially intact. The catalog is still deletable; what a rebuild from
|
||
sidecars cannot recover — collections and albums, which nothing in the filesystem records — is what
|
||
the sync exists to carry.
|
||
|
||
### 8.3 Two hazards the implementation must handle
|
||
|
||
**A WAL database is not one file.** Committed transactions can sit in `catalog.sqlite-wal` with the
|
||
main file lagging, so copying `catalog.sqlite` alone uploads a torn snapshot — internally consistent
|
||
as of some older point, silently missing everything since. The upload therefore never copies the
|
||
live file. It builds the snapshot in an empty file. It attaches the catalog, creates each table
|
||
from the catalog's own schema, and fills it with `INSERT … SELECT`, all inside one transaction.
|
||
That transaction holds a single read snapshot of the catalog, so concurrent writers are serialised
|
||
rather than raced, as the backup API did before. The file is then checked with `quick_check` before
|
||
it goes anywhere.
|
||
|
||
**The face crops stay out of the upload.** A crop is a ~5 KB JPEG on each `faces` row. On a 19k-face
|
||
library they are 96 MB of a 158 MB catalog. The face shards carry them to other devices, once each.
|
||
The merge reads a remote face's box and model to match it to a local one — and, where the boxes cannot decide, its embedding — never its pixels. No
|
||
device adopts a downloaded catalog as its own: a fresh device starts empty and takes faces, crops
|
||
included, from the shards. So the snapshot's `crop` is NULL, and a merge never writes a local
|
||
crop. They were first stripped (2026-08) by copying the whole file with the backup API, setting
|
||
`crop` to NULL and `VACUUM`ing. That wrote the file about three times to upload 50 MB. Since #71
|
||
the snapshot is built without them. Its header is kept as it was (`user_version`, page size and
|
||
the WAL flag), so every earlier build merges it unchanged. NFR-R2 backups still use the backup API
|
||
and keep the crops, because a backup is a file the user may have to live on. `album_folders` is
|
||
dropped from the built file for the reason §8.2 gives.
|
||
|
||
**Integer primary keys are not identities.** Two devices each allocate `collections.id = 1` for
|
||
different collections, so a row-level merge keyed on the integer id would collide them. Collections
|
||
therefore carry a **UUID**, and membership maps across devices by **image content hash**. The
|
||
integer ids stay local and are never compared across catalogs.
|
||
|
||
### 8.4 Merge rules
|
||
|
||
| Concern | Rule | Why |
|
||
|---|---|---|
|
||
| Which collection wins | Higher `revision` — a counter bumped per local edit. `modified` only breaks an exact tie | A device with a skewed clock cannot silently overwrite real work. The same reason FR-NC-9 avoids mtime for sidecars |
|
||
| Membership | **Set union**, not last-writer-wins | Two devices adding different images to one collection keep both. The exception — a removal racing an addition — resolves toward the addition, which is recoverable by removing it again. A lost addition is not |
|
||
| Deletion | Tombstone (`deleted = 1`) carrying a revision | Without it, merging against a device that still holds the collection resurrects it. With a revision, deletion competes on equal footing with a rename |
|
||
| An image the remote has and we do not | Skip the membership row | It joins on a later merge, once a scan has catalogued the file. Not an error |
|
||
| A remote from a newer schema | Decline before attaching | Attempting it would fail mid-transaction rather than declining cleanly |
|
||
| People with the same name | Folded after each merge when their faces agree ([faces.md §19](faces.md)) | Names typed separately on two devices otherwise stay two people for ever |
|
||
|
||
Merging is idempotent: running it twice reports no changes the second time. That property is tested,
|
||
because a merge that oscillates would upload on every sync forever.
|
||
|
||
### 8.5 What was rejected
|
||
|
||
**Replace-if-newer.** The literal reading of "sync the file and take the newer one". Rejected
|
||
because it is not a merge: whichever device syncs second loses every collection the first did not
|
||
have. Binary SQLite files do not merge, so "newer wins" means "older is destroyed".
|
||
|
||
**A `collections.drsc` sidecar at the library root.** The alternative that would have kept §6.12
|
||
untouched, merging as text the way edit sidecars do. Viable, and cheaper in machinery, but it means
|
||
a second serialisation format and a second merge implementation for the same data. Recorded here
|
||
because if the SQLite path proves troublesome, this is the fallback with a known shape.
|
||
|
||
---
|
||
|
||
## 9. What this document does not settle
|
||
|
||
- **FTS.** `Selector::Text` is a `LIKE` scan over filename and keywords. Adequate at 50k; if free
|
||
text over description and title becomes a real workflow, an FTS5 table is the answer, and it is
|
||
additive.
|
||
- **Smart collection materialisation.** Currently evaluated on read. If a smart collection's
|
||
membership needs to be *stable* — for manual ordering, or for a pinned set that must not shift
|
||
under the user — it needs materialising with an invalidation rule. Deferred until there is a
|
||
concrete need.
|
||
- **Multi-root capture-time collisions.** FR-CAT-11 detects duplicates on import, and FR-CAT-11a
|
||
consolidates the copies one root holds in several folders; the same image catalogued under two
|
||
roots is a related but distinct case, not yet specified.
|
||
- **Timeline granularity selection.** Which bucket size the UI picks for a given zoom is a UI
|
||
concern, but the catalog should probably suggest one from the query's date span rather than have
|
||
the UI guess.
|
||
|
||
---
|
||
|
||
## 10. People and faces
|
||
|
||
Specified by FR-CULL-8 … FR-CULL-12, NFR-SEC-5, [architecture.md §6.4](architecture.md). Gated on
|
||
spike S14 and decision D13 — the runtime and the model licences are unresolved, so this is the shape
|
||
of the subsystem, not a build order.
|
||
|
||
[faces.md](faces.md) names the models this shape is filled in with, and adds one column to §10.1's
|
||
`faces` table (`crop_px`) that the calibration in its §8 depends on.
|
||
|
||
### 10.1 Schema (a v5 migration)
|
||
|
||
```sql
|
||
CREATE TABLE people (
|
||
id INTEGER PRIMARY KEY,
|
||
uuid TEXT NOT NULL UNIQUE, -- merge identity, not the name (ARCH §6.3)
|
||
name TEXT NOT NULL,
|
||
-- Tombstone-by-redirect. A merged person must outlive its merge, or a
|
||
-- device that still has it resurrects it — same hazard collections have.
|
||
merged_into INTEGER REFERENCES people(id) ON DELETE SET NULL,
|
||
created INTEGER NOT NULL,
|
||
revision INTEGER NOT NULL DEFAULT 1,
|
||
modified INTEGER NOT NULL
|
||
);
|
||
|
||
CREATE TABLE faces (
|
||
id INTEGER PRIMARY KEY,
|
||
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
|
||
-- Normalised to the image's long edge, so a face survives the proxy it was
|
||
-- found on being regenerated at another resolution.
|
||
x REAL NOT NULL, y REAL NOT NULL, w REAL NOT NULL, h REAL NOT NULL,
|
||
landmarks BLOB, -- 5 × (x, y) f32, the alignment input
|
||
detector_confidence REAL NOT NULL,
|
||
embedding BLOB NOT NULL, -- 512 × f16, the raw model output; re-normalised on load
|
||
-- Length of that vector: the model's own reading of how recognisable the
|
||
-- crop was, and the gate on whether this face may be compared *against*
|
||
-- (faces.md §6, §9). NULL for a face stored as a unit vector before it
|
||
-- was kept.
|
||
quality REAL,
|
||
-- What the eyes are doing (FR-CULL-8a, faces.md §17): per eye P(open),
|
||
-- the source pixels across its box and the sharpness of the patch the
|
||
-- classifier saw; and P(sunglasses). All seven or none; NULL is "never
|
||
-- read", which every filter treats as unknown rather than as closed.
|
||
-- The verdict -- open, closed, sunglasses, unclear -- is a rule in
|
||
-- dr_face::eyes, not a column.
|
||
eye_right REAL,
|
||
eye_right_px REAL,
|
||
eye_right_sharp REAL,
|
||
eye_left REAL,
|
||
eye_left_px REAL,
|
||
eye_left_sharp REAL,
|
||
sunglasses REAL,
|
||
-- The 106 dense landmarks the eyes were read from, packed as 16-bit
|
||
-- fixed point over the frame: 424 bytes (schema V18). Kept so the next
|
||
-- per-face pass runs from the catalog rather than from the original.
|
||
landmarks_dense BLOB,
|
||
-- Which model produced this. An embedding is only comparable to others
|
||
-- from the same model; mixing them silently yields nonsense similarities.
|
||
model_id TEXT NOT NULL,
|
||
detected_at INTEGER NOT NULL
|
||
);
|
||
CREATE INDEX faces_image ON faces(image_id);
|
||
-- Covers the eyes-open filter's subquery. Without it every check read the
|
||
-- whole face row -- the eye columns sit after the blobs -- and one count
|
||
-- took 24 s on the reference library (schema V17).
|
||
CREATE INDEX faces_eyes ON faces(image_id, eye_right, eye_right_px, eye_right_sharp,
|
||
eye_left, eye_left_px, eye_left_sharp, sunglasses);
|
||
|
||
CREATE TABLE face_person (
|
||
face_id INTEGER PRIMARY KEY REFERENCES faces(id) ON DELETE CASCADE,
|
||
person_id INTEGER NOT NULL REFERENCES people(id) ON DELETE CASCADE,
|
||
-- Calibrated P(this face is this person), never a raw cosine (FR-CULL-9).
|
||
probability REAL NOT NULL,
|
||
-- The user said so. Never overwritten by a later inference pass.
|
||
confirmed INTEGER NOT NULL DEFAULT 0
|
||
);
|
||
CREATE INDEX face_person_person ON face_person(person_id, confirmed);
|
||
```
|
||
|
||
Three things in that schema are load-bearing:
|
||
|
||
**`model_id` on every face.** Embeddings from different models are not comparable — this is the one
|
||
mistake that produces plausible-looking garbage rather than an error. Storing the model with the
|
||
embedding means a model change is detectable and re-indexable, instead of quietly poisoning every
|
||
similarity in the library.
|
||
|
||
**Normalised bounding boxes.** Detection runs on whichever proxy exists (FR-CULL-8). Storing pixel
|
||
coordinates would bind a face to a resolution that the cache is entitled to evict and regenerate
|
||
differently.
|
||
|
||
**`confirmed` as a column, not a probability of 1.0.** A confirmation is a different kind of fact
|
||
from a confident guess, and collapsing them loses the ability to recompute suggestions without
|
||
touching user data.
|
||
|
||
### 10.2 Why clustering is not a job kind
|
||
|
||
Detection is per-image and parallel, so it is a job (`DetectFaces`, coalesced per image like any
|
||
other). Clustering is a *whole-library* operation over the embeddings detection produced — it has no
|
||
natural `subject_id`, and running it per-image would rebuild the world on every photograph.
|
||
|
||
It therefore runs as a debounced library-level pass, triggered when detection has been idle and the
|
||
face count has moved materially since the last clustering. The same reasoning as sidecar writes: the
|
||
work is cheap to defer, expensive to repeat, and nobody is waiting on it.
|
||
|
||
### 10.3 The calibration lives with the library
|
||
|
||
FR-CULL-9 requires similarity to be a calibrated probability, fitted from this library's own faces.
|
||
That fit is a property of the catalog and its model, so it is stored alongside — a small table
|
||
holding the fit parameters, its validity flag, and a hash of the face set it was derived from, so a
|
||
materially changed library recomputes rather than trusting a stale fit.
|
||
|
||
When the fit is not valid — a library with too few faces to have positive pairs — the UI says the
|
||
confidence is unavailable. It does not fall back to an untuned default dressed up as a measurement.
|
||
|
||
### 10.4 What this does not settle
|
||
|
||
- **Which model, and which runtime.** D13. Everything above holds regardless of the answer, which is
|
||
why it is specified in terms of "a 512-d embedding from a stated model" rather than a named one.
|
||
- **The clustering algorithm.** Density-based over the calibrated distance is the obvious starting
|
||
point, but the parameters are an S14 question, not a design-time one.
|
||
- **Whether embeddings sync.** NFR-SEC-5 permits it, opt-in. The shard mechanism in §7.3 is the
|
||
obvious carrier if they do, but nothing here depends on that decision.
|
||
- **Faces in trashed images.** FR-CAT-15's trash moves files; whether their faces stay indexed and
|
||
keep contributing to clusters is unspecified. Probably they should be excluded from suggestions but
|
||
not deleted, so a restore does not re-index.
|
||
|
||
---
|
||
|
||
## 10a. Bursts and near-duplicates
|
||
|
||
Specified by FR-CULL-5, implemented in `dr_catalog::bursts` (a v11 migration) with the pass that
|
||
feeds it in `dr_ui::bursts`.
|
||
|
||
A burst is a run of frames that are **adjacent in time and look like the frame before them**. Both
|
||
halves are load-bearing. Time alone groups a whole wedding ceremony, because a photographer working
|
||
steadily never leaves the gap that would end the run. Similarity alone groups a studio setup shot
|
||
across two days, which is a project rather than a moment. The bounds are two seconds and eight bits
|
||
of a 64-bit difference hash, and the reasoning for each figure is in the module.
|
||
|
||
**Two seconds, for a burst that fires ten frames in one.** `images.captured_at` is whole seconds:
|
||
EXIF's `DateTimeOriginal` has no sub-second field, and `SubSecTimeOriginal` is optional and widely
|
||
omitted. Ten frames of a burst therefore arrive sharing a timestamp, and any threshold finer than a
|
||
second is a threshold on information the catalog does not have. Where the pace really is faster than
|
||
two seconds, the similarity bound is what separates the frames.
|
||
|
||
**The signal is a perceptual hash of the thumbnail, not of the original.** `images.perceptual_hash`
|
||
is filled from the 256px thumbnails §7 already stores — vastly more resolution than a 9×8 reduction
|
||
uses — so a library that has been browsed has already paid for its signatures and no RAW is decoded
|
||
for this. The consequence is stated rather than hidden: an image with no thumbnail gets no
|
||
signature, and a frame with no signature never joins a burst. It is picked up by the next pass.
|
||
|
||
**It is a pass, not a job kind**, for exactly the reason §10.2 gives for face clustering: a burst is
|
||
a property of a *run* of frames and has no natural `subject_id`, so a per-image job would rebuild
|
||
the world once per photograph. It runs when the thumbnail sweep finishes, which is the first moment
|
||
the signatures can all be computed.
|
||
|
||
**A newly found burst arrives open.** The pass marks frames; it never takes them off the screen.
|
||
Collapsing on discovery would be tidier and would also mean a background pass removing photographs
|
||
from under someone part way through a cull. Folding a burst up is the user's act, it is remembered
|
||
(`burst_expanded`), and a burst that is already known keeps whatever state it is in — so the pass
|
||
that follows the next import does not spring open a morning's work.
|
||
|
||
**Nothing here ranks a frame.** The representative of a collapsed burst is its *earliest* frame,
|
||
which is a fact about the clock rather than a judgement about the photograph. FR-CULL-5 names the
|
||
failure this avoids — rejecting the only frame of an important moment because someone blinked — and
|
||
the only judgement in the subsystem is the user's own choice of representative, which lives in its
|
||
own table (`burst_pick`) so that rebuilding the grouping cannot erase it. Same argument as
|
||
`people.ignored` in §10.
|
||
|
||
**What the collapse costs the grid.** Which rows a collapsed burst hides has to be decided by the
|
||
query rather than by the cells, because the grid is a window (`LIMIT n OFFSET k`) and the frames it
|
||
hides are mostly not loaded. So the predicate joins `VISIBLE` in every query that lists or counts
|
||
cells, under the same discipline: present in four places of five, the header's count, the
|
||
scrollbar, the shift-click range and the scrub's ordinal stop describing the same list.
|
||
|
||
**What this does not settle.** Bursts are local: the tables ride along in the uploaded catalog
|
||
snapshot and nothing on the far side reads them, so a second device rebuilds its own grouping from
|
||
its own signatures. Making `burst_pick` cross-device is a merge question of the same shape as §8.4's
|
||
and is not answered here.
|
||
|
||
---
|
||
|
||
## 11. Requirements touched
|
||
|
||
| ID | How this document addresses it |
|
||
|---|---|
|
||
| FR-CAT-1 | §3 incremental scan, cancellable and resumable via §6 jobs |
|
||
| FR-CAT-3 | §7 thumbnail pyramid, two size classes, embedded-preview fast path |
|
||
| FR-CAT-4 | §4.1 windowed queries, memory independent of catalog size |
|
||
| FR-CAT-5 | §3.5 two-pass metadata |
|
||
| FR-CAT-6 | §4.3 indexed filter compilation, §5 selectors |
|
||
| FR-CAT-7 | §2 collections schema, §5 manual and smart, §7a hierarchy, membership and editing |
|
||
| FR-CAT-9 | §3.4 the offline/deleted distinction and the sweep guard |
|
||
| FR-CAT-11 | §3.5 lazy content hashing |
|
||
| FR-NC-3 | §7.2 range-extract for remote thumbnails |
|
||
| FR-NC-6a | §5 shared selector type |
|
||
| FR-NC-6c | §3.5 metadata honesty, §7.2 availability-driven sourcing |
|
||
| NFR-P1 | §3.1 one stat per directory, not per file |
|
||
| NFR-P3 | §7.1 on-demand generation |
|
||
| NFR-ARCH-2 | §6.3 priority classes shared with the GPU scheduler |
|
||
| FR-CULL-5 | §10a burst grouping: capture-time proximity and image similarity, collapse without selection |
|
||
| NFR-ARCH-3 | §4.3 query cancellation, §6 job cancellation |
|
||
| NFR-RES-4 | §7.3 LRU cap, eviction order |
|
||
| FR-CULL-8 | §10.1 `faces` schema, §6.1 `DetectFaces` job kind on the proxy tier |
|
||
| FR-CULL-9 | §10.3 per-library calibration, stored with its validity and source hash |
|
||
| FR-CULL-10 | §10.1 `people` and `face_person`, merge-by-redirect, §10.2 clustering as a library pass |
|
||
| FR-CULL-11 | §5 `Selector::Person`, confirmed-only by default |
|
||
| FR-CULL-12 | §10.1 derived data in the catalog; names to the sidecar, UUID as merge identity |
|
||
| FR-CULL-8a | §10.1 the seven eye columns, derived like the embedding; the filter term is `RatingFilter::eyes_open` |
|
||
| NFR-SEC-5 | §10.4 sync left undecided and off; nothing in §10 emits an embedding |
|