dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Sync face data as sealed shards, so a second device does not re-index

Browse Source 
Indexing 23,500 images is about two hours of CPU, and the result is
byte-identical on every device: the same model over the same proxy
produces the same embedding. Paying for it once per account rather than
once per device is the point.

Shards rather than the catalog snapshot, because the snapshot goes up
whole on every sync and a fully indexed library carries roughly 30 MB of
embeddings. That is exactly the cost the thumbnail store's 25 MB cap
exists to bound, so face shards use the same cap -- imported from
dr_thumbs rather than restated, since the number is a statement about
sync cost and the two must not drift apart.

The split follows the one already there: bulk immutable data in sealed
shards, small mutable data in the catalog snapshot. Faces, landmarks,
embeddings and run markers shard; people, names and assignments ride the
catalog and merge by uuid.

Keyed on oc:fileid throughout, never on image_id, because a row id means
nothing on another device.

The run marker travels with the faces it describes. Without it a
receiving device cannot tell an image with no faces from one never
examined, and would re-detect every landscape it had just adopted.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-26 22:41:50 +02:00
co-authored by Claude Opus 5
parent 26a1eb7e28
commit 96d07da15f
7 changed files with 1179 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/traceability.md
+5 -5
View File
@@ -9,8 +9,8 @@ Denominators are parsed from [`requirements.md`](requirements.md) at run time, n
| Metric | Value |
|---|---|
| Source files scanned | 242 |
| TRACES tags found | 628 |
| Source files scanned | 243 |
| TRACES tags found | 629 |
| Requirements defined | 177 |
| Requirements covered | 97 |
| **Coverage** | **54.8%** (97/177) |
@@ -54,7 +54,7 @@ _None._
| FR-CULL-12 | [`core/dr-catalog/src/faces.rs:1`](../core/dr-catalog/src/faces.rs#L1), [`core/dr-catalog/src/schema.rs:384`](../core/dr-catalog/src/schema.rs#L384), [`ui/dr-ui/src/identity.rs:1`](../ui/dr-ui/src/identity.rs#L1), [`ui/dr-ui/ui/identity.slint:1`](../ui/dr-ui/ui/identity.slint#L1) |
| FR-CULL-2 | [`core/dr-decode/src/locate.rs:1`](../core/dr-decode/src/locate.rs#L1), [`core/dr-decode/src/preview.rs:161`](../core/dr-decode/src/preview.rs#L161), [`ui/dr-ui/src/import.rs:464`](../ui/dr-ui/src/import.rs#L464) |
| FR-CULL-4 | [`core/dr-catalog/src/rating.rs:1`](../core/dr-catalog/src/rating.rs#L1), [`core/dr-pipeline/src/sidecar.rs:137`](../core/dr-pipeline/src/sidecar.rs#L137), [`ui/dr-ui/src/library.rs:203`](../ui/dr-ui/src/library.rs#L203), [`ui/dr-ui/src/library.rs:364`](../ui/dr-ui/src/library.rs#L364) |
| FR-CULL-8 | [`core/dr-catalog/src/faces.rs:1`](../core/dr-catalog/src/faces.rs#L1), [`core/dr-catalog/src/schema.rs:343`](../core/dr-catalog/src/schema.rs#L343), [`core/dr-catalog/src/schema.rs:384`](../core/dr-catalog/src/schema.rs#L384), [`ui/dr-ui/src/faces.rs:1`](../ui/dr-ui/src/faces.rs#L1) |
| FR-CULL-8 | [`core/dr-catalog/src/face_shard.rs:1`](../core/dr-catalog/src/face_shard.rs#L1), [`core/dr-catalog/src/faces.rs:1`](../core/dr-catalog/src/faces.rs#L1), [`core/dr-catalog/src/schema.rs:343`](../core/dr-catalog/src/schema.rs#L343), [`core/dr-catalog/src/schema.rs:384`](../core/dr-catalog/src/schema.rs#L384), [`ui/dr-ui/src/faces.rs:1`](../ui/dr-ui/src/faces.rs#L1) |
| FR-CULL-9 | [`core/dr-catalog/src/faces.rs:1`](../core/dr-catalog/src/faces.rs#L1), [`core/dr-catalog/src/schema.rs:384`](../core/dr-catalog/src/schema.rs#L384), [`ui/dr-ui/src/faces.rs:1`](../ui/dr-ui/src/faces.rs#L1), [`ui/dr-ui/src/identity_ui.rs:1`](../ui/dr-ui/src/identity_ui.rs#L1) |
| FR-DEV-2 | [`core/dr-pipeline/src/operation.rs:352`](../core/dr-pipeline/src/operation.rs#L352) |
| FR-DEV-3 | [`core/dr-gpu/src/adjust.rs:2167`](../core/dr-gpu/src/adjust.rs#L2167), [`core/dr-gpu/src/adjust.rs:651`](../core/dr-gpu/src/adjust.rs#L651), [`core/dr-gpu/src/adjust.rs:770`](../core/dr-gpu/src/adjust.rs#L770), [`core/dr-gpu/src/adjust.rs:84`](../core/dr-gpu/src/adjust.rs#L84), [`core/dr-gpu/tests/tone_curve.rs:1`](../core/dr-gpu/tests/tone_curve.rs#L1), [`core/dr-pipeline/src/detail.rs:364`](../core/dr-pipeline/src/detail.rs#L364), [`core/dr-pipeline/src/detail.rs:439`](../core/dr-pipeline/src/detail.rs#L439), [`core/dr-pipeline/src/framing.rs:188`](../core/dr-pipeline/src/framing.rs#L188), [`core/dr-pipeline/src/framing.rs:602`](../core/dr-pipeline/src/framing.rs#L602), [`core/dr-pipeline/src/graph.rs:154`](../core/dr-pipeline/src/graph.rs#L154), [`core/dr-pipeline/src/graph.rs:457`](../core/dr-pipeline/src/graph.rs#L457), [`core/dr-pipeline/src/mask.rs:120`](../core/dr-pipeline/src/mask.rs#L120), [`core/dr-pipeline/src/operation.rs:299`](../core/dr-pipeline/src/operation.rs#L299), [`core/dr-pipeline/src/operation.rs:473`](../core/dr-pipeline/src/operation.rs#L473), [`core/dr-pipeline/src/ops/capture_sharpen.rs:1`](../core/dr-pipeline/src/ops/capture_sharpen.rs#L1), [`core/dr-pipeline/src/ops/capture_sharpen.rs:207`](../core/dr-pipeline/src/ops/capture_sharpen.rs#L207), [`core/dr-pipeline/src/ops/curve.rs:1`](../core/dr-pipeline/src/ops/curve.rs#L1), [`core/dr-pipeline/src/ops/curve.rs:218`](../core/dr-pipeline/src/ops/curve.rs#L218), [`core/dr-pipeline/src/ops/curve.rs:631`](../core/dr-pipeline/src/ops/curve.rs#L631), [`core/dr-pipeline/src/ops/curve.rs:99`](../core/dr-pipeline/src/ops/curve.rs#L99), [`core/dr-pipeline/src/ops/local_contrast.rs:1`](../core/dr-pipeline/src/ops/local_contrast.rs#L1), [`core/dr-pipeline/src/ops/noise_reduction.rs:1`](../core/dr-pipeline/src/ops/noise_reduction.rs#L1), [`core/dr-pipeline/src/ops/noise_reduction.rs:270`](../core/dr-pipeline/src/ops/noise_reduction.rs#L270), [`core/dr-pipeline/src/sidecar.rs:1343`](../core/dr-pipeline/src/sidecar.rs#L1343), [`core/dr-pipeline/src/sidecar.rs:1399`](../core/dr-pipeline/src/sidecar.rs#L1399), [`core/dr-pipeline/src/sidecar.rs:158`](../core/dr-pipeline/src/sidecar.rs#L158), [`core/dr-pipeline/tests/tone_curve.rs:1`](../core/dr-pipeline/tests/tone_curve.rs#L1), [`ui/dr-ui/src/develop.rs:101`](../ui/dr-ui/src/develop.rs#L101), [`ui/dr-ui/src/develop.rs:1111`](../ui/dr-ui/src/develop.rs#L1111), [`ui/dr-ui/src/develop.rs:132`](../ui/dr-ui/src/develop.rs#L132), [`ui/dr-ui/src/develop.rs:1584`](../ui/dr-ui/src/develop.rs#L1584), [`ui/dr-ui/src/develop.rs:1599`](../ui/dr-ui/src/develop.rs#L1599), [`ui/dr-ui/src/develop.rs:1621`](../ui/dr-ui/src/develop.rs#L1621), [`ui/dr-ui/src/develop.rs:1753`](../ui/dr-ui/src/develop.rs#L1753), [`ui/dr-ui/src/develop.rs:1831`](../ui/dr-ui/src/develop.rs#L1831), [`ui/dr-ui/src/develop.rs:238`](../ui/dr-ui/src/develop.rs#L238), [`ui/dr-ui/src/develop.rs:270`](../ui/dr-ui/src/develop.rs#L270), [`ui/dr-ui/src/develop.rs:2762`](../ui/dr-ui/src/develop.rs#L2762), [`ui/dr-ui/src/develop.rs:3194`](../ui/dr-ui/src/develop.rs#L3194), [`ui/dr-ui/src/develop.rs:3248`](../ui/dr-ui/src/develop.rs#L3248), [`ui/dr-ui/src/develop.rs:3292`](../ui/dr-ui/src/develop.rs#L3292), [`ui/dr-ui/src/develop.rs:3342`](../ui/dr-ui/src/develop.rs#L3342), [`ui/dr-ui/src/develop.rs:506`](../ui/dr-ui/src/develop.rs#L506), [`ui/dr-ui/src/develop.rs:544`](../ui/dr-ui/src/develop.rs#L544), [`ui/dr-ui/src/lib.rs:1323`](../ui/dr-ui/src/lib.rs#L1323), [`ui/dr-ui/src/lib.rs:1982`](../ui/dr-ui/src/lib.rs#L1982), [`ui/dr-ui/src/lib.rs:293`](../ui/dr-ui/src/lib.rs#L293), [`ui/dr-ui/src/library.rs:411`](../ui/dr-ui/src/library.rs#L411), [`ui/dr-ui/src/masks_ui.rs:218`](../ui/dr-ui/src/masks_ui.rs#L218), [`ui/dr-ui/src/masks_ui.rs:41`](../ui/dr-ui/src/masks_ui.rs#L41), [`ui/dr-ui/src/masks_ui.rs:816`](../ui/dr-ui/src/masks_ui.rs#L816), [`ui/dr-ui/src/masks_ui.rs:930`](../ui/dr-ui/src/masks_ui.rs#L930), [`ui/dr-ui/src/segmentation.rs:218`](../ui/dr-ui/src/segmentation.rs#L218), [`ui/dr-ui/ui/app.slint:1985`](../ui/dr-ui/ui/app.slint#L1985), [`ui/dr-ui/ui/app.slint:949`](../ui/dr-ui/ui/app.slint#L949) |
@@ -92,7 +92,7 @@ _None._
| FR-NC-6a | [`core/dr-catalog/src/cache.rs:1`](../core/dr-catalog/src/cache.rs#L1), [`core/dr-catalog/src/schema.rs:555`](../core/dr-catalog/src/schema.rs#L555), [`core/dr-catalog/src/schema.rs:956`](../core/dr-catalog/src/schema.rs#L956), [`core/dr-types/src/selector.rs:1`](../core/dr-types/src/selector.rs#L1), [`core/dr-types/src/settings.rs:1`](../core/dr-types/src/settings.rs#L1), [`ui/dr-ui/src/collections_ui.rs:3195`](../ui/dr-ui/src/collections_ui.rs#L3195), [`ui/dr-ui/src/collections_ui.rs:621`](../ui/dr-ui/src/collections_ui.rs#L621), [`ui/dr-ui/src/collections_ui.rs:683`](../ui/dr-ui/src/collections_ui.rs#L683), [`ui/dr-ui/src/lib.rs:1658`](../ui/dr-ui/src/lib.rs#L1658), [`ui/dr-ui/src/lib.rs:2455`](../ui/dr-ui/src/lib.rs#L2455), [`ui/dr-ui/src/library.rs:1324`](../ui/dr-ui/src/library.rs#L1324), [`ui/dr-ui/src/library.rs:1347`](../ui/dr-ui/src/library.rs#L1347), [`ui/dr-ui/src/library.rs:1505`](../ui/dr-ui/src/library.rs#L1505), [`ui/dr-ui/src/library_ui.rs:1065`](../ui/dr-ui/src/library_ui.rs#L1065), [`ui/dr-ui/src/library_ui.rs:1138`](../ui/dr-ui/src/library_ui.rs#L1138), [`ui/dr-ui/src/library_ui.rs:1239`](../ui/dr-ui/src/library_ui.rs#L1239), [`ui/dr-ui/src/library_ui.rs:1294`](../ui/dr-ui/src/library_ui.rs#L1294), [`ui/dr-ui/src/library_ui.rs:1412`](../ui/dr-ui/src/library_ui.rs#L1412), [`ui/dr-ui/src/library_ui.rs:1531`](../ui/dr-ui/src/library_ui.rs#L1531), [`ui/dr-ui/src/library_ui.rs:2058`](../ui/dr-ui/src/library_ui.rs#L2058), [`ui/dr-ui/src/library_ui.rs:267`](../ui/dr-ui/src/library_ui.rs#L267), [`ui/dr-ui/src/library_ui.rs:278`](../ui/dr-ui/src/library_ui.rs#L278), [`ui/dr-ui/src/library_ui.rs:286`](../ui/dr-ui/src/library_ui.rs#L286), [`ui/dr-ui/src/library_ui.rs:298`](../ui/dr-ui/src/library_ui.rs#L298), [`ui/dr-ui/src/library_ui.rs:307`](../ui/dr-ui/src/library_ui.rs#L307), [`ui/dr-ui/src/library_ui.rs:399`](../ui/dr-ui/src/library_ui.rs#L399), [`ui/dr-ui/src/library_ui.rs:409`](../ui/dr-ui/src/library_ui.rs#L409), [`ui/dr-ui/src/library_ui.rs:451`](../ui/dr-ui/src/library_ui.rs#L451), [`ui/dr-ui/src/library_ui.rs:514`](../ui/dr-ui/src/library_ui.rs#L514), [`ui/dr-ui/src/library_ui.rs:5239`](../ui/dr-ui/src/library_ui.rs#L5239), [`ui/dr-ui/src/library_ui.rs:5257`](../ui/dr-ui/src/library_ui.rs#L5257), [`ui/dr-ui/src/library_ui.rs:5269`](../ui/dr-ui/src/library_ui.rs#L5269), [`ui/dr-ui/src/library_ui.rs:545`](../ui/dr-ui/src/library_ui.rs#L545), [`ui/dr-ui/src/library_ui.rs:557`](../ui/dr-ui/src/library_ui.rs#L557), [`ui/dr-ui/src/settings_store.rs:1`](../ui/dr-ui/src/settings_store.rs#L1), [`ui/dr-ui/src/settings_ui.rs:1`](../ui/dr-ui/src/settings_ui.rs#L1), [`ui/dr-ui/ui/app.slint:2490`](../ui/dr-ui/ui/app.slint#L2490), [`ui/dr-ui/ui/app.slint:612`](../ui/dr-ui/ui/app.slint#L612), [`ui/dr-ui/ui/collections.slint:249`](../ui/dr-ui/ui/collections.slint#L249), [`ui/dr-ui/ui/collections.slint:369`](../ui/dr-ui/ui/collections.slint#L369), [`ui/dr-ui/ui/collections.slint:52`](../ui/dr-ui/ui/collections.slint#L52), [`ui/dr-ui/ui/collections.slint:682`](../ui/dr-ui/ui/collections.slint#L682), [`ui/dr-ui/ui/collections.slint:84`](../ui/dr-ui/ui/collections.slint#L84), [`ui/dr-ui/ui/icons.slint:260`](../ui/dr-ui/ui/icons.slint#L260), [`ui/dr-ui/ui/library.slint:974`](../ui/dr-ui/ui/library.slint#L974) |
| FR-NC-6b | [`ui/dr-ui/src/library_ui.rs:1294`](../ui/dr-ui/src/library_ui.rs#L1294) |
| FR-NC-6c | [`core/dr-sync-nextcloud/src/desktop_client.rs:30`](../core/dr-sync-nextcloud/src/desktop_client.rs#L30), [`core/dr-types/src/lib.rs:119`](../core/dr-types/src/lib.rs#L119), [`core/dr-types/src/lib.rs:201`](../core/dr-types/src/lib.rs#L201), [`ui/dr-ui/src/activity.rs:1`](../ui/dr-ui/src/activity.rs#L1), [`ui/dr-ui/src/collections_ui.rs:3195`](../ui/dr-ui/src/collections_ui.rs#L3195), [`ui/dr-ui/src/collections_ui.rs:621`](../ui/dr-ui/src/collections_ui.rs#L621), [`ui/dr-ui/src/collections_ui.rs:683`](../ui/dr-ui/src/collections_ui.rs#L683), [`ui/dr-ui/src/library_ui.rs:1065`](../ui/dr-ui/src/library_ui.rs#L1065), [`ui/dr-ui/src/library_ui.rs:1138`](../ui/dr-ui/src/library_ui.rs#L1138), [`ui/dr-ui/ui/collections.slint:249`](../ui/dr-ui/ui/collections.slint#L249), [`ui/dr-ui/ui/collections.slint:682`](../ui/dr-ui/ui/collections.slint#L682), [`ui/dr-ui/ui/icons.slint:260`](../ui/dr-ui/ui/icons.slint#L260) |
| FR-NC-7 | [`core/dr-sync-nextcloud/src/lib.rs:95`](../core/dr-sync-nextcloud/src/lib.rs#L95), [`ui/dr-ui/src/derived_sync.rs:1`](../ui/dr-ui/src/derived_sync.rs#L1), [`ui/dr-ui/src/library.rs:2624`](../ui/dr-ui/src/library.rs#L2624), [`ui/dr-ui/src/library_ui.rs:3608`](../ui/dr-ui/src/library_ui.rs#L3608), [`ui/dr-ui/ui/settings.slint:340`](../ui/dr-ui/ui/settings.slint#L340) |
| FR-NC-7 | [`core/dr-catalog/src/face_shard.rs:1`](../core/dr-catalog/src/face_shard.rs#L1), [`core/dr-sync-nextcloud/src/lib.rs:95`](../core/dr-sync-nextcloud/src/lib.rs#L95), [`ui/dr-ui/src/derived_sync.rs:1`](../ui/dr-ui/src/derived_sync.rs#L1), [`ui/dr-ui/src/library.rs:2624`](../ui/dr-ui/src/library.rs#L2624), [`ui/dr-ui/src/library_ui.rs:3608`](../ui/dr-ui/src/library_ui.rs#L3608), [`ui/dr-ui/ui/settings.slint:340`](../ui/dr-ui/ui/settings.slint#L340) |
| FR-NC-7a | [`core/dr-ingest/src/layout.rs:1`](../core/dr-ingest/src/layout.rs#L1), [`core/dr-sync/src/upload.rs:1`](../core/dr-sync/src/upload.rs#L1), [`core/dr-sync/src/upload.rs:40`](../core/dr-sync/src/upload.rs#L40), [`core/dr-types/src/settings.rs:116`](../core/dr-types/src/settings.rs#L116), [`ui/dr-ui/src/import.rs:1`](../ui/dr-ui/src/import.rs#L1), [`ui/dr-ui/src/import.rs:97`](../ui/dr-ui/src/import.rs#L97), [`ui/dr-ui/src/import_ui.rs:1`](../ui/dr-ui/src/import_ui.rs#L1), [`ui/dr-ui/src/lib.rs:1050`](../ui/dr-ui/src/lib.rs#L1050), [`ui/dr-ui/ui/import.slint:5`](../ui/dr-ui/ui/import.slint#L5) |
| FR-NC-7b | [`core/dr-ingest/src/lib.rs:733`](../core/dr-ingest/src/lib.rs#L733), [`core/dr-sync/src/upload.rs:1`](../core/dr-sync/src/upload.rs#L1), [`ui/dr-ui/src/import.rs:123`](../ui/dr-ui/src/import.rs#L123), [`ui/dr-ui/src/import.rs:337`](../ui/dr-ui/src/import.rs#L337), [`ui/dr-ui/src/import.rs:585`](../ui/dr-ui/src/import.rs#L585), [`ui/dr-ui/src/import.rs:97`](../ui/dr-ui/src/import.rs#L97), [`ui/dr-ui/src/import_ui.rs:1`](../ui/dr-ui/src/import_ui.rs#L1), [`ui/dr-ui/src/lib.rs:1050`](../ui/dr-ui/src/lib.rs#L1050) |
| FR-NC-8 | [`core/dr-pipeline/src/sidecar.rs:120`](../core/dr-pipeline/src/sidecar.rs#L120), [`core/dr-pipeline/src/sidecar.rs:90`](../core/dr-pipeline/src/sidecar.rs#L90), [`ui/dr-ui/src/lib.rs:1627`](../ui/dr-ui/src/lib.rs#L1627), [`ui/dr-ui/src/library.rs:364`](../ui/dr-ui/src/library.rs#L364), [`ui/dr-ui/src/library_ui.rs:466`](../ui/dr-ui/src/library_ui.rs#L466) |
@@ -125,7 +125,7 @@ _None._
| NFR-R7 | [`core/dr-gpu/src/error.rs:1`](../core/dr-gpu/src/error.rs#L1) |
| NFR-R8 | [`core/dr-gpu/src/error.rs:1`](../core/dr-gpu/src/error.rs#L1) |
| NFR-RES-1 | [`core/dr-pipeline/src/history.rs:55`](../core/dr-pipeline/src/history.rs#L55), [`ui/dr-ui/src/lib.rs:62`](../ui/dr-ui/src/lib.rs#L62) |
| NFR-RES-4 | [`core/dr-catalog/src/cache.rs:1`](../core/dr-catalog/src/cache.rs#L1), [`core/dr-catalog/src/schema.rs:555`](../core/dr-catalog/src/schema.rs#L555), [`core/dr-thumbs/src/codec.rs:1`](../core/dr-thumbs/src/codec.rs#L1), [`core/dr-thumbs/src/lib.rs:1`](../core/dr-thumbs/src/lib.rs#L1), [`core/dr-thumbs/src/lib.rs:376`](../core/dr-thumbs/src/lib.rs#L376), [`ui/dr-ui/src/library.rs:2598`](../ui/dr-ui/src/library.rs#L2598) |
| NFR-RES-4 | [`core/dr-catalog/src/cache.rs:1`](../core/dr-catalog/src/cache.rs#L1), [`core/dr-catalog/src/face_shard.rs:1`](../core/dr-catalog/src/face_shard.rs#L1), [`core/dr-catalog/src/schema.rs:555`](../core/dr-catalog/src/schema.rs#L555), [`core/dr-thumbs/src/codec.rs:1`](../core/dr-thumbs/src/codec.rs#L1), [`core/dr-thumbs/src/lib.rs:1`](../core/dr-thumbs/src/lib.rs#L1), [`core/dr-thumbs/src/lib.rs:376`](../core/dr-thumbs/src/lib.rs#L376), [`ui/dr-ui/src/library.rs:2598`](../ui/dr-ui/src/library.rs#L2598) |
| NFR-SEC-1 | [`core/dr-decode/src/error.rs:1`](../core/dr-decode/src/error.rs#L1) |
| NFR-SEC-5 | [`core/dr-catalog/src/faces.rs:1`](../core/dr-catalog/src/faces.rs#L1), [`core/dr-catalog/src/schema.rs:384`](../core/dr-catalog/src/schema.rs#L384), [`ui/dr-ui/src/faces.rs:1`](../ui/dr-ui/src/faces.rs#L1), [`ui/dr-ui/src/identity.rs:1`](../ui/dr-ui/src/identity.rs#L1), [`ui/dr-ui/src/identity_ui.rs:1`](../ui/dr-ui/src/identity_ui.rs#L1), [`ui/dr-ui/ui/identity.slint:1`](../ui/dr-ui/ui/identity.slint#L1) |
| R1 | [`tools/traceability/src/lib.rs:495`](../tools/traceability/src/lib.rs#L495), [`tools/traceability/src/lib.rs:499`](../tools/traceability/src/lib.rs#L499) |