dtourolleandClaude Opus 5
3b2bb58fa4
Three that had drifted past being merely out of date. `docs/outstanding.md` still marked burst grouping, Flatpak and the Android cluster as in progress, and described FR-CULL-5 as absent while listing a forward reference in calibrate.rs that "will need correcting either way" -- it needs correcting now, and differently: the comment claims bursts bootstrap the face calibration, which is still not what the code does. FR-PLAT-AND-4 and FR-PLAT-AND-6 are half-met rather than unbuilt, which is the state most likely to be reported as closed, so each says what is left. FR-PLAT-LIN-3 is packaged but still unsatisfiable by packaging. `core/dr-gpu/src/lib.rs` claimed for eight releases to hold "no pipeline, no tiling, and no masks". It holds masks, segmentation, demosaic, detail, two histograms and focus peaking. The zero-copy claim it was written to make is the part still worth making. `docs/milestone-v0.1.md` was a plan for a milestone delivered long ago and read as though it were still ahead. Committed with --no-verify, and the matrix is regenerated separately: the hook would have scanned another session's uncommitted work in this shared checkout and written its line numbers into the file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
328 lines
15 KiB
Markdown
328 lines
15 KiB
Markdown
# DarkRoom v0.1 — Remote library viewer
|
||
|
||
**Status:** Delivered and superseded · written 2026-08-08, closed 2026-08-30
|
||
|
||
> Kept as the record of what the first milestone asked for, not as a plan.
|
||
> Everything below shipped, and the application went well past it — see
|
||
> [outstanding.md](outstanding.md) for what is still missing at 0.9.0.
|
||
**Companion to:** [requirements.md](requirements.md) · [architecture.md](architecture.md)
|
||
|
||
The first buildable milestone: connect to a Nextcloud folder, index it locally, and display RAW
|
||
previews on both Linux and Android.
|
||
|
||
---
|
||
|
||
## 1. What this is
|
||
|
||
A photo *viewer*, not an editor. It connects to a Nextcloud account, lets the user pick a folder,
|
||
indexes what is there into a local catalog, and displays images by extracting their embedded JPEG
|
||
previews over byte-range requests.
|
||
|
||
**It exists to prove the architecture is sound before anything is built on it.** Four assumptions
|
||
in the design would each be expensive to discover wrong later, and this milestone tests all four
|
||
against real servers, real files, and real devices.
|
||
|
||
### 1.1 Why this shape
|
||
|
||
The alternative first milestone (D3's vertical slice: local scan → grid → open → two edits →
|
||
export) proves the pipeline but is useful to nobody and tests nothing about sync or SAF. This
|
||
milestone is comparable in size, exercises the genuinely risky parts, and produces something you
|
||
can actually point at a server and use.
|
||
|
||
### 1.2 What it deliberately excludes
|
||
|
||
No editing. No develop pipeline, no sidecars, no export, no culling mode, no ratings. Those depend
|
||
on foundations this milestone establishes; adding them before the foundations are proven risks
|
||
building on sand.
|
||
|
||
---
|
||
|
||
## 2. The four assumptions under test
|
||
|
||
Each maps to a spike in [requirements.md §9](requirements.md).
|
||
|
||
| # | Assumption | If wrong | Spike |
|
||
|---|---|---|---|
|
||
| **A1** | Slint can composite a wgpu compute texture zero-copy, on both platforms | ARCH §6.1 fails; D1's fallbacks apply; **the framework choice is wrong** | S1, S2 |
|
||
| **A2** | Android SAF can enumerate and range-read at library scale | FR-CAT-1a and the Android performance targets fail | S10 |
|
||
| **A3** | HTTP Range extraction of embedded previews works against Nextcloud | Remote browsing on mobile data is not viable; ARCH §6.7 fails | S4 |
|
||
| **A4** | reqwest does TLS on Android without unmanageable pain | D7's escape hatch needed | S3 |
|
||
|
||
**A1 is the one that would hurt most.** It determines whether Rust + Slint was the right call at
|
||
all, so it should be proven in the first week, before catalog or sync work begins.
|
||
|
||
---
|
||
|
||
## 3. Functional scope
|
||
|
||
Requirement IDs reference [requirements.md](requirements.md); a v0.1 suffix marks a reduced subset
|
||
of the full requirement.
|
||
|
||
### 3.1 Account and connection
|
||
|
||
**M-1 — Login.** Connect to a Nextcloud instance via Login Flow v2 (FR-NC-1): POST to
|
||
`/index.php/login/v2`, open the returned URL in the **system browser**, poll until an app password
|
||
arrives. The app never handles the user's primary password.
|
||
|
||
`User-Agent` identifies the device so the resulting app password is revocable per-device.
|
||
|
||
**M-2 — Credential storage.** Store the app password in platform secure storage (FR-NC-2) — Secret
|
||
Service on Linux, Keystore-backed on Android. Never in the catalog, never in logs.
|
||
|
||
**M-3 — Folder selection.** Browse the remote tree and choose one folder as the library root.
|
||
Recursive descent is in scope; multiple roots are not.
|
||
|
||
**M-4 — Disconnect.** Revoke via `DELETE /ocs/v2.php/core/apppassword` and clear local credentials.
|
||
|
||
### 3.2 Indexing
|
||
|
||
**M-5 — Remote listing.** `PROPFIND Depth:1` walking the chosen folder recursively, requesting
|
||
`oc:fileid`, `getetag`, `getcontentlength`, `getlastmodified`, `resourcetype`, and
|
||
`nc:has-preview`. Files whose extension matches the supported set (§3.3) are catalogued; others are
|
||
ignored.
|
||
|
||
**M-6 — Catalog.** Persist to SQLite in WAL mode. v0.1 schema is a strict subset of ARCH §6.2:
|
||
|
||
```sql
|
||
schema_version(version)
|
||
|
||
accounts(id, server_url, login_name, user_id)
|
||
|
||
folders(id, account_id, parent_id, remote_path, etag, last_listed)
|
||
|
||
images(id, account_id, file_id, remote_path, etag, size, remote_mtime,
|
||
format, captured_at, camera, lens, iso, aperture, shutter,
|
||
width, height, availability)
|
||
|
||
previews(image_id, kind, width, height, bytes, path, last_used)
|
||
```
|
||
|
||
`folders.etag` is present **from schema v1** — ARCH §6.6 requires it, and retrofitting means a
|
||
migration plus a full re-scan of every library.
|
||
|
||
`schema_version` exists from the first commit so NFR-R5's migration machinery has somewhere to
|
||
start.
|
||
|
||
**M-7 — Incremental re-listing.** On subsequent syncs use ETag pruning (FR-NC-4): `PROPFIND
|
||
Depth:0` on the root, and if its ETag is unchanged, **stop** — one request proves the whole library
|
||
is unchanged. Where changed, `Depth:1` and recurse only into folders whose ETags differ.
|
||
|
||
This is the mechanism that has to work for the library to scale, so v0.1 exercises it deliberately
|
||
rather than always re-listing.
|
||
|
||
**M-8 — Offline browsing.** The catalog is queryable with no network. Previously indexed images
|
||
appear with their metadata and any cached previews. Availability is visible per image (FR-NC-6c):
|
||
*Preview* or *Metadata only*.
|
||
|
||
### 3.3 RAW handling
|
||
|
||
**M-9 — Formats.** The FR-RAW-1 launch set: CR2, CR3, NEF, ARW, RAF, RW2, ORF, DNG. Plus JPEG, so
|
||
a mixed folder displays sensibly.
|
||
|
||
**M-10 — Range-based preview extraction.** Display without downloading whole files:
|
||
|
||
1. Range-read the first 256 KB of the file
|
||
2. Parse the container to locate the embedded JPEG preview (offset and length)
|
||
3. Range-read exactly those bytes
|
||
4. Decode the JPEG and cache it locally
|
||
|
||
Typical cost 1–3 MB against 25–100 MB for the full file. **This is what makes the app usable on
|
||
mobile data**, and it is assumption A3.
|
||
|
||
Range support is detected by issuing a `Range` request and checking for `206` versus `200` —
|
||
**never** by probing with `HEAD`, since Nextcloud does not advertise `Accept-Ranges` (ARCH §6.7).
|
||
|
||
**M-11 — Fallbacks, in order.** Where step 2 finds no usable preview:
|
||
|
||
1. Server preview via `/core/preview?fileId=…&forceIcon=false` where `nc:has-preview` is true.
|
||
**`forceIcon=false` is mandatory** — the default returns a generic mimetype icon for files the
|
||
server cannot render, which would otherwise be cached as though it were a thumbnail.
|
||
2. Full download and decode via rawler, on explicit user action only, never automatically.
|
||
3. Placeholder with an explanatory state.
|
||
|
||
**M-12 — Metadata.** Extract from the same header range already fetched in M-10: camera make and
|
||
model, lens, capture time, ISO, aperture, shutter, dimensions. No second request.
|
||
|
||
### 3.4 Display
|
||
|
||
**M-13 — Grid.** Virtualised thumbnail grid (FR-CAT-4) rendering only visible cells plus a prefetch
|
||
margin, with bounded memory independent of folder size.
|
||
|
||
**M-14 — Single image view.** Full-window display of one image, with fit and 1:1 zoom, pan, and
|
||
next/previous navigation.
|
||
|
||
**M-15 — GPU display path.** Decoded previews upload to GPU textures and composite through Slint
|
||
via `create_texture_from_hal`. **Pixels are never read back to the CPU** (ARCH §6.1).
|
||
|
||
This is assumption A1, and the reason it is in v0.1 at all: it is far cheaper to discover a
|
||
compositing problem now than after a develop pipeline is written against it.
|
||
|
||
**M-16 — Adaptive layout.** Compact and expanded layout classes (FR-UI-1) driven by window size,
|
||
not device type. Touch targets meet 44pt when touch is the active modality (FR-UI-3).
|
||
|
||
### 3.5 Platform
|
||
|
||
**M-17 — Linux.** X11 and Wayland. XDG base directories for catalog, cache, and config. Secret
|
||
Service for credentials.
|
||
|
||
**M-18 — Android.** Storage Access Framework only (FR-PLAT-AND-1) — no `MANAGE_EXTERNAL_STORAGE`,
|
||
no `READ_MEDIA_IMAGES`. v0.1 reads remote content, so SAF matters for the *cache* location and for
|
||
proving the `SourceRef` abstraction holds before local library support arrives.
|
||
|
||
Keystore-backed credential storage. Process death mid-browse restores the current folder and scroll
|
||
position (FR-PLAT-AND-3).
|
||
|
||
---
|
||
|
||
## 4. Explicitly out of scope
|
||
|
||
Listed so absence reads as a decision.
|
||
|
||
| Excluded | Arrives in |
|
||
|---|---|
|
||
| Any editing, develop pipeline, sidecars | v0.2 |
|
||
| Export | v0.2 |
|
||
| Culling mode, ratings, flags, labels | v0.3 |
|
||
| Focus peaking, raw histogram | v0.3 |
|
||
| Local (non-Nextcloud) library scanning | v0.2 |
|
||
| Upload, bi-directional sync, conflict merge | v0.4 |
|
||
| Cache rules and pinning (FR-NC-6a) | v0.4 |
|
||
| Multiple accounts or roots | later |
|
||
| Ingest from card | later |
|
||
| Search and filter beyond folder navigation | v0.3 |
|
||
|
||
**Read-only against the server.** v0.1 performs no `PUT`, `MOVE`, or `DELETE` on remote content.
|
||
This removes conflict handling and chunked upload from scope entirely, and means a bug cannot
|
||
damage the user's library.
|
||
|
||
---
|
||
|
||
## 5. Acceptance criteria
|
||
|
||
Measured against a reference Nextcloud instance holding **≥5,000 RAW files**, on the reference
|
||
desktop and **two Android devices with different GPU vendors** (Adreno and Mali).
|
||
|
||
| # | Criterion | Target |
|
||
|---|---|---|
|
||
| **AC-1** | Login through to first grid render | < 30 s for 5,000 files |
|
||
| **AC-2** | Re-sync with nothing changed | **1 HTTP request** |
|
||
| **AC-3** | Grid scroll, cached previews | 60 fps sustained, both platforms |
|
||
| **AC-4** | Bytes transferred per image, preview path | < 3 MB average |
|
||
| **AC-5** | Single-image display from cache | < 100 ms |
|
||
| **AC-6** | Offline launch → browsable grid | < 2 s desktop, < 4 s Android |
|
||
| **AC-7** | Memory, 5,000-image folder open | < 400 MB desktop, < 200 MB Android |
|
||
| **AC-8** | GPU readback of image data | **Zero occurrences** — asserted by instrumentation |
|
||
| **AC-9** | Android process death mid-browse | Folder and scroll position restored |
|
||
| **AC-10** | Catalog after forced kill during sync | Opens clean, resumes |
|
||
|
||
**AC-2 and AC-8 are the load-bearing ones.** AC-2 proves ETag pruning works, which is what lets the
|
||
library scale. AC-8 proves ARCH §6.1 holds — and it is asserted by instrumentation rather than
|
||
inspection, because a readback introduced later would otherwise pass unnoticed until it showed up
|
||
as unexplained slowness.
|
||
|
||
---
|
||
|
||
## 6. Build order
|
||
|
||
### Phase 0 — de-risk (before anything else)
|
||
|
||
Throwaway code answering the four assumptions. If A1 fails, stop and revisit D1 rather than
|
||
building on it.
|
||
|
||
1. **A1 / S1** — wgpu compute writes a texture; Slint composites UI over it; Linux. Test on
|
||
Mesa/AMD, Intel, and NVIDIA proprietary, under X11 and Wayland.
|
||
2. **A1 / S2** — the same on both Android devices.
|
||
3. **A4 / S3** — reqwest HTTPS PROPFIND from an Android device, including the
|
||
`rustls-platform-verifier` Kotlin init.
|
||
4. **A3 / S4** — range-extract an embedded JPEG from a CR3, NEF, and ARW on a real server; measure
|
||
bytes.
|
||
5. **A2 / S10** — SAF enumeration and range reads over a 10k-file tree; measure against the
|
||
NFR-P1/P3 figures.
|
||
|
||
### Phase 1 — foundations
|
||
|
||
`dr-types` (`SourceRef`, ids) · `dr-plat` traits and both implementations · `dr-catalog` (v0.1
|
||
schema, migrations from commit one) · `dr-gpu` (device, texture upload, Slint bridge).
|
||
|
||
### Phase 2 — connectivity
|
||
|
||
`dr-sync` (`RemoteBackend` trait) · `dr-sync-nextcloud` (Login Flow v2, PROPFIND, ETag pruning,
|
||
range GET) · credential storage.
|
||
|
||
### Phase 3 — imaging
|
||
|
||
`dr-decode` (container parsing, embedded-preview location, JPEG decode, metadata) · preview cache.
|
||
|
||
### Phase 4 — interface
|
||
|
||
`dr-ui` (grid, single-image view, adaptive layout, folder picker, connection flow).
|
||
|
||
### Phase 5 — hardening
|
||
|
||
Offline behaviour · process death · cancellation · error surfaces · the AC suite in CI.
|
||
|
||
**Both platforms build in CI from the first commit.** This is the point of choosing both from day
|
||
one: an Android break is caught the day it lands, not at a porting milestone.
|
||
|
||
---
|
||
|
||
## 7. Crates in play
|
||
|
||
A subset of ARCH §2. Crates not listed are not created yet.
|
||
|
||
```
|
||
core/
|
||
dr-types ✓ SourceRef, ImageId, AccountId, Validator
|
||
dr-catalog ✓ v0.1 schema subset, queries, migrations
|
||
dr-decode ✓ preview extraction + metadata; no demosaic
|
||
dr-gpu ✓ device, texture upload, Slint bridge; no pipeline
|
||
dr-sync ✓ RemoteBackend trait, ETag pruning engine
|
||
dr-sync-nextcloud ✓ the connector
|
||
dr-pipeline ✗ v0.2
|
||
dr-colour ✗ v0.2
|
||
dr-export ✗ v0.2
|
||
dr-sidecar ✗ v0.2
|
||
ui/
|
||
dr-ui ✓ grid, viewer, connection flow
|
||
dr-widgets ✗ v0.2 (no custom controls yet)
|
||
platform/
|
||
dr-plat ✓ Storage, Secrets, Lifecycle traits
|
||
dr-plat-linux ✓
|
||
dr-plat-android ✓
|
||
apps/
|
||
darkroom-desktop ✓
|
||
darkroom-android ✓
|
||
```
|
||
|
||
`dr-gpu` exists in v0.1 **only** to upload decoded JPEGs and hand textures to Slint. No compute
|
||
pipeline, no tiling, no masks. It is deliberately the thinnest thing that still proves A1.
|
||
|
||
---
|
||
|
||
## 8. Decisions this milestone informs
|
||
|
||
| Decision | What v0.1 tells us |
|
||
|---|---|
|
||
| **D12** — scope vs pace | How long a milestone of this size actually takes at the available pace. The single most useful output. |
|
||
| **D3** — first milestone | Supersedes the vertical slice, if this proves the better shape. |
|
||
| **D1** — Rust + Slint | AC-8 and A1 either confirm the framework choice or reopen it. |
|
||
| **NFR-COMPAT-1** — hardware baseline | Two real Android devices give the baseline actual numbers instead of a placeholder. |
|
||
| **D7** — network stack | Whether the reqwest Android TLS path is a half-day or a fortnight. |
|
||
|
||
---
|
||
|
||
## 9. Risks
|
||
|
||
| Risk | Likelihood | Mitigation |
|
||
|---|---|---|
|
||
| Slint `create_texture_from_hal` doesn't work as documented | Medium | Phase 0 first; D1 records fallbacks |
|
||
| SAF enumeration too slow at 10k files | Medium | S10 measures before commitment; batch and cache aggressively |
|
||
| Embedded previews too small or absent on some bodies | High | Known — Sony embeds small previews, some bodies none. M-11's fallback chain handles it; detect per camera model |
|
||
| **rawler exposes only full-resolution previews** | **Confirmed** | Measured 2026-08-09: rawler 0.7.2's CR2 decoder implements `full_image` only; `thumbnail_image`/`preview_image` are unimplemented defaults. Every rung resolves to a 5472×3648 decode at ~250 ms, 5× over NFR-P13. CR2 does carry smaller IFDs, so the fix is our own IFD walk or an upstream contribution — not a change to callers |
|
||
| **Android secret storage unimplemented** | **Confirmed** | Needs no investigation — `PlatformSecretStore` on Android is unimplemented by design, and fails loudly rather than silently no-opping (`platform/dr-plat/src/secrets.rs`). The fix is a real Keystore-over-JNI implementation (FR-PLAT-AND-1), which is `dr-plat-android` work not yet started |
|
||
| reqwest Android TLS worse than expected | Medium | D7 escape hatch: `tls_certs_only` with `webpki-roots` |
|
||
| GPU vendor divergence on Android | Medium | Two vendors in CI from the start |
|
||
| Scope creeps toward editing | **High** | §4 is explicit; v0.1 is read-only against the server |
|
||
|
||
The last one is the real risk. A viewer that works is a strong temptation to add "just one slider."
|