dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Make storage pluggable, and prove it with a folder backend

Browse Source 
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.

A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:

- **Capabilities** — already there, and the reason the engine can drive
  two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
  whatever form its connector addresses, with no server in it. Loads
  every existing config unchanged (`backend` defaults to `nextcloud`,
  `endpoint` is stored under its historical `server` key), and
  `Account::namespace()` reproduces the old catalog directory byte for
  byte, because changing it would abandon a catalog, its thumbnail
  shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
  `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
  names a connector.

`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.

Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.

`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.

docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
This commit is contained in:
dtourolle
2026-08-29 09:57:52 +02:00
parent 1b8b7998a2
commit f12aece07e
40 changed files with 3617 additions and 960 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture.md
+74 -6
View File
@@ -56,8 +56,9 @@ darkroom/
│ ├── dr-gpu wgpu device, tile scheduler, WGSL shaders, mask rasteriser
│ ├── dr-colour lcms2 bindings, camera profiles, working-space transforms
│ ├── dr-export encoders, resampling, output sizing
│ ├── dr-sync RemoteBackend trait, sync engine, cache rules, merge
│ └── dr-sync-nextcloud the only backend implementation (§8.4)
│ ├── dr-sync RemoteBackend + BackendProvider, Account, sync engine, merge
│ ├── dr-sync-nextcloud WebDAV, oc:fileid, chunked v2, Login Flow v2 (§8.4)
│ └── dr-sync-folder a plain directory: disk, mount, synced folder (§8.4a)
├── ui/
│ ├── dr-ui Slint components, adaptive layout, descriptor→control mapping
│ └── dr-widgets custom controls per WidgetKind (curve, wheel, crop, brush)
@@ -572,9 +573,34 @@ recovery. Panics in decode are caught at the boundary, since RAW parsing handles
## 8. Sync architecture
Sync is pluggable. `dr-sync` defines a `RemoteBackend` trait; **only the Nextcloud connector is
implemented**, but the boundary is designed so S3, generic WebDAV, or a self-hosted photo server can
be added without touching the sync engine.
Sync is pluggable. `dr-sync` defines a `RemoteBackend` trait and the capability model the engine
adapts to; two connectors implement it — `dr-sync-nextcloud` and `dr-sync-folder` — and S3, generic
WebDAV, or a self-hosted photo server can be added without touching the engine.
**`docs/storage.md` is the contract**: the four traits a connector meets, the four steps to add one,
and what each shipped connector actually declares. This section says *why* the seam is shaped the
way it is; that document says how to use it.
### 8.0 A trait is not a seam
Worth stating because this was got wrong for a release. `RemoteBackend` existed from the start and
the code above it still knew it was talking to Nextcloud: seven files in `dr-ui` constructed a
`NextcloudBackend` directly, ten functions took one by concrete type, an account *was* a server URL
beside a DAV user id, and the local cache directory was named after a hostname. The abstraction was
real and bought nothing.
Pluggable storage needs four things, and only the first is a trait over operations:
| | What | Where |
|---|---|---|
| 1 | Operations | `RemoteBackend` (§8.3) |
| 2 | Capabilities — what is *cheap*, so the engine adapts rather than assumes | `Capabilities` (§8.1) |
| 3 | Configuration — what an account is, with no server in it | `Account`, `Connection` |
| 4 | Registration — how a connector is discovered without being named | `BackendProvider`, `BackendRegistry` |
`ui/dr-ui/src/remote.rs` is the only file above `dr-sync` that names a connector. `dr-sync` itself
depends on none of them, so a build that only wants a folder library does not compile a TLS stack to
get one.
### 8.1 Why capability negotiation, not a common denominator
@@ -702,7 +728,8 @@ Design notes worth keeping:
### 8.4 The Nextcloud connector
The only implementation. Mapping to the trait:
The reference implementation, and the one whose peculiarities the capability model exists to keep.
Mapping to the trait:
| Trait method | Nextcloud |
|---|---|
@@ -739,6 +766,47 @@ never `Depth: infinity`. Two implementation details worth keeping:
and proves nothing about children, so it is pure overhead; a test asserts zero probes in that
case. Only `PropagatingEtags` makes an unchanged parent prove an unchanged subtree.
### 8.4a The folder connector
A plain directory: a local disk, a network mount, an external drive, or the folder a Nextcloud
desktop client already syncs. No server, no account, no credential — which makes it the route that
works on a machine with no secrets daemon.
It exists for two reasons. It is genuinely useful, and a second implementation is the only way to
find out whether the first was an abstraction or a description. Adding it is what turned §8.0's four
items from a claim into a fact.
| Trait method | Folder |
|---|---|
| `capabilities` | `LocalEtags`, **no** stable ids, ranges, no chunking, conditional |
| `list` | `read_dir` + `metadata`; validator is `size`-`mtime` |
| `dir_validator` | `Unsupported` — a directory's mtime does not propagate |
| `delta` | `Unsupported` — a folder keeps no change feed |
| `get` + range | `seek` + `take`; a short read past the end is not an error |
| `put` | write to a temporary beside the destination, `rename` over it |
| `put` `IfAbsent` | `O_CREAT | O_EXCL` — genuinely atomic |
| `put` `IfMatch` | `stat`, compare, then the same rename — narrows the race, does not close it |
| `delete` | files, and *empty* directories only — see below |
| `move_to` | `rename`, falling back to copy + unlink across a mount boundary |
| auth | none |
Three things worth carrying forward:
- **`LocalEtags` is the honest answer, and it costs nothing.** A POSIX directory's mtime describes
its own entry list and nothing below it, so there is no propagation to exploit and the engine
walks the tree every scan. The walk that was expensive was expensive because it was 50k
`PROPFIND`s; 50k `stat` calls take well under a second. This is the capability model paying for
itself — one engine, two backends, each running at the speed it actually runs at.
- **Identity is a path hash, not an inode.** An inode is stable across a rename but differs between
devices and is reused after a delete, so two machines would disagree about which photograph a
thumbnail belonged to and a recycled inode would attach an old thumbnail to a new image.
Re-deriving a thumbnail is a cost; showing the wrong one is a bug. `stable_ids: false` reports the
consequence.
- **`delete` is deliberately not recursive**, unlike WebDAV's `DELETE` on a collection. There is no
server-side trash behind a local folder, so a caller with a wrong path would have no way back.
Nothing in the engine deletes a directory — the soft delete is a `move_to` (§FR-CAT-15) — so the
guard is free.
### 8.5 Sidecar conflict resolution
`put` with `Precondition::IfMatch(validator)`. On precondition failure: