# Storage backends How DarkRoom talks to wherever a library lives, and what it takes to add somewhere new. This document is the contract. `docs/architecture.md` §8 says why sync is built on capability negotiation rather than a common denominator; this says what the seam actually is, where each piece lives, and what a third connector has to do. --- ## 1. What "pluggable" has to mean A trait alone does not make storage pluggable. `RemoteBackend` existed from the first release and every layer above it still knew it was talking to Nextcloud: seven files in `dr-ui` constructed a `NextcloudBackend` directly, ten functions took one by concrete type, the account model was a server URL beside a DAV user id, and the local cache directory was named after a hostname. The abstraction was real and bought nothing, because everything that *reached* a backend was still shaped like one product. Pluggable means all four of these, not just the first: 1. **Operations** — what a backend can do. `RemoteBackend`. 2. **Capabilities** — what it can do *cheaply*, so the engine adapts instead of assuming. `Capabilities`. 3. **Configuration** — what an account is, with no server in it. `Account`. 4. **Registration** — how the application discovers a connector at all, without naming it. `BackendProvider` + `BackendRegistry`. Two connectors ship. Nextcloud is unchanged and keeps every one of its peculiarities — those are the point of the capability model, not an embarrassment it has to hide. The folder connector serves a plain directory and exists partly because it is genuinely useful and partly because a second implementation is the only way to find out whether the first was an abstraction. --- ## 2. Where each piece lives ``` core/dr-sync/ the contract, and nothing that speaks a protocol ├─ types.rs RemotePath, RemoteId, RemoteEntry, Validator, … ├─ capability.rs Capabilities, ChangeDetection, ServerPreviews ├─ error.rs RemoteError — the one error every caller handles ├─ account.rs Account, AccountStore, Secret, Connection ├─ provider.rs BackendProvider, BackendRegistry, SignIn ├─ lib.rs RemoteBackend, SyncStrategy ├─ scan.rs the walk, driven by capabilities ├─ upload.rs where an original is placed └─ reachability.rs online/offline, inferred from observed results core/dr-sync-nextcloud/ WebDAV, oc:fileid, chunked upload v2, Login Flow v2 core/dr-sync-folder/ a directory on a filesystem ui/dr-ui/src/remote.rs the registry — the ONLY file above dr-sync that names a connector ``` `dr-sync` depends on no connector. That is deliberate and load-bearing: a build that only wants a folder library must not compile a TLS stack to get one, and the registry therefore lives in the crate that already depends on everything — the interface. --- ## 3. The four traits and types a connector meets ### 3.1 `RemoteBackend` — operations ```rust #[async_trait] pub trait RemoteBackend: Send + Sync { fn capabilities(&self) -> &Capabilities; fn name(&self) -> &str; // discovery async fn list(&self, dir: &RemotePath, since: Option<&Validator>) -> Result, RemoteError>; async fn dir_validator(&self, dir: &RemotePath) -> Result; async fn delta(&self, cursor: &Cursor) -> Result<(Vec, Cursor), RemoteError>; // transfer async fn get(&self, id: &RemoteId, range: Option>) -> Result, RemoteError>; async fn get_reporting(&self, id: &RemoteId, // defaulted progress: &(dyn Fn(u64, Option) + Send + Sync)) -> Result, RemoteError>; async fn put(&self, path: &RemotePath, body: Vec, precond: Option) -> Result; async fn put_many(&self, items: Vec<(RemotePath, Vec)>) // defaulted -> Result>, RemoteError>; async fn delete(&self, id: &RemoteId, precond: Option) -> Result<(), RemoteError>; async fn move_to(&self, from: &RemoteId, to: &RemotePath) -> Result<(), RemoteError>; async fn create_dir(&self, path: &RemotePath) -> Result<(), RemoteError>; // optional async fn thumbnail(&self, id: &RemoteId, size: u32) // defaulted to None -> Result>, RemoteError>; } ``` Rules that are not obvious from the signatures: - **`dir_validator` and `delta` are capability-gated.** Return `RemoteError::Unsupported` unless your `ChangeDetection` is `PropagatingEtags` or `DeltaCursor` respectively. Answering `dir_validator` with something that does not actually propagate is worse than refusing: it lets a caller prune a subtree whose contents changed, and hides those changes for as long as the folder list holds still. - **`get` takes an optional range, and it is a hint.** A backend without cheap ranges may return the whole object; the caller slices. Correctness holds either way and `Capabilities::range_reads` says whether it was cheap. - **`get_reporting` is for the one transfer somebody is watching.** An original opened in develop is tens of megabytes, and "downloading" alone for that long reads as stuck. It reports the bytes received and the length the server declared, if it declared one. The default is `get` whole and one report at the end, which is right for a backend whose read is local; Nextcloud overrides it to read the body chunk by chunk. The develop view reads the figures by path from the in-flight registry in `ui/dr-ui/src/library/thumbnails_fetch.rs`, because a step along the roll usually lands on a frame the prefetcher is already fetching, and falls back on the catalog's file length when no length was declared. - **Chunked upload is not in the trait.** It is an implementation detail of `put`, chosen by body size. Exposing it would leak one server's protocol. - **`move_to` must preserve identity where the backend has stable ids.** This is what a soft delete uses (`FR-CAT-15`): a move implemented as copy + delete allocates a new id, orphaning the thumbnail shard and turning a restore into a full re-download. - **`create_dir` makes parents and succeeds if the directory exists.** Callers use it to guarantee a destination, not to claim they created one. It is also the server browser's `New folder` (§5.3), which then lists the parent again rather than inserting the name it asked for: the server may have normalised it. ### 3.2 `Capabilities` — what is cheap The engine reads these once at connect time and picks a `SyncStrategy`. See ARCH §8.1–8.2 for the tiers. The two that change behaviour rather than speed: | Absent | Consequence the engine handles | |---|---| | `range_reads` | Embedded-preview extraction is impossible; browsing falls back to server previews or full download, and is refused on a metered connection | | `conditional_write` | Sidecar conflict detection falls back to revision counters inside the sidecar — narrows the race, does not close it. Reported as a reduced-safety mode | **Declare what is true, not what is flattering.** A backend claiming `PropagatingEtags` it does not have does not merely run slowly; it silently hides changes. ### 3.3 `Account` — configuration with no server in it ```rust pub struct Account { pub backend: String, // BackendProvider::id; defaults to "nextcloud" on load pub endpoint: String, // stored as "server" — a URL, a path, a bucket pub login: String, // empty where the connector has no notion of a user pub user_id: String, // connector-defined sub-address; Nextcloud's DAV segment pub root: String, // the folder chosen as the library root pub formats: Vec, pub last_scan: Option, } ``` Everything but `backend` is the connector's to interpret. Code above `dr-sync` reads these for display and for cache keys, never for meaning. Two properties are load-bearing: - **The on-disk form is backwards compatible.** `backend` defaults to `"nextcloud"` and `endpoint` is stored under its historical key `server`, so every account written before there was a choice loads unchanged. A config the app refuses to parse is an account the user has to set up again. - **`Account::namespace()` is frozen for Nextcloud.** It names the directory holding the catalog, the thumbnail shards, the sidecar spool and the export outbox. Changing it does not lose that data, it *abandons* it — silently, as an upgrade — and costs a full rescan on top. The Nextcloud form is reproduced byte for byte from what `catalog_path` computed before; every other backend is prefixed by its connector id, and long endpoints are truncated with a hash tail so two deep paths cannot collide inside one filesystem's 255-byte component limit. ### 3.4 `Connection` and `Secret` — the credential split ```rust pub struct Connection { pub account: Account, pub secret: Option } ``` Credentials go to platform secure storage (`FR-NC-2`, `NFR-SEC-2`). Never the catalog, never the config file, never a log line. `AccountStore` writes the account as plain JSON and the secret to the keyring, which is what lets the app show "signed in as duncan, watching /PhotosRaw" before it has touched the keyring at all. `Secret`'s inner string is reachable only through `expose()`, and its `Debug` prints `Secret(***)`. That closes the indirect leak — a `{:?}` on any struct that happens to hold a connection — by construction rather than by review. `Connection` is also what replaced a pair of arguments (credentials, user id) threaded together through fifteen signatures in an order that could be swapped. ### 3.5 `BackendProvider` — registration ```rust pub trait BackendProvider: Send + Sync { fn id(&self) -> &'static str; // written to Account::backend fn display_name(&self) -> &'static str; fn endpoint_label(&self) -> &'static str; // "Server" / "Folder" fn endpoint_placeholder(&self) -> &'static str; fn sign_in(&self) -> SignIn; fn normalise_endpoint(&self, input: &str) -> Result; fn upgrade_endpoint(&self, stored: &str) -> Option; // defaulted: None fn account_for(&self, endpoint: &str) -> Result; // defaulted fn connect(&self, conn: &Connection) -> Result, RemoteError>; } pub enum SignIn { /// A handshake the user completes outside the app, yielding a credential. Browser, /// The endpoint is the whole account. No credential, no waiting state. EndpointOnly, } ``` - **`id` is on-disk configuration.** Changing it after anyone has an account orphans that account. Pick it once. - **`normalise_endpoint` is where a bad endpoint is *rejected*,** before an account is written for a library that does not exist. Its error string is shown to the user, so it says what to fix rather than naming a type. The Nextcloud provider upgrades `http://` to `https://` here (`NFR-SEC-3`); the folder provider canonicalises the path, so two spellings of one directory do not become two accounts indexing the same photographs. - **`upgrade_endpoint` is for accounts already saved,** and is not a second call to `normalise_endpoint`: the folder provider's needs the path to exist, so running it on every launch would fail a library on an unplugged disk. `AccountStore::upgrade_endpoints` runs it at launch; only Nextcloud answers, rewriting an `http://` account an older build saved to `https://` (#65). The keyring entry is filed under the endpoint, so the secret is copied across before the record changes, and a move that would change `namespace()` is refused rather than performed. Below both, the Nextcloud client sets `https_only`, so no URL it sends — a typed one, the login flow's poll endpoint, a redirect — can carry the app password in the clear, and the login flow keeps the address the user typed rather than the server's idea of its own URL. - **`connect` is synchronous and cheap.** It validates configuration and builds a client; it does not talk to the remote. Workers call it per task. - **`SignIn` is a shape, not a method.** It would be tidier to expose `async fn sign_in()`, and wrong: Login Flow v2 is a browser handshake the user completes elsewhere while the app polls, so it is not one call, it does not finish on our schedule, and the screen has to render a URL and a waiting state in the middle of it. `SignIn` tells the launch screen which of the two shapes to draw; the flow stays where its protocol is. **Credentials are deliberately not abstracted.** An app password, an OAuth token and a bucket key pair have no useful common shape, and inventing one before a third backend exists would produce a wrong answer confidently. The general form is `Connection` — an account plus an opaque secret — and each connector translates that into what its protocol needs (`NextcloudProvider::credentials`). --- ## 4. Adding a backend 1. **Implement `RemoteBackend`** over your protocol, in a new `core/dr-sync-` crate depending on `dr-sync` and nothing else of ours. 2. **Declare `Capabilities` honestly.** Start from `Capabilities::minimal()` and raise only what you can actually deliver. 3. **Implement `BackendProvider`** beside it. 4. **Register it** in `ui/dr-ui/src/remote.rs::registry()` and add the crate to `ui/dr-ui/Cargo.toml`. That is the whole list. Nothing else in `dr-ui` changes, because nothing else in `dr-ui` names a connector. **Two things to get right, because they are silent when wrong:** - **Identity.** `RemoteEntry::id` should be `RemoteId::Stable(u64)` wherever you can produce a `u64` that names the same photograph on every device looking at the same library. The catalog keys the thumbnail shards and the face index on it (`catalog.md` §10.1), and an entry without one gets neither. Set `Capabilities::stable_ids` only if that id also survives a rename — the two are different questions and only the second is a capability. - **Path safety.** A `RemotePath` is built from names on the remote and from a catalog another device wrote. If you resolve one against a real filesystem, reject `..` before you open anything. Register a test double the same way — `BackendRegistry::register` replaces an existing id rather than shadowing it — so an integration test can stand a fake server behind `"nextcloud"` without the registry knowing it happened. --- ## 5. The connectors that ship ### 5.1 Nextcloud (`dr-sync-nextcloud`, id `"nextcloud"`) Unchanged by the abstraction, peculiarities intact — see ARCH §8.4 for the full mapping. What matters here is that none of them had to be given up to make room for a second backend: | | | |---|---| | `change_detection` | `PropagatingEtags` — the one-request no-op sync | | `stable_ids` | yes, `oc:fileid`, survives server-side rename and move | | `range_reads` | yes, detected by `206` vs `200`, never `HEAD` | | `chunked_upload` | v2, 5 MB – 5 GB, `MKCOL` → `PUT` chunks → `MOVE .file` | | `bulk_upload` | yes, `POST /remote.php/dav/bulk` | | `conditional_write` | yes, `If-Match` | | `server_previews` | `CommonFormatsOnly` — stock Nextcloud renders no RAW | | `get_reporting` | overridden: the body is read chunk by chunk, against `Content-Length` | | sign-in | `SignIn::Browser`, Login Flow v2, system browser, app password | Also kept: the `oc:permissions` probe on a refused `PUT`, which is what distinguishes a create-only share from a bad credential; the `423 Locked` retry classification; and the bundled ISRG Root YE certificate. ### 5.2 Folder (`dr-sync-folder`, id `"folder"`) A local disk, an NFS or SMB mount, an external drive, or the directory a Nextcloud desktop client already syncs. No server, no account, no credential — which makes it the route that works on a machine with no secrets daemon at all. | | | |---|---| | `change_detection` | `LocalEtags` — see below | | `stable_ids` | **no** — the id is a path hash and does not survive a rename | | `range_reads` | yes, `seek` + `take` | | `chunked_upload` | none; a write is a write | | `bulk_upload` | no | | `conditional_write` | yes, with a documented residual race | | `server_previews` | `None` | | `get_reporting` | the default: a local read, reported once when it is done | | sign-in | `SignIn::EndpointOnly`, the folder chosen in the platform's dialogue (§5.3) | **Why `LocalEtags` and not `PropagatingEtags`.** A POSIX directory's mtime changes when its own entry list changes and at no other time — not when a child's contents are edited, and not for a grandchild. There is nothing to propagate, so `dir_validator` returns `Unsupported` and the engine walks the tree every scan. Which costs almost nothing, because the walk that was expensive was expensive for a reason this backend does not have. **Measured 2026-08-28**, `cargo run -p dr-sync-folder --example scan`: a full uncached walk of 2,299 images across 233 directories completed in **137 ms**, and 380 images across 13 directories in **29 ms** — the same engine, the same `Depth: 1`-per-directory walk, with no pruning at all. The Nextcloud connector's comparable figure is 34.1 s for 17,185 RAWs across 334 directories *with* pruning available (ARCH §8.4). The capability model is what lets one engine drive both at the speed each actually runs at, instead of forcing the fast one down to the slow one's interface. **Identity is a hash of the path relative to the library root**, FNV-1a 64 (written out, because `DefaultHasher` is explicitly unstable between Rust releases and this value is written into the catalog). It gives the catalog a `u64` that names a photograph, is the same on every device looking at the same folder, and does not change when the file is edited. It does not survive a rename, and `stable_ids: false` says so: a moved photograph is seen as a delete and an add, and its thumbnail is derived again. The alternative — keying on the inode — is stable across a rename but *differs between devices* and is reused by the filesystem after a delete. Two machines would disagree about which photograph a thumbnail belonged to, and a recycled inode would silently attach an old thumbnail to a new image. Re-deriving a thumbnail is a cost; showing the wrong one is a bug. **Conditional writes.** `IfAbsent` is genuinely atomic (`O_CREAT | O_EXCL`). `IfMatch` is compare-then-swap: a `stat`, then a write to a temporary beside the destination and a `rename` over it. A POSIX filesystem has no compare-and-swap, so the race is narrowed to the microseconds between the two syscalls rather than closed — still far tighter than the fallback the engine uses for a backend that declares no conditional write at all, which spans a whole read-modify-write. The capability is declared, and the residual race is documented at the call site. **Two deliberate divergences from WebDAV semantics:** - **`delete` is not recursive.** A folder library is the user's own photographs on their own disk with no server-side trash behind it, so a caller that passed the wrong path would have no way back. Deleting a non-empty directory returns `RemoteError::Configuration`. Nothing in the engine deletes a directory — the soft delete is a `move_to` into the trash folder — so the guard is free. - **Every filesystem call runs on the blocking pool.** On a local disk that is overkill; on the NFS mount this backend is most useful over, a stalled server would otherwise wedge the async worker that made the call and every other request sharing it. **Failure classification** matters as much as the operations. A vanished mount (`ESTALE`, `ENOTCONN`, `EIO`) maps to `RemoteError::Network`, which is what puts the app into offline mode and leaves the catalog readable — exactly as a dead server does. A permissions problem maps to `PermissionDenied` and does *not*, because going offline over one forbidden file would hide a fixable problem behind a network banner. An endpoint that is not a directory at all maps to `RemoteError::Configuration`: nothing was unreachable and no credential was wrong, so neither of the other two would send the user anywhere useful. ### 5.3 Choosing folders, and the folders exports go to **Folders are pointed at, never typed** (FR-EXP-6). On the desktop, `ui/dr-ui/src/folder_dialog.rs` asks the platform: the XDG desktop portal's FileChooser on Linux, through `rfd`, and the common item dialogue on Windows. That is how the folder connector's endpoint is chosen on the launch screen, and the path it returns still goes through `normalise_endpoint` as a typed one did. A folder on the server is chosen in the in-app browser, which lists with `list` and makes a folder with `create_dir`; the launch screen and the album sheet share it through `ui/dr-ui/src/remote_folders.rs`, which keeps both round trips off the interface thread. Android has no filesystem dialogue, so its library folder is still typed, and an album's folder there comes from the Storage Access Framework's tree picker. **An album is where exports go** (FR-EXP-10), and never inside the library: a JPEG written into the tree a scan catalogues comes back as a photograph beside the RAW it was made from. Its folder is one of two kinds (`dr_catalog::albums::Place`): - **On the server,** relative to the *account* root, not the library root. The browser refuses a folder inside the library and says why; on a server whose whole account is the library, every folder is inside it, and it says that instead. Exports reach it through the outbox like any upload, and a queued file's `.dest` record gains a third line saying its folder is relative to the account — a third line rather than a leading slash, because a record written before albums may carry a stray slash and has to keep the meaning it was written with. - **On this device,** a path, or on Android a SAF tree URI with a persisted grant, written through `DocumentsContract` (`ui/dr-ui/src/saf.rs`). A provider renames on a collision by itself, so the album records the name it was given rather than the one asked for. A server folder lives on the album row and syncs with it. A device folder lives in `album_folders`, which the merge never reads and the upload snapshot drops, because a path or a grant on one device means nothing on another: an album made on the desktop arrives on the tablet with no folder until one is chosen there. --- ## 6. Virtual filesystems A sync client in virtual-files mode leaves a **placeholder** where a file is catalogued but not downloaded. On Linux — the only mode it supports — that means `IMG.CR2` does not exist at all and `IMG.CR2.nextcloud` does, holding one byte. ARCH §9.0 measured a real machine: 121,785 placeholders against 10,267 materialised files. A folder library that ignores this is not merely degraded, it is dangerous. Before the handling below existed, the folder connector catalogued every stub as a 1-byte image, gave it an identity that changed the moment it was downloaded, and — worst — reported a dehydrated *sidecar* as absent, which made the sidecar writer create a fresh document over an existing one and discard every edit another device had put there. ### 6.1 Three questions, one trait Everything else about a synced folder is an ordinary directory, so this is not a second connector. `dr_sync_folder::Vfs` asks only what differs: ```rust pub trait Vfs: Send + Sync { fn name(&self) -> &'static str; fn is_placeholder(&self, on_disk: &str) -> bool; fn real_name<'a>(&self, on_disk: &'a str) -> &'a str; fn placeholder_name(&self, name: &str) -> Cow<'_, str>; fn can_materialise(&self) -> bool; // defaulted false fn materialise(&self, local: &Path) -> Result<(), RemoteError>; // defaulted fn dematerialise(&self, local: &Path) -> Result<(), RemoteError>; // defaulted } ``` `NoVfs` for a plain directory; `dr_sync_nextcloud::NextcloudVfs` for a synced one, wrapping the `DesktopClient` socket. A third convention is a third impl. **Why not a `folder-vfs` provider.** The interesting capability is not a property of the backend: the same directory can materialise on demand while the client is running and cannot when it is down, so it must be computed per connection either way. Registering two providers would ask the user to choose between two things that differ by whether a background process is up. The convention is detected instead, per connection, by a hook the registry supplies (`FolderProvider::with_vfs_detector`) — which is what keeps `dr-sync-folder` free of any client's protocol. ### 6.2 What the backend reports | | | |---|---| | `RemoteEntry::path` | the photograph's name, never the stub's — so identity survives a download | | `RemoteEntry::materialised` | `false` on a stub; the catalog maps it to `Availability::Offline` | | `RemoteEntry::size` | `0` on a stub, meaning *unknown* — see below | | `get` on a stub | `RemoteError::NotMaterialised`, **never** `NotFound` and never the stub's one byte | | `put` over a stub, unconditional | **replaces it** — the whole file is being written, so there is nothing in the stub to keep, and the placeholder is removed after the content lands | | `put` over a stub, `IfMatch` | `NotMaterialised` — a stub's validator describes the placeholder, so nothing here can satisfy the guard; the caller fetches and retries | | `put` over a stub, `IfAbsent` | `PreconditionFailed` — the file *is* there, only its content is elsewhere | | `move_to` a stub | moves the stub and keeps it a stub — culling without downloading is ordinary | | `delete` a stub | deletes it; a photograph is deleted whether or not its bytes are here | | `capabilities().materialisation` | `OnDemand` with a client, `Placeholders` without, `Always` on a plain folder | **Size is genuinely unknown.** A Linux suffix-mode stub is one byte and carries no record of what it stands for. The client's `._sync_*.db` has the real size, but that is a private schema and reading it would couple us to their migrations. FR-NC-6c wants a transfer size quoted before an operation starts; for a stub the honest answer is that it cannot be, and the interface should say so rather than report one byte or invent an estimate silently. ### 6.3 Hydration is a borrow The rule: **a file is returned to the state it was found in.** What a pass downloaded is released; what the user already had is left alone. `BorrowPool` enforces it. ```rust let pool = BorrowPool::new(); { let held = pool.borrow(&backend, &path).await?; // downloads only if absent // ... read it, thumbnail it, index its faces ... } // borrow ends let stats = pool.release_all(&backend).await; // dehydrates only what it hydrated ``` Three properties that are not obvious: - **Reference counted.** The thumbnail pass and the face pass meet on the same RAW. Without counting, the first to finish dehydrates the file the second is reading; with it, the transfer is paid once and released when the last borrower is done. - **Prior state is read before asking.** After `materialise` there is no way to tell what the pass brought from what was already there, so it is recorded first. Getting this wrong silently undoes a pin, and "my pinned trip evaporated after an indexing run" is the failure that would make people stop trusting the feature. - **Being unsure is not symmetric.** `borrow_known(.., Some(true))` keeps a file that might have been ours — costing disk. `Some(false)` releases one that might have been the user's. An uncertain caller passes `true` or `None`, never a guess at `false`. A borrow against a plain folder or a server backend short-circuits and does nothing, so a pass written for a VFS library runs unchanged everywhere rather than growing two code paths. **Measured 2026-08-29**, `cargo run -p dr-sync-folder --example vfs_cycle`: a library of 100 photographs at 25 MB each, 90 of them dehydrated and 10 the user keeps. A pass over all 100, borrowing and releasing as it goes: | | | |---|---| | on disk at rest | 250 MB | | **peak during the pass** | **275 MB** — the resting set plus one photograph | | without borrowing | 2,500 MB | | on disk afterwards | 250 MB | | of the 10 the user already had | 10 still there | The peak is the working set, not the library, and the release is selective. ### 6.4 Derived state is dehydrated too Shards, the catalog snapshot and the place record live in `.darkroom-derived/` **inside the library folder**, so a sync client dehydrates them exactly as it dehydrates a photograph. Unlike a photograph, none of them can be skipped: a shard that will not open is a peer's thumbnails never merging, and a catalog snapshot that will not open is their collections. The folder holds three kinds of thing: | File | What it is | How two devices reconcile it | | --- | --- | --- | | `shard--NNNN.sqlite` | Thumbnail and face shards | Sealed and immutable; a name match is a content match, so existence is the whole protocol | | `catalog.sqlite` | The catalog snapshot, for its collections | Read-modify-write: **merge theirs, then push the union** | | `place.json` | Where the photographer was (FR-UI-8) | Replace: the newer timestamp wins outright | `place.json` is the odd one out, and deliberately. Everything else there is *derived* — a faster way to learn something the device could have worked out for itself from the originals and the sidecars — so losing it costs time. A place is a fact only the other device knew, and losing it costs a scroll. That is why it is exchanged last in the pass, why its failures are logged rather than reported, and why it is the one file here that is replaced rather than merged: two devices cannot both be where the photographer is, so there is nothing of theirs inside ours to preserve. It still refuses to upload over a copy it could not read, for a smaller version of the reason below: a record we have not compared against might be the newer one, and overwriting it would move the other device's photographer without ever having seen where they were. `derived_sync::read_derived` fetches on demand rather than giving up. More important is what happens when it *cannot*: The catalog sync is a read-modify-write over a file another device also writes. It was shaped `if let Ok(bytes) = backend.get(..)`, which folded every failure into "there is no remote catalog" and carried straight on to the upload — so a dehydrated snapshot meant pushing ours over theirs unmerged, taking their collections and members with it. The same shape as the sidecar bug in §6, and the same fix: a read that fails for any reason other than `NotFound` **stops the upload**. That is why `NotFound` and `NotMaterialised` had to be separate errors. One means "yours is the whole truth, write it"; the other means "do not dare". ### 6.5 Release means dehydrate, never delete The single most dangerous thing in this feature. A synced folder is not a cache: deleting a materialised file inside it propagates the deletion to the server and removes the photograph from every device the user owns. `Vfs` and `RemoteBackend::dematerialise` both say so, and an implementation that cannot dehydrate returns `Unsupported` rather than approximating it. This is also why the originals cache (`dr_catalog::cache`) cannot simply be pointed at a VFS library: `Cache::release` deletes bytes, which is right for a copy under `originals/` and catastrophic in place. ### 6.6 Which photographs stay downloaded The user's half of the bargain: a pass borrows for a moment, but *some* of the library should stay local — the trip you are about to take, the shoot you are working on. That is a **pin**, and it is the pin the originals cache already had (`dr_catalog::cache`, FR-NC-6a). Nothing parallel was built, because the model was already the right one: | Cache concept | On a placeholder library | |---|---| | `tier_desired` | what the user asked to keep hydrated | | `tier_actual` | what is actually materialised | | `pending_pins()` | the work list — what to hydrate next, resumable | | pinned rows are never evicted | a pinned collection is never dehydrated | | passive rows, LRU under a budget | what a pass borrowed, released when it finishes | So "keep this collection hydrated" is `Cache::pin`, and the existing pin worker drives it — except that on a placeholder library it calls `materialise` instead of downloading a copy. **Why not a copy.** The original materialises *in the library folder*. Copying it under `originals/` as well would hold every pinned photograph twice, and the copy would be the half the budget could evict while the real disk cost stayed. `Cache::record_in_place` records the bookkeeping with **`path = NULL`**, and that null is load-bearing: `release` deletes the file a row names, and a row that names none deletes nothing. The safety property is structural rather than remembered. Unpinning therefore frees nothing by itself — the bytes are not ours to delete. `spawn_dehydrate` asks the client to take them back, which is what actually returns the disk. --- ## 7. What the abstraction does not yet cover Stated so the next person does not have to rediscover it. - **Multiple accounts at once.** `AccountStore` holds a list and the launch screen uses the most recent. Nothing in the model prevents two open libraries; the interface has no place to show them. - **Per-backend settings.** A connector has no way to contribute a settings page. Anything configurable is on the `Account` or is not configurable. - **Capability probing at runtime.** `Capabilities` is fixed at construction. Nextcloud's `server_previews` should really be probed per account — a server with `camerarawpreviews` installed can render RAW — and today it is assumed to be `CommonFormatsOnly`. - **A general notion of an account.** Credentials stay connector-specific on purpose (§3.5). A third connector with an OAuth flow will need a third `SignIn` variant, and that is the right place for it to appear. - **A quoted cost before a hydrating pass.** FR-NC-6c wants the transfer size stated before an operation that needs absent data. A placeholder reports no size (§6.2), so the honest figure for "index this library" is a count and not a byte total. The interface should say *n photographs, size unknown until fetched* rather than estimate one silently — and it does not say anything yet. - **Metadata-only placeholders.** Windows and macOS express these in filesystem metadata rather than in the name, and carry the real size there. `Vfs` asks its questions about a *name*, which is all the one convention this project has met needs. Supporting them means widening the trait to take a `Metadata`, and doing that before anyone has run this on those platforms would be guessing. - **Hydration during browsing, deliberately.** It stays forbidden (ARCH §9.0 finding 3). A grid cell whose content is absent shows as not-downloaded; only a pass the user asked for may fetch.