dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Make storage pluggable, and prove it with a folder backend

Browse Source 
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.

A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:

- **Capabilities** — already there, and the reason the engine can drive
  two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
  whatever form its connector addresses, with no server in it. Loads
  every existing config unchanged (`backend` defaults to `nextcloud`,
  `endpoint` is stored under its historical `server` key), and
  `Account::namespace()` reproduces the old catalog directory byte for
  byte, because changing it would abandon a catalog, its thumbnail
  shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
  `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
  names a connector.

`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.

Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.

`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.

docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
This commit is contained in:
dtourolle
2026-08-29 09:57:52 +02:00
parent 1b8b7998a2
commit f12aece07e
40 changed files with 3617 additions and 960 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-sync/src/provider.rs
+278
View File
@@ -0,0 +1,278 @@
// TRACES: FR-NC-12
//! How a connector announces itself.
//!
//! [`RemoteBackend`] says what a backend can *do* once it is open.
//! [`BackendProvider`] says everything the application needs before that: what
//! to call it, what a library location looks like, whether signing in involves
//! a browser, and how to turn a stored [`Account`] into a live backend.
//!
//! Together they are the whole contract. Adding a storage layer is:
//!
//! 1. implement [`RemoteBackend`] over your protocol,
//! 2. implement [`BackendProvider`] beside it,
//! 3. register it in `dr_ui::remote`.
//!
//! Nothing above that module names a connector, so nothing above it changes.
//!
//! # Why sign-in is a shape rather than a method
//!
//! It would be tidier for a provider to expose `async fn sign_in()` and let
//! the launch screen await it. It would also be wrong: Nextcloud's Login Flow
//! v2 is a browser handshake the user completes elsewhere while the app polls,
//! so it is not one call, it does not finish on our schedule, and the screen
//! has to render a URL and a waiting state in the middle of it. A folder needs
//! none of that. [`SignIn`] names which of those two shapes the screen must
//! draw, and the flow itself stays where its protocol is.
use std::sync::Arc;
use crate::{Account, Connection, RemoteBackend, RemoteError};
/// What establishing an account involves.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SignIn {
/// A handshake the user completes outside the app, yielding a credential
/// the app then stores. Nextcloud's Login Flow v2.
///
/// The connector drives it; the launch screen only shows the waiting
/// state, because what happens in the middle is protocol-specific.
Browser,
/// The endpoint is the whole account. Nothing to authenticate, nothing to
/// store in the keyring, no waiting state to draw — a local folder.
EndpointOnly,
}
impl SignIn {
/// Whether an account of this shape has a credential in secure storage.
pub fn needs_secret(self) -> bool {
matches!(self, SignIn::Browser)
}
}
/// TRACES: FR-NC-12
/// A storage connector, described well enough to configure without naming it.
///
/// Implementations are held in an [`Arc`] inside a [`BackendRegistry`] and
/// must be usable from any thread: the launch screen reads them on the UI
/// thread and workers open connections from them on their own.
pub trait BackendProvider: Send + Sync {
/// The stable identifier written to [`Account::backend`].
///
/// **It is on-disk configuration.** Changing it after anyone has an
/// account orphans that account, so pick it once.
fn id(&self) -> &'static str;
/// What to call this in the interface. "Nextcloud", "Folder".
fn display_name(&self) -> &'static str;
/// What to label the endpoint field: "Server address", "Folder".
fn endpoint_label(&self) -> &'static str;
/// An example endpoint, for the empty field.
fn endpoint_placeholder(&self) -> &'static str;
/// How an account of this kind is established.
fn sign_in(&self) -> SignIn;
/// Turn what the user typed into the form that gets stored.
///
/// Two jobs, and the second is the important one: this is where a bad
/// endpoint is *rejected*, before an account is written for a library that
/// does not exist. The error is shown to the user, so it says what is
/// wrong rather than naming a type.
fn normalise_endpoint(&self, input: &str) -> Result<String, String>;
/// Build an account from a normalised endpoint alone.
///
/// Only meaningful for [`SignIn::EndpointOnly`]; a browser flow produces
/// its account from what the handshake returned, so the default here
/// refuses rather than inventing one.
fn account_for(&self, endpoint: &str) -> Result<Account, RemoteError> {
let _ = endpoint;
Err(RemoteError::Unsupported(
"this backend establishes an account through its sign-in flow",
))
}
/// Open a live backend.
///
/// Cheap and synchronous: it validates configuration and constructs a
/// client, and does not talk to the remote. Workers call it per task, so
/// anything expensive here is paid over and over.
fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError>;
}
/// TRACES: FR-NC-12 | FR-NC-13
/// The connectors this build has.
///
/// One instance is built at startup and consulted by everything that needs a
/// backend. The registry is the *only* thing that knows connectors exist,
/// which is what keeps the layers above free of them.
#[derive(Clone, Default)]
pub struct BackendRegistry {
providers: Vec<Arc<dyn BackendProvider>>,
}
impl BackendRegistry {
pub fn new() -> Self {
Self::default()
}
/// Add a connector.
///
/// Later registrations of an id replace earlier ones, so a build can
/// substitute a connector — a test double for a real server — without the
/// registry needing to know it happened.
pub fn register(&mut self, provider: Arc<dyn BackendProvider>) -> &mut Self {
let id = provider.id();
self.providers.retain(|p| p.id() != id);
self.providers.push(provider);
self
}
/// The connector for an id.
pub fn get(&self, id: &str) -> Option<&Arc<dyn BackendProvider>> {
self.providers.iter().find(|p| p.id() == id)
}
/// The connector an account names, or a message naming the account's.
///
/// The error case is real rather than defensive: a configuration file can
/// outlive the build that wrote it, and a user moving between a full
/// desktop build and a cut-down one will have accounts this binary cannot
/// serve. Saying which backend is missing is the difference between that
/// and "could not open library".
pub fn for_account(&self, account: &Account) -> Result<&Arc<dyn BackendProvider>, RemoteError> {
self.get(&account.backend).ok_or_else(|| {
RemoteError::Configuration(format!(
"no storage backend named {:?} in this build",
account.backend
))
})
}
/// Open the backend an account is configured for.
pub fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
self.for_account(&conn.account)?.connect(conn)
}
/// Every connector, in registration order. What the launch screen offers.
pub fn providers(&self) -> &[Arc<dyn BackendProvider>] {
&self.providers
}
}
impl std::fmt::Debug for BackendRegistry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BackendRegistry")
.field(
"providers",
&self.providers.iter().map(|p| p.id()).collect::<Vec<_>>(),
)
.finish()
}
}
#[cfg(test)]
mod tests {
use super::*;
struct Stub(&'static str);
impl BackendProvider for Stub {
fn id(&self) -> &'static str {
self.0
}
fn display_name(&self) -> &'static str {
"Stub"
}
fn endpoint_label(&self) -> &'static str {
"Where"
}
fn endpoint_placeholder(&self) -> &'static str {
"somewhere"
}
fn sign_in(&self) -> SignIn {
SignIn::EndpointOnly
}
fn normalise_endpoint(&self, input: &str) -> Result<String, String> {
if input.trim().is_empty() {
Err("say where the library is".into())
} else {
Ok(input.trim().to_string())
}
}
fn connect(&self, _conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
Err(RemoteError::Unsupported("stub"))
}
}
fn registry() -> BackendRegistry {
let mut r = BackendRegistry::new();
r.register(Arc::new(Stub("alpha")));
r.register(Arc::new(Stub("beta")));
r
}
#[test]
fn a_registered_backend_is_found_by_id() {
assert_eq!(registry().get("beta").map(|p| p.id()), Some("beta"));
}
#[test]
fn registering_an_id_twice_replaces_rather_than_shadows() {
let mut r = registry();
r.register(Arc::new(Stub("alpha")));
assert_eq!(r.providers().len(), 2, "{r:?}");
}
#[test]
fn an_account_for_a_missing_backend_says_which_one() {
// A config can outlive the build that wrote it. "could not open
// library" would send the user to check their server.
let account = Account::new("s3", "bucket");
let err = match registry().for_account(&account) {
Err(e) => e.to_string(),
Ok(p) => panic!("a backend this build has no connector for: {}", p.id()),
};
assert!(err.contains("s3"), "{err}");
}
#[test]
fn an_endpoint_only_backend_needs_no_credential() {
assert!(!SignIn::EndpointOnly.needs_secret());
assert!(SignIn::Browser.needs_secret());
}
#[test]
fn a_browser_backend_refuses_to_invent_an_account() {
// Building one from an endpoint would skip the handshake and store an
// account with no credential, which fails later and further away.
struct Interactive;
impl BackendProvider for Interactive {
fn id(&self) -> &'static str {
"i"
}
fn display_name(&self) -> &'static str {
"I"
}
fn endpoint_label(&self) -> &'static str {
"Server"
}
fn endpoint_placeholder(&self) -> &'static str {
""
}
fn sign_in(&self) -> SignIn {
SignIn::Browser
}
fn normalise_endpoint(&self, i: &str) -> Result<String, String> {
Ok(i.into())
}
fn connect(&self, _: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
Err(RemoteError::Unsupported("stub"))
}
}
assert!(Interactive.account_for("https://x").is_err());
}
}