dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Make storage pluggable, and prove it with a folder backend

Browse Source 
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.

A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:

- **Capabilities** — already there, and the reason the engine can drive
  two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
  whatever form its connector addresses, with no server in it. Loads
  every existing config unchanged (`backend` defaults to `nextcloud`,
  `endpoint` is stored under its historical `server` key), and
  `Account::namespace()` reproduces the old catalog directory byte for
  byte, because changing it would abandon a catalog, its thumbnail
  shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
  `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
  names a connector.

`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.

Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.

`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.

docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
This commit is contained in:
dtourolle
2026-08-29 09:57:52 +02:00
parent 1b8b7998a2
commit f12aece07e
40 changed files with 3617 additions and 960 deletions
Download Patch File Download Diff File Expand all files Collapse all files
core/dr-sync/Cargo.toml
+5
View File
@@ -7,7 +7,12 @@ license.workspace = true
[dependencies]
dr-types.workspace = true
# Accounts keep their credential in platform secure storage, never in the
# config file they are otherwise written to (NFR-SEC-2).
dr-plat.workspace = true
async-trait.workspace = true
serde.workspace = true
serde_json.workspace = true
thiserror.workspace = true
log.workspace = true
core/dr-sync/src/account.rs
+804
View File
@@ -0,0 +1,804 @@
// TRACES: FR-NC-12 | FR-NC-2
//! What a configured library *is*, with no connector in it.
//!
//! Before this existed, "an account" meant a Nextcloud server URL, a login
//! name and a DAV user id, and that shape reached every layer above:
//! `dr-ui` stored it, keyed its caches off it, threaded it through a dozen
//! worker threads and handed it to a constructor named after one product.
//! [`RemoteBackend`](crate::RemoteBackend) was abstract; everything that
//! *reached* a backend was not, so a second connector had nowhere to live.
//!
//! An [`Account`] is what remains once the product is taken out: somewhere a
//! library lives ([`endpoint`](Account::endpoint)), a folder inside it
//! ([`root`](Account::root)), and the settings the scan needs. What an
//! endpoint means is the connector's business — a URL for Nextcloud, a
//! directory for a plain folder, a bucket for whatever comes next.
//!
//! # The split that has to survive
//!
//! Credentials go to platform secure storage (FR-NC-2). Never the catalog,
//! never a file, never a log line. Everything else is ordinary configuration
//! written as plain JSON. That split is what lets the app show "signed in as
//! duncan, watching /PhotosRaw" before it has touched the keyring — and it is
//! why a [`Connection`] carries the two halves separately rather than as one
//! blob.
use std::path::{Path, PathBuf};
use dr_plat::{SecretError, SecretRef, SecretStore};
use dr_types::{Format, FormatFilter};
use serde::{Deserialize, Serialize};
use crate::RemoteError;
/// The connector every account had before there was a choice.
///
/// Named here, in connector-neutral code, for exactly one reason:
/// [`Account::namespace`] must keep producing the same string for these
/// accounts as the hard-coded Nextcloud version did. That string is a
/// directory name holding a catalog, thumbnail shards, un-uploaded sidecars
/// and an export outbox. Changing it does not lose that data, it *abandons*
/// it — silently, as an upgrade — and costs a full rescan of the library on
/// top.
///
/// Nothing else in this crate branches on a connector's identity, and nothing
/// else should.
pub const LEGACY_BACKEND: &str = "nextcloud";
/// Where configuration is written, when the platform has told us.
///
/// Android has no `$HOME` and no XDG directories, so the guess below resolves
/// to a path the app cannot write. Nothing failed loudly: the account list went
/// to a doomed path, so credentials survived only as long as the process did and
/// backgrounding the app lost the account (ARCH §6.9 — no core API may assume a
/// filesystem path on Android).
///
/// The platform layer sets this once at startup, before any store is opened.
static DATA_DIR: std::sync::OnceLock<PathBuf> = std::sync::OnceLock::new();
/// TRACES: FR-NC-2
/// Declare the per-app directory configuration belongs in.
///
/// Call before opening any store; later calls are ignored rather than racing.
/// On Android this is `AndroidApp::internal_data_path`, which is private to the
/// app and survives being backgrounded. Desktop needs no call — the XDG
/// fallback is correct there.
pub fn set_data_dir(dir: PathBuf) {
let _ = DATA_DIR.set(dir);
}
/// The directory configuration lives in.
pub fn config_dir() -> PathBuf {
if let Some(d) = DATA_DIR.get() {
return d.clone();
}
std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| PathBuf::from(std::env::var("HOME").unwrap_or_default()).join(".config"))
.join("darkroom")
}
/// TRACES: FR-NC-12
/// A configured library, minus its credential.
///
/// Every field but [`backend`](Self::backend) is interpreted by the connector
/// that owns it. Code above this layer reads them for display and for cache
/// keys and never for meaning.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
pub struct Account {
/// Which connector serves this library, as
/// [`BackendProvider::id`](crate::BackendProvider::id).
///
/// Defaulted rather than required, because every account written before
/// there was a choice omits it and every one of them is a Nextcloud
/// account. A missing field here must load, not fail — a config the app
/// refuses to parse is an account the user has to set up again.
#[serde(default = "legacy_backend")]
pub backend: String,
/// Where the library lives, in whatever form the connector addresses:
/// `https://cloud.example` for Nextcloud, `/mnt/photos` for a folder.
///
/// Stored under its historical name so existing configuration loads
/// unchanged.
#[serde(rename = "server")]
pub endpoint: String,
/// Who we are, where that means anything. Empty for connectors with no
/// notion of a user — it is shown, and used to key the credential.
#[serde(default)]
pub login: String,
/// A connector-defined sub-address. Nextcloud's DAV path segment, which
/// may differ from `login` because a login can be an email address while
/// the user id is something else. Empty where the connector has no use
/// for one.
#[serde(default)]
pub user_id: String,
/// The folder chosen as the library root, relative to the endpoint. Empty
/// means the endpoint itself.
#[serde(default)]
pub root: String,
/// Which formats the scan looks for (the tick-boxes).
#[serde(default)]
pub formats: Vec<String>,
/// Unix seconds of the last completed scan, for display.
#[serde(default)]
pub last_scan: Option<i64>,
}
fn legacy_backend() -> String {
LEGACY_BACKEND.to_string()
}
impl Account {
/// A bare account for `backend` at `endpoint`, with nothing chosen yet.
pub fn new(backend: impl Into<String>, endpoint: impl Into<String>) -> Self {
Self {
backend: backend.into(),
endpoint: endpoint.into(),
login: String::new(),
user_id: String::new(),
root: String::new(),
formats: Vec::new(),
last_scan: None,
}
}
pub fn with_login(mut self, login: impl Into<String>, user_id: impl Into<String>) -> Self {
self.login = login.into();
self.user_id = user_id.into();
self
}
/// Whether two records name the same account.
///
/// The identity the store deduplicates on. Endpoint and login together,
/// because one server can hold two accounts and one machine can hold two
/// folders — but the *same* pair twice is the same library reconfigured,
/// not a second one.
pub fn is_same_as(&self, other: &Account) -> bool {
self.backend == other.backend
&& self.endpoint == other.endpoint
&& self.login == other.login
}
/// The stored format selection, defaulting to every supported format.
///
/// An unconfigured account must find everything rather than nothing.
pub fn format_filter(&self) -> FormatFilter {
if self.formats.is_empty() {
FormatFilter::all()
} else {
FormatFilter::from_formats(
self.formats
.iter()
.filter_map(|s| Format::from_extension(&s.to_ascii_lowercase())),
)
}
}
pub fn set_format_filter(&mut self, filter: &FormatFilter) {
self.formats = filter
.iter()
.map(|f| format!("{f:?}").to_lowercase())
.collect();
}
/// Where this account's credential lives, for connectors that need one.
pub fn secret_ref(&self) -> SecretRef {
SecretRef::app_password(&self.endpoint, &self.login)
}
/// A short description for the UI.
///
/// Reads for both shapes without asking the connector: "duncan on
/// cloud.example/PhotosRaw" where there is a login, and just the location
/// where there is not — a folder library has no user to name, and
/// inventing one ("(local) on /mnt/photos") would be worse than saying
/// where it is.
pub fn describe(&self) -> String {
let place = self
.endpoint
.trim_start_matches("https://")
.trim_start_matches("http://");
let place = if self.root.is_empty() {
place.to_string()
} else {
format!("{}/{}", place.trim_end_matches('/'), self.root)
};
if self.login.is_empty() {
place
} else {
format!("{} on {place}", self.login)
}
}
/// TRACES: FR-NC-10 | NFR-R1
/// The directory name this account's local data hangs off.
///
/// Not a display string and not stable across a change of endpoint: it is
/// the key for the catalog, the thumbnail shards, the sidecar spool and
/// the export outbox. Two accounts must never collide here — one would
/// index the other's library — and one account must produce the same
/// answer on every launch, forever, or its data is abandoned in place.
///
/// The Nextcloud form is reproduced byte for byte from what
/// `catalog_path` computed before accounts were multi-backend
/// ([`LEGACY_BACKEND`]). Everything else is prefixed by its connector, so
/// a folder library at `/srv/photos` and a hypothetical S3 bucket of the
/// same name cannot land in one directory.
pub fn namespace(&self) -> String {
let slug = slugify(
self.endpoint
.trim_start_matches("https://")
.trim_start_matches("http://"),
);
if self.backend == LEGACY_BACKEND {
// Frozen. See LEGACY_BACKEND.
return format!("{slug}-{}", self.user_id);
}
let tail = if self.user_id.is_empty() {
String::new()
} else {
format!("-{}", slugify(&self.user_id))
};
let name = format!("{}-{slug}{tail}", slugify(&self.backend));
shorten(&name)
}
}
/// Everything that is not `[A-Za-z0-9]`, flattened to `-`.
///
/// Not an escape and not reversible: the result names a directory, and the
/// only property it needs is that it is a legal filename on every platform
/// the app runs on.
fn slugify(s: &str) -> String {
s.chars()
.map(|c| if c.is_ascii_alphanumeric() { c } else { '-' })
.collect()
}
/// Cap a namespace at a length every filesystem accepts.
///
/// A folder endpoint is an absolute path and can be far longer than a server
/// URL — deep enough to exceed the 255-byte component limit on ext4 and APFS
/// alike, at which point creating the catalog directory fails and the library
/// cannot be opened at all. Truncating alone would make two deep paths under
/// one parent collide, so the discarded tail is replaced by a hash of the
/// whole.
fn shorten(name: &str) -> String {
const MAX: usize = 96;
if name.len() <= MAX {
return name.to_string();
}
let head: String = name.chars().take(MAX - 17).collect();
format!("{head}-{:016x}", fnv1a64(name.as_bytes()))
}
/// FNV-1a, 64-bit.
///
/// Written out rather than taken from `DefaultHasher`, whose output is
/// explicitly not stable between Rust releases. This one keys a directory that
/// must be found again after a toolchain upgrade.
fn fnv1a64(bytes: &[u8]) -> u64 {
let mut h: u64 = 0xcbf2_9ce4_8422_2325;
for b in bytes {
h ^= *b as u64;
h = h.wrapping_mul(0x0000_0100_0000_01b3);
}
h
}
/// TRACES: FR-NC-2 | NFR-SEC-2
/// A credential, kept out of logs by construction.
///
/// The inner string is reachable only through [`expose`](Secret::expose), so
/// the ways a secret leaks — a `{:?}` on a struct that happens to contain one,
/// a `Display` in an error message — do not compile into a leak. NFR-SEC-2 is
/// the requirement; this is the part of it that a reviewer cannot forget to
/// apply.
#[derive(Clone, PartialEq, Eq)]
pub struct Secret(String);
impl Secret {
pub fn new(value: impl Into<String>) -> Self {
Secret(value.into())
}
/// The credential itself. Every call site is a place to check.
pub fn expose(&self) -> &str {
&self.0
}
}
impl std::fmt::Debug for Secret {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str("Secret(***)")
}
}
/// Everything needed to open a backend, in one movable value.
///
/// Workers run on their own threads and each one needs its own way in, so this
/// is `Clone` and owns what it holds. It replaced a pair of arguments —
/// credentials and a user id — that had to be threaded together through
/// fifteen functions and could be passed in the wrong order.
#[derive(Debug, Clone)]
pub struct Connection {
pub account: Account,
/// `None` where the connector needs no credential, which is the ordinary
/// state of a folder library rather than a failure to load one.
pub secret: Option<Secret>,
}
impl Connection {
pub fn new(account: Account, secret: Option<Secret>) -> Self {
Self { account, secret }
}
/// The credential, or [`RemoteError::Unauthenticated`].
///
/// For connectors that require one: turning the absence into the error the
/// caller already handles saves every implementation writing the same
/// `ok_or`.
pub fn require_secret(&self) -> Result<&Secret, RemoteError> {
self.secret.as_ref().ok_or(RemoteError::Unauthenticated)
}
}
/// TRACES: FR-NC-1 | FR-NC-2 | M-1 | M-2
/// Loads and saves accounts, keeping credentials in secure storage.
pub struct AccountStore {
config_path: PathBuf,
secrets: Box<dyn SecretStore>,
}
/// What is written to disk. Versioned so a format change is a migration
/// rather than a parse failure.
#[derive(Debug, Default, Serialize, Deserialize)]
struct ConfigFile {
#[serde(default = "one")]
version: u32,
/// Named `sessions` on disk because that is what it has always been
/// called there, and renaming the key would orphan every existing config.
#[serde(default)]
sessions: Vec<Account>,
}
fn one() -> u32 {
1
}
impl AccountStore {
/// Open the store at the platform config location.
///
/// Linux: `$XDG_CONFIG_HOME/darkroom/sessions.json`, falling back to
/// `~/.config` (FR-PLAT-LIN-1).
pub fn open(secrets: Box<dyn SecretStore>) -> Self {
Self::open_at(config_dir().join("sessions.json"), secrets)
}
/// Where configuration lives, for callers that need to sit files beside it.
pub fn data_dir() -> PathBuf {
config_dir()
}
/// Open at an explicit path — used by tests, and by anything wanting a
/// non-default config location.
pub fn open_at(config_path: PathBuf, secrets: Box<dyn SecretStore>) -> Self {
Self {
config_path,
secrets,
}
}
pub fn config_path(&self) -> &Path {
&self.config_path
}
/// Whether credentials can be remembered at all.
///
/// Where false the UI should say sign-in will not persist, rather than
/// letting the user discover it next launch.
pub fn can_remember(&self) -> bool {
self.secrets.is_available()
}
/// Every configured account. Missing or unreadable config yields an empty
/// list rather than an error — a first run is not a failure.
pub fn list(&self) -> Vec<Account> {
self.read_config().sessions
}
/// The most recently configured account, if any.
pub fn current(&self) -> Option<Account> {
self.read_config().sessions.into_iter().next_back()
}
/// Persist an account and its credential.
///
/// The credential goes to secure storage first: if that fails there is no
/// point recording an account that cannot authenticate. `None` is the
/// ordinary case for a connector that needs no credential, and stores
/// nothing rather than an empty secret.
pub fn save(&self, account: &Account, secret: Option<&Secret>) -> Result<(), AccountError> {
if let Some(s) = secret {
self.secrets.store(&account.secret_ref(), s.expose())?;
}
let mut config = self.read_config();
config.sessions.retain(|a| !a.is_same_as(account));
config.sessions.push(account.clone());
self.write_config(&config)
}
/// Update an account's settings, leaving its credential untouched.
pub fn update(&self, account: &Account) -> Result<(), AccountError> {
let mut config = self.read_config();
match config.sessions.iter_mut().find(|a| a.is_same_as(account)) {
Some(existing) => *existing = account.clone(),
None => config.sessions.push(account.clone()),
}
self.write_config(&config)
}
/// Rebuild a connection for an account, fetching its credential.
///
/// `needs_secret` is the connector's answer, passed in rather than
/// inferred: an account with an empty login might be a folder library or
/// might be a broken Nextcloud record, and guessing turns the second into
/// a silent unauthenticated connection instead of an error the user can
/// act on.
///
/// [`SecretError::NotFound`] means the credential was revoked or the
/// keyring was cleared — the caller re-runs the sign-in.
pub fn connection(
&self,
account: &Account,
needs_secret: bool,
) -> Result<Connection, AccountError> {
let secret = if needs_secret {
Some(Secret::new(self.secrets.retrieve(&account.secret_ref())?))
} else {
None
};
Ok(Connection::new(account.clone(), secret))
}
/// Forget an account and delete its credential.
///
/// The credential is removed even if the config write fails, so a logout
/// never leaves a usable secret behind. A connector that stores none
/// reports [`SecretError::NotFound`], which is not a failure to forget.
pub fn forget(&self, account: &Account) -> Result<(), AccountError> {
let deleted = match self.secrets.delete(&account.secret_ref()) {
Err(SecretError::NotFound) => Ok(()),
other => other,
};
let mut config = self.read_config();
config.sessions.retain(|a| !a.is_same_as(account));
let written = self.write_config(&config);
deleted?;
written
}
fn read_config(&self) -> ConfigFile {
std::fs::read_to_string(&self.config_path)
.ok()
.and_then(|t| serde_json::from_str(&t).ok())
.unwrap_or_default()
}
fn write_config(&self, config: &ConfigFile) -> Result<(), AccountError> {
if let Some(parent) = self.config_path.parent() {
std::fs::create_dir_all(parent)?;
}
let json = serde_json::to_string_pretty(config)?;
// Write and rename, so an interrupted save cannot truncate an
// existing config.
let tmp = self.config_path.with_extension("tmp");
std::fs::write(&tmp, json)?;
std::fs::rename(&tmp, &self.config_path)?;
Ok(())
}
}
#[derive(Debug, thiserror::Error)]
pub enum AccountError {
#[error("secure storage: {0}")]
Secret(#[from] SecretError),
#[error("config io: {0}")]
Io(#[from] std::io::Error),
#[error("config format: {0}")]
Serde(#[from] serde_json::Error),
#[error(transparent)]
Remote(#[from] RemoteError),
}
#[cfg(test)]
mod tests {
use super::*;
use dr_plat::EphemeralSecretStore;
fn nextcloud() -> Account {
Account::new(LEGACY_BACKEND, "https://cloud.example").with_login("duncan", "duncan")
}
fn folder() -> Account {
Account::new("folder", "/mnt/photos")
}
fn store_in(dir: &Path) -> AccountStore {
AccountStore::open_at(
dir.join("sessions.json"),
Box::new(EphemeralSecretStore::new()),
)
}
fn tmpdir(name: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("darkroom-account-test-{name}"));
let _ = std::fs::remove_dir_all(&d);
std::fs::create_dir_all(&d).unwrap();
d
}
#[test]
fn a_saved_account_survives_reopening() {
let dir = tmpdir("survives");
let store = store_in(&dir);
let mut a = nextcloud();
a.root = "PhotosRaw".into();
store.save(&a, Some(&Secret::new("token"))).unwrap();
let reloaded = store.current().expect("account persisted");
assert_eq!(reloaded.login, "duncan");
assert_eq!(reloaded.root, "PhotosRaw");
}
#[test]
fn the_credential_never_reaches_the_config_file() {
// NFR-SEC-2: the whole point of the split.
let dir = tmpdir("nocreds");
let store = store_in(&dir);
store
.save(&nextcloud(), Some(&Secret::new("secret-token")))
.unwrap();
let text = std::fs::read_to_string(dir.join("sessions.json")).unwrap();
assert!(!text.contains("secret-token"), "credential leaked to disk");
assert!(text.contains("duncan"), "account metadata should be there");
}
#[test]
fn a_secret_does_not_print_itself() {
// The leak this closes is indirect: a `{:?}` on any struct holding a
// connection used to print the app password.
let c = Connection::new(nextcloud(), Some(Secret::new("hunter2")));
let printed = format!("{c:?}");
assert!(!printed.contains("hunter2"), "credential leaked to a log");
}
#[test]
fn credentials_round_trip_through_secure_storage() {
let dir = tmpdir("roundtrip");
let store = store_in(&dir);
let a = nextcloud();
store.save(&a, Some(&Secret::new("secret-token"))).unwrap();
let conn = store.connection(&a, true).unwrap();
assert_eq!(conn.require_secret().unwrap().expose(), "secret-token");
}
#[test]
fn a_credentialless_account_connects_without_touching_the_keyring() {
// A folder library must open on a machine with no secrets daemon at
// all — asking for a credential it does not have would fail the one
// backend that needs nothing.
let dir = tmpdir("nosecret");
let store = store_in(&dir);
let a = folder();
store.save(&a, None).unwrap();
let conn = store.connection(&a, false).unwrap();
assert!(conn.secret.is_none());
assert!(matches!(
conn.require_secret(),
Err(RemoteError::Unauthenticated)
));
}
#[test]
fn forgetting_removes_both_halves() {
let dir = tmpdir("forget");
let store = store_in(&dir);
let a = nextcloud();
store.save(&a, Some(&Secret::new("token"))).unwrap();
store.forget(&a).unwrap();
assert!(store.current().is_none());
assert!(matches!(
store.connection(&a, true),
Err(AccountError::Secret(SecretError::NotFound))
));
}
#[test]
fn forgetting_a_credentialless_account_is_not_an_error() {
// There is no secret to delete, and reporting the absence as a failure
// would leave a folder library that cannot be signed out of.
let dir = tmpdir("forget-folder");
let store = store_in(&dir);
let a = folder();
store.save(&a, None).unwrap();
store.forget(&a).unwrap();
assert!(store.current().is_none());
}
#[test]
fn two_backends_at_the_same_endpoint_are_two_accounts() {
let dir = tmpdir("twobackends");
let store = store_in(&dir);
store.save(&Account::new("folder", "/mnt/p"), None).unwrap();
store.save(&Account::new("webdav", "/mnt/p"), None).unwrap();
assert_eq!(store.list().len(), 2);
}
#[test]
fn saving_the_same_account_twice_does_not_duplicate_it() {
let dir = tmpdir("dedupe");
let store = store_in(&dir);
let mut a = nextcloud();
store.save(&a, Some(&Secret::new("token"))).unwrap();
a.root = "Photos".into();
store.save(&a, Some(&Secret::new("token"))).unwrap();
assert_eq!(store.list().len(), 1);
assert_eq!(store.current().unwrap().root, "Photos");
}
#[test]
fn a_missing_config_is_a_first_run_not_an_error() {
let dir = tmpdir("firstrun");
let store = store_in(&dir);
assert!(store.list().is_empty());
assert!(store.current().is_none());
}
#[test]
fn a_corrupt_config_does_not_prevent_starting() {
// Better to present a first-run state than to refuse to launch.
let dir = tmpdir("corrupt");
std::fs::write(dir.join("sessions.json"), "{ not json").unwrap();
let store = store_in(&dir);
assert!(store.list().is_empty());
}
#[test]
fn a_config_written_before_backends_existed_still_loads() {
// The upgrade path. Every account written by an earlier version omits
// `backend`, and refusing to parse one would make an upgrade look
// like a signed-out app with a library that has to be set up again.
let dir = tmpdir("legacy");
std::fs::write(
dir.join("sessions.json"),
r#"{"version":1,"sessions":[{"server":"https://cloud.example",
"login":"duncan","user_id":"duncan","root":"PhotosRaw",
"formats":[],"last_scan":null}]}"#,
)
.unwrap();
let a = store_in(&dir).current().expect("legacy account loads");
assert_eq!(a.backend, LEGACY_BACKEND);
assert_eq!(a.endpoint, "https://cloud.example");
assert_eq!(a.root, "PhotosRaw");
}
#[test]
fn a_legacy_account_keeps_the_directory_its_data_is_already_in() {
// Frozen deliberately: this string names the directory holding the
// catalog, the thumbnail shards and un-uploaded sidecars. A change
// here abandons all three and forces a full rescan.
let a = Account::new(LEGACY_BACKEND, "https://cloud.example.com").with_login("d", "duncan");
assert_eq!(a.namespace(), "cloud-example-com-duncan");
}
#[test]
fn a_new_backend_cannot_collide_with_a_legacy_one() {
let ns = Account::new("folder", "/mnt/photos").namespace();
assert!(ns.starts_with("folder-"), "{ns}");
assert_ne!(ns, Account::new(LEGACY_BACKEND, "/mnt/photos").namespace());
}
#[test]
fn two_folders_never_share_a_directory() {
// Two libraries in one catalog would index each other's images.
assert_ne!(
Account::new("folder", "/mnt/photos/2025").namespace(),
Account::new("folder", "/mnt/photos/2026").namespace()
);
}
#[test]
fn a_very_deep_folder_still_yields_a_legal_directory_name() {
// Past 255 bytes the catalog directory cannot be created at all, and
// the library simply fails to open.
let deep = format!("/{}", vec!["a-rather-long-folder-name"; 40].join("/"));
let a = Account::new("folder", &deep);
let ns = a.namespace();
assert!(ns.len() <= 96, "{} chars", ns.len());
// Truncation alone would make these two the same directory.
let b = Account::new("folder", format!("{deep}/second"));
assert_ne!(ns, b.namespace());
}
#[test]
fn describe_reads_for_an_account_with_no_user() {
// A folder library has nobody to name; "(none) on /mnt/photos" would
// be worse than saying where it is.
let mut a = folder();
assert_eq!(a.describe(), "/mnt/photos");
a.root = "2026".into();
assert_eq!(a.describe(), "/mnt/photos/2026");
}
#[test]
fn describe_is_readable_and_hides_the_scheme() {
let mut a = nextcloud();
assert_eq!(a.describe(), "duncan on cloud.example");
a.root = "PhotosRaw".into();
assert_eq!(a.describe(), "duncan on cloud.example/PhotosRaw");
}
#[test]
fn format_selection_round_trips() {
let mut a = nextcloud();
a.set_format_filter(&FormatFilter::from_formats([Format::Cr2, Format::Dng]));
let f = a.format_filter();
assert!(f.allows(Format::Cr2));
assert!(f.allows(Format::Dng));
assert!(!f.allows(Format::Nef));
}
#[test]
fn an_unset_filter_means_every_format() {
// Never "no formats", which would silently find nothing.
let f = nextcloud().format_filter();
assert!(f.allows(Format::Cr2));
assert!(f.allows(Format::Jpeg));
}
#[test]
fn updating_settings_leaves_the_credential_alone() {
let dir = tmpdir("update");
let store = store_in(&dir);
let mut a = nextcloud();
store.save(&a, Some(&Secret::new("secret-token"))).unwrap();
a.root = "Elsewhere".into();
store.update(&a).unwrap();
assert_eq!(store.current().unwrap().root, "Elsewhere");
assert_eq!(
store
.connection(&a, true)
.unwrap()
.require_secret()
.unwrap()
.expose(),
"secret-token"
);
}
}
core/dr-sync/src/error.rs
+15
View File
@@ -41,6 +41,21 @@ pub enum RemoteError {
#[error("operation unsupported by this backend: {0}")]
Unsupported(&'static str),
/// The account is configured wrongly, or for a backend this build has no
/// connector for.
///
/// **Not a network failure and not an auth failure**, which is why it is
/// its own variant. A folder library whose directory has been unmounted,
/// or an account naming a backend a cut-down build was not compiled with,
/// produces a request that never leaves the process — reporting either as
/// `Network` would put the app into offline mode and tell the user their
/// connection is down, and reporting them as `AuthFailed` would send them
/// to re-enter a credential that is fine. The message names what is wrong
/// with the configuration, because that is the only thing that will fix
/// it.
#[error("account misconfigured: {0}")]
Configuration(String),
/// A conditional write failed: the remote changed underneath us. Triggers
/// the sidecar merge path (ARCH §8.5).
#[error("precondition failed — remote was modified")]
core/dr-sync/src/lib.rs
+12 -3
View File
@@ -1,9 +1,14 @@
//! Pluggable remote storage for DarkRoom.
//!
//! Defines the [`RemoteBackend`] trait and the capability model the sync
//! engine adapts to. Only the Nextcloud connector is implemented
//! (`dr-sync-nextcloud`), but the boundary is designed so other backends can
//! be added without touching the engine.
//! engine adapts to, plus the pieces that let the application hold a backend
//! without naming one: an [`Account`] that is configuration rather than a
//! server, and a [`BackendProvider`] registry that turns one into a live
//! connection.
//!
//! Two connectors ship: `dr-sync-nextcloud` and `dr-sync-folder`. Adding a
//! third is implementing those two traits and registering the result — see
//! [`provider`] for the whole contract.
//!
//! # Why capabilities rather than a common denominator
//!
@@ -20,15 +25,19 @@ use std::ops::Range;
use async_trait::async_trait;
pub mod account;
pub mod capability;
pub mod error;
pub mod provider;
pub mod reachability;
pub mod scan;
pub mod types;
pub mod upload;
pub use account::{Account, AccountError, AccountStore, Connection, Secret, LEGACY_BACKEND};
pub use capability::{Capabilities, ChangeDetection, ChunkConstraints, ServerPreviews};
pub use error::RemoteError;
pub use provider::{BackendProvider, BackendRegistry, SignIn};
pub use reachability::{Connectivity, Reachability};
pub use scan::{scan, ScanProgress, ScanResult};
pub use types::{
core/dr-sync/src/provider.rs
+278
View File
@@ -0,0 +1,278 @@
// TRACES: FR-NC-12
//! How a connector announces itself.
//!
//! [`RemoteBackend`] says what a backend can *do* once it is open.
//! [`BackendProvider`] says everything the application needs before that: what
//! to call it, what a library location looks like, whether signing in involves
//! a browser, and how to turn a stored [`Account`] into a live backend.
//!
//! Together they are the whole contract. Adding a storage layer is:
//!
//! 1. implement [`RemoteBackend`] over your protocol,
//! 2. implement [`BackendProvider`] beside it,
//! 3. register it in `dr_ui::remote`.
//!
//! Nothing above that module names a connector, so nothing above it changes.
//!
//! # Why sign-in is a shape rather than a method
//!
//! It would be tidier for a provider to expose `async fn sign_in()` and let
//! the launch screen await it. It would also be wrong: Nextcloud's Login Flow
//! v2 is a browser handshake the user completes elsewhere while the app polls,
//! so it is not one call, it does not finish on our schedule, and the screen
//! has to render a URL and a waiting state in the middle of it. A folder needs
//! none of that. [`SignIn`] names which of those two shapes the screen must
//! draw, and the flow itself stays where its protocol is.
use std::sync::Arc;
use crate::{Account, Connection, RemoteBackend, RemoteError};
/// What establishing an account involves.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum SignIn {
/// A handshake the user completes outside the app, yielding a credential
/// the app then stores. Nextcloud's Login Flow v2.
///
/// The connector drives it; the launch screen only shows the waiting
/// state, because what happens in the middle is protocol-specific.
Browser,
/// The endpoint is the whole account. Nothing to authenticate, nothing to
/// store in the keyring, no waiting state to draw — a local folder.
EndpointOnly,
}
impl SignIn {
/// Whether an account of this shape has a credential in secure storage.
pub fn needs_secret(self) -> bool {
matches!(self, SignIn::Browser)
}
}
/// TRACES: FR-NC-12
/// A storage connector, described well enough to configure without naming it.
///
/// Implementations are held in an [`Arc`] inside a [`BackendRegistry`] and
/// must be usable from any thread: the launch screen reads them on the UI
/// thread and workers open connections from them on their own.
pub trait BackendProvider: Send + Sync {
/// The stable identifier written to [`Account::backend`].
///
/// **It is on-disk configuration.** Changing it after anyone has an
/// account orphans that account, so pick it once.
fn id(&self) -> &'static str;
/// What to call this in the interface. "Nextcloud", "Folder".
fn display_name(&self) -> &'static str;
/// What to label the endpoint field: "Server address", "Folder".
fn endpoint_label(&self) -> &'static str;
/// An example endpoint, for the empty field.
fn endpoint_placeholder(&self) -> &'static str;
/// How an account of this kind is established.
fn sign_in(&self) -> SignIn;
/// Turn what the user typed into the form that gets stored.
///
/// Two jobs, and the second is the important one: this is where a bad
/// endpoint is *rejected*, before an account is written for a library that
/// does not exist. The error is shown to the user, so it says what is
/// wrong rather than naming a type.
fn normalise_endpoint(&self, input: &str) -> Result<String, String>;
/// Build an account from a normalised endpoint alone.
///
/// Only meaningful for [`SignIn::EndpointOnly`]; a browser flow produces
/// its account from what the handshake returned, so the default here
/// refuses rather than inventing one.
fn account_for(&self, endpoint: &str) -> Result<Account, RemoteError> {
let _ = endpoint;
Err(RemoteError::Unsupported(
"this backend establishes an account through its sign-in flow",
))
}
/// Open a live backend.
///
/// Cheap and synchronous: it validates configuration and constructs a
/// client, and does not talk to the remote. Workers call it per task, so
/// anything expensive here is paid over and over.
fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError>;
}
/// TRACES: FR-NC-12 | FR-NC-13
/// The connectors this build has.
///
/// One instance is built at startup and consulted by everything that needs a
/// backend. The registry is the *only* thing that knows connectors exist,
/// which is what keeps the layers above free of them.
#[derive(Clone, Default)]
pub struct BackendRegistry {
providers: Vec<Arc<dyn BackendProvider>>,
}
impl BackendRegistry {
pub fn new() -> Self {
Self::default()
}
/// Add a connector.
///
/// Later registrations of an id replace earlier ones, so a build can
/// substitute a connector — a test double for a real server — without the
/// registry needing to know it happened.
pub fn register(&mut self, provider: Arc<dyn BackendProvider>) -> &mut Self {
let id = provider.id();
self.providers.retain(|p| p.id() != id);
self.providers.push(provider);
self
}
/// The connector for an id.
pub fn get(&self, id: &str) -> Option<&Arc<dyn BackendProvider>> {
self.providers.iter().find(|p| p.id() == id)
}
/// The connector an account names, or a message naming the account's.
///
/// The error case is real rather than defensive: a configuration file can
/// outlive the build that wrote it, and a user moving between a full
/// desktop build and a cut-down one will have accounts this binary cannot
/// serve. Saying which backend is missing is the difference between that
/// and "could not open library".
pub fn for_account(&self, account: &Account) -> Result<&Arc<dyn BackendProvider>, RemoteError> {
self.get(&account.backend).ok_or_else(|| {
RemoteError::Configuration(format!(
"no storage backend named {:?} in this build",
account.backend
))
})
}
/// Open the backend an account is configured for.
pub fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
self.for_account(&conn.account)?.connect(conn)
}
/// Every connector, in registration order. What the launch screen offers.
pub fn providers(&self) -> &[Arc<dyn BackendProvider>] {
&self.providers
}
}
impl std::fmt::Debug for BackendRegistry {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.debug_struct("BackendRegistry")
.field(
"providers",
&self.providers.iter().map(|p| p.id()).collect::<Vec<_>>(),
)
.finish()
}
}
#[cfg(test)]
mod tests {
use super::*;
struct Stub(&'static str);
impl BackendProvider for Stub {
fn id(&self) -> &'static str {
self.0
}
fn display_name(&self) -> &'static str {
"Stub"
}
fn endpoint_label(&self) -> &'static str {
"Where"
}
fn endpoint_placeholder(&self) -> &'static str {
"somewhere"
}
fn sign_in(&self) -> SignIn {
SignIn::EndpointOnly
}
fn normalise_endpoint(&self, input: &str) -> Result<String, String> {
if input.trim().is_empty() {
Err("say where the library is".into())
} else {
Ok(input.trim().to_string())
}
}
fn connect(&self, _conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
Err(RemoteError::Unsupported("stub"))
}
}
fn registry() -> BackendRegistry {
let mut r = BackendRegistry::new();
r.register(Arc::new(Stub("alpha")));
r.register(Arc::new(Stub("beta")));
r
}
#[test]
fn a_registered_backend_is_found_by_id() {
assert_eq!(registry().get("beta").map(|p| p.id()), Some("beta"));
}
#[test]
fn registering_an_id_twice_replaces_rather_than_shadows() {
let mut r = registry();
r.register(Arc::new(Stub("alpha")));
assert_eq!(r.providers().len(), 2, "{r:?}");
}
#[test]
fn an_account_for_a_missing_backend_says_which_one() {
// A config can outlive the build that wrote it. "could not open
// library" would send the user to check their server.
let account = Account::new("s3", "bucket");
let err = match registry().for_account(&account) {
Err(e) => e.to_string(),
Ok(p) => panic!("a backend this build has no connector for: {}", p.id()),
};
assert!(err.contains("s3"), "{err}");
}
#[test]
fn an_endpoint_only_backend_needs_no_credential() {
assert!(!SignIn::EndpointOnly.needs_secret());
assert!(SignIn::Browser.needs_secret());
}
#[test]
fn a_browser_backend_refuses_to_invent_an_account() {
// Building one from an endpoint would skip the handshake and store an
// account with no credential, which fails later and further away.
struct Interactive;
impl BackendProvider for Interactive {
fn id(&self) -> &'static str {
"i"
}
fn display_name(&self) -> &'static str {
"I"
}
fn endpoint_label(&self) -> &'static str {
"Server"
}
fn endpoint_placeholder(&self) -> &'static str {
""
}
fn sign_in(&self) -> SignIn {
SignIn::Browser
}
fn normalise_endpoint(&self, i: &str) -> Result<String, String> {
Ok(i.into())
}
fn connect(&self, _: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
Err(RemoteError::Unsupported("stub"))
}
}
assert!(Interactive.account_for("https://x").is_err());
}
}