Make storage pluggable, and prove it with a folder backend
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.
A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:
- **Capabilities** — already there, and the reason the engine can drive
two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
whatever form its connector addresses, with no server in it. Loads
every existing config unchanged (`backend` defaults to `nextcloud`,
`endpoint` is stored under its historical `server` key), and
`Account::namespace()` reproduces the old catalog directory byte for
byte, because changing it would abandon a catalog, its thumbnail
shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
`ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
names a connector.
`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.
Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.
`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.
docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
This commit is contained in:
+11
-13
@@ -62,7 +62,7 @@ use std::time::Duration;
|
||||
|
||||
use dr_export::{Encoded, NameContext};
|
||||
use dr_sync::RemotePath;
|
||||
use dr_sync_nextcloud::AppCredentials;
|
||||
use dr_sync::{Account, Connection};
|
||||
use dr_types::{ExportSettings, ExportTarget};
|
||||
|
||||
use crate::AppWindow;
|
||||
@@ -72,8 +72,8 @@ use crate::AppWindow;
|
||||
/// Beside the catalog, for the reason in the module docs. Per account,
|
||||
/// because the destination folder is a path on one particular server and an
|
||||
/// entry queued for one account is meaningless to another.
|
||||
pub fn outbox_dir(server: &str, user_id: &str) -> PathBuf {
|
||||
crate::library::catalog_path(server, user_id)
|
||||
pub fn outbox_dir(account: &Account) -> PathBuf {
|
||||
crate::library::catalog_path(account)
|
||||
.parent()
|
||||
.map(|p| p.join("outbox"))
|
||||
.unwrap_or_else(|| std::env::temp_dir().join("darkroom-outbox"))
|
||||
@@ -146,7 +146,7 @@ impl Placed {
|
||||
),
|
||||
// Named as queued rather than exported: the file is real and
|
||||
// finished, but it is not yet where the user asked for it, and
|
||||
// saying "exported to Nextcloud" before it has uploaded would be
|
||||
// saying "exported to the library" before it has uploaded would be
|
||||
// a claim the app cannot keep if the disk is pulled.
|
||||
Placed::Queued { remote_dir, .. } => {
|
||||
let dir = if remote_dir.is_empty() {
|
||||
@@ -318,8 +318,7 @@ pub enum UploadMessage {
|
||||
/// network error would burn the whole queue against a server that is not
|
||||
/// answering, and the next pass costs nothing.
|
||||
pub fn spawn_upload(
|
||||
creds: AppCredentials,
|
||||
user_id: String,
|
||||
conn: Connection,
|
||||
root: String,
|
||||
outbox: PathBuf,
|
||||
) -> std::sync::mpsc::Receiver<UploadMessage> {
|
||||
@@ -339,7 +338,7 @@ pub fn spawn_upload(
|
||||
};
|
||||
|
||||
rt.block_on(async {
|
||||
let backend = match crate::remote::connect(&creds, &user_id) {
|
||||
let backend = match crate::remote::connect(&conn) {
|
||||
Ok(b) => b,
|
||||
Err(e) => {
|
||||
let _ = tx.send(UploadMessage::Finished {
|
||||
@@ -486,7 +485,7 @@ pub struct BatchRequest {
|
||||
/// Credentials for the account the library is open on. `None` where no
|
||||
/// library is open, which is fine for a [`Source::Rendered`] and fatal for
|
||||
/// anything that has to be fetched.
|
||||
pub creds: Option<(AppCredentials, String)>,
|
||||
pub conn: Option<Connection>,
|
||||
pub settings: ExportSettings,
|
||||
pub outbox: PathBuf,
|
||||
pub sidecar_cache: PathBuf,
|
||||
@@ -692,7 +691,7 @@ fn render_from_library(
|
||||
cache: Option<crate::library::CacheContext>,
|
||||
cancel: &Cancel,
|
||||
) -> Option<Result<RenderedItem, ItemError>> {
|
||||
let Some((creds, user_id)) = request.creds.clone() else {
|
||||
let Some(conn) = request.conn.clone() else {
|
||||
return Some(Err(ItemError::Fetch("no library is open".into())));
|
||||
};
|
||||
let Some(gpu) = request.gpu.as_ref() else {
|
||||
@@ -706,13 +705,12 @@ fn render_from_library(
|
||||
// RAW, so it costs nothing to have in hand by the time there is a session
|
||||
// to apply it to.
|
||||
let sidecar_rx = crate::library::spawn_sidecar_fetch(
|
||||
creds.clone(),
|
||||
user_id.clone(),
|
||||
conn.clone(),
|
||||
path.to_string(),
|
||||
request.sidecar_cache.clone(),
|
||||
request.offline,
|
||||
);
|
||||
let bytes_rx = crate::library::spawn_full_fetch(creds, user_id, path.to_string(), cache);
|
||||
let bytes_rx = crate::library::spawn_full_fetch(conn, path.to_string(), cache);
|
||||
|
||||
let bytes = match wait_for(&bytes_rx, cancel) {
|
||||
Waited::Got(Ok(bytes)) => bytes,
|
||||
@@ -1332,7 +1330,7 @@ mod tests {
|
||||
fn request(settings: ExportSettings, sources: Vec<Source>) -> BatchRequest {
|
||||
BatchRequest {
|
||||
sources,
|
||||
creds: None,
|
||||
conn: None,
|
||||
settings,
|
||||
outbox: std::env::temp_dir().join("dr-batch-test-outbox"),
|
||||
sidecar_cache: std::env::temp_dir().join("dr-batch-test-sidecars"),
|
||||
|
||||
Reference in New Issue
Block a user