dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Make storage pluggable, and prove it with a folder backend

Browse Source 
`RemoteBackend` existed from the first release and bought nothing it was
designed for. Seven files in `dr-ui` constructed a `NextcloudBackend`
directly, an account *was* a server URL beside a DAV user id, the local
cache directory was named after a hostname, and the launch screen knew
that signing in meant a browser handshake. The trait was real; the seam
was documentation.

A trait over operations is only a quarter of it. Pluggable storage needs
four things, and this adds the other three:

- **Capabilities** — already there, and the reason the engine can drive
  two backends at the speed each actually runs at.
- **Configuration** — `dr_sync::Account`: where a library lives, in
  whatever form its connector addresses, with no server in it. Loads
  every existing config unchanged (`backend` defaults to `nextcloud`,
  `endpoint` is stored under its historical `server` key), and
  `Account::namespace()` reproduces the old catalog directory byte for
  byte, because changing it would abandon a catalog, its thumbnail
  shards, and the sidecars holding unsynced offline work.
- **Registration** — `BackendProvider` and `BackendRegistry`.
  `ui/dr-ui/src/remote.rs` is now the only file above `dr-sync` that
  names a connector.

`Connection` (an account plus an optional `Secret`) replaces the
credentials-and-user-id pair that was threaded through fifteen
signatures in an order that could be swapped. `Secret`'s inner string is
reachable only through `expose()` and its `Debug` prints `Secret(***)`,
so the indirect leak — a `{:?}` on anything holding one — no longer
compiles into a leak.

Nextcloud is unchanged and keeps every peculiarity: propagating ETags,
chunked upload v2, `oc:fileid`, the `oc:permissions` probe on a refused
PUT, the 423 retry classification, Login Flow v2. Those are what the
capability model exists to serve, not something to hide.

`dr-sync-folder` is the second connector: a local disk, a network mount,
an external drive, or a folder a Nextcloud client already syncs. No
account, no credential — the route that works where no secrets daemon
does. It declares `LocalEtags` rather than claiming propagation a POSIX
directory cannot provide, which costs nothing because 50k `stat` calls
are not 50k PROPFINDs. Identity is a path hash, not an inode: an inode
survives a rename but differs between devices and is reused after a
delete, so two machines would disagree about which photograph a
thumbnail belonged to. Re-deriving a thumbnail is a cost; showing the
wrong one is a bug.

docs/storage.md is the contract — the traits, the four steps to add a
backend, and what each connector declares. ARCH §8.0 and §8.4a, and
FR-NC-13, say why.
This commit is contained in:
dtourolle
2026-08-29 09:57:52 +02:00
parent 1b8b7998a2
commit f12aece07e
40 changed files with 3617 additions and 960 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/lib.rs
+37 -49
View File
@@ -406,10 +406,10 @@ fn batch_request(
export::BatchRequest {
sources,
creds: library.credentials(),
conn: library.credentials(),
settings: stored.export,
outbox: match library.session() {
Some((_, s)) => export::outbox_dir(&s.server, &s.user_id),
Some(c) => export::outbox_dir(&c.account),
// No account, so no outbox — a device export still works, and a
// remote one is refused by `place` rather than here, so the message
// names the setting rather than the plumbing.
@@ -434,15 +434,16 @@ fn drain_outbox(library: &Rc<library_ui::LibraryController>) {
if library.is_offline() {
return;
}
let Some((creds, session)) = library.session() else {
let Some(conn) = library.session() else {
return;
};
let outbox = export::outbox_dir(&session.server, &session.user_id);
let outbox = export::outbox_dir(&conn.account);
if export::pending_count(&outbox) == 0 {
return;
}
let rx = export::spawn_upload(creds, session.user_id.clone(), session.root.clone(), outbox);
let root = conn.account.root.clone();
let rx = export::spawn_upload(conn, root, outbox);
std::thread::spawn(move || {
while let Ok(msg) = rx.recv() {
match msg {
@@ -472,7 +473,9 @@ fn refresh_export_label(window: &AppWindow, settings: &Rc<settings_ui::SettingsC
let remote = stored.export.target == dr_types::ExportTarget::Remote;
window.set_export_label(
if remote {
"Export to Nextcloud"
// Not the connector's name: this is a Nextcloud account for some
// libraries and a folder on a mount for others.
"Export to the library"
} else {
"Export"
}
@@ -1036,13 +1039,10 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
library.catalog(),
activity.clone(),
move || {
let (_, session) = lib_store.session()?;
dr_thumbs::ThumbStore::open(&library::thumbs_dir(
&session.server,
&session.user_id,
))
.ok()
.map(std::rc::Rc::new)
let conn = lib_store.session()?;
dr_thumbs::ThumbStore::open(&library::thumbs_dir(&conn.account))
.ok()
.map(std::rc::Rc::new)
},
// The weights are not shipped and are not a build input
// (docs/faces.md §2): the user puts them beside the catalog,
@@ -1050,24 +1050,21 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
{
let lib = library.clone();
move || {
let (_, session) = lib.session()?;
library::face_models(&session.server, &session.user_id)
let conn = lib.session()?;
library::face_models(&conn.account)
}
},
// The sweep opens its own connection on its own thread, so it
// takes paths rather than the handles this screen holds — and
// credentials, because it fetches the pixels it indexes rather
// a connection, because it fetches the pixels it indexes rather
// than reading whatever the grid happened to leave behind.
{
let lib = library.clone();
move || {
let (creds, session) = lib.session()?;
Some((
creds,
session.user_id.clone(),
library::catalog_path(&session.server, &session.user_id),
library::thumbs_dir(&session.server, &session.user_id),
))
let conn = lib.session()?;
let catalog = library::catalog_path(&conn.account);
let thumbs = library::thumbs_dir(&conn.account);
Some((conn, catalog, thumbs))
}
},
);
@@ -1100,7 +1097,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// Before anything opens a store: an upgrade must not
// abandon a catalog, its thumbnails, or the offline
// ratings and edits waiting beside them.
library::migrate_legacy_cache_data(&session.server, &session.user_id);
library::migrate_legacy_cache_data(&session);
library_ui::open(
&window,
library.clone(),
@@ -1147,23 +1144,22 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
&window,
import,
move || {
let (creds, session) = library_for_context.session()?;
let conn = library_for_context.session()?;
Some(import_ui::Context {
catalog: library::catalog_path(&session.server, &session.user_id),
library_label: session.root.clone(),
catalog: library::catalog_path(&conn.account),
library_label: conn.account.root.clone(),
// The same formats the scan looks for. An import that took
// types the library then ignores would copy files off the
// card that never appear in the grid.
filter: session.format_filter(),
filter: conn.account.format_filter(),
upload: Some(import::Upload {
credentials: creds,
user_id: session.user_id.clone(),
library: session.root.clone(),
library: conn.account.root.clone(),
// The same shard store the grid reads and the sync
// pushes, so a thumbnail made during an import is the
// one every other client gets.
thumbs: library::thumbs_dir(&session.server, &session.user_id),
staging: import::staging_dir(&session.server, &session.user_id),
thumbs: library::thumbs_dir(&conn.account),
staging: import::staging_dir(&conn.account),
conn,
}),
})
},
@@ -1218,14 +1214,8 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
library: &Rc<library_ui::LibraryController>,
path: String| {
match library.session() {
Some((creds, session)) => {
settings_ui::spawn_folder_list(
weak.clone(),
ctl.clone(),
creds,
session.user_id.clone(),
path,
);
Some(conn) => {
settings_ui::spawn_folder_list(weak.clone(), ctl.clone(), conn, path);
}
None => {
// No account, so nothing to browse. Said plainly rather
@@ -1364,14 +1354,13 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
let lib = library.clone();
let catalog = library.catalog();
move |w: &AppWindow| {
let store = lib.session().and_then(|(_, s)| {
dr_thumbs::ThumbStore::open(&library::thumbs_dir(&s.server, &s.user_id))
.ok()
let store = lib.session().and_then(|c| {
dr_thumbs::ThumbStore::open(&library::thumbs_dir(&c.account)).ok()
});
identity_ui::refresh_coverage(w, &catalog, store.as_ref());
w.set_identity_model_missing(
lib.session()
.and_then(|(_, s)| library::face_models(&s.server, &s.user_id))
.and_then(|c| library::face_models(&c.account))
.is_none(),
);
}
@@ -1780,7 +1769,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
w.set_index(0);
w.set_total(1);
let Some((creds, user_id)) = library.credentials() else {
let Some(conn) = library.credentials() else {
w.set_load_error("no library session".into());
return;
};
@@ -1809,8 +1798,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// photograph is, instead of the image appearing at its defaults
// and visibly changing a moment later.
let sidecar_rx = library::spawn_sidecar_fetch(
creds.clone(),
user_id.clone(),
conn.clone(),
path.clone(),
library.sidecar_cache_dir().unwrap_or_default(),
library.is_offline(),
@@ -1829,7 +1817,7 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
log::info!("fetching {path} for develop");
w.set_load_error("Downloading…".into());
let rx = library::spawn_full_fetch(creds, user_id, path.clone(), cache);
let rx = library::spawn_full_fetch(conn, path.clone(), cache);
// The one transfer the user is actively waiting on. It gets a row
// like any other, so a download that is still running after they