dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Choose the export folder by walking the server, not by typing it

Browse Source 
The destination for a Nextcloud export was a text field. Nobody recalls the
exact spelling of a path three levels down, and getting it wrong does not
fail — `create_dir` makes whatever was typed, so a misremembered folder
becomes a new one at the root and the exports are somewhere nobody looks.

So it is picked the way the library root is picked, using the same
`FolderBrowser` model the launch screen drives: up, into, and "use this
folder", confirming the folder currently *shown* rather than one selected in
the list. Same rule in both places, so the phrase means one thing.

The model is shared; the worker is not. `settings_ui::spawn_folder_list` is a
near-twin of the launch screen's, because that one reaches into the
`LaunchController` for its session and reports onto the launch screen's error
line, while this one is handed credentials and writes to the settings page.
Factoring them together needs a function taking both controllers or a trait
implemented twice to abstract two call sites — more machinery than the twenty
lines it saves. What matters is shared already: navigation behaves identically
because both drive the same model.

The callbacks are wired in `lib.rs` rather than in `settings_ui::wire`,
because listing a remote folder needs credentials and the settings page holds
no session on purpose — it is reachable before a library is opened and must
not depend on one existing. With no account the picker says to sign in first,
rather than showing an empty list that reads as a server with no folders.

Details that are decisions rather than accidents: the picker opens at the
library root rather than at whatever half-typed path is in the field, which
would list nothing and look broken. The listing area is a fixed 180px, since a
folder with sixty children would otherwise push the rest of the settings page
off the bottom. "Up" is disabled at the root rather than hidden, so the row
does not jump as the user navigates. A failed listing leaves the picker open
on the folder it was showing — where the user had got to is not something to
discard over a dropped request. And the chosen folder saves immediately like
every other setting on a page that has no Save button.

The poll timer lives on the controller for the reason `LaunchController` keeps
its own there: a `slint::Timer` stops when dropped, so one local to the
function that starts it would be collected before the listing arrived.

Carries in-flight work from a parallel session — a segmentation pass in
dr-gpu, a sidecar cache, and the develop panel's continuing changes.

1020 tests pass, fmt clean. One clippy warning remains and is not mine:
`sidecar_cache::dir` is unused while that work is in progress.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-17 07:12:01 +02:00
co-authored by Claude Opus 5
parent e00c99b864
commit cb1d2be240
16 changed files with 3292 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/library.rs
+391 -96
View File
@@ -28,6 +28,8 @@ use dr_catalog::{Catalog, JobKind, Priority};
use dr_sync::{RemoteBackend, RemoteId, RemotePath};
use dr_sync_nextcloud::{AppCredentials, NextcloudBackend};
use dr_thumbs::ThumbStore;
use crate::sidecar_cache::SidecarCache;
use dr_types::FormatFilter;
/// Largest preview worth fetching whole.
@@ -344,6 +346,7 @@ pub fn sidecar_path(image_path: &str) -> String {
format!("{stem}.{}", dr_pipeline::sidecar::EXTENSION)
}
/// TRACES: FR-CAT-8 | FR-CAT-9 | FR-NC-10
/// Persist amendments to sidecars beside their images.
///
/// # Why this reads before it writes
@@ -355,76 +358,135 @@ pub fn sidecar_path(image_path: &str) -> String {
/// parsed, amended, and written back; a fetch that 404s simply means there is
/// no sidecar yet and a new one is created.
///
/// # Why the local write is the commit point
///
/// FR-CAT-9 requires that edits made offline *queue and apply when the source
/// returns*. So every amendment is written to the local cache first and the
/// upload is best-effort: an entry stays marked pending until the server has
/// actually taken it, and [`spawn_outbox_drain`] retries the marked ones later.
///
/// This is what makes `offline` a parameter rather than a reason to skip. It
/// was one: a cull or a paste made with no connection used to be dropped
/// entirely, which for a pasted edit meant it survived nowhere at all — the
/// catalog holds no parameters. Now the two cases differ only in whether the
/// upload is attempted.
///
/// # Why failure here is logged rather than surfaced
///
/// The catalog write has already succeeded by the time this runs, so the star
/// is on screen and will survive a restart. A network failure costs
/// *durability across a catalog rebuild*, not the judgement itself, and
/// interrupting a cull with an error dialog per frame would be far worse than
/// the risk. The count of failures is reported once, at the end.
/// The write has already succeeded locally by the time the network is touched,
/// so nothing is lost by a failure and there is nothing for the user to do
/// about it. Interrupting a cull with an error dialog per frame would be far
/// worse than the risk. The counts are reported once, at the end.
pub fn spawn_sidecar_writes(
creds: AppCredentials,
user_id: String,
writes: Vec<SidecarWrite>,
cache_dir: PathBuf,
offline: bool,
) -> Receiver<SidecarMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
failed: writes.len(),
last_error: Some(e.to_string()),
});
return;
let cache = SidecarCache::open(cache_dir);
// The runtime and the backend are only needed to *upload*. Offline,
// neither is built — and a failure to build either is not a failure to
// record the edit, it just means every write is queued instead.
let rt = if offline {
None
} else {
match crate::net_runtime::build() {
Ok(rt) => Some(rt),
Err(e) => {
log::debug!("no runtime for sidecar upload ({e}); queueing");
None
}
}
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
failed: writes.len(),
last_error: Some(e.to_string()),
});
return;
}
};
let (mut written, mut failed) = (0usize, 0usize);
let mut last_error = None;
let run = |backend: Option<&NextcloudBackend>| {
let mut report = SidecarReport::default();
for w in &writes {
match write_one_sidecar(&backend, w).await {
Ok(()) => written += 1,
match write_one_sidecar(backend, &cache, w) {
Ok(Outcome::Uploaded) => report.written += 1,
Ok(Outcome::Queued) => report.queued += 1,
Err(e) => {
log::debug!("sidecar for {}: {e}", w.image_path);
last_error = Some(e);
failed += 1;
report.last_error = Some(e);
report.failed += 1;
}
}
}
report
};
let _ = tx.send(SidecarMessage::Finished {
written,
failed,
last_error,
});
let report = match rt {
None => run(None),
Some(rt) => rt.block_on(async {
match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => {
let mut report = SidecarReport::default();
for w in &writes {
match write_one_sidecar_online(&b, &cache, w).await {
Ok(Outcome::Uploaded) => report.written += 1,
Ok(Outcome::Queued) => report.queued += 1,
Err(e) => {
log::debug!("sidecar for {}: {e}", w.image_path);
report.last_error = Some(e);
report.failed += 1;
}
}
}
report
}
// No backend: the edits are still recorded locally and
// will go up with the next drain.
Err(e) => {
log::debug!("no backend for sidecar upload ({e}); queueing");
run(None)
}
}
}),
};
let _ = tx.send(SidecarMessage::Finished {
written: report.written,
queued: report.queued,
failed: report.failed,
last_error: report.last_error,
});
});
rx
}
/// What one write ended up doing.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Outcome {
/// Recorded locally and accepted by the server.
Uploaded,
/// Recorded locally, still in the outbox.
Queued,
}
/// Running totals for a batch, so the loop bodies stay readable.
#[derive(Debug, Default)]
struct SidecarReport {
written: usize,
queued: usize,
failed: usize,
last_error: Option<String>,
}
/// The outcome of a batch of sidecar writes.
#[derive(Debug)]
pub enum SidecarMessage {
Finished {
written: usize,
/// Recorded locally but not yet on the server — offline, or an upload
/// that failed. These are retried by [`spawn_outbox_drain`], so this
/// is a count of *deferred* work rather than of losses.
queued: usize,
failed: usize,
/// Reported once rather than per file: a network that is down fails
/// every write with the same message, and forty identical lines in the
@@ -433,44 +495,14 @@ pub enum SidecarMessage {
},
}
/// Read-modify-write one sidecar.
async fn write_one_sidecar(backend: &NextcloudBackend, w: &SidecarWrite) -> Result<(), String> {
let path = RemotePath::new(sidecar_path(&w.image_path));
let id = RemoteId::Path(path.clone());
// An existing sidecar may hold an edit. Absent is the normal case on a
// library that has never been edited, and is not an error.
let existing = backend.get(&id, None).await.ok();
let mut sidecar = existing
.as_deref()
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.and_then(|text| match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
// A corrupt sidecar is *not* overwritten silently: that would
// destroy an edit this build merely failed to understand. The
// judgement stays in the catalog and the file is left alone.
Err(e) => {
log::warn!(
"sidecar at {} is unreadable ({e}); not overwriting",
path.as_str()
);
None
}
})
.unwrap_or_default();
// A parse failure above means we must not touch the file at all.
if existing.is_some() && sidecar.versions.is_empty() && existing.as_deref() != Some(b"") {
// Distinguish "empty file" from "unparseable": only the latter is a
// refusal, and it already logged.
let text = existing
.as_deref()
.map(|b| String::from_utf8_lossy(b).into_owned())
.unwrap_or_default();
if dr_pipeline::Sidecar::parse(&text).is_err() {
return Err("existing sidecar is unreadable".into());
}
}
/// Apply an amendment to a document, returning the new one.
///
/// Split out from both write paths so that online and offline produce
/// *identical* documents: the only thing that differs between them is which
/// base was read and whether an upload follows. A second copy of this for the
/// offline case is how the two would come to disagree about what a paste means.
fn amend(base: dr_pipeline::Sidecar, w: &SidecarWrite) -> dr_pipeline::Sidecar {
let mut sidecar = base;
// Amend the version this write belongs to, creating it if the file did
// not have one. The uuid comes from the catalog, so the same photograph
@@ -509,12 +541,234 @@ async fn write_one_sidecar(backend: &NextcloudBackend, w: &SidecarWrite) -> Resu
version.modified = now_secs();
sidecar.put(version);
sidecar
}
/// TRACES: FR-CAT-9
/// Record an amendment with no server to send it to.
///
/// The base is whatever the cache holds, which is either what the server last
/// had or what earlier offline writes have already built on top of it. Either
/// way the result is queued, and the drain reconciles it with the server's own
/// copy when the connection returns — that reconciliation is a *merge*
/// (FR-NC-9), not an overwrite, so building on a possibly-stale base here does
/// not cost another device's work.
fn write_one_sidecar(
_backend: Option<&NextcloudBackend>,
cache: &SidecarCache,
w: &SidecarWrite,
) -> Result<Outcome, String> {
let path = sidecar_path(&w.image_path);
let base = cache.load(&path).unwrap_or_default();
cache.store(&path, &amend(base, w), true)?;
Ok(Outcome::Queued)
}
/// TRACES: FR-CAT-8 | FR-CAT-9
/// Read-modify-write one sidecar, with a server to read from and send to.
async fn write_one_sidecar_online(
backend: &NextcloudBackend,
cache: &SidecarCache,
w: &SidecarWrite,
) -> Result<Outcome, String> {
let path_str = sidecar_path(&w.image_path);
let path = RemotePath::new(path_str.clone());
let id = RemoteId::Path(path.clone());
// An existing sidecar may hold an edit. Absent is the normal case on a
// library that has never been edited, and is not an error.
let existing = backend.get(&id, None).await.ok();
// A corrupt sidecar is *not* overwritten: that would destroy an edit this
// build merely failed to understand. Refused before anything is written,
// locally or remotely, so the cache cannot end up holding a document that
// silently discarded the file's real contents.
if let Some(bytes) = existing.as_deref() {
if !bytes.is_empty() {
let text = String::from_utf8_lossy(bytes);
if dr_pipeline::Sidecar::parse(&text).is_err() {
return Err(format!("sidecar at {path_str} is unreadable"));
}
}
}
let base = existing
.as_deref()
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.and_then(|text| dr_pipeline::Sidecar::parse(&text).ok())
// No sidecar on the server. The cache may still hold queued offline
// work for this image, and taking `default()` here would drop it.
.or_else(|| cache.load(&path_str))
.unwrap_or_default();
let sidecar = amend(base, w);
// Locally first: this is the commit point, and an upload that fails after
// it leaves the edit queued rather than lost.
cache.store(&path_str, &sidecar, true)?;
backend
.put(&path, sidecar.to_text().into_bytes(), None)
.await
.map(|_| ())
.map_err(|e| e.to_string())
.map_err(|e| e.to_string())?;
// Accepted by the server, so it leaves the outbox. The document stays
// cached, which is what lets the next offline open still show the edit.
cache.store(&path_str, &sidecar, false)?;
Ok(Outcome::Uploaded)
}
/// TRACES: FR-CAT-9 | FR-NC-9 | FR-NC-10
/// Upload everything the outbox is still holding.
///
/// # Why this merges rather than uploads
///
/// A queued edit was built on whatever this device last saw. While it sat in
/// the outbox another device may have edited the same photograph, and simply
/// PUTting the local document would discard that work — the precise failure
/// FR-NC-9's node-level merge exists to prevent. So each entry is reconciled
/// against the server's current copy before it goes up, and disjoint edits
/// (a crop made here, an exposure change made there) both survive.
///
/// # Why an entry stays queued on failure
///
/// The marker is cleared only after the server has taken the bytes. A drain
/// interrupted halfway leaves the rest of the outbox exactly as it was, so
/// nothing depends on this running to completion.
pub fn spawn_outbox_drain(
creds: AppCredentials,
user_id: String,
cache_dir: PathBuf,
) -> Receiver<SidecarMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let cache = SidecarCache::open(cache_dir);
let queued = cache.pending();
if queued.is_empty() {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: 0,
failed: 0,
last_error: None,
});
return;
}
log::info!("draining {} queued sidecar(s)", queued.len());
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: queued.len(),
failed: 0,
last_error: Some(e.to_string()),
});
return;
}
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: queued.len(),
failed: 0,
last_error: Some(e.to_string()),
});
return;
}
};
let mut report = SidecarReport::default();
for path_str in &queued {
match drain_one(&backend, &cache, path_str).await {
Ok(()) => report.written += 1,
Err(e) => {
log::debug!("draining {path_str}: {e}");
report.last_error = Some(e);
report.failed += 1;
// Still queued — the marker was never cleared.
report.queued += 1;
}
}
}
let _ = tx.send(SidecarMessage::Finished {
written: report.written,
queued: report.queued,
failed: report.failed,
last_error: report.last_error,
});
});
});
rx
}
/// Reconcile one queued sidecar with the server and upload it.
async fn drain_one(
backend: &NextcloudBackend,
cache: &SidecarCache,
path_str: &str,
) -> Result<(), String> {
let Some(mut local) = cache.load(path_str) else {
// The document went while the drain was running. Nothing to send.
return Ok(());
};
let path = RemotePath::new(path_str.to_string());
let id = RemoteId::Path(path.clone());
let remote = backend.get(&id, None).await.ok();
if let Some(bytes) = remote.as_deref() {
if !bytes.is_empty() {
let text = String::from_utf8_lossy(bytes);
match dr_pipeline::Sidecar::parse(&text) {
Ok(remote) => merge_into(&mut local, &remote),
// Unreadable on the server. Uploading over it would destroy an
// edit this build failed to understand, so the entry stays
// queued rather than being resolved destructively.
Err(e) => return Err(format!("remote sidecar is unreadable ({e})")),
}
}
}
backend
.put(&path, local.to_text().into_bytes(), None)
.await
.map_err(|e| e.to_string())?;
cache.store(path_str, &local, false)
}
/// TRACES: FR-NC-9
/// Merge the server's copy into ours, version by version.
///
/// No common ancestor is available — the outbox stores the result, not the
/// base it was built from — so the merge runs with `None`, which treats every
/// key either side holds as changed. Disjoint keys therefore still both
/// survive, and a key both sides set resolves by revision exactly as it would
/// with a base. What is lost without one is the ability to see a *deletion*:
/// a parameter reset to default on the other device reads as absent rather
/// than as removed, so our value stands. That is the same direction of caution
/// the judgement merge takes — an edit is preserved rather than erased.
fn merge_into(local: &mut dr_pipeline::Sidecar, remote: &dr_pipeline::Sidecar) {
for (uuid, their_version) in &remote.versions {
match local.versions.get(uuid).cloned() {
Some(mut ours) => {
ours.merge(their_version, None);
local.put(ours);
}
// A version only the server has — another device's virtual copy
// (FR-CAT-12). Keeping it is what stops one device's upload from
// deleting another's work.
None => local.put(their_version.clone()),
}
}
}
/// Where the catalog for an account lives.
@@ -1100,44 +1354,85 @@ pub fn spawn_sidecar_fetch(
creds: AppCredentials,
user_id: String,
image_path: String,
cache_dir: PathBuf,
offline: bool,
) -> Receiver<Option<dr_pipeline::Sidecar>> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = match crate::net_runtime::build() {
Ok(e) => e,
Err(e) => {
log::debug!("sidecar fetch runtime: {e}");
let _ = tx.send(None);
return;
let cache = SidecarCache::open(cache_dir);
let path_str = sidecar_path(&image_path);
// TRACES: FR-CAT-9 | FR-NC-10
// The cache wins outright when it is holding work the server has not
// seen. Fetching in that state would answer with a document *older*
// than the edit sitting in the outbox, and opening the photograph
// would silently show it without the change the user just made —
// which the next save would then write back over the top of.
if cache.is_pending(&path_str) {
log::debug!("{path_str} has queued local edits; opening from the cache");
let _ = tx.send(cache.load(&path_str));
return;
}
// Offline there is nothing to ask, and the cache is the whole answer.
let rt = if offline {
None
} else {
match crate::net_runtime::build() {
Ok(e) => Some(e),
Err(e) => {
log::debug!("sidecar fetch runtime: {e}");
None
}
}
};
let Some(rt) = rt else {
let _ = tx.send(cache.load(&path_str));
return;
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
log::debug!("sidecar fetch backend: {e}");
let _ = tx.send(None);
let _ = tx.send(cache.load(&path_str));
return;
}
};
let path = RemotePath::new(sidecar_path(&image_path));
let path = RemotePath::new(path_str.clone());
let id = RemoteId::Path(path.clone());
// A 404 is the normal case on a library that has never been
// edited, so this is `ok()` rather than an error path.
let parsed = backend.get(&id, None).await.ok().and_then(|bytes| {
let text = String::from_utf8_lossy(&bytes).into_owned();
match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
Err(e) => {
log::warn!("sidecar at {} is unreadable ({e})", path.as_str());
None
}
let Ok(bytes) = backend.get(&id, None).await else {
// Unreachable, or no such file. The cache cannot tell those
// apart and does not need to: either way it holds the best
// answer this device has.
let _ = tx.send(cache.load(&path_str));
return;
};
let text = String::from_utf8_lossy(&bytes).into_owned();
let parsed = match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
Err(e) => {
log::warn!("sidecar at {} is unreadable ({e})", path.as_str());
None
}
});
};
// Populate the cache from what the server said, so the *next*
// open of this photograph works with no connection. Clean rather
// than pending: this content came from the server, so there is
// nothing to send back.
if let Some(sidecar) = parsed.as_ref() {
if let Err(e) = cache.store(&path_str, sidecar, false) {
log::debug!("caching {path_str}: {e}");
}
}
let _ = tx.send(parsed);
});