dtourolle/DarkRoom
1
0
Fork 0
Code Issues 39 Pull Requests Actions Packages Projects Releases 20 Wiki Activity

Choose the export folder by walking the server, not by typing it

Browse Source 
The destination for a Nextcloud export was a text field. Nobody recalls the
exact spelling of a path three levels down, and getting it wrong does not
fail — `create_dir` makes whatever was typed, so a misremembered folder
becomes a new one at the root and the exports are somewhere nobody looks.

So it is picked the way the library root is picked, using the same
`FolderBrowser` model the launch screen drives: up, into, and "use this
folder", confirming the folder currently *shown* rather than one selected in
the list. Same rule in both places, so the phrase means one thing.

The model is shared; the worker is not. `settings_ui::spawn_folder_list` is a
near-twin of the launch screen's, because that one reaches into the
`LaunchController` for its session and reports onto the launch screen's error
line, while this one is handed credentials and writes to the settings page.
Factoring them together needs a function taking both controllers or a trait
implemented twice to abstract two call sites — more machinery than the twenty
lines it saves. What matters is shared already: navigation behaves identically
because both drive the same model.

The callbacks are wired in `lib.rs` rather than in `settings_ui::wire`,
because listing a remote folder needs credentials and the settings page holds
no session on purpose — it is reachable before a library is opened and must
not depend on one existing. With no account the picker says to sign in first,
rather than showing an empty list that reads as a server with no folders.

Details that are decisions rather than accidents: the picker opens at the
library root rather than at whatever half-typed path is in the field, which
would list nothing and look broken. The listing area is a fixed 180px, since a
folder with sixty children would otherwise push the rest of the settings page
off the bottom. "Up" is disabled at the root rather than hidden, so the row
does not jump as the user navigates. A failed listing leaves the picker open
on the folder it was showing — where the user had got to is not something to
discard over a dropped request. And the chosen folder saves immediately like
every other setting on a page that has no Save button.

The poll timer lives on the controller for the reason `LaunchController` keeps
its own there: a `slint::Timer` stops when dropped, so one local to the
function that starts it would be collected before the listing arrived.

Carries in-flight work from a parallel session — a segmentation pass in
dr-gpu, a sidecar cache, and the develop panel's continuing changes.

1020 tests pass, fmt clean. One clippy warning remains and is not mine:
`sidecar_cache::dir` is unused while that work is in progress.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
dtourolle
2026-08-17 07:12:01 +02:00
co-authored by Claude Opus 5
parent e00c99b864
commit cb1d2be240
16 changed files with 3292 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ui/dr-ui/src/develop.rs
+33 -1
View File
@@ -89,6 +89,32 @@ impl DevelopSession {
}
}
/// The empty choices model, shared by every row that is not an enum.
///
/// **One identity, deliberately reused.** `ModelRc` compares by *identity*, not
/// by contents, and `sync_rows` decides which controls to invalidate by
/// comparing each fresh row against the one on screen. Handing out a brand-new
/// empty model per row per call therefore makes every row differ from itself on
/// every parameter event, so the panel rewrites all of them.
///
/// That is not merely wasteful, it breaks dragging. An operation with several
/// parameters renders them through a repeater whose model is read off the
/// group's head row; rewriting that row re-evaluates the repeater, which
/// rebuilds its items and destroys the `TouchArea` holding the gesture. The
/// slider takes the press, jumps once, and then goes dead under the finger —
/// and only for multi-parameter operations, since a lone parameter has no inner
/// repeater to rebuild.
///
/// `sync_rows` already carries the identical warning about a curve's `points`.
/// This is the same hazard arriving through a second field.
fn no_choices() -> slint::ModelRc<slint::SharedString> {
thread_local! {
static EMPTY: slint::ModelRc<slint::SharedString> =
slint::ModelRc::new(slint::VecModel::from(Vec::<slint::SharedString>::new()));
}
EMPTY.with(Clone::clone)
}
/// Whether this frontend has an implementation of `widget` **anywhere**.
///
/// "Anywhere" is doing real work: a widget may be drawn in the panel, as the
@@ -269,7 +295,13 @@ pub(crate) fn rows_from(caps: &[OpCapability]) -> Vec<ParamRow> {
unit: unit.into(),
// Only curve rows carry points.
points: slint::ModelRc::new(slint::VecModel::from(Vec::<f32>::new())),
choices: slint::ModelRc::new(slint::VecModel::from(choices)),
// The shared empty model unless this row really has choices —
// see `no_choices` for why the identity matters.
choices: if choices.is_empty() {
no_choices()
} else {
slint::ModelRc::new(slint::VecModel::from(choices))
},
});
}
}
ui/dr-ui/src/lib.rs
+129 -2
View File
@@ -28,6 +28,7 @@ mod net_runtime;
mod presets;
mod settings_store;
mod settings_ui;
mod sidecar_cache;
mod trash;
use std::cell::RefCell;
@@ -768,6 +769,127 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
library.set_keep_opened_originals(stored.cache.keep_opened_originals);
}
// --- the export folder picker ----------------------------------------
//
// Wired here rather than inside `settings_ui::wire` because listing a
// remote folder needs credentials, and the settings page deliberately
// holds no session — it is reachable before a library is opened and must
// not depend on one existing.
{
let weak = window.as_weak();
let ctl = settings.clone();
let library = library.clone();
// Every entry point needs the same three things, so they are fetched
// once here rather than at four call sites.
let start = move |ctl: &Rc<settings_ui::SettingsController>,
weak: &slint::Weak<AppWindow>,
library: &Rc<library_ui::LibraryController>,
path: String| {
match library.session() {
Some((creds, session)) => {
settings_ui::spawn_folder_list(
weak.clone(),
ctl.clone(),
creds,
session.user_id.clone(),
path,
);
}
None => {
// No account, so nothing to browse. Said plainly rather
// than left as an empty list, which would read as a server
// with no folders on it.
ctl.set_error("Sign in to a library before choosing a folder on it.");
ctl.browser.replace(None);
}
}
};
{
let (weak, ctl, library, start) =
(weak.clone(), ctl.clone(), library.clone(), start);
window.on_settings_browse_open_picker(move || {
let Some(w) = weak.upgrade() else { return };
// Opens on the library root rather than on whatever the
// destination field happens to contain: a half-typed path
// would list nothing and look like a broken picker.
ctl.browser.replace(Some(launch::FolderBrowser {
path: String::new(),
entries: Vec::new(),
loading: true,
}));
start(&ctl, &weak, &library, String::new());
settings_ui::render(&w, &ctl);
});
}
{
let (weak, ctl, library, start) =
(weak.clone(), ctl.clone(), library.clone(), start);
window.on_settings_browse_into(move |name| {
let Some(w) = weak.upgrade() else { return };
let path = {
let mut browser = ctl.browser.borrow_mut();
let Some(b) = browser.as_mut() else { return };
let path = b.child_path(&name);
b.path = path.clone();
b.entries.clear();
b.loading = true;
path
};
start(&ctl, &weak, &library, path);
settings_ui::render(&w, &ctl);
});
}
{
let (weak, ctl, library, start) =
(weak.clone(), ctl.clone(), library.clone(), start);
window.on_settings_browse_up(move || {
let Some(w) = weak.upgrade() else { return };
let path = {
let mut browser = ctl.browser.borrow_mut();
let Some(b) = browser.as_mut() else { return };
let Some(path) = b.parent_path() else { return };
b.path = path.clone();
b.entries.clear();
b.loading = true;
path
};
start(&ctl, &weak, &library, path);
settings_ui::render(&w, &ctl);
});
}
{
let (weak, ctl) = (weak.clone(), ctl.clone());
window.on_settings_browse_confirm(move || {
let Some(w) = weak.upgrade() else { return };
// The folder being *shown* is the one chosen, matching the
// library picker — so "use this one" means the same thing in
// both places rather than depending on a selection the list
// does not have.
let chosen = ctl.browser.borrow().as_ref().map(|b| b.path.clone());
if let Some(path) = chosen {
ctl.set_destination(path);
}
ctl.browser.replace(None);
settings_ui::render(&w, &ctl);
refresh_export_label(&w, &ctl);
});
}
{
let (weak, ctl) = (weak.clone(), ctl.clone());
window.on_settings_browse_cancel(move || {
let Some(w) = weak.upgrade() else { return };
ctl.browser.replace(None);
settings_ui::render(&w, &ctl);
});
}
}
let lib = library.clone();
let ctl = settings.clone();
let weak = window.as_weak();
@@ -1134,8 +1256,13 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
// it to — and starting it here means the edit is ready when the
// photograph is, instead of the image appearing at its defaults
// and visibly changing a moment later.
let sidecar_rx =
library::spawn_sidecar_fetch(creds.clone(), user_id.clone(), path.clone());
let sidecar_rx = library::spawn_sidecar_fetch(
creds.clone(),
user_id.clone(),
path.clone(),
library.sidecar_cache_dir().unwrap_or_default(),
library.is_offline(),
);
// TRACES: FR-NC-6a
// The cache is consulted first, so a second open of the same
ui/dr-ui/src/library.rs
+391 -96
View File
@@ -28,6 +28,8 @@ use dr_catalog::{Catalog, JobKind, Priority};
use dr_sync::{RemoteBackend, RemoteId, RemotePath};
use dr_sync_nextcloud::{AppCredentials, NextcloudBackend};
use dr_thumbs::ThumbStore;
use crate::sidecar_cache::SidecarCache;
use dr_types::FormatFilter;
/// Largest preview worth fetching whole.
@@ -344,6 +346,7 @@ pub fn sidecar_path(image_path: &str) -> String {
format!("{stem}.{}", dr_pipeline::sidecar::EXTENSION)
}
/// TRACES: FR-CAT-8 | FR-CAT-9 | FR-NC-10
/// Persist amendments to sidecars beside their images.
///
/// # Why this reads before it writes
@@ -355,76 +358,135 @@ pub fn sidecar_path(image_path: &str) -> String {
/// parsed, amended, and written back; a fetch that 404s simply means there is
/// no sidecar yet and a new one is created.
///
/// # Why the local write is the commit point
///
/// FR-CAT-9 requires that edits made offline *queue and apply when the source
/// returns*. So every amendment is written to the local cache first and the
/// upload is best-effort: an entry stays marked pending until the server has
/// actually taken it, and [`spawn_outbox_drain`] retries the marked ones later.
///
/// This is what makes `offline` a parameter rather than a reason to skip. It
/// was one: a cull or a paste made with no connection used to be dropped
/// entirely, which for a pasted edit meant it survived nowhere at all — the
/// catalog holds no parameters. Now the two cases differ only in whether the
/// upload is attempted.
///
/// # Why failure here is logged rather than surfaced
///
/// The catalog write has already succeeded by the time this runs, so the star
/// is on screen and will survive a restart. A network failure costs
/// *durability across a catalog rebuild*, not the judgement itself, and
/// interrupting a cull with an error dialog per frame would be far worse than
/// the risk. The count of failures is reported once, at the end.
/// The write has already succeeded locally by the time the network is touched,
/// so nothing is lost by a failure and there is nothing for the user to do
/// about it. Interrupting a cull with an error dialog per frame would be far
/// worse than the risk. The counts are reported once, at the end.
pub fn spawn_sidecar_writes(
creds: AppCredentials,
user_id: String,
writes: Vec<SidecarWrite>,
cache_dir: PathBuf,
offline: bool,
) -> Receiver<SidecarMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
failed: writes.len(),
last_error: Some(e.to_string()),
});
return;
let cache = SidecarCache::open(cache_dir);
// The runtime and the backend are only needed to *upload*. Offline,
// neither is built — and a failure to build either is not a failure to
// record the edit, it just means every write is queued instead.
let rt = if offline {
None
} else {
match crate::net_runtime::build() {
Ok(rt) => Some(rt),
Err(e) => {
log::debug!("no runtime for sidecar upload ({e}); queueing");
None
}
}
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
failed: writes.len(),
last_error: Some(e.to_string()),
});
return;
}
};
let (mut written, mut failed) = (0usize, 0usize);
let mut last_error = None;
let run = |backend: Option<&NextcloudBackend>| {
let mut report = SidecarReport::default();
for w in &writes {
match write_one_sidecar(&backend, w).await {
Ok(()) => written += 1,
match write_one_sidecar(backend, &cache, w) {
Ok(Outcome::Uploaded) => report.written += 1,
Ok(Outcome::Queued) => report.queued += 1,
Err(e) => {
log::debug!("sidecar for {}: {e}", w.image_path);
last_error = Some(e);
failed += 1;
report.last_error = Some(e);
report.failed += 1;
}
}
}
report
};
let _ = tx.send(SidecarMessage::Finished {
written,
failed,
last_error,
});
let report = match rt {
None => run(None),
Some(rt) => rt.block_on(async {
match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => {
let mut report = SidecarReport::default();
for w in &writes {
match write_one_sidecar_online(&b, &cache, w).await {
Ok(Outcome::Uploaded) => report.written += 1,
Ok(Outcome::Queued) => report.queued += 1,
Err(e) => {
log::debug!("sidecar for {}: {e}", w.image_path);
report.last_error = Some(e);
report.failed += 1;
}
}
}
report
}
// No backend: the edits are still recorded locally and
// will go up with the next drain.
Err(e) => {
log::debug!("no backend for sidecar upload ({e}); queueing");
run(None)
}
}
}),
};
let _ = tx.send(SidecarMessage::Finished {
written: report.written,
queued: report.queued,
failed: report.failed,
last_error: report.last_error,
});
});
rx
}
/// What one write ended up doing.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum Outcome {
/// Recorded locally and accepted by the server.
Uploaded,
/// Recorded locally, still in the outbox.
Queued,
}
/// Running totals for a batch, so the loop bodies stay readable.
#[derive(Debug, Default)]
struct SidecarReport {
written: usize,
queued: usize,
failed: usize,
last_error: Option<String>,
}
/// The outcome of a batch of sidecar writes.
#[derive(Debug)]
pub enum SidecarMessage {
Finished {
written: usize,
/// Recorded locally but not yet on the server — offline, or an upload
/// that failed. These are retried by [`spawn_outbox_drain`], so this
/// is a count of *deferred* work rather than of losses.
queued: usize,
failed: usize,
/// Reported once rather than per file: a network that is down fails
/// every write with the same message, and forty identical lines in the
@@ -433,44 +495,14 @@ pub enum SidecarMessage {
},
}
/// Read-modify-write one sidecar.
async fn write_one_sidecar(backend: &NextcloudBackend, w: &SidecarWrite) -> Result<(), String> {
let path = RemotePath::new(sidecar_path(&w.image_path));
let id = RemoteId::Path(path.clone());
// An existing sidecar may hold an edit. Absent is the normal case on a
// library that has never been edited, and is not an error.
let existing = backend.get(&id, None).await.ok();
let mut sidecar = existing
.as_deref()
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.and_then(|text| match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
// A corrupt sidecar is *not* overwritten silently: that would
// destroy an edit this build merely failed to understand. The
// judgement stays in the catalog and the file is left alone.
Err(e) => {
log::warn!(
"sidecar at {} is unreadable ({e}); not overwriting",
path.as_str()
);
None
}
})
.unwrap_or_default();
// A parse failure above means we must not touch the file at all.
if existing.is_some() && sidecar.versions.is_empty() && existing.as_deref() != Some(b"") {
// Distinguish "empty file" from "unparseable": only the latter is a
// refusal, and it already logged.
let text = existing
.as_deref()
.map(|b| String::from_utf8_lossy(b).into_owned())
.unwrap_or_default();
if dr_pipeline::Sidecar::parse(&text).is_err() {
return Err("existing sidecar is unreadable".into());
}
}
/// Apply an amendment to a document, returning the new one.
///
/// Split out from both write paths so that online and offline produce
/// *identical* documents: the only thing that differs between them is which
/// base was read and whether an upload follows. A second copy of this for the
/// offline case is how the two would come to disagree about what a paste means.
fn amend(base: dr_pipeline::Sidecar, w: &SidecarWrite) -> dr_pipeline::Sidecar {
let mut sidecar = base;
// Amend the version this write belongs to, creating it if the file did
// not have one. The uuid comes from the catalog, so the same photograph
@@ -509,12 +541,234 @@ async fn write_one_sidecar(backend: &NextcloudBackend, w: &SidecarWrite) -> Resu
version.modified = now_secs();
sidecar.put(version);
sidecar
}
/// TRACES: FR-CAT-9
/// Record an amendment with no server to send it to.
///
/// The base is whatever the cache holds, which is either what the server last
/// had or what earlier offline writes have already built on top of it. Either
/// way the result is queued, and the drain reconciles it with the server's own
/// copy when the connection returns — that reconciliation is a *merge*
/// (FR-NC-9), not an overwrite, so building on a possibly-stale base here does
/// not cost another device's work.
fn write_one_sidecar(
_backend: Option<&NextcloudBackend>,
cache: &SidecarCache,
w: &SidecarWrite,
) -> Result<Outcome, String> {
let path = sidecar_path(&w.image_path);
let base = cache.load(&path).unwrap_or_default();
cache.store(&path, &amend(base, w), true)?;
Ok(Outcome::Queued)
}
/// TRACES: FR-CAT-8 | FR-CAT-9
/// Read-modify-write one sidecar, with a server to read from and send to.
async fn write_one_sidecar_online(
backend: &NextcloudBackend,
cache: &SidecarCache,
w: &SidecarWrite,
) -> Result<Outcome, String> {
let path_str = sidecar_path(&w.image_path);
let path = RemotePath::new(path_str.clone());
let id = RemoteId::Path(path.clone());
// An existing sidecar may hold an edit. Absent is the normal case on a
// library that has never been edited, and is not an error.
let existing = backend.get(&id, None).await.ok();
// A corrupt sidecar is *not* overwritten: that would destroy an edit this
// build merely failed to understand. Refused before anything is written,
// locally or remotely, so the cache cannot end up holding a document that
// silently discarded the file's real contents.
if let Some(bytes) = existing.as_deref() {
if !bytes.is_empty() {
let text = String::from_utf8_lossy(bytes);
if dr_pipeline::Sidecar::parse(&text).is_err() {
return Err(format!("sidecar at {path_str} is unreadable"));
}
}
}
let base = existing
.as_deref()
.map(|bytes| String::from_utf8_lossy(bytes).into_owned())
.and_then(|text| dr_pipeline::Sidecar::parse(&text).ok())
// No sidecar on the server. The cache may still hold queued offline
// work for this image, and taking `default()` here would drop it.
.or_else(|| cache.load(&path_str))
.unwrap_or_default();
let sidecar = amend(base, w);
// Locally first: this is the commit point, and an upload that fails after
// it leaves the edit queued rather than lost.
cache.store(&path_str, &sidecar, true)?;
backend
.put(&path, sidecar.to_text().into_bytes(), None)
.await
.map(|_| ())
.map_err(|e| e.to_string())
.map_err(|e| e.to_string())?;
// Accepted by the server, so it leaves the outbox. The document stays
// cached, which is what lets the next offline open still show the edit.
cache.store(&path_str, &sidecar, false)?;
Ok(Outcome::Uploaded)
}
/// TRACES: FR-CAT-9 | FR-NC-9 | FR-NC-10
/// Upload everything the outbox is still holding.
///
/// # Why this merges rather than uploads
///
/// A queued edit was built on whatever this device last saw. While it sat in
/// the outbox another device may have edited the same photograph, and simply
/// PUTting the local document would discard that work — the precise failure
/// FR-NC-9's node-level merge exists to prevent. So each entry is reconciled
/// against the server's current copy before it goes up, and disjoint edits
/// (a crop made here, an exposure change made there) both survive.
///
/// # Why an entry stays queued on failure
///
/// The marker is cleared only after the server has taken the bytes. A drain
/// interrupted halfway leaves the rest of the outbox exactly as it was, so
/// nothing depends on this running to completion.
pub fn spawn_outbox_drain(
creds: AppCredentials,
user_id: String,
cache_dir: PathBuf,
) -> Receiver<SidecarMessage> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let cache = SidecarCache::open(cache_dir);
let queued = cache.pending();
if queued.is_empty() {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: 0,
failed: 0,
last_error: None,
});
return;
}
log::info!("draining {} queued sidecar(s)", queued.len());
let rt = match crate::net_runtime::build() {
Ok(rt) => rt,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: queued.len(),
failed: 0,
last_error: Some(e.to_string()),
});
return;
}
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
let _ = tx.send(SidecarMessage::Finished {
written: 0,
queued: queued.len(),
failed: 0,
last_error: Some(e.to_string()),
});
return;
}
};
let mut report = SidecarReport::default();
for path_str in &queued {
match drain_one(&backend, &cache, path_str).await {
Ok(()) => report.written += 1,
Err(e) => {
log::debug!("draining {path_str}: {e}");
report.last_error = Some(e);
report.failed += 1;
// Still queued — the marker was never cleared.
report.queued += 1;
}
}
}
let _ = tx.send(SidecarMessage::Finished {
written: report.written,
queued: report.queued,
failed: report.failed,
last_error: report.last_error,
});
});
});
rx
}
/// Reconcile one queued sidecar with the server and upload it.
async fn drain_one(
backend: &NextcloudBackend,
cache: &SidecarCache,
path_str: &str,
) -> Result<(), String> {
let Some(mut local) = cache.load(path_str) else {
// The document went while the drain was running. Nothing to send.
return Ok(());
};
let path = RemotePath::new(path_str.to_string());
let id = RemoteId::Path(path.clone());
let remote = backend.get(&id, None).await.ok();
if let Some(bytes) = remote.as_deref() {
if !bytes.is_empty() {
let text = String::from_utf8_lossy(bytes);
match dr_pipeline::Sidecar::parse(&text) {
Ok(remote) => merge_into(&mut local, &remote),
// Unreadable on the server. Uploading over it would destroy an
// edit this build failed to understand, so the entry stays
// queued rather than being resolved destructively.
Err(e) => return Err(format!("remote sidecar is unreadable ({e})")),
}
}
}
backend
.put(&path, local.to_text().into_bytes(), None)
.await
.map_err(|e| e.to_string())?;
cache.store(path_str, &local, false)
}
/// TRACES: FR-NC-9
/// Merge the server's copy into ours, version by version.
///
/// No common ancestor is available — the outbox stores the result, not the
/// base it was built from — so the merge runs with `None`, which treats every
/// key either side holds as changed. Disjoint keys therefore still both
/// survive, and a key both sides set resolves by revision exactly as it would
/// with a base. What is lost without one is the ability to see a *deletion*:
/// a parameter reset to default on the other device reads as absent rather
/// than as removed, so our value stands. That is the same direction of caution
/// the judgement merge takes — an edit is preserved rather than erased.
fn merge_into(local: &mut dr_pipeline::Sidecar, remote: &dr_pipeline::Sidecar) {
for (uuid, their_version) in &remote.versions {
match local.versions.get(uuid).cloned() {
Some(mut ours) => {
ours.merge(their_version, None);
local.put(ours);
}
// A version only the server has — another device's virtual copy
// (FR-CAT-12). Keeping it is what stops one device's upload from
// deleting another's work.
None => local.put(their_version.clone()),
}
}
}
/// Where the catalog for an account lives.
@@ -1100,44 +1354,85 @@ pub fn spawn_sidecar_fetch(
creds: AppCredentials,
user_id: String,
image_path: String,
cache_dir: PathBuf,
offline: bool,
) -> Receiver<Option<dr_pipeline::Sidecar>> {
let (tx, rx) = std::sync::mpsc::channel();
std::thread::spawn(move || {
let rt = match crate::net_runtime::build() {
Ok(e) => e,
Err(e) => {
log::debug!("sidecar fetch runtime: {e}");
let _ = tx.send(None);
return;
let cache = SidecarCache::open(cache_dir);
let path_str = sidecar_path(&image_path);
// TRACES: FR-CAT-9 | FR-NC-10
// The cache wins outright when it is holding work the server has not
// seen. Fetching in that state would answer with a document *older*
// than the edit sitting in the outbox, and opening the photograph
// would silently show it without the change the user just made —
// which the next save would then write back over the top of.
if cache.is_pending(&path_str) {
log::debug!("{path_str} has queued local edits; opening from the cache");
let _ = tx.send(cache.load(&path_str));
return;
}
// Offline there is nothing to ask, and the cache is the whole answer.
let rt = if offline {
None
} else {
match crate::net_runtime::build() {
Ok(e) => Some(e),
Err(e) => {
log::debug!("sidecar fetch runtime: {e}");
None
}
}
};
let Some(rt) = rt else {
let _ = tx.send(cache.load(&path_str));
return;
};
rt.block_on(async {
let backend = match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => b,
Err(e) => {
log::debug!("sidecar fetch backend: {e}");
let _ = tx.send(None);
let _ = tx.send(cache.load(&path_str));
return;
}
};
let path = RemotePath::new(sidecar_path(&image_path));
let path = RemotePath::new(path_str.clone());
let id = RemoteId::Path(path.clone());
// A 404 is the normal case on a library that has never been
// edited, so this is `ok()` rather than an error path.
let parsed = backend.get(&id, None).await.ok().and_then(|bytes| {
let text = String::from_utf8_lossy(&bytes).into_owned();
match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
Err(e) => {
log::warn!("sidecar at {} is unreadable ({e})", path.as_str());
None
}
let Ok(bytes) = backend.get(&id, None).await else {
// Unreachable, or no such file. The cache cannot tell those
// apart and does not need to: either way it holds the best
// answer this device has.
let _ = tx.send(cache.load(&path_str));
return;
};
let text = String::from_utf8_lossy(&bytes).into_owned();
let parsed = match dr_pipeline::Sidecar::parse(&text) {
Ok(s) => Some(s),
Err(e) => {
log::warn!("sidecar at {} is unreadable ({e})", path.as_str());
None
}
});
};
// Populate the cache from what the server said, so the *next*
// open of this photograph works with no connection. Clean rather
// than pending: this content came from the server, so there is
// nothing to send back.
if let Some(sidecar) = parsed.as_ref() {
if let Err(e) = cache.store(&path_str, sidecar, false) {
log::debug!("caching {path_str}: {e}");
}
}
let _ = tx.send(parsed);
});
ui/dr-ui/src/library_ui.rs
+157 -16
View File
@@ -143,6 +143,20 @@ pub struct LibraryController {
/// a drop all reload the window, and every one of them must honour it or
/// the filter silently lapses.
filter: RefCell<library::RatingFilter>,
/// TRACES: FR-CAT-9
/// Drains the outbox uploader. Held so that a repaint while one is
/// already running does not start a second against the same channel —
/// this handle *is* the "a drain is in flight" state.
outbox_timer: RefCell<Option<slint::Timer>>,
/// Whether the outbox might hold something, so the common case costs a
/// boolean rather than a directory walk.
///
/// `refresh_offline` runs on scan progress as well as on a genuine
/// connectivity change, so the drain is *asked* far more often than there
/// is anything to send. Starts `true` so the first ask after launch does
/// walk — edits queued in a previous session are exactly the ones that
/// need sending, and nothing in memory knows about them.
outbox_maybe_dirty: std::cell::Cell<bool>,
/// Drains the sidecar writer. Held so a second judgement replaces the
/// timer rather than leaving two draining the same finished channel.
sidecar_timer: RefCell<Option<slint::Timer>>,
@@ -236,6 +250,8 @@ impl LibraryController {
sidecar_timer: RefCell::new(None),
generation: std::cell::Cell::new(0),
reachability: RefCell::new(dr_sync::Reachability::new()),
outbox_timer: RefCell::new(None),
outbox_maybe_dirty: std::cell::Cell::new(true),
pin_timer: RefCell::new(None),
local_only: std::cell::Cell::new(false),
// The catalog's own floor until the settings page reports what the
@@ -346,6 +362,22 @@ impl LibraryController {
.ok()
}
/// TRACES: FR-CAT-9 | FR-NC-10
/// Where cached sidecars and the upload outbox live for the open library.
///
/// Beside the catalog and the originals cache, for the same reason those
/// two sit together: all three are per-account and are discarded together.
/// A separate directory rather than a subfolder of `originals` because the
/// two have opposite lifetimes — originals are evicted under a budget
/// (FR-NC-6a), and a queued edit must never be.
pub fn sidecar_cache_dir(&self) -> Option<PathBuf> {
let borrow = self.session.borrow();
let (_, session, _) = borrow.as_ref()?;
library::catalog_path(&session.server, &session.user_id)
.parent()
.map(|p| p.join("sidecars"))
}
/// TRACES: FR-NC-6a
/// Where cached originals live for the open library.
///
@@ -1035,6 +1067,100 @@ fn refresh_offline(window: &AppWindow, ctl: &Rc<LibraryController>) {
if offline {
window.set_library_error("".into());
}
drop(reach);
// TRACES: FR-CAT-9
// Back online: send whatever the outbox is still holding.
//
// Hung off the one function that paints connectivity rather than off each
// of the seven places that move it — a drain that had to be remembered at
// every call site is a drain that will be forgotten at one of them, and
// the symptom is an edit that stays queued until the app is restarted.
//
// `start_outbox_drain` is a no-op when the outbox is empty and while one
// is already running, so calling it on every repaint costs a directory
// walk that finds nothing.
if !offline {
start_outbox_drain(window, ctl);
}
}
/// TRACES: FR-CAT-9 | FR-NC-10
/// Upload the sidecars queued while this device had no connection.
///
/// Guarded on the timer rather than on a flag: the timer *is* the "a drain is
/// running" state, and a second one started beside it would drain the same
/// channel twice.
fn start_outbox_drain(window: &AppWindow, ctl: &Rc<LibraryController>) {
if ctl.outbox_timer.borrow().is_some() {
return;
}
if !ctl.outbox_maybe_dirty.get() {
return;
}
let Some(cache_dir) = ctl.sidecar_cache_dir() else {
return;
};
if crate::sidecar_cache::SidecarCache::open(cache_dir.clone())
.pending()
.is_empty()
{
// Nothing there. Recorded so the next hundred repaints skip the walk;
// a write that queues sets it again.
ctl.outbox_maybe_dirty.set(false);
return;
}
let Some((creds, session, _)) = ctl.session.borrow().clone() else {
return;
};
let rx = library::spawn_outbox_drain(creds, session.user_id.clone(), cache_dir);
let job = ctl
.activity
.begin(crate::activity::Kind::Upload, "Uploading queued edits");
let timer = slint::Timer::default();
let weak = window.as_weak();
let ctl_cb = ctl.clone();
timer.start(
slint::TimerMode::Repeated,
std::time::Duration::from_millis(250),
move || {
let Some(w) = weak.upgrade() else { return };
match rx.try_recv() {
Ok(library::SidecarMessage::Finished {
written,
queued,
failed,
last_error,
}) => {
if failed > 0 {
log::warn!(
"{failed} queued sidecar(s) still undelivered: {}",
last_error.clone().unwrap_or_default()
);
// Not `fail`: the edits are still safely queued, and
// reporting this as a loss would be wrong.
job.finish(format!("{written} uploaded · {queued} still queued"));
} else if written > 0 {
log::info!("{written} queued sidecar(s) uploaded");
job.finish(format!("{written} queued edit(s) uploaded"));
w.set_library_status(format!("{written} queued edit(s) uploaded").into());
} else {
job.finish_quietly();
}
ctl_cb.outbox_maybe_dirty.set(queued > 0);
stop(&ctl_cb.outbox_timer);
}
Err(std::sync::mpsc::TryRecvError::Empty) => {}
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
job.finish_quietly();
stop(&ctl_cb.outbox_timer);
}
}
},
);
*ctl.outbox_timer.borrow_mut() = Some(timer);
}
/// A coarse "how long ago", for the offline banner.
@@ -1581,29 +1707,27 @@ pub(crate) fn start_sidecar_writes(
}
// TRACES: FR-CAT-9
// Offline, this would stall on a timeout per sidecar, on the very
// keystroke path a cull is built for speed on (FR-CULL-1). The judgement
// itself is safe either way — `apply_judgement` has already committed it
// to the catalog, which is what the grid reads and what survives a
// restart.
// Offline is passed down rather than used to skip.
//
// What is deferred is the sidecar, and with it the guarantee that the
// judgement survives a *catalog rebuild* (ARCH §6.12). There is no queue
// behind this yet, so a rating made offline is written to its sidecar only
// when that image is judged again while connected. That is a real gap
// rather than a hidden one: it trades a durability property that already
// depends on the network for a cull that stays responsive without it.
if ctl.is_offline() {
log::debug!("offline: skipping {} sidecar write(s)", writes.len());
return;
}
// It used to skip, and the reasoning was that a cull stays responsive
// because the rating is safe in the catalog. That held for judgements and
// not for edits: the catalog stores no parameters, so a pasted edit made
// offline survived nowhere at all. Every write now commits to the local
// sidecar cache first and the upload is best-effort, which keeps the
// keystroke path off the network — the original concern — without the
// write being conditional on it.
let offline = ctl.is_offline();
let Some((creds, session, _)) = ctl.session.borrow().clone() else {
return;
};
let Some(cache_dir) = ctl.sidecar_cache_dir() else {
return;
};
let count = writes.len();
let rx = library::spawn_sidecar_writes(creds, session.user_id.clone(), writes);
let rx =
library::spawn_sidecar_writes(creds, session.user_id.clone(), writes, cache_dir, offline);
let timer = slint::Timer::default();
let weak = window.as_weak();
@@ -1628,10 +1752,27 @@ pub(crate) fn start_sidecar_writes(
match rx.try_recv() {
Ok(library::SidecarMessage::Finished {
written,
queued,
failed,
last_error,
}) => {
if failed == 0 && queued > 0 {
// Recorded locally, waiting for the server. Said out
// loud because the user has just made an edit with no
// connection and deserves to know it is safe — the
// old behaviour here was to drop it silently.
log::debug!("{queued} sidecar(s) queued for upload");
ctl_cb.outbox_maybe_dirty.set(true);
job.finish_quietly();
w.set_library_status(
format!("{queued} edit(s) saved · will upload when back online").into(),
);
stop(&ctl_cb.sidecar_timer);
return;
}
if failed > 0 {
// A failed upload left the edit in the outbox.
ctl_cb.outbox_maybe_dirty.set(true);
log::warn!(
"{failed} sidecar write(s) failed: {}",
last_error.clone().unwrap_or_default()
ui/dr-ui/src/settings_ui.rs
+178
View File
@@ -45,6 +45,23 @@ pub struct SettingsController {
/// How much disk the cache is currently using, as a label. Supplied by
/// whoever owns the catalog — this module has no connection to query.
usage_label: RefCell<String>,
/// TRACES: FR-EXP-6
/// The remote folder picker, while it is open.
///
/// The same [`FolderBrowser`](crate::launch::FolderBrowser) the launch
/// screen uses to choose a library root, reused rather than reimplemented:
/// it browses a remote tree and nothing about it is specific to what the
/// chosen folder is *for*. `None` means the picker is closed, which is
/// also the only state a device destination ever has — a path on this
/// machine is typed or chosen by the platform, not walked over WebDAV.
pub browser: RefCell<Option<crate::launch::FolderBrowser>>,
/// Polls the folder listing while one is in flight.
///
/// Held here rather than in the function that starts it: a `slint::Timer`
/// stops the moment it is dropped, so a timer local to `spawn_folder_list`
/// would be collected before the listing it is waiting on ever arrived.
/// The same place `LaunchController` keeps its own poll timer.
poll_timer: RefCell<Option<slint::Timer>>,
}
impl SettingsController {
@@ -56,6 +73,8 @@ impl SettingsController {
store,
error: RefCell::new(None),
usage_label: RefCell::new(String::new()),
browser: RefCell::new(None),
poll_timer: RefCell::new(None),
})
}
@@ -64,6 +83,21 @@ impl SettingsController {
self.settings.borrow().clone()
}
/// Adopt a folder chosen in the picker as the export destination.
///
/// Goes through `edit` like every other change, so it is saved the moment
/// it is chosen — the page has no Save button and a destination that
/// survived only until the window closed would be the one setting that
/// behaved differently from all the others.
pub fn set_destination(&self, path: String) {
self.edit(|s| s.export.destination = path);
}
/// Report a failure onto the page's error line.
pub fn set_error(&self, message: impl Into<String>) {
*self.error.borrow_mut() = Some(message.into());
}
/// Show what the cache is holding. Empty hides the line.
pub fn set_usage_label(&self, label: String) {
*self.usage_label.borrow_mut() = label;
@@ -173,6 +207,40 @@ pub fn render(window: &AppWindow, controller: &SettingsController) {
.into(),
);
// --- the remote folder picker --------------------------------------
{
let browser = controller.browser.borrow();
window.set_settings_browse_open(browser.is_some());
match browser.as_ref() {
Some(b) => {
// The root is shown as a word rather than as an empty string,
// which would read as a control that had lost its value.
window.set_settings_browse_path(
if b.path.is_empty() {
"Library root".to_string()
} else {
b.path.clone()
}
.into(),
);
window.set_settings_browse_loading(b.loading);
window.set_settings_browse_at_root(b.parent_path().is_none());
window.set_settings_browse_entries(slint::ModelRc::new(slint::VecModel::from(
b.entries
.iter()
.map(|e| slint::SharedString::from(e.as_str()))
.collect::<Vec<_>>(),
)));
}
None => {
window.set_settings_browse_entries(slint::ModelRc::new(slint::VecModel::from(
Vec::<slint::SharedString>::new(),
)));
window.set_settings_browse_loading(false);
}
}
}
window.set_settings_error(controller.error.borrow().clone().unwrap_or_default().into());
}
@@ -490,6 +558,112 @@ where
}
}
/// TRACES: FR-EXP-6
/// List the folders under `path`, for the export destination picker.
///
/// A near-twin of `launch_ui::spawn_folder_list` and deliberately not shared
/// with it. That one reaches into the `LaunchController` for its session and
/// reports failures onto the launch screen's error line; this one is handed
/// credentials and writes to the settings page. Factoring them together would
/// mean a function taking both controllers, or a trait implemented twice to
/// abstract two call sites — more machinery than the twenty lines it saves.
///
/// The *model* is shared, which is the part that matters: both drive a
/// [`FolderBrowser`](crate::launch::FolderBrowser), so navigation behaves
/// identically in both places.
pub fn spawn_folder_list(
weak: slint::Weak<AppWindow>,
ctl: Rc<SettingsController>,
creds: dr_sync_nextcloud::AppCredentials,
user_id: String,
path: String,
) {
use dr_sync::{RemoteBackend, RemotePath};
use dr_sync_nextcloud::NextcloudBackend;
let (tx, rx) = std::sync::mpsc::channel::<Result<Vec<String>, String>>();
std::thread::spawn(move || {
// Multi-thread, for the reason the login worker records: a
// current-thread runtime left reqwest's connection future unpolled on
// Android, and the await never resolved.
let rt = tokio::runtime::Builder::new_multi_thread()
.worker_threads(1)
.enable_io()
.enable_time()
.build();
let Ok(rt) = rt else {
let _ = tx.send(Err("runtime".into()));
return;
};
rt.block_on(async {
match NextcloudBackend::new(&creds, &user_id) {
Ok(b) => match b.list(&RemotePath::new(&path), None).await {
Ok(entries) => {
let mut dirs: Vec<String> = entries
.iter()
.filter(|e| e.kind == dr_sync::EntryKind::Directory)
.map(|e| e.path.name().to_string())
.collect();
dirs.sort_by_key(|d| d.to_ascii_lowercase());
let _ = tx.send(Ok(dirs));
}
Err(e) => {
let _ = tx.send(Err(e.to_string()));
}
},
Err(e) => {
let _ = tx.send(Err(e.to_string()));
}
}
});
});
let timer = slint::Timer::default();
let ctl_cb = ctl.clone();
timer.start(
slint::TimerMode::Repeated,
std::time::Duration::from_millis(150),
move || {
let Some(w) = weak.upgrade() else { return };
match rx.try_recv() {
Ok(Ok(dirs)) => {
if let Some(b) = ctl_cb.browser.borrow_mut().as_mut() {
b.entries = dirs;
b.loading = false;
}
render(&w, &ctl_cb);
}
Ok(Err(e)) => {
// The picker stays open showing the folder it was on. A
// listing that failed is not a reason to discard where the
// user had navigated to.
if let Some(b) = ctl_cb.browser.borrow_mut().as_mut() {
b.loading = false;
}
*ctl_cb.error.borrow_mut() = Some(format!("Could not list folders: {e}"));
render(&w, &ctl_cb);
}
Err(std::sync::mpsc::TryRecvError::Empty) => return,
Err(std::sync::mpsc::TryRecvError::Disconnected) => {
if let Some(b) = ctl_cb.browser.borrow_mut().as_mut() {
b.loading = false;
}
render(&w, &ctl_cb);
}
}
// The channel has delivered, so there is nothing left to poll
// for. Stopping it here rather than leaving it running is what
// keeps a page opened and closed twenty times from accumulating
// twenty timers.
if let Some(t) = ctl_cb.poll_timer.borrow().as_ref() {
t.stop();
}
},
);
*ctl.poll_timer.borrow_mut() = Some(timer);
}
#[cfg(test)]
mod tests {
use super::*;
@@ -514,6 +688,8 @@ mod tests {
store,
error: RefCell::new(None),
usage_label: RefCell::new(String::new()),
browser: RefCell::new(None),
poll_timer: RefCell::new(None),
})
}
@@ -584,6 +760,8 @@ mod tests {
store: SettingsStore::open_at(blocker.join("settings.json")),
error: RefCell::new(None),
usage_label: RefCell::new(String::new()),
browser: RefCell::new(None),
poll_timer: RefCell::new(None),
};
broken.edit(|s| s.export.quality = 50);
ui/dr-ui/src/sidecar_cache.rs
+420
View File
@@ -0,0 +1,420 @@
//! TRACES: FR-CAT-9 | FR-NC-10 | FR-CAT-8
//! A local copy of every sidecar this device has seen, and the outbox of the
//! ones the server has not.
//!
//! # Why a cache at all
//!
//! FR-CAT-9: *"edits queue and apply when the source returns."* Before this,
//! a rating or a pasted edit made with no connection was dropped — the
//! judgement survived in the catalog, which is disposable (ARCH §6.12), and a
//! pasted edit survived nowhere at all. An edit that vanishes because the
//! train went into a tunnel is the worst failure an authoritative store can
//! have, and it is silent.
//!
//! So the local file is the **commit point** and the upload is best-effort.
//! Writing is always a local write first; the network attempt follows and, if
//! it fails or was never possible, the entry stays marked and is retried when
//! the server comes back. Online and offline are therefore the same code path
//! differing only in whether the upload is attempted, rather than two paths
//! where one quietly does less.
//!
//! # Why the filesystem is the queue
//!
//! The catalog has a `jobs` table with backoff and coalescing, and
//! `JobKind::WriteSidecar` was reserved for this. It is deliberately not used:
//! the catalog is rebuildable and may be deleted at any time, and a queue of
//! *unuploaded edits* is the one thing in this system that cannot be
//! reconstructed from anywhere else. A pending marker sitting beside the
//! document it refers to survives a catalog deletion, an app reinstall that
//! keeps app data, and a crash — because there is no separate index that could
//! disagree with it.
//!
//! It is also debuggable in the way the sidecar format itself is meant to be:
//! the cache mirrors the server's layout, so the file holding an edit that
//! failed to upload is at the path you would guess, and a `.pending` marker
//! beside it says why it is still there.
//!
//! # Why there is no budget
//!
//! Unlike cached originals (FR-NC-6a), sidecars are hundreds of bytes. A
//! hundred-thousand-image library is tens of megabytes, and evicting one would
//! cost a round-trip to re-read an edit the user is about to open. The
//! originals cache exists because RAW files are 30 MB each; this does not have
//! the problem that motivates one.
use std::path::{Path, PathBuf};
use dr_pipeline::Sidecar;
/// Suffix marking a cached sidecar the server has not yet accepted.
///
/// A separate zero-byte file rather than a flag inside the document: the
/// document's bytes are what gets uploaded, and a marker inside it would have
/// to be stripped on the way out — one more chance to upload something subtly
/// different from what was stored.
const PENDING_SUFFIX: &str = ".pending";
/// The on-disk sidecar cache for one library.
pub struct SidecarCache {
dir: PathBuf,
}
impl SidecarCache {
/// Open the cache rooted at `dir`. Nothing is created until a write.
pub fn open(dir: PathBuf) -> Self {
Self { dir }
}
/// Where a remote sidecar is cached.
///
/// Mirrors the server's layout, so `photos/2024/a.drsc` is cached at
/// `<dir>/photos/2024/a.drsc`. Keyed on the **sidecar's** remote path
/// rather than the image's, because that is what an upload addresses —
/// deriving one from the other in two places is how they come to disagree.
///
/// Returns `None` for a path that would escape the cache directory. The
/// path comes from a server response, so it is not this process's to
/// trust: a `..` component would let a hostile or merely broken server
/// name a file anywhere this app can write.
pub fn path_for(&self, remote_sidecar_path: &str) -> Option<PathBuf> {
// No directory means no library is open. Every path would otherwise
// be relative and land in the process's working directory, which for a
// desktop launch is wherever the user happened to be standing.
if self.dir.as_os_str().is_empty() {
return None;
}
let mut out = self.dir.clone();
let mut depth = 0usize;
for part in remote_sidecar_path.split('/') {
match part {
// Empty from a leading or doubled slash, and `.`, both mean
// "here" and are simply skipped.
"" | "." => continue,
".." => return None,
// A Windows-style drive or a backslash cannot appear in a
// WebDAV path segment and would not mean what it looks like.
p if p.contains('\\') => return None,
p => {
out.push(p);
depth += 1;
}
}
}
(depth > 0).then_some(out)
}
/// The cached document, if there is one.
///
/// An unreadable cache entry answers `None` rather than an error: the
/// caller's fallback is to treat the photograph as unedited, and a cache
/// is by definition reconstructible from the server.
pub fn load(&self, remote_sidecar_path: &str) -> Option<Sidecar> {
let path = self.path_for(remote_sidecar_path)?;
let text = std::fs::read_to_string(&path).ok()?;
match Sidecar::parse(&text) {
Ok(s) => Some(s),
Err(e) => {
log::warn!("cached sidecar at {} is unreadable ({e})", path.display());
None
}
}
}
/// Write a document into the cache.
///
/// `pending` records whether the server has this content. Passing `false`
/// after a successful upload is what takes an entry out of the outbox, so
/// the marker is *removed* here rather than only in a separate call — one
/// function owns the pair, and they cannot drift apart.
pub fn store(
&self,
remote_sidecar_path: &str,
sidecar: &Sidecar,
pending: bool,
) -> Result<(), String> {
let path = self
.path_for(remote_sidecar_path)
.ok_or_else(|| format!("unsafe sidecar path {remote_sidecar_path}"))?;
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent).map_err(|e| e.to_string())?;
}
// Write and rename, so an interrupted save cannot truncate an edit
// that was already safely on disk — the same discipline the settings
// store and the local sidecar writer use.
let tmp = path.with_extension("drsc.tmp");
std::fs::write(&tmp, sidecar.to_text()).map_err(|e| e.to_string())?;
std::fs::rename(&tmp, &path).map_err(|e| e.to_string())?;
// The marker is written *after* the document. The other order would
// leave a window where a crash produced a marker pointing at content
// that was never stored, and the drain would upload a stale document
// believing it to be the queued one.
let marker = marker_for(&path);
if pending {
std::fs::write(&marker, b"").map_err(|e| e.to_string())?;
} else if marker.exists() {
std::fs::remove_file(&marker).map_err(|e| e.to_string())?;
}
Ok(())
}
/// Whether this entry is waiting to be uploaded.
pub fn is_pending(&self, remote_sidecar_path: &str) -> bool {
self.path_for(remote_sidecar_path)
.is_some_and(|p| marker_for(&p).exists())
}
/// Every sidecar waiting to be uploaded, as remote paths.
///
/// Walks the tree rather than consulting an index, which is the property
/// that makes the queue survive a catalog deletion: the markers *are* the
/// queue, so there is nothing to fall out of step with them.
///
/// Sorted, so a drain that is interrupted resumes in the same order rather
/// than retrying whichever entry the directory happened to yield first.
pub fn pending(&self) -> Vec<String> {
let mut out = Vec::new();
collect_pending(&self.dir, &self.dir, &mut out);
out.sort();
out
}
}
/// The marker path for a cached document.
fn marker_for(path: &Path) -> PathBuf {
let mut s = path.as_os_str().to_os_string();
s.push(PENDING_SUFFIX);
PathBuf::from(s)
}
/// Recursively gather remote paths whose marker exists.
///
/// A missing or unreadable directory contributes nothing rather than failing
/// the walk: a cache that has never been written has no directory at all, and
/// that is the normal state on a fresh install.
fn collect_pending(root: &Path, dir: &Path, out: &mut Vec<String>) {
let Ok(entries) = std::fs::read_dir(dir) else {
return;
};
for entry in entries.flatten() {
let path = entry.path();
if path.is_dir() {
collect_pending(root, &path, out);
continue;
}
// The marker names the document; the document names the remote path.
let Some(name) = path.file_name().and_then(|n| n.to_str()) else {
continue;
};
let Some(stem) = name.strip_suffix(PENDING_SUFFIX) else {
continue;
};
let document = path.with_file_name(stem);
// A marker whose document has gone is not a queued upload; it is
// debris. Skipped rather than reported, since there is nothing to send
// and nothing the user could do about it.
if !document.exists() {
continue;
}
if let Ok(rel) = document.strip_prefix(root) {
// Back to a remote path: the cache mirrors the server's layout, so
// the relative path *is* the remote path.
let remote: Vec<&str> = rel.iter().filter_map(|c| c.to_str()).collect();
if !remote.is_empty() {
out.push(remote.join("/"));
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use dr_pipeline::sidecar::Version;
fn tempdir(name: &str) -> PathBuf {
let dir = std::env::temp_dir().join(format!(
"dr-sidecar-cache-{name}-{}-{:?}",
std::process::id(),
std::thread::current().id()
));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
dir
}
/// A cache and the directory it is rooted at.
fn cache(name: &str) -> (SidecarCache, PathBuf) {
let root = tempdir(name).join("sidecars");
(SidecarCache::open(root.clone()), root)
}
fn rated(stars: u8) -> Sidecar {
let mut s = Sidecar::new();
s.put(Version {
uuid: "u1".into(),
name: "Default".into(),
is_default: true,
revision: 1,
rating: stars,
..Default::default()
});
s
}
#[test]
fn the_cache_mirrors_the_servers_layout() {
// What makes an entry findable by hand when an upload has gone wrong,
// and what lets `pending` recover a remote path with no index.
let (c, _d) = cache("layout");
let p = c.path_for("photos/2024/a.drsc").unwrap();
assert!(
p.ends_with("sidecars/photos/2024/a.drsc"),
"{}",
p.display()
);
}
#[test]
fn a_document_survives_a_store_and_a_load() {
let (c, _d) = cache("round-trip");
c.store("photos/a.drsc", &rated(4), true).unwrap();
let back = c.load("photos/a.drsc").expect("cached");
assert_eq!(back.default_version().unwrap().rating, 4);
}
#[test]
fn an_absent_entry_loads_as_nothing() {
let (c, _d) = cache("absent");
assert!(c.load("photos/never-seen.drsc").is_none());
}
#[test]
fn a_pending_write_appears_in_the_outbox() {
// The whole point: an edit made offline is queued rather than dropped.
let (c, _d) = cache("outbox");
c.store("photos/a.drsc", &rated(3), true).unwrap();
assert!(c.is_pending("photos/a.drsc"));
assert_eq!(c.pending(), vec!["photos/a.drsc".to_string()]);
}
#[test]
fn a_stored_upload_leaves_the_outbox() {
let (c, _d) = cache("drained");
c.store("photos/a.drsc", &rated(3), true).unwrap();
c.store("photos/a.drsc", &rated(3), false).unwrap();
assert!(!c.is_pending("photos/a.drsc"));
assert!(c.pending().is_empty());
// And the content is still cached, so an offline open still shows the
// edit. Clearing the marker must not clear the document.
assert_eq!(
c.load("photos/a.drsc")
.unwrap()
.default_version()
.unwrap()
.rating,
3
);
}
#[test]
fn the_outbox_finds_entries_nested_at_any_depth() {
// The walk is what stands in for an index, so it has to reach an entry
// wherever the server's tree put it.
let (c, _d) = cache("nested");
c.store("a.drsc", &rated(1), true).unwrap();
c.store("photos/b.drsc", &rated(1), true).unwrap();
c.store("photos/2024/spain/c.drsc", &rated(1), true)
.unwrap();
c.store("photos/d.drsc", &rated(1), false).unwrap();
assert_eq!(
c.pending(),
vec![
"a.drsc".to_string(),
"photos/2024/spain/c.drsc".to_string(),
"photos/b.drsc".to_string(),
]
);
}
#[test]
fn an_empty_cache_has_an_empty_outbox() {
// A fresh install has no directory at all; the walk must not fail.
let (c, _d) = cache("fresh");
assert!(c.pending().is_empty());
}
#[test]
fn a_marker_without_its_document_is_not_a_queued_upload() {
// Debris from an interrupted write. There is nothing to send, and
// reporting it as queued would leave the outbox permanently non-empty.
let (c, _d) = cache("debris");
c.store("photos/a.drsc", &rated(1), true).unwrap();
std::fs::remove_file(c.path_for("photos/a.drsc").unwrap()).unwrap();
assert!(c.pending().is_empty());
}
#[test]
fn a_path_escaping_the_cache_is_refused() {
// The remote path comes from a server response and is not this
// process's to trust.
let (c, _d) = cache("escape");
assert!(c.path_for("../../etc/passwd.drsc").is_none());
assert!(c.path_for("photos/../../../a.drsc").is_none());
assert!(c.path_for("").is_none());
assert!(c.store("../evil.drsc", &rated(1), true).is_err());
}
#[test]
fn a_cache_with_no_directory_is_inert() {
// What a caller holds before a library is open. Writing relative to
// the working directory would scatter sidecars wherever the app was
// launched from.
let c = SidecarCache::open(PathBuf::new());
assert!(c.path_for("photos/a.drsc").is_none());
assert!(c.load("photos/a.drsc").is_none());
assert!(c.store("photos/a.drsc", &rated(1), true).is_err());
assert!(c.pending().is_empty());
}
#[test]
fn a_leading_slash_is_not_an_absolute_path() {
// WebDAV paths often arrive with one. Treating it as absolute would
// push the write to the filesystem root.
let (c, root) = cache("leading-slash");
let p = c.path_for("/photos/a.drsc").unwrap();
assert!(p.starts_with(&root), "{}", p.display());
}
#[test]
fn a_corrupt_cache_entry_reads_as_absent_rather_than_failing() {
// A cache is reconstructible from the server by definition, so the
// fallback is to fetch — never to refuse to open the photograph.
let (c, _d) = cache("corrupt");
c.store("photos/a.drsc", &rated(1), false).unwrap();
std::fs::write(c.path_for("photos/a.drsc").unwrap(), "drsc 99\n").unwrap();
assert!(c.load("photos/a.drsc").is_none());
}
#[test]
fn storing_leaves_no_temporary_file_behind() {
let (c, root) = cache("no-temp");
c.store("photos/a.drsc", &rated(1), true).unwrap();
let leftovers: Vec<_> = std::fs::read_dir(root.join("photos"))
.unwrap()
.filter_map(|e| e.ok())
.map(|e| e.file_name().to_string_lossy().to_string())
.filter(|n| n.ends_with(".tmp"))
.collect();
assert!(leftovers.is_empty(), "left {leftovers:?} behind");
}
}