Also carries in-flight work that shared these files: the zoom structure-key fix in the adjust pipeline, nearest-neighbour filtering past 1:1, the timeline scrub marker correction, the 423-Locked retry in the metadata sweep, and the thumbnail size-class migration. # Offline mode (FR-CAT-9) The app previously assumed the server was reachable and treated its absence as a series of unrelated per-operation failures. A launch without a connection produced an empty grid, even with a complete catalog on disk and every thumbnail already in the shards. Reachability is now inferred from traffic the app was already making, rather than probed for. `RemoteError::indicates_offline` draws the line that makes this possible: a dead connection is offline, a 403 or a 500 is not — the server answered, so blanking the library over one forbidden file would be a worse error than the one being reported. `Reachability` turns those outcomes into a state, so a library browsing happily never issues a probe at all. Going offline takes one failure, because the user is already experiencing it. Coming back requires evidence — a completed scan or a fetched thumbnail — with a capped exponential backoff behind the manual retry, so twelve sweep lanes failing together do not schedule twelve immediate probes. What keeps working: the catalog opens even when the scan that normally provides it failed, so the grid fills from the last successful scan. Thumbnails come from the shards. Rating, flagging and collecting are catalog writes that never touched the network. What stops is opening an original that was never stored locally, and it now says so in those words instead of reporting "network error: connection refused" over a photograph. Work that is pure network is refused rather than left to fail slowly: the metadata sweep, derived sync, and sidecar writes. The sweep would otherwise spend a timeout per image across the whole library while the progress bar implied something was happening. Deferring sidecars is a real gap rather than a hidden one — a rating made offline reaches its sidecar only when that image is judged again while connected — and it is recorded as such at the call site. # The "On this device" filter A chip beside the rating filters, narrowing the grid to images whose original is held locally. It composes with the rating terms rather than replacing them, so "five-star frames I can actually edit on this train" is one filter. The predicate is SQL, like the rating terms and for the same reason: the count in the header has to agree with the cells drawn. It reads `image_cache.tier_actual`, which nothing writes yet — the next commit fills it. Until then the chip honestly reports zero. `Tier` gains an explicit on-disk encoding. The variants are ordered by generosity and the derived `Ord` invites reordering them, which would silently reinterpret every cached row; the round-trip test is what holds the two in agreement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
231 lines
8.8 KiB
Rust
231 lines
8.8 KiB
Rust
//! Pluggable remote storage for DarkRoom.
|
|
//!
|
|
//! Defines the [`RemoteBackend`] trait and the capability model the sync
|
|
//! engine adapts to. Only the Nextcloud connector is implemented
|
|
//! (`dr-sync-nextcloud`), but the boundary is designed so other backends can
|
|
//! be added without touching the engine.
|
|
//!
|
|
//! # Why capabilities rather than a common denominator
|
|
//!
|
|
//! Nextcloud's fast path relies on a behaviour that is *not* a WebDAV
|
|
//! guarantee: directory ETags propagate up the tree, so an unchanged root
|
|
//! ETag proves nothing anywhere in the library changed. That single property
|
|
//! turns a no-op sync over 50k images into one HTTP request.
|
|
//!
|
|
//! A trait built to what every backend can do would force full enumeration
|
|
//! every time — the exact cost the design exists to avoid. So backends
|
|
//! declare what they support and the engine picks a strategy (ARCH §8.1).
|
|
|
|
use std::ops::Range;
|
|
|
|
use async_trait::async_trait;
|
|
|
|
pub mod capability;
|
|
pub mod error;
|
|
pub mod reachability;
|
|
pub mod scan;
|
|
pub mod types;
|
|
|
|
pub use capability::{Capabilities, ChangeDetection, ChunkConstraints, ServerPreviews};
|
|
pub use error::RemoteError;
|
|
pub use reachability::{Connectivity, Reachability};
|
|
pub use scan::{scan, ScanProgress, ScanResult};
|
|
pub use types::{
|
|
Cursor, EntryKind, Identity, Precondition, RemoteChange, RemoteEntry, RemoteId, RemotePath,
|
|
Validator,
|
|
};
|
|
|
|
/// TRACES: FR-NC-12
|
|
/// A remote storage backend.
|
|
///
|
|
/// Implementations are expected to be cheap to clone or to be used behind an
|
|
/// `Arc`; the engine may call them concurrently.
|
|
#[async_trait]
|
|
pub trait RemoteBackend: Send + Sync {
|
|
/// What this backend supports. Read once at connect time and used to pick
|
|
/// a sync strategy.
|
|
fn capabilities(&self) -> &Capabilities;
|
|
|
|
/// Human-readable backend name, for logs and the UI.
|
|
fn name(&self) -> &str;
|
|
|
|
// ---- discovery --------------------------------------------------------
|
|
|
|
/// List one directory level.
|
|
///
|
|
/// `since` carries the validator the caller last saw, so backends able to
|
|
/// skip unchanged entries may do so. Backends that cannot simply ignore
|
|
/// it.
|
|
async fn list(
|
|
&self,
|
|
dir: &RemotePath,
|
|
since: Option<&Validator>,
|
|
) -> Result<Vec<RemoteEntry>, RemoteError>;
|
|
|
|
/// Fetch a directory's validator without listing its contents.
|
|
///
|
|
/// The cheap probe that makes ETag pruning work: one request against the
|
|
/// root answers "did anything change?". Backends without
|
|
/// [`ChangeDetection::PropagatingEtags`] return
|
|
/// [`RemoteError::Unsupported`].
|
|
async fn dir_validator(&self, dir: &RemotePath) -> Result<Validator, RemoteError>;
|
|
|
|
/// Ask what changed since a cursor.
|
|
///
|
|
/// Only meaningful for [`ChangeDetection::DeltaCursor`] backends; others
|
|
/// return [`RemoteError::Unsupported`].
|
|
async fn delta(&self, cursor: &Cursor) -> Result<(Vec<RemoteChange>, Cursor), RemoteError>;
|
|
|
|
// ---- transfer ---------------------------------------------------------
|
|
|
|
/// Fetch an object, optionally a byte range.
|
|
///
|
|
/// The range is a hint, not a guarantee: backends without range support
|
|
/// may return the whole object, and the caller slices. Correctness holds
|
|
/// either way; [`Capabilities::range_reads`] says whether it was cheap.
|
|
async fn get(&self, id: &RemoteId, range: Option<Range<u64>>) -> Result<Vec<u8>, RemoteError>;
|
|
|
|
/// Upload, optionally guarded by a precondition.
|
|
///
|
|
/// Backends handle chunking internally based on body size — chunked
|
|
/// upload is an implementation detail, not part of this interface, since
|
|
/// exposing it would leak one server's protocol into the abstraction.
|
|
async fn put(
|
|
&self,
|
|
path: &RemotePath,
|
|
body: Vec<u8>,
|
|
precond: Option<Precondition>,
|
|
) -> Result<Validator, RemoteError>;
|
|
|
|
/// Upload many small objects.
|
|
///
|
|
/// Defaults to sequential [`put`](Self::put) calls; backends with a bulk
|
|
/// endpoint override it. Sidecars are the motivating case — hundreds of
|
|
/// a few KB each.
|
|
async fn put_many(
|
|
&self,
|
|
items: Vec<(RemotePath, Vec<u8>)>,
|
|
) -> Result<Vec<Result<Validator, RemoteError>>, RemoteError> {
|
|
let mut out = Vec::with_capacity(items.len());
|
|
for (path, body) in items {
|
|
out.push(self.put(&path, body, None).await);
|
|
}
|
|
Ok(out)
|
|
}
|
|
|
|
/// Delete an object.
|
|
async fn delete(&self, id: &RemoteId, precond: Option<Precondition>)
|
|
-> Result<(), RemoteError>;
|
|
|
|
/// Move an object, keeping its identity.
|
|
///
|
|
/// TRACES: FR-CAT-15
|
|
/// **The stable id must survive.** This is what a soft delete uses to put a
|
|
/// photograph in the trash folder, and what a restore uses to bring it back.
|
|
/// A move implemented as copy-then-delete would allocate a *new*
|
|
/// `oc:fileid`, which orphans the thumbnail shard entry and the sidecar
|
|
/// mapping and turns a restore into a full re-download. WebDAV `MOVE` is one
|
|
/// request and preserves the id, which is why this is its own method rather
|
|
/// than something the caller composes.
|
|
///
|
|
/// Creates missing parent directories of `to`: the trash folder does not
|
|
/// exist until the first image is trashed, and requiring the caller to
|
|
/// create it separately makes the first trash of every library a two-step
|
|
/// dance with a failure mode in the middle.
|
|
async fn move_to(&self, from: &RemoteId, to: &RemotePath) -> Result<(), RemoteError>;
|
|
|
|
/// Create a directory, and any missing parents.
|
|
///
|
|
/// Succeeds if it already exists — callers use this to guarantee a
|
|
/// destination, not to claim they created it.
|
|
async fn create_dir(&self, path: &RemotePath) -> Result<(), RemoteError>;
|
|
|
|
// ---- optional ---------------------------------------------------------
|
|
|
|
/// Server-rendered thumbnail, where available.
|
|
///
|
|
/// `Ok(None)` means the server has no preview for this object — which is
|
|
/// common for RAW, since stock Nextcloud ships no RAW preview provider
|
|
/// (ARCH §6.7). Callers must have a local fallback.
|
|
async fn thumbnail(&self, _id: &RemoteId, _size: u32) -> Result<Option<Vec<u8>>, RemoteError> {
|
|
Ok(None)
|
|
}
|
|
}
|
|
|
|
/// TRACES: FR-NC-4 | FR-NC-12
|
|
/// Which sync strategy the engine should use for a backend.
|
|
///
|
|
/// Derived from capabilities at connect time. Reported to the user so a slow
|
|
/// backend is visibly slow rather than mysteriously slow (FR-NC-12).
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum SyncStrategy {
|
|
/// Ask the server what changed. Cheapest.
|
|
Delta,
|
|
/// Probe the root ETag; recurse only where it differs. One request when
|
|
/// nothing changed.
|
|
EtagPruning,
|
|
/// Enumerate the tree, using per-entry ETags to avoid re-downloading.
|
|
FullListing,
|
|
/// Enumerate and compare modification times. Degraded — clock skew and
|
|
/// second-granularity timestamps both cause misses.
|
|
TimestampCompare,
|
|
}
|
|
|
|
impl SyncStrategy {
|
|
pub fn for_capabilities(caps: &Capabilities) -> Self {
|
|
match caps.change_detection {
|
|
ChangeDetection::DeltaCursor => SyncStrategy::Delta,
|
|
ChangeDetection::PropagatingEtags => SyncStrategy::EtagPruning,
|
|
ChangeDetection::LocalEtags => SyncStrategy::FullListing,
|
|
ChangeDetection::Timestamps => SyncStrategy::TimestampCompare,
|
|
}
|
|
}
|
|
|
|
/// A short description for the UI.
|
|
pub fn describe(self) -> &'static str {
|
|
match self {
|
|
SyncStrategy::Delta => "server change feed",
|
|
SyncStrategy::EtagPruning => "incremental (ETag pruning)",
|
|
SyncStrategy::FullListing => "full listing, cached by ETag",
|
|
SyncStrategy::TimestampCompare => "full listing by timestamp (degraded)",
|
|
}
|
|
}
|
|
|
|
/// Whether this strategy can prove "nothing changed" cheaply.
|
|
///
|
|
/// Where false, every sync costs at least one request per folder, which
|
|
/// the UI should warn about on large libraries.
|
|
pub fn has_cheap_noop(self) -> bool {
|
|
matches!(self, SyncStrategy::Delta | SyncStrategy::EtagPruning)
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn strategy_follows_capability() {
|
|
let mut caps = Capabilities::minimal();
|
|
assert_eq!(
|
|
SyncStrategy::for_capabilities(&caps),
|
|
SyncStrategy::TimestampCompare
|
|
);
|
|
|
|
caps.change_detection = ChangeDetection::PropagatingEtags;
|
|
assert_eq!(
|
|
SyncStrategy::for_capabilities(&caps),
|
|
SyncStrategy::EtagPruning
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn only_delta_and_pruning_have_cheap_noop() {
|
|
assert!(SyncStrategy::Delta.has_cheap_noop());
|
|
assert!(SyncStrategy::EtagPruning.has_cheap_noop());
|
|
// These cost at least one request per folder, every time.
|
|
assert!(!SyncStrategy::FullListing.has_cheap_noop());
|
|
assert!(!SyncStrategy::TimestampCompare.has_cheap_noop());
|
|
}
|
|
}
|