The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none.
189 lines
6.8 KiB
Rust
189 lines
6.8 KiB
Rust
//! What a backend can do.
|
|
//!
|
|
//! Declared rather than assumed, because the operations that matter most for
|
|
//! performance are not universal (ARCH §8.1).
|
|
|
|
/// TRACES: FR-NC-4
|
|
/// How a backend reports what changed.
|
|
///
|
|
/// This is the single most consequential capability: it determines whether a
|
|
/// no-op sync over a large library costs one request or thousands.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum ChangeDetection {
|
|
/// The backend maintains a change feed; we present a cursor and receive
|
|
/// what changed since. Cheapest possible.
|
|
DeltaCursor,
|
|
|
|
/// Directory ETags propagate upward, so an unchanged parent proves an
|
|
/// unchanged subtree. Nextcloud. One request proves a whole library
|
|
/// unchanged.
|
|
PropagatingEtags,
|
|
|
|
/// ETags exist per entry but do not propagate. The tree must be walked,
|
|
/// though ETags still prevent re-downloading unchanged content.
|
|
LocalEtags,
|
|
|
|
/// Modification times only. Walk and compare — vulnerable to clock skew
|
|
/// and coarse timestamp granularity.
|
|
Timestamps,
|
|
}
|
|
|
|
/// Constraints on chunked upload.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub struct ChunkConstraints {
|
|
pub min_chunk: u64,
|
|
pub max_chunk: u64,
|
|
/// Bodies at or below this go in a single request.
|
|
pub single_shot_below: u64,
|
|
pub max_chunks: u32,
|
|
}
|
|
|
|
/// TRACES: FR-NC-6c
|
|
/// Whether every listed object's content is actually reachable.
|
|
///
|
|
/// Every backend but a virtual-filesystem folder answers [`Always`](Self::Always).
|
|
/// A VFS folder is the case this exists for: the sync client leaves a
|
|
/// placeholder where a file is catalogued but not downloaded, so the name is
|
|
/// listable and the bytes are not (ARCH §9.0).
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum Materialisation {
|
|
/// Listing an object means its content can be read. Every server backend,
|
|
/// and a plain directory.
|
|
Always,
|
|
|
|
/// Some objects are placeholders, and nothing this process can do will
|
|
/// change that — the sync client is not running, or the platform offers no
|
|
/// way to ask. Such an object reads as
|
|
/// [`RemoteError::NotMaterialised`](crate::RemoteError::NotMaterialised)
|
|
/// and is shown as offline rather than broken.
|
|
Placeholders,
|
|
|
|
/// Some objects are placeholders, and this backend can ask for their
|
|
/// content — and give it back.
|
|
///
|
|
/// **Whole-file, and that is the whole difficulty.** Hydration has two
|
|
/// states, one byte or all bytes, so using it to fill a grid transfers the
|
|
/// entire library to produce thumbnails (ARCH §9.0). It belongs to the
|
|
/// originals tier — an image opened in develop, exported, or deliberately
|
|
/// pinned — and to passes the user has asked for and been quoted a price
|
|
/// on. Never to browsing.
|
|
OnDemand,
|
|
}
|
|
|
|
impl Materialisation {
|
|
/// Whether content can be fetched on request.
|
|
pub fn can_materialise(self) -> bool {
|
|
matches!(self, Materialisation::OnDemand)
|
|
}
|
|
|
|
/// Whether some objects may have no content locally.
|
|
pub fn has_placeholders(self) -> bool {
|
|
!matches!(self, Materialisation::Always)
|
|
}
|
|
}
|
|
|
|
/// TRACES: FR-NC-3
|
|
/// Whether the server can render thumbnails, and for what.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub enum ServerPreviews {
|
|
/// No server-side rendering.
|
|
None,
|
|
/// Common web formats only. **This is stock Nextcloud** — no RAW preview
|
|
/// provider ships with it, so RAW thumbnails must come from local
|
|
/// embedded-preview extraction (ARCH §6.7).
|
|
CommonFormatsOnly,
|
|
/// RAW included, e.g. via the `camerarawpreviews` app or an Imaginary
|
|
/// backend. Detected per-account, never assumed.
|
|
IncludingRaw,
|
|
}
|
|
|
|
/// The full capability set.
|
|
#[derive(Debug, Clone)]
|
|
pub struct Capabilities {
|
|
pub change_detection: ChangeDetection,
|
|
/// Identity survives server-side rename and move, so a move is not
|
|
/// mistaken for delete-plus-add of a large file.
|
|
pub stable_ids: bool,
|
|
/// Byte-range reads. Without these, embedded-preview extraction is
|
|
/// impossible and remote browsing must download whole files.
|
|
pub range_reads: bool,
|
|
pub chunked_upload: Option<ChunkConstraints>,
|
|
/// Many small objects in one request.
|
|
pub bulk_upload: bool,
|
|
/// Conditional write (If-Match), for conflict-safe sidecar updates.
|
|
pub conditional_write: bool,
|
|
pub server_previews: ServerPreviews,
|
|
/// Whether a listed object's content is necessarily present.
|
|
pub materialisation: Materialisation,
|
|
}
|
|
|
|
impl Capabilities {
|
|
/// The weakest backend the engine will still drive: listing and whole-file
|
|
/// transfer, nothing more. Everything degrades but stays correct.
|
|
pub fn minimal() -> Self {
|
|
Self {
|
|
change_detection: ChangeDetection::Timestamps,
|
|
stable_ids: false,
|
|
range_reads: false,
|
|
chunked_upload: None,
|
|
bulk_upload: false,
|
|
conditional_write: false,
|
|
server_previews: ServerPreviews::None,
|
|
// The weakest backend still answers for everything it lists;
|
|
// placeholders are a property a backend opts into.
|
|
materialisation: Materialisation::Always,
|
|
}
|
|
}
|
|
|
|
/// Whether remote browsing can avoid downloading whole files.
|
|
///
|
|
/// Where false, the UI must not browse a remote library on a metered
|
|
/// connection without explicit consent (FR-NC-12).
|
|
pub fn can_browse_cheaply(&self) -> bool {
|
|
self.range_reads || matches!(self.server_previews, ServerPreviews::IncludingRaw)
|
|
}
|
|
|
|
/// Whether sidecar conflicts can be detected reliably.
|
|
///
|
|
/// Without conditional writes the engine falls back to comparing revision
|
|
/// counters inside the sidecar, which narrows the race but does not close
|
|
/// it — surfaced as a reduced-safety mode.
|
|
pub fn safe_concurrent_writes(&self) -> bool {
|
|
self.conditional_write
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn minimal_backend_degrades_but_stays_usable() {
|
|
let caps = Capabilities::minimal();
|
|
assert!(!caps.can_browse_cheaply());
|
|
assert!(!caps.safe_concurrent_writes());
|
|
}
|
|
|
|
#[test]
|
|
fn server_raw_previews_substitute_for_range_reads() {
|
|
// A server that renders RAW thumbnails makes browsing cheap even
|
|
// without range support.
|
|
let caps = Capabilities {
|
|
server_previews: ServerPreviews::IncludingRaw,
|
|
..Capabilities::minimal()
|
|
};
|
|
assert!(caps.can_browse_cheaply());
|
|
}
|
|
|
|
#[test]
|
|
fn common_format_previews_do_not_help_raw() {
|
|
// Stock Nextcloud: previews exist, but not for RAW, so range reads
|
|
// remain the only cheap path.
|
|
let caps = Capabilities {
|
|
server_previews: ServerPreviews::CommonFormatsOnly,
|
|
..Capabilities::minimal()
|
|
};
|
|
assert!(!caps.can_browse_cheaply());
|
|
}
|
|
}
|