The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none.
457 lines
16 KiB
Rust
457 lines
16 KiB
Rust
//! PROPFIND request bodies and multistatus parsing.
|
|
//!
|
|
//! Hand-rolled rather than using a WebDAV crate, because the properties that
|
|
//! matter here are Nextcloud extensions — `oc:fileid` for stable identity
|
|
//! (FR-NC-5) and `nc:has-preview` for the preview probe — and no general
|
|
//! client exposes them.
|
|
|
|
use dr_sync::{EntryKind, RemoteEntry, RemoteError, RemoteId, RemotePath, Validator};
|
|
use quick_xml::events::Event;
|
|
use quick_xml::Reader;
|
|
|
|
/// Body requesting every property the catalog stores.
|
|
///
|
|
/// One request per directory level, so asking for everything at once is
|
|
/// cheaper than a second round trip for metadata.
|
|
pub const PROPFIND_BODY: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
|
|
<d:propfind xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
|
|
<d:prop>
|
|
<d:getetag/>
|
|
<d:getlastmodified/>
|
|
<d:getcontentlength/>
|
|
<d:resourcetype/>
|
|
<oc:fileid/>
|
|
<oc:size/>
|
|
<nc:has-preview/>
|
|
</d:prop>
|
|
</d:propfind>"#;
|
|
|
|
/// Body requesting only the ETag.
|
|
///
|
|
/// The cheap probe behind ETag pruning: one of these against the root answers
|
|
/// "did anything change anywhere?" (ARCH §8.1). Asking for less matters —
|
|
/// this runs on every sync.
|
|
pub const PROPFIND_ETAG_ONLY: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
|
|
<d:propfind xmlns:d="DAV:">
|
|
<d:prop><d:getetag/></d:prop>
|
|
</d:propfind>"#;
|
|
|
|
/// One parsed `<d:response>` element.
|
|
#[derive(Debug, Default, Clone)]
|
|
struct Response {
|
|
href: String,
|
|
etag: Option<String>,
|
|
last_modified: Option<String>,
|
|
content_length: Option<u64>,
|
|
is_collection: bool,
|
|
file_id: Option<u64>,
|
|
has_preview: bool,
|
|
}
|
|
|
|
/// TRACES: FR-NC-4 | FR-NC-5 | M-5
|
|
/// Parse a multistatus body into entries.
|
|
///
|
|
/// `base` is the DAV prefix (`/remote.php/dav/files/<user>/`) stripped from
|
|
/// each href so paths are library-relative.
|
|
///
|
|
/// The entry for the requested directory itself is skipped: a `Depth: 1`
|
|
/// response includes it, and returning it would make every listing look like
|
|
/// it contains itself.
|
|
pub fn parse_multistatus(xml: &str, base: &str) -> Result<Vec<RemoteEntry>, RemoteError> {
|
|
let responses = parse_responses(xml)?;
|
|
let self_href = normalise(&percent_decode(
|
|
responses.first().map(|r| r.href.as_str()).unwrap_or(""),
|
|
));
|
|
|
|
let mut out = Vec::new();
|
|
for r in &responses {
|
|
let href = normalise(&percent_decode(&r.href));
|
|
if href == self_href {
|
|
continue;
|
|
}
|
|
let rel = href.strip_prefix(&normalise(base)).unwrap_or(&href);
|
|
let path = RemotePath::new(rel.trim_matches('/'));
|
|
|
|
// An entry with no ETag is unusable — nothing can detect its changes.
|
|
let Some(etag) = r.etag.as_ref() else {
|
|
continue;
|
|
};
|
|
|
|
out.push(RemoteEntry {
|
|
id: match r.file_id {
|
|
Some(id) => RemoteId::Stable(id),
|
|
None => RemoteId::Path(path.clone()),
|
|
},
|
|
path,
|
|
kind: if r.is_collection {
|
|
EntryKind::Directory
|
|
} else {
|
|
EntryKind::File
|
|
},
|
|
validator: Validator::new(etag),
|
|
size: r.content_length.unwrap_or(0),
|
|
modified: r.last_modified.as_deref().and_then(parse_http_date),
|
|
has_preview: r.has_preview,
|
|
// Everything WebDAV lists can be fetched; placeholders are a
|
|
// property of a locally synced folder, not of the server.
|
|
materialised: true,
|
|
});
|
|
}
|
|
Ok(out)
|
|
}
|
|
|
|
/// TRACES: FR-NC-4 | M-7
|
|
/// Extract the ETag of the *first* response — the requested resource itself.
|
|
///
|
|
/// Used by `dir_validator` for the pruning probe.
|
|
pub fn parse_self_etag(xml: &str) -> Result<Validator, RemoteError> {
|
|
let responses = parse_responses(xml)?;
|
|
responses
|
|
.first()
|
|
.and_then(|r| r.etag.as_ref())
|
|
.map(Validator::new)
|
|
.ok_or_else(|| RemoteError::Protocol("no ETag in multistatus".into()))
|
|
}
|
|
|
|
fn parse_responses(xml: &str) -> Result<Vec<Response>, RemoteError> {
|
|
let mut reader = Reader::from_str(xml);
|
|
reader.config_mut().trim_text(true);
|
|
|
|
let mut out: Vec<Response> = Vec::new();
|
|
let mut cur = Response::default();
|
|
let mut in_response = false;
|
|
let mut path: Vec<String> = Vec::new();
|
|
let mut text = String::new();
|
|
|
|
loop {
|
|
match reader.read_event() {
|
|
Ok(Event::Start(e)) => {
|
|
let name = local_name(e.name().as_ref());
|
|
path.push(name.clone());
|
|
match name.as_str() {
|
|
"response" => {
|
|
in_response = true;
|
|
cur = Response::default();
|
|
}
|
|
// resourcetype containing <collection/> marks a directory.
|
|
"collection" if in_response => cur.is_collection = true,
|
|
_ => {}
|
|
}
|
|
text.clear();
|
|
}
|
|
Ok(Event::Empty(e)) => {
|
|
// Self-closing <d:collection/> is the common spelling.
|
|
if local_name(e.name().as_ref()) == "collection" && in_response {
|
|
cur.is_collection = true;
|
|
}
|
|
}
|
|
Ok(Event::Text(t)) => {
|
|
// quick-xml 0.41 replaced `unescape` with `xml_content`,
|
|
// which both decodes bytes and resolves entity references.
|
|
// Nextcloud sends no XML declaration version, which is
|
|
// exactly the implicit-1.0 case.
|
|
text = t
|
|
.xml_content(quick_xml::XmlVersion::Implicit1_0)
|
|
.unwrap_or_default()
|
|
.to_string();
|
|
}
|
|
Ok(Event::End(e)) => {
|
|
let name = local_name(e.name().as_ref());
|
|
if in_response {
|
|
match name.as_str() {
|
|
"href" => cur.href = text.clone(),
|
|
"getetag" => cur.etag = Some(text.clone()),
|
|
"getlastmodified" => cur.last_modified = Some(text.clone()),
|
|
"getcontentlength" => cur.content_length = text.parse().ok(),
|
|
"fileid" => cur.file_id = text.parse().ok(),
|
|
"has-preview" => cur.has_preview = text.eq_ignore_ascii_case("true"),
|
|
"response" => {
|
|
in_response = false;
|
|
out.push(std::mem::take(&mut cur));
|
|
}
|
|
_ => {}
|
|
}
|
|
}
|
|
path.pop();
|
|
text.clear();
|
|
}
|
|
Ok(Event::Eof) => {
|
|
// A truncated document reaches EOF with elements still open.
|
|
// quick-xml's streaming reader does not treat that as an
|
|
// error, so check explicitly — a half-parsed multistatus must
|
|
// not be mistaken for an empty directory.
|
|
if !path.is_empty() {
|
|
return Err(RemoteError::Protocol(format!(
|
|
"unexpected end of XML with {} element(s) unclosed",
|
|
path.len()
|
|
)));
|
|
}
|
|
break;
|
|
}
|
|
Err(e) => return Err(RemoteError::Protocol(e.to_string())),
|
|
_ => {}
|
|
}
|
|
}
|
|
|
|
Ok(out)
|
|
}
|
|
|
|
/// Strip any XML namespace prefix: `d:getetag` → `getetag`.
|
|
///
|
|
/// Servers vary in the prefixes they use, so matching on the local name is the
|
|
/// only stable approach.
|
|
fn local_name(raw: &[u8]) -> String {
|
|
let s = String::from_utf8_lossy(raw);
|
|
s.rsplit(':').next().unwrap_or(&s).to_ascii_lowercase()
|
|
}
|
|
|
|
/// Collapse duplicate slashes and ensure a single trailing slash.
|
|
fn normalise(path: &str) -> String {
|
|
let mut out = String::with_capacity(path.len() + 1);
|
|
let mut last_slash = false;
|
|
for c in path.chars() {
|
|
if c == '/' {
|
|
if !last_slash {
|
|
out.push(c);
|
|
}
|
|
last_slash = true;
|
|
} else {
|
|
out.push(c);
|
|
last_slash = false;
|
|
}
|
|
}
|
|
if !out.ends_with('/') {
|
|
out.push('/');
|
|
}
|
|
out
|
|
}
|
|
|
|
/// Decode percent-escapes in an href.
|
|
///
|
|
/// Servers escape spaces and non-ASCII, so a raw href does not match a stored
|
|
/// path without this.
|
|
fn percent_decode(s: &str) -> String {
|
|
let b = s.as_bytes();
|
|
let mut out = Vec::with_capacity(b.len());
|
|
let mut i = 0;
|
|
while i < b.len() {
|
|
if b[i] == b'%' && i + 2 < b.len() {
|
|
if let Ok(v) = u8::from_str_radix(&s[i + 1..i + 3], 16) {
|
|
out.push(v);
|
|
i += 3;
|
|
continue;
|
|
}
|
|
}
|
|
out.push(b[i]);
|
|
i += 1;
|
|
}
|
|
String::from_utf8_lossy(&out).into_owned()
|
|
}
|
|
|
|
/// Parse an RFC 1123 date into Unix seconds.
|
|
///
|
|
/// Hand-rolled to avoid a chrono dependency for one field. Returns `None`
|
|
/// rather than guessing on an unparseable date — a wrong timestamp is worse
|
|
/// than a missing one, since it drives change detection on Tier 4 backends.
|
|
fn parse_http_date(s: &str) -> Option<i64> {
|
|
// "Wed, 09 Aug 2026 06:35:41 GMT"
|
|
let parts: Vec<&str> = s.split_whitespace().collect();
|
|
if parts.len() < 5 {
|
|
return None;
|
|
}
|
|
let day: i64 = parts[1].parse().ok()?;
|
|
let month = match parts[2] {
|
|
"Jan" => 1,
|
|
"Feb" => 2,
|
|
"Mar" => 3,
|
|
"Apr" => 4,
|
|
"May" => 5,
|
|
"Jun" => 6,
|
|
"Jul" => 7,
|
|
"Aug" => 8,
|
|
"Sep" => 9,
|
|
"Oct" => 10,
|
|
"Nov" => 11,
|
|
"Dec" => 12,
|
|
_ => return None,
|
|
};
|
|
let year: i64 = parts[3].parse().ok()?;
|
|
let hms: Vec<&str> = parts[4].split(':').collect();
|
|
if hms.len() != 3 {
|
|
return None;
|
|
}
|
|
let (h, m, sec): (i64, i64, i64) = (
|
|
hms[0].parse().ok()?,
|
|
hms[1].parse().ok()?,
|
|
hms[2].parse().ok()?,
|
|
);
|
|
|
|
// Days since the Unix epoch, via a civil-from-days algorithm.
|
|
let y = if month <= 2 { year - 1 } else { year };
|
|
let era = if y >= 0 { y } else { y - 399 } / 400;
|
|
let yoe = y - era * 400;
|
|
let mp = (month + 9) % 12;
|
|
let doy = (153 * mp + 2) / 5 + day - 1;
|
|
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
|
|
let days = era * 146_097 + doe - 719_468;
|
|
|
|
Some(days * 86_400 + h * 3_600 + m * 60 + sec)
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
const BASE: &str = "/remote.php/dav/files/duncan/";
|
|
|
|
/// A realistic Nextcloud Depth:1 response: the directory itself, a
|
|
/// subdirectory, and a file.
|
|
const SAMPLE: &str = r#"<?xml version="1.0"?>
|
|
<d:multistatus xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
|
|
<d:response>
|
|
<d:href>/remote.php/dav/files/duncan/Photos/</d:href>
|
|
<d:propstat>
|
|
<d:prop>
|
|
<d:getetag>"abc123"</d:getetag>
|
|
<d:resourcetype><d:collection/></d:resourcetype>
|
|
<oc:fileid>1001</oc:fileid>
|
|
</d:prop>
|
|
<d:status>HTTP/1.1 200 OK</d:status>
|
|
</d:propstat>
|
|
</d:response>
|
|
<d:response>
|
|
<d:href>/remote.php/dav/files/duncan/Photos/2026/</d:href>
|
|
<d:propstat>
|
|
<d:prop>
|
|
<d:getetag>"def456"</d:getetag>
|
|
<d:resourcetype><d:collection/></d:resourcetype>
|
|
<oc:fileid>1002</oc:fileid>
|
|
</d:prop>
|
|
</d:propstat>
|
|
</d:response>
|
|
<d:response>
|
|
<d:href>/remote.php/dav/files/duncan/Photos/_MG_4130.CR2</d:href>
|
|
<d:propstat>
|
|
<d:prop>
|
|
<d:getetag>"ghi789"</d:getetag>
|
|
<d:getlastmodified>Wed, 09 Aug 2026 06:35:41 GMT</d:getlastmodified>
|
|
<d:getcontentlength>34489068</d:getcontentlength>
|
|
<d:resourcetype/>
|
|
<oc:fileid>1003</oc:fileid>
|
|
<nc:has-preview>false</nc:has-preview>
|
|
</d:prop>
|
|
</d:propstat>
|
|
</d:response>
|
|
</d:multistatus>"#;
|
|
|
|
#[test]
|
|
fn parses_entries_and_skips_the_directory_itself() {
|
|
let entries = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
// Three responses, but the first describes the requested directory.
|
|
assert_eq!(entries.len(), 2);
|
|
assert_eq!(entries[0].path.as_str(), "Photos/2026");
|
|
assert_eq!(entries[1].path.as_str(), "Photos/_MG_4130.CR2");
|
|
}
|
|
|
|
#[test]
|
|
fn distinguishes_directories_from_files() {
|
|
let e = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
assert_eq!(e[0].kind, EntryKind::Directory);
|
|
assert_eq!(e[1].kind, EntryKind::File);
|
|
}
|
|
|
|
#[test]
|
|
fn uses_fileid_as_stable_identity() {
|
|
// FR-NC-5: a server-side move must not look like delete-plus-add of a
|
|
// 34 MB file.
|
|
let e = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
assert_eq!(e[1].id, RemoteId::Stable(1003));
|
|
}
|
|
|
|
#[test]
|
|
fn normalises_etag_quoting() {
|
|
// Nextcloud quotes ETags; comparing raw strings against an unquoted
|
|
// stored value would make every sync look like a change.
|
|
let e = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
assert_eq!(e[1].validator.as_str(), "ghi789");
|
|
}
|
|
|
|
#[test]
|
|
fn reads_size_and_preview_flag() {
|
|
let e = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
assert_eq!(e[1].size, 34_489_068);
|
|
// Stock Nextcloud reports no preview for RAW (ARCH §6.7).
|
|
assert!(!e[1].has_preview);
|
|
}
|
|
|
|
#[test]
|
|
fn parses_last_modified() {
|
|
let e = parse_multistatus(SAMPLE, BASE).unwrap();
|
|
// 2026-08-09T06:35:41Z, verified against a reference implementation
|
|
// rather than computed by hand.
|
|
assert_eq!(e[1].modified, Some(1_786_257_341));
|
|
}
|
|
|
|
#[test]
|
|
fn extracts_the_self_etag_for_pruning() {
|
|
// The one-request probe that proves a library unchanged.
|
|
assert_eq!(parse_self_etag(SAMPLE).unwrap().as_str(), "abc123");
|
|
}
|
|
|
|
#[test]
|
|
fn handles_percent_encoded_hrefs() {
|
|
let xml = r#"<d:multistatus xmlns:d="DAV:">
|
|
<d:response><d:href>/remote.php/dav/files/duncan/Photos/</d:href>
|
|
<d:propstat><d:prop><d:getetag>"a"</d:getetag>
|
|
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
|
|
<d:response><d:href>/remote.php/dav/files/duncan/Photos/My%20Trip%202026/</d:href>
|
|
<d:propstat><d:prop><d:getetag>"b"</d:getetag>
|
|
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
|
|
</d:multistatus>"#;
|
|
let e = parse_multistatus(xml, BASE).unwrap();
|
|
assert_eq!(e[0].path.as_str(), "Photos/My Trip 2026");
|
|
}
|
|
|
|
#[test]
|
|
fn tolerates_unknown_namespace_prefixes() {
|
|
// Prefixes are arbitrary; matching must be on the local name.
|
|
let xml = r#"<multistatus xmlns="DAV:">
|
|
<response><href>/remote.php/dav/files/duncan/x/</href>
|
|
<propstat><prop><getetag>"root"</getetag>
|
|
<resourcetype><collection/></resourcetype></prop></propstat></response>
|
|
<response><href>/remote.php/dav/files/duncan/x/a.jpg</href>
|
|
<propstat><prop><getetag>"one"</getetag><resourcetype/></prop></propstat></response>
|
|
</multistatus>"#;
|
|
let e = parse_multistatus(xml, BASE).unwrap();
|
|
assert_eq!(e.len(), 1);
|
|
assert_eq!(e[0].path.as_str(), "x/a.jpg");
|
|
}
|
|
|
|
#[test]
|
|
fn entries_without_an_etag_are_dropped() {
|
|
// Nothing can detect changes to them, so cataloguing one would create
|
|
// a row that never syncs correctly.
|
|
let xml = r#"<d:multistatus xmlns:d="DAV:">
|
|
<d:response><d:href>/remote.php/dav/files/duncan/x/</d:href>
|
|
<d:propstat><d:prop><d:getetag>"root"</d:getetag>
|
|
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
|
|
<d:response><d:href>/remote.php/dav/files/duncan/x/broken.jpg</d:href>
|
|
<d:propstat><d:prop><d:resourcetype/></d:prop>
|
|
<d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response>
|
|
</d:multistatus>"#;
|
|
assert!(parse_multistatus(xml, BASE).unwrap().is_empty());
|
|
}
|
|
|
|
#[test]
|
|
fn malformed_xml_is_an_error_not_a_panic() {
|
|
assert!(parse_multistatus("<d:multistatus><unclosed>", BASE).is_err());
|
|
}
|
|
|
|
#[test]
|
|
fn http_dates_parse_or_return_none() {
|
|
assert_eq!(parse_http_date("Thu, 01 Jan 1970 00:00:00 GMT"), Some(0));
|
|
assert_eq!(parse_http_date("not a date"), None);
|
|
assert_eq!(parse_http_date(""), None);
|
|
}
|
|
}
|