Files
DarkRoom/core/dr-sync-nextcloud/src/propfind.rs
T
dtourolle c102ba9df2 Treat a placeholder as the photograph, not as a one-byte file
The folder connector was pointed at a Nextcloud VFS tree and got three
things wrong, the first of which loses work.

**A dehydrated sidecar read as absent.** `a.drsc` does not exist when the
client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed,
`.ok()` swallowed the `NotFound`, and the sidecar writer took that for
"there is no sidecar yet" and wrote a fresh document over the existing
one. Every edit another device had put there went with it. That function's
own doc comment calls this the exact loss the format's unknown-key
preservation exists to prevent.

**A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this
machine at 121,785 placeholders against 10,267 real files — so a folder
library on a synced tree was ~92% broken rows.

**Identity changed on hydration**, so downloading a photograph looked like
a delete and an add, orphaning its thumbnail and its face rows.

Entries now carry the photograph's own name and a `materialised` flag;
`get` on a stub returns the new `RemoteError::NotMaterialised`, which is
distinct from `NotFound` precisely because the sidecar writer must treat
them differently — it fetches the sidecar and merges, or leaves the entry
queued.

Hydration is a **borrow**. `BorrowPool` records what was on disk before it
asked, so `release_all` dehydrates only what a pass brought and leaves
what the user already had. Reference counted: the thumbnail pass and the
face pass meet on the same RAW, and without counting the first to finish
dehydrates the file the second is reading. A borrow against a plain folder
or a server does nothing, so a pass written for VFS runs everywhere.

Releasing means asking the client to dehydrate and never deleting: a
deletion inside a synced tree propagates to the server and removes the
photograph from every device.

Not a second backend — the capability is per *connection*, not per type,
since the same folder hydrates only while the client runs. The convention
arrives through a detector the registry supplies, so `dr-sync-folder`
still knows nothing about any client's protocol.

ARCH §9.0a records this as an amendment: finding 3 rejected hydration
because it costs 100× a range read, and that comparison assumed a
connector was available. A folder library has none.
2026-08-29 09:57:52 +02:00

457 lines
16 KiB
Rust

//! PROPFIND request bodies and multistatus parsing.
//!
//! Hand-rolled rather than using a WebDAV crate, because the properties that
//! matter here are Nextcloud extensions — `oc:fileid` for stable identity
//! (FR-NC-5) and `nc:has-preview` for the preview probe — and no general
//! client exposes them.
use dr_sync::{EntryKind, RemoteEntry, RemoteError, RemoteId, RemotePath, Validator};
use quick_xml::events::Event;
use quick_xml::Reader;
/// Body requesting every property the catalog stores.
///
/// One request per directory level, so asking for everything at once is
/// cheaper than a second round trip for metadata.
pub const PROPFIND_BODY: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:prop>
<d:getetag/>
<d:getlastmodified/>
<d:getcontentlength/>
<d:resourcetype/>
<oc:fileid/>
<oc:size/>
<nc:has-preview/>
</d:prop>
</d:propfind>"#;
/// Body requesting only the ETag.
///
/// The cheap probe behind ETag pruning: one of these against the root answers
/// "did anything change anywhere?" (ARCH §8.1). Asking for less matters —
/// this runs on every sync.
pub const PROPFIND_ETAG_ONLY: &str = r#"<?xml version="1.0" encoding="UTF-8"?>
<d:propfind xmlns:d="DAV:">
<d:prop><d:getetag/></d:prop>
</d:propfind>"#;
/// One parsed `<d:response>` element.
#[derive(Debug, Default, Clone)]
struct Response {
href: String,
etag: Option<String>,
last_modified: Option<String>,
content_length: Option<u64>,
is_collection: bool,
file_id: Option<u64>,
has_preview: bool,
}
/// TRACES: FR-NC-4 | FR-NC-5 | M-5
/// Parse a multistatus body into entries.
///
/// `base` is the DAV prefix (`/remote.php/dav/files/<user>/`) stripped from
/// each href so paths are library-relative.
///
/// The entry for the requested directory itself is skipped: a `Depth: 1`
/// response includes it, and returning it would make every listing look like
/// it contains itself.
pub fn parse_multistatus(xml: &str, base: &str) -> Result<Vec<RemoteEntry>, RemoteError> {
let responses = parse_responses(xml)?;
let self_href = normalise(&percent_decode(
responses.first().map(|r| r.href.as_str()).unwrap_or(""),
));
let mut out = Vec::new();
for r in &responses {
let href = normalise(&percent_decode(&r.href));
if href == self_href {
continue;
}
let rel = href.strip_prefix(&normalise(base)).unwrap_or(&href);
let path = RemotePath::new(rel.trim_matches('/'));
// An entry with no ETag is unusable — nothing can detect its changes.
let Some(etag) = r.etag.as_ref() else {
continue;
};
out.push(RemoteEntry {
id: match r.file_id {
Some(id) => RemoteId::Stable(id),
None => RemoteId::Path(path.clone()),
},
path,
kind: if r.is_collection {
EntryKind::Directory
} else {
EntryKind::File
},
validator: Validator::new(etag),
size: r.content_length.unwrap_or(0),
modified: r.last_modified.as_deref().and_then(parse_http_date),
has_preview: r.has_preview,
// Everything WebDAV lists can be fetched; placeholders are a
// property of a locally synced folder, not of the server.
materialised: true,
});
}
Ok(out)
}
/// TRACES: FR-NC-4 | M-7
/// Extract the ETag of the *first* response — the requested resource itself.
///
/// Used by `dir_validator` for the pruning probe.
pub fn parse_self_etag(xml: &str) -> Result<Validator, RemoteError> {
let responses = parse_responses(xml)?;
responses
.first()
.and_then(|r| r.etag.as_ref())
.map(Validator::new)
.ok_or_else(|| RemoteError::Protocol("no ETag in multistatus".into()))
}
fn parse_responses(xml: &str) -> Result<Vec<Response>, RemoteError> {
let mut reader = Reader::from_str(xml);
reader.config_mut().trim_text(true);
let mut out: Vec<Response> = Vec::new();
let mut cur = Response::default();
let mut in_response = false;
let mut path: Vec<String> = Vec::new();
let mut text = String::new();
loop {
match reader.read_event() {
Ok(Event::Start(e)) => {
let name = local_name(e.name().as_ref());
path.push(name.clone());
match name.as_str() {
"response" => {
in_response = true;
cur = Response::default();
}
// resourcetype containing <collection/> marks a directory.
"collection" if in_response => cur.is_collection = true,
_ => {}
}
text.clear();
}
Ok(Event::Empty(e)) => {
// Self-closing <d:collection/> is the common spelling.
if local_name(e.name().as_ref()) == "collection" && in_response {
cur.is_collection = true;
}
}
Ok(Event::Text(t)) => {
// quick-xml 0.41 replaced `unescape` with `xml_content`,
// which both decodes bytes and resolves entity references.
// Nextcloud sends no XML declaration version, which is
// exactly the implicit-1.0 case.
text = t
.xml_content(quick_xml::XmlVersion::Implicit1_0)
.unwrap_or_default()
.to_string();
}
Ok(Event::End(e)) => {
let name = local_name(e.name().as_ref());
if in_response {
match name.as_str() {
"href" => cur.href = text.clone(),
"getetag" => cur.etag = Some(text.clone()),
"getlastmodified" => cur.last_modified = Some(text.clone()),
"getcontentlength" => cur.content_length = text.parse().ok(),
"fileid" => cur.file_id = text.parse().ok(),
"has-preview" => cur.has_preview = text.eq_ignore_ascii_case("true"),
"response" => {
in_response = false;
out.push(std::mem::take(&mut cur));
}
_ => {}
}
}
path.pop();
text.clear();
}
Ok(Event::Eof) => {
// A truncated document reaches EOF with elements still open.
// quick-xml's streaming reader does not treat that as an
// error, so check explicitly — a half-parsed multistatus must
// not be mistaken for an empty directory.
if !path.is_empty() {
return Err(RemoteError::Protocol(format!(
"unexpected end of XML with {} element(s) unclosed",
path.len()
)));
}
break;
}
Err(e) => return Err(RemoteError::Protocol(e.to_string())),
_ => {}
}
}
Ok(out)
}
/// Strip any XML namespace prefix: `d:getetag` → `getetag`.
///
/// Servers vary in the prefixes they use, so matching on the local name is the
/// only stable approach.
fn local_name(raw: &[u8]) -> String {
let s = String::from_utf8_lossy(raw);
s.rsplit(':').next().unwrap_or(&s).to_ascii_lowercase()
}
/// Collapse duplicate slashes and ensure a single trailing slash.
fn normalise(path: &str) -> String {
let mut out = String::with_capacity(path.len() + 1);
let mut last_slash = false;
for c in path.chars() {
if c == '/' {
if !last_slash {
out.push(c);
}
last_slash = true;
} else {
out.push(c);
last_slash = false;
}
}
if !out.ends_with('/') {
out.push('/');
}
out
}
/// Decode percent-escapes in an href.
///
/// Servers escape spaces and non-ASCII, so a raw href does not match a stored
/// path without this.
fn percent_decode(s: &str) -> String {
let b = s.as_bytes();
let mut out = Vec::with_capacity(b.len());
let mut i = 0;
while i < b.len() {
if b[i] == b'%' && i + 2 < b.len() {
if let Ok(v) = u8::from_str_radix(&s[i + 1..i + 3], 16) {
out.push(v);
i += 3;
continue;
}
}
out.push(b[i]);
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
/// Parse an RFC 1123 date into Unix seconds.
///
/// Hand-rolled to avoid a chrono dependency for one field. Returns `None`
/// rather than guessing on an unparseable date — a wrong timestamp is worse
/// than a missing one, since it drives change detection on Tier 4 backends.
fn parse_http_date(s: &str) -> Option<i64> {
// "Wed, 09 Aug 2026 06:35:41 GMT"
let parts: Vec<&str> = s.split_whitespace().collect();
if parts.len() < 5 {
return None;
}
let day: i64 = parts[1].parse().ok()?;
let month = match parts[2] {
"Jan" => 1,
"Feb" => 2,
"Mar" => 3,
"Apr" => 4,
"May" => 5,
"Jun" => 6,
"Jul" => 7,
"Aug" => 8,
"Sep" => 9,
"Oct" => 10,
"Nov" => 11,
"Dec" => 12,
_ => return None,
};
let year: i64 = parts[3].parse().ok()?;
let hms: Vec<&str> = parts[4].split(':').collect();
if hms.len() != 3 {
return None;
}
let (h, m, sec): (i64, i64, i64) = (
hms[0].parse().ok()?,
hms[1].parse().ok()?,
hms[2].parse().ok()?,
);
// Days since the Unix epoch, via a civil-from-days algorithm.
let y = if month <= 2 { year - 1 } else { year };
let era = if y >= 0 { y } else { y - 399 } / 400;
let yoe = y - era * 400;
let mp = (month + 9) % 12;
let doy = (153 * mp + 2) / 5 + day - 1;
let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy;
let days = era * 146_097 + doe - 719_468;
Some(days * 86_400 + h * 3_600 + m * 60 + sec)
}
#[cfg(test)]
mod tests {
use super::*;
const BASE: &str = "/remote.php/dav/files/duncan/";
/// A realistic Nextcloud Depth:1 response: the directory itself, a
/// subdirectory, and a file.
const SAMPLE: &str = r#"<?xml version="1.0"?>
<d:multistatus xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns" xmlns:nc="http://nextcloud.org/ns">
<d:response>
<d:href>/remote.php/dav/files/duncan/Photos/</d:href>
<d:propstat>
<d:prop>
<d:getetag>"abc123"</d:getetag>
<d:resourcetype><d:collection/></d:resourcetype>
<oc:fileid>1001</oc:fileid>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<d:response>
<d:href>/remote.php/dav/files/duncan/Photos/2026/</d:href>
<d:propstat>
<d:prop>
<d:getetag>"def456"</d:getetag>
<d:resourcetype><d:collection/></d:resourcetype>
<oc:fileid>1002</oc:fileid>
</d:prop>
</d:propstat>
</d:response>
<d:response>
<d:href>/remote.php/dav/files/duncan/Photos/_MG_4130.CR2</d:href>
<d:propstat>
<d:prop>
<d:getetag>"ghi789"</d:getetag>
<d:getlastmodified>Wed, 09 Aug 2026 06:35:41 GMT</d:getlastmodified>
<d:getcontentlength>34489068</d:getcontentlength>
<d:resourcetype/>
<oc:fileid>1003</oc:fileid>
<nc:has-preview>false</nc:has-preview>
</d:prop>
</d:propstat>
</d:response>
</d:multistatus>"#;
#[test]
fn parses_entries_and_skips_the_directory_itself() {
let entries = parse_multistatus(SAMPLE, BASE).unwrap();
// Three responses, but the first describes the requested directory.
assert_eq!(entries.len(), 2);
assert_eq!(entries[0].path.as_str(), "Photos/2026");
assert_eq!(entries[1].path.as_str(), "Photos/_MG_4130.CR2");
}
#[test]
fn distinguishes_directories_from_files() {
let e = parse_multistatus(SAMPLE, BASE).unwrap();
assert_eq!(e[0].kind, EntryKind::Directory);
assert_eq!(e[1].kind, EntryKind::File);
}
#[test]
fn uses_fileid_as_stable_identity() {
// FR-NC-5: a server-side move must not look like delete-plus-add of a
// 34 MB file.
let e = parse_multistatus(SAMPLE, BASE).unwrap();
assert_eq!(e[1].id, RemoteId::Stable(1003));
}
#[test]
fn normalises_etag_quoting() {
// Nextcloud quotes ETags; comparing raw strings against an unquoted
// stored value would make every sync look like a change.
let e = parse_multistatus(SAMPLE, BASE).unwrap();
assert_eq!(e[1].validator.as_str(), "ghi789");
}
#[test]
fn reads_size_and_preview_flag() {
let e = parse_multistatus(SAMPLE, BASE).unwrap();
assert_eq!(e[1].size, 34_489_068);
// Stock Nextcloud reports no preview for RAW (ARCH §6.7).
assert!(!e[1].has_preview);
}
#[test]
fn parses_last_modified() {
let e = parse_multistatus(SAMPLE, BASE).unwrap();
// 2026-08-09T06:35:41Z, verified against a reference implementation
// rather than computed by hand.
assert_eq!(e[1].modified, Some(1_786_257_341));
}
#[test]
fn extracts_the_self_etag_for_pruning() {
// The one-request probe that proves a library unchanged.
assert_eq!(parse_self_etag(SAMPLE).unwrap().as_str(), "abc123");
}
#[test]
fn handles_percent_encoded_hrefs() {
let xml = r#"<d:multistatus xmlns:d="DAV:">
<d:response><d:href>/remote.php/dav/files/duncan/Photos/</d:href>
<d:propstat><d:prop><d:getetag>"a"</d:getetag>
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
<d:response><d:href>/remote.php/dav/files/duncan/Photos/My%20Trip%202026/</d:href>
<d:propstat><d:prop><d:getetag>"b"</d:getetag>
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
</d:multistatus>"#;
let e = parse_multistatus(xml, BASE).unwrap();
assert_eq!(e[0].path.as_str(), "Photos/My Trip 2026");
}
#[test]
fn tolerates_unknown_namespace_prefixes() {
// Prefixes are arbitrary; matching must be on the local name.
let xml = r#"<multistatus xmlns="DAV:">
<response><href>/remote.php/dav/files/duncan/x/</href>
<propstat><prop><getetag>"root"</getetag>
<resourcetype><collection/></resourcetype></prop></propstat></response>
<response><href>/remote.php/dav/files/duncan/x/a.jpg</href>
<propstat><prop><getetag>"one"</getetag><resourcetype/></prop></propstat></response>
</multistatus>"#;
let e = parse_multistatus(xml, BASE).unwrap();
assert_eq!(e.len(), 1);
assert_eq!(e[0].path.as_str(), "x/a.jpg");
}
#[test]
fn entries_without_an_etag_are_dropped() {
// Nothing can detect changes to them, so cataloguing one would create
// a row that never syncs correctly.
let xml = r#"<d:multistatus xmlns:d="DAV:">
<d:response><d:href>/remote.php/dav/files/duncan/x/</d:href>
<d:propstat><d:prop><d:getetag>"root"</d:getetag>
<d:resourcetype><d:collection/></d:resourcetype></d:prop></d:propstat></d:response>
<d:response><d:href>/remote.php/dav/files/duncan/x/broken.jpg</d:href>
<d:propstat><d:prop><d:resourcetype/></d:prop>
<d:status>HTTP/1.1 404 Not Found</d:status></d:propstat></d:response>
</d:multistatus>"#;
assert!(parse_multistatus(xml, BASE).unwrap().is_empty());
}
#[test]
fn malformed_xml_is_an_error_not_a_panic() {
assert!(parse_multistatus("<d:multistatus><unclosed>", BASE).is_err());
}
#[test]
fn http_dates_parse_or_return_none() {
assert_eq!(parse_http_date("Thu, 01 Jan 1970 00:00:00 GMT"), Some(0));
assert_eq!(parse_http_date("not a date"), None);
assert_eq!(parse_http_date(""), None);
}
}