Do not push a catalog over one we could not read

`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped

    if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {

which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.

The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.

Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.

And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
This commit is contained in:
2026-08-29 10:36:26 +02:00
parent 702d83c218
commit 4168d67cfa
5 changed files with 190 additions and 26 deletions
+42 -9
View File
@@ -588,15 +588,36 @@ impl RemoteBackend for FolderBackend {
body: Vec<u8>,
precond: Option<Precondition>,
) -> Result<Validator, RemoteError> {
let (local, materialised) = self.locate(path)?;
if !materialised {
// Writing `a.drsc` while `a.drsc.nextcloud` sits beside it creates
// two files for one document and hands the sync client a conflict
// to resolve — in favour of whichever it sees last. The caller
// must materialise it and merge, which is what the typed error is
// for.
return Err(RemoteError::NotMaterialised(local.display().to_string()));
}
let (found, materialised) = self.locate(path)?;
// Where the content belongs, which is not where a placeholder for it
// sits — suffix-mode VFS gives the two different names.
let local = self.resolve(path)?;
// A stub is still this file, so what to do about it depends entirely
// on what the caller is promising.
let replaces = if materialised {
None
} else {
match &precond {
// Nothing here can satisfy it: the validator on a placeholder
// describes the placeholder. The caller fetches the content
// and tries again, which is what the typed error asks for.
Some(Precondition::IfMatch(_)) => {
return Err(RemoteError::NotMaterialised(found.display().to_string()))
}
// Something *is* there — the file exists, only its content is
// elsewhere — so a create-if-absent must fail.
Some(Precondition::IfAbsent) => return Err(RemoteError::PreconditionFailed),
// An unconditional write replaces the whole file, so there is
// nothing in the stub worth reading and no reason to download
// it first. Refusing here instead was a mistake: derived state
// lives in the library folder and the client dehydrates it
// like anything else, so a refusal meant sync could never
// write to a folder it had been away from.
None => Some(found),
}
};
blocking(move || {
let what = local.display().to_string();
if let Some(parent) = local.parent() {
@@ -658,6 +679,18 @@ impl RemoteBackend for FolderBackend {
return Err(map_io(e, &what));
}
// The stub goes only once the content is safely in place. The
// other order risks leaving neither, and in a synced tree an
// absence is a deletion the client would propagate.
if let Some(stub) = replaces {
if let Err(e) = std::fs::remove_file(&stub) {
// The content landed, so the write succeeded; a leftover
// placeholder beside it is untidy rather than harmful, and
// the client reconciles the pair on its next pass.
log::warn!("removing placeholder {}: {e}", stub.display());
}
}
let meta = std::fs::metadata(&local).map_err(|e| map_io(e, &what))?;
Ok(validator_of(&meta))
})
+46 -5
View File
@@ -630,19 +630,60 @@ async fn reading_a_placeholder_is_distinguishable_from_a_missing_file() {
}
#[tokio::test]
async fn writing_over_a_placeholder_is_refused() {
// Writing `a.drsc` beside `a.drsc.stub` makes two files for one document
// and hands the sync client a conflict it resolves arbitrarily.
async fn an_unconditional_write_replaces_a_placeholder() {
// Derived state — shards, the catalog snapshot — lives in the library
// folder, so the client dehydrates it like anything else. Refusing here
// meant sync could never write to a folder it had been away from. The
// whole file is being replaced, so there is nothing in the stub to keep.
let t = Tmp::new("vfs-write");
t.file("a.drsc.stub", &[0u8]);
let b = with_stubs(&t);
b.put(&RemotePath::new("a.drsc"), b"<new/>".to_vec(), None)
.await
.unwrap();
assert_eq!(std::fs::read(t.0.join("a.drsc")).unwrap(), b"<new/>");
// And exactly one file for one document: a leftover stub beside it is a
// conflict the client would resolve in favour of whichever it saw last.
assert!(!t.0.join("a.drsc.stub").exists(), "placeholder left behind");
assert_eq!(
names(&b.list(&RemotePath::root(), None).await.unwrap()),
vec!["a.drsc"]
);
}
#[tokio::test]
async fn a_conditional_write_over_a_placeholder_asks_for_the_content_first() {
// `IfMatch` guards a read-modify-write. A stub's validator describes the
// placeholder, not the document, so nothing here can satisfy it — and
// quietly writing anyway is how the other device's edits are lost.
let t = Tmp::new("vfs-write-cond");
t.file("a.drsc.stub", &[0u8]);
let b = with_stubs(&t);
let e = b
.put(&RemotePath::new("a.drsc"), b"<new/>".to_vec(), None)
.put(
&RemotePath::new("a.drsc"),
b"<new/>".to_vec(),
Some(Precondition::IfMatch(Validator::new("whatever"))),
)
.await
.unwrap_err();
assert!(matches!(e, RemoteError::NotMaterialised(_)), "{e:?}");
assert!(!t.0.join("a.drsc").exists(), "no rival file created");
assert!(!t.0.join("a.drsc").exists(), "nothing written");
// And a create-if-absent fails, because the file *is* there — only its
// content is elsewhere.
let e = b
.put(
&RemotePath::new("a.drsc"),
b"<new/>".to_vec(),
Some(Precondition::IfAbsent),
)
.await
.unwrap_err();
assert!(matches!(e, RemoteError::PreconditionFailed), "{e:?}");
}
#[tokio::test]