Do not push a catalog over one we could not read

`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped

    if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {

which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.

The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.

Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.

And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
This commit is contained in:
2026-08-29 10:36:26 +02:00
parent 702d83c218
commit 4168d67cfa
5 changed files with 190 additions and 26 deletions
+27 -3
View File
@@ -418,7 +418,9 @@ free of any client's protocol.
| `RemoteEntry::materialised` | `false` on a stub; the catalog maps it to `Availability::Offline` |
| `RemoteEntry::size` | `0` on a stub, meaning *unknown* — see below |
| `get` on a stub | `RemoteError::NotMaterialised`, **never** `NotFound` and never the stub's one byte |
| `put` over a stub | refused; writing a rival file hands the client a conflict to resolve arbitrarily |
| `put` over a stub, unconditional | **replaces it** — the whole file is being written, so there is nothing in the stub to keep, and the placeholder is removed after the content lands |
| `put` over a stub, `IfMatch` | `NotMaterialised` — a stub's validator describes the placeholder, so nothing here can satisfy the guard; the caller fetches and retries |
| `put` over a stub, `IfAbsent` | `PreconditionFailed` — the file *is* there, only its content is elsewhere |
| `move_to` a stub | moves the stub and keeps it a stub — culling without downloading is ordinary |
| `delete` a stub | deletes it; a photograph is deleted whether or not its bytes are here |
| `capabilities().materialisation` | `OnDemand` with a client, `Placeholders` without, `Always` on a plain folder |
@@ -480,7 +482,29 @@ keeps. A pass over all 100, borrowing and releasing as it goes:
The peak is the working set, not the library, and the release is selective.
### 6.4 Release means dehydrate, never delete
### 6.4 Derived state is dehydrated too
Shards and the catalog snapshot live in `.darkroom-derived/` **inside the
library folder**, so a sync client dehydrates them exactly as it dehydrates a
photograph. Unlike a photograph, none of them can be skipped: a shard that will
not open is a peer's thumbnails never merging, and a catalog snapshot that will
not open is their collections.
`derived_sync::read_derived` fetches on demand rather than giving up. More
important is what happens when it *cannot*:
The catalog sync is a read-modify-write over a file another device also writes.
It was shaped `if let Ok(bytes) = backend.get(..)`, which folded every failure
into "there is no remote catalog" and carried straight on to the upload — so a
dehydrated snapshot meant pushing ours over theirs unmerged, taking their
collections and members with it. The same shape as the sidecar bug in §6, and
the same fix: a read that fails for any reason other than `NotFound` **stops the
upload**.
That is why `NotFound` and `NotMaterialised` had to be separate errors. One
means "yours is the whole truth, write it"; the other means "do not dare".
### 6.5 Release means dehydrate, never delete
The single most dangerous thing in this feature. A synced folder is not a
cache: deleting a materialised file inside it propagates the deletion to the
@@ -492,7 +516,7 @@ This is also why the originals cache (`dr_catalog::cache`) cannot simply be
pointed at a VFS library: `Cache::release` deletes bytes, which is right for a
copy under `originals/` and catastrophic in place.
### 6.5 Which photographs stay downloaded
### 6.6 Which photographs stay downloaded
The user's half of the bargain: a pass borrows for a moment, but *some* of the
library should stay local — the trip you are about to take, the shoot you are