Do not push a catalog over one we could not read
`sync_catalog` is a read-modify-write over a file another device also
writes: take theirs, merge, push the union. It was shaped
if let Ok(bytes) = backend.get(&RemoteId::Path(target), None).await {
which folds *every* failure into "there is no remote catalog" and carries
straight on to the upload. On a placeholder library the snapshot in
`.darkroom-derived/` is dehydrated like anything else, so the read failed
every time and each sync pushed our catalog over theirs unmerged —
taking the other device's collections and their members with it.
The same shape as the sidecar bug, and the same fix: a read that fails
for anything other than `NotFound` stops the upload and says why. An
unreadable or unopenable snapshot stops it too — "will not parse" is not
"is not there". This is what `NotFound` and `NotMaterialised` being
separate errors is *for*: one means ours is the whole truth, the other
means do not dare.
Shard downloads go through the same fetch-on-demand read. They logged
and skipped before, which on a library the client keeps dehydrated is
every shard, every pass, and a peer's thumbnails and faces silently
never arriving.
And `put` over a placeholder now replaces it rather than refusing.
Refusing was over-cautious of me: derived state lives inside the library
folder, so a folder the client had dehydrated could never be written to
again. An unconditional write replaces the whole file, so there is
nothing in the stub to keep — content first, then the placeholder, since
in a synced tree an absence is a deletion that propagates. `IfMatch`
still refuses, because a stub's validator describes the stub; `IfAbsent`
fails, because the file is there and only its content is not.
This commit is contained in:
+27
-3
@@ -418,7 +418,9 @@ free of any client's protocol.
|
||||
| `RemoteEntry::materialised` | `false` on a stub; the catalog maps it to `Availability::Offline` |
|
||||
| `RemoteEntry::size` | `0` on a stub, meaning *unknown* — see below |
|
||||
| `get` on a stub | `RemoteError::NotMaterialised`, **never** `NotFound` and never the stub's one byte |
|
||||
| `put` over a stub | refused; writing a rival file hands the client a conflict to resolve arbitrarily |
|
||||
| `put` over a stub, unconditional | **replaces it** — the whole file is being written, so there is nothing in the stub to keep, and the placeholder is removed after the content lands |
|
||||
| `put` over a stub, `IfMatch` | `NotMaterialised` — a stub's validator describes the placeholder, so nothing here can satisfy the guard; the caller fetches and retries |
|
||||
| `put` over a stub, `IfAbsent` | `PreconditionFailed` — the file *is* there, only its content is elsewhere |
|
||||
| `move_to` a stub | moves the stub and keeps it a stub — culling without downloading is ordinary |
|
||||
| `delete` a stub | deletes it; a photograph is deleted whether or not its bytes are here |
|
||||
| `capabilities().materialisation` | `OnDemand` with a client, `Placeholders` without, `Always` on a plain folder |
|
||||
@@ -480,7 +482,29 @@ keeps. A pass over all 100, borrowing and releasing as it goes:
|
||||
The peak is the working set, not the library, and the release is selective.
|
||||
|
||||
|
||||
### 6.4 Release means dehydrate, never delete
|
||||
### 6.4 Derived state is dehydrated too
|
||||
|
||||
Shards and the catalog snapshot live in `.darkroom-derived/` **inside the
|
||||
library folder**, so a sync client dehydrates them exactly as it dehydrates a
|
||||
photograph. Unlike a photograph, none of them can be skipped: a shard that will
|
||||
not open is a peer's thumbnails never merging, and a catalog snapshot that will
|
||||
not open is their collections.
|
||||
|
||||
`derived_sync::read_derived` fetches on demand rather than giving up. More
|
||||
important is what happens when it *cannot*:
|
||||
|
||||
The catalog sync is a read-modify-write over a file another device also writes.
|
||||
It was shaped `if let Ok(bytes) = backend.get(..)`, which folded every failure
|
||||
into "there is no remote catalog" and carried straight on to the upload — so a
|
||||
dehydrated snapshot meant pushing ours over theirs unmerged, taking their
|
||||
collections and members with it. The same shape as the sidecar bug in §6, and
|
||||
the same fix: a read that fails for any reason other than `NotFound` **stops the
|
||||
upload**.
|
||||
|
||||
That is why `NotFound` and `NotMaterialised` had to be separate errors. One
|
||||
means "yours is the whole truth, write it"; the other means "do not dare".
|
||||
|
||||
### 6.5 Release means dehydrate, never delete
|
||||
|
||||
The single most dangerous thing in this feature. A synced folder is not a
|
||||
cache: deleting a materialised file inside it propagates the deletion to the
|
||||
@@ -492,7 +516,7 @@ This is also why the originals cache (`dr_catalog::cache`) cannot simply be
|
||||
pointed at a VFS library: `Cache::release` deletes bytes, which is right for a
|
||||
copy under `originals/` and catastrophic in place.
|
||||
|
||||
### 6.5 Which photographs stay downloaded
|
||||
### 6.6 Which photographs stay downloaded
|
||||
|
||||
The user's half of the bargain: a pass borrows for a moment, but *some* of the
|
||||
library should stay local — the trip you are about to take, the shoot you are
|
||||
|
||||
Reference in New Issue
Block a user