Borrow the library to index it, and give it back
The passes that need every photograph's bytes — thumbnails, face indexing — now borrow each one and release it at the end. On a placeholder library that is the difference between peak disk being the working set and being the whole library. Including on cancellation, which was nearly missed: the face sweep returns mid-loop when the user presses Stop, and without releasing there the disk is spent and nothing is delivered for it. `materialise` now answers whether *it* fetched the content. The pool used to work that out by listing a file's parent directory — one listing per file across a library — when the backend already had to `stat` it to decide whether to ask. One syscall instead of a directory walk, and it removes the bug class the tests found earlier: a file at the library root has no `parent()`, so every one of them read as already-downloaded. **Pinning is the retention control**, and it drives the model the catalog already had rather than a second one. `tier_desired` is what the user asked to keep hydrated, `pending_pins` is the resumable work list, and a pinned collection is never dehydrated for the same reason it was never evicted. It was in fact *broken* here before: `get` on a stub failed, and the pin worker logged "one unreadable file must not abandon the whole pin" and silently did nothing. Pinned originals on such a library are recorded with `path = NULL` (`Cache::record_in_place`) rather than copied under `originals/`. Two reasons, and the second is the important one. A copy would hold every pinned photograph twice, with the budget able to evict the half that was not costing the disk. And `release` deletes the file a row names — so a row that names none cannot delete anything, which puts the one catastrophic operation out of reach by construction rather than by remembering not to call it. Deleting a materialised file inside a synced tree removes the photograph from the server and every other device. Handing disk back is `spawn_dehydrate`, which asks the client. Two gaps written down rather than papered over (docs/storage.md §7): a hydrating pass cannot yet quote its cost, because a stub reports no size; and the two sweeps hold separate pools, so a library indexed for both fetches twice.
This commit is contained in:
@@ -477,6 +477,40 @@ This is also why the originals cache (`dr_catalog::cache`) cannot simply be
|
||||
pointed at a VFS library: `Cache::release` deletes bytes, which is right for a
|
||||
copy under `originals/` and catastrophic in place.
|
||||
|
||||
### 6.5 Which photographs stay downloaded
|
||||
|
||||
The user's half of the bargain: a pass borrows for a moment, but *some* of the
|
||||
library should stay local — the trip you are about to take, the shoot you are
|
||||
working on.
|
||||
|
||||
That is a **pin**, and it is the pin the originals cache already had
|
||||
(`dr_catalog::cache`, FR-NC-6a). Nothing parallel was built, because the model
|
||||
was already the right one:
|
||||
|
||||
| Cache concept | On a placeholder library |
|
||||
|---|---|
|
||||
| `tier_desired` | what the user asked to keep hydrated |
|
||||
| `tier_actual` | what is actually materialised |
|
||||
| `pending_pins()` | the work list — what to hydrate next, resumable |
|
||||
| pinned rows are never evicted | a pinned collection is never dehydrated |
|
||||
| passive rows, LRU under a budget | what a pass borrowed, released when it finishes |
|
||||
|
||||
So "keep this collection hydrated" is `Cache::pin`, and the existing pin worker
|
||||
drives it — except that on a placeholder library it calls `materialise` instead
|
||||
of downloading a copy.
|
||||
|
||||
**Why not a copy.** The original materialises *in the library folder*. Copying
|
||||
it under `originals/` as well would hold every pinned photograph twice, and the
|
||||
copy would be the half the budget could evict while the real disk cost stayed.
|
||||
`Cache::record_in_place` records the bookkeeping with **`path = NULL`**, and
|
||||
that null is load-bearing: `release` deletes the file a row names, and a row
|
||||
that names none deletes nothing. The safety property is structural rather than
|
||||
remembered.
|
||||
|
||||
Unpinning therefore frees nothing by itself — the bytes are not ours to delete.
|
||||
`spawn_dehydrate` asks the client to take them back, which is what actually
|
||||
returns the disk.
|
||||
|
||||
---
|
||||
|
||||
## 7. What the abstraction does not yet cover
|
||||
@@ -495,3 +529,16 @@ Stated so the next person does not have to rediscover it.
|
||||
- **A general notion of an account.** Credentials stay connector-specific on
|
||||
purpose (§3.5). A third connector with an OAuth flow will need a third `SignIn`
|
||||
variant, and that is the right place for it to appear.
|
||||
- **A quoted cost before a hydrating pass.** FR-NC-6c wants the transfer size
|
||||
stated before an operation that needs absent data. A placeholder reports no
|
||||
size (§6.2), so the honest figure for "index this library" is a count and not
|
||||
a byte total. The interface should say *n photographs, size unknown until
|
||||
fetched* rather than estimate one silently — and it does not say anything yet.
|
||||
- **Metadata-only placeholders.** Windows and macOS express these in filesystem
|
||||
metadata rather than in the name, and carry the real size there. `Vfs` asks
|
||||
its questions about a *name*, which is all the one convention this project has
|
||||
met needs. Supporting them means widening the trait to take a `Metadata`, and
|
||||
doing that before anyone has run this on those platforms would be guessing.
|
||||
- **Hydration during browsing, deliberately.** It stays forbidden (ARCH §9.0
|
||||
finding 3). A grid cell whose content is absent shows as not-downloaded; only
|
||||
a pass the user asked for may fetch.
|
||||
|
||||
Reference in New Issue
Block a user