Measure the folder walk against a real tree, not a claim
The folder connector declares `LocalEtags`, which means the engine walks the whole library on every scan with no pruning. That is the honest capability, and the argument for it being affordable was so far an assertion about `stat` versus `PROPFIND`. `--example scan` runs the real path — `dr_sync::scan` over the connector, then a ranged read of the kind the thumbnail worker makes. Read-only; it never writes into the folder it is pointed at. 2,299 images across 233 directories in 137 ms, and 380 across 13 in 29 ms. Against 34.1 s for 17,185 RAWs over WebDAV *with* pruning available. Recorded in docs/storage.md §5.2 and ARCH §8.4a, because a capability trade-off argued from a number nobody measured is the kind that gets quietly reversed later.
This commit is contained in:
@@ -794,9 +794,11 @@ Three things worth carrying forward:
|
||||
|
||||
- **`LocalEtags` is the honest answer, and it costs nothing.** A POSIX directory's mtime describes
|
||||
its own entry list and nothing below it, so there is no propagation to exploit and the engine
|
||||
walks the tree every scan. The walk that was expensive was expensive because it was 50k
|
||||
`PROPFIND`s; 50k `stat` calls take well under a second. This is the capability model paying for
|
||||
itself — one engine, two backends, each running at the speed it actually runs at.
|
||||
walks the tree every scan. **Measured 2026-08-28**: a full uncached walk of 2,299 images
|
||||
across 233 directories took **137 ms**, with no pruning at all — against 34.1 s for 17,185 RAWs
|
||||
over WebDAV *with* pruning (§8.4). The walk that was expensive was expensive because it was
|
||||
thousands of `PROPFIND`s. This is the capability model paying for itself — one engine, two
|
||||
backends, each running at the speed it actually runs at.
|
||||
- **Identity is a path hash, not an inode.** An inode is stable across a rename but differs between
|
||||
devices and is reused after a delete, so two machines would disagree about which photograph a
|
||||
thumbnail belonged to and a recycled inode would attach an old thumbnail to a new image.
|
||||
|
||||
+10
-4
@@ -309,10 +309,16 @@ changes when its own entry list changes and at no other time — not when a
|
||||
child's contents are edited, and not for a grandchild. There is nothing to
|
||||
propagate, so `dir_validator` returns `Unsupported` and the engine walks the
|
||||
tree every scan. Which costs almost nothing, because the walk that was expensive
|
||||
was expensive for a reason this backend does not have: fifty thousand `stat`
|
||||
calls against a filesystem take well under a second, and fifty thousand
|
||||
`PROPFIND`s do not. The capability model is what lets both be driven by the same
|
||||
engine at the speed each actually runs at.
|
||||
was expensive for a reason this backend does not have.
|
||||
|
||||
**Measured 2026-08-28**, `cargo run -p dr-sync-folder --example scan`: a full
|
||||
uncached walk of 2,299 images across 233 directories completed in **137 ms**,
|
||||
and 380 images across 13 directories in **29 ms** — the same engine, the same
|
||||
`Depth: 1`-per-directory walk, with no pruning at all. The Nextcloud connector's
|
||||
comparable figure is 34.1 s for 17,185 RAWs across 334 directories *with*
|
||||
pruning available (ARCH §8.4). The capability model is what lets one engine
|
||||
drive both at the speed each actually runs at, instead of forcing the fast one
|
||||
down to the slow one's interface.
|
||||
|
||||
**Identity is a hash of the path relative to the library root**, FNV-1a 64
|
||||
(written out, because `DefaultHasher` is explicitly unstable between Rust
|
||||
|
||||
Reference in New Issue
Block a user