Treat a placeholder as the photograph, not as a one-byte file

The folder connector was pointed at a Nextcloud VFS tree and got three
things wrong, the first of which loses work.

**A dehydrated sidecar read as absent.** `a.drsc` does not exist when the
client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed,
`.ok()` swallowed the `NotFound`, and the sidecar writer took that for
"there is no sidecar yet" and wrote a fresh document over the existing
one. Every edit another device had put there went with it. That function's
own doc comment calls this the exact loss the format's unknown-key
preservation exists to prevent.

**A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this
machine at 121,785 placeholders against 10,267 real files — so a folder
library on a synced tree was ~92% broken rows.

**Identity changed on hydration**, so downloading a photograph looked like
a delete and an add, orphaning its thumbnail and its face rows.

Entries now carry the photograph's own name and a `materialised` flag;
`get` on a stub returns the new `RemoteError::NotMaterialised`, which is
distinct from `NotFound` precisely because the sidecar writer must treat
them differently — it fetches the sidecar and merges, or leaves the entry
queued.

Hydration is a **borrow**. `BorrowPool` records what was on disk before it
asked, so `release_all` dehydrates only what a pass brought and leaves
what the user already had. Reference counted: the thumbnail pass and the
face pass meet on the same RAW, and without counting the first to finish
dehydrates the file the second is reading. A borrow against a plain folder
or a server does nothing, so a pass written for VFS runs everywhere.

Releasing means asking the client to dehydrate and never deleting: a
deletion inside a synced tree propagates to the server and removes the
photograph from every device.

Not a second backend — the capability is per *connection*, not per type,
since the same folder hydrates only while the client runs. The convention
arrives through a detector the registry supplies, so `dr-sync-folder`
still knows nothing about any client's protocol.

ARCH §9.0a records this as an amendment: finding 3 rejected hydration
because it costs 100× a range read, and that comparison assumed a
connector was available. A folder library has none.
This commit is contained in:
2026-08-29 09:57:52 +02:00
parent 6c363cee97
commit c102ba9df2
22 changed files with 1555 additions and 53 deletions
+164 -4
View File
@@ -482,7 +482,7 @@ async fn an_upload_lands_where_the_engine_places_it() {
#[test]
fn an_endpoint_is_checked_before_an_account_is_written_for_it() {
let t = Tmp::new("provider");
let p = FolderProvider;
let p = FolderProvider::new();
assert!(p.normalise_endpoint(" ").is_err(), "empty");
assert!(p.normalise_endpoint("Pictures").is_err(), "relative");
@@ -504,7 +504,7 @@ fn two_spellings_of_one_folder_become_one_account() {
// Otherwise the same photographs are indexed twice, into two catalogs.
let t = Tmp::new("canonical");
t.file("sub/a.CR2", b"x");
let p = FolderProvider;
let p = FolderProvider::new();
let direct = p
.normalise_endpoint(&t.0.join("sub").to_string_lossy())
.unwrap();
@@ -516,7 +516,7 @@ fn two_spellings_of_one_folder_become_one_account() {
#[test]
fn a_folder_account_needs_no_credential() {
let p = FolderProvider;
let p = FolderProvider::new();
assert_eq!(p.sign_in(), SignIn::EndpointOnly);
assert!(!p.sign_in().needs_secret());
@@ -532,9 +532,169 @@ fn the_registry_opens_a_folder_account() {
// with nothing in between naming this crate.
let t = Tmp::new("registry");
let mut registry = dr_sync::BackendRegistry::new();
registry.register(std::sync::Arc::new(FolderProvider));
registry.register(std::sync::Arc::new(FolderProvider::new()));
let account = Account::new(BACKEND_ID, t.0.to_string_lossy());
let backend = registry.connect(&Connection::new(account, None)).unwrap();
assert_eq!(backend.name(), "Folder");
}
// --- virtual filesystems --------------------------------------------------
//
// A suffix-mode convention, matching the only one Linux supports. The
// behaviour under test is what the *backend* does with it; the borrow cycle
// has its own tests beside the pool.
struct SuffixVfs;
impl Vfs for SuffixVfs {
fn name(&self) -> &'static str {
"suffix"
}
fn is_placeholder(&self, on_disk: &str) -> bool {
on_disk.ends_with(".stub")
}
fn real_name<'a>(&self, on_disk: &'a str) -> &'a str {
on_disk.strip_suffix(".stub").unwrap_or(on_disk)
}
fn placeholder_name(&self, name: &str) -> std::borrow::Cow<'_, str> {
std::borrow::Cow::Owned(format!("{name}.stub"))
}
}
fn with_stubs(t: &Tmp) -> FolderBackend {
FolderBackend::with_vfs(&t.0, std::sync::Arc::new(SuffixVfs)).unwrap()
}
#[tokio::test]
async fn a_placeholder_is_listed_under_the_photographs_own_name() {
// The catalog records this as `source_ref`, and identity is derived from
// it. Reporting the stub's name gives the same photograph two identities
// and a name no other device recognises.
let t = Tmp::new("vfs-name");
t.file("shoot/IMG_0001.CR2.stub", &[0u8]);
let b = with_stubs(&t);
let entries = b.list(&RemotePath::new("shoot"), None).await.unwrap();
assert_eq!(entries[0].path.as_str(), "shoot/IMG_0001.CR2");
assert!(!entries[0].materialised, "the content is not here");
// One byte is not the photograph's size, and putting it in the catalog
// would claim a 30 MB RAW is a single byte.
assert_eq!(entries[0].size, 0, "unknown, not one");
}
#[tokio::test]
async fn identity_survives_a_download() {
// The failure this prevents: downloading a photograph looked like a
// delete and an add, which orphaned its thumbnail and its face rows.
let t = Tmp::new("vfs-identity");
t.file("a.CR2.stub", &[0u8]);
let b = with_stubs(&t);
let before = b.list(&RemotePath::root(), None).await.unwrap()[0]
.id
.clone();
std::fs::remove_file(t.0.join("a.CR2.stub")).unwrap();
std::fs::write(t.0.join("a.CR2"), vec![3u8; 4096]).unwrap();
let after = b.list(&RemotePath::root(), None).await.unwrap()[0]
.id
.clone();
assert_eq!(before, after, "the same photograph throughout");
}
#[tokio::test]
async fn reading_a_placeholder_is_distinguishable_from_a_missing_file() {
// The distinction the sidecar writer depends on: "not here" is fetchable
// and "not found" means create a new one. Conflating them overwrites an
// existing sidecar with a fresh document.
let t = Tmp::new("vfs-read");
t.file("a.drsc.stub", &[0u8]);
let b = with_stubs(&t);
let stub = b
.get(&RemoteId::Path(RemotePath::new("a.drsc")), None)
.await
.unwrap_err();
assert!(matches!(stub, RemoteError::NotMaterialised(_)), "{stub:?}");
let absent = b
.get(&RemoteId::Path(RemotePath::new("nothing.drsc")), None)
.await
.unwrap_err();
assert!(matches!(absent, RemoteError::NotFound(_)), "{absent:?}");
// And emphatically not the stub's one byte, which is what made a
// dehydrated sidecar parse as an empty document.
assert!(!matches!(stub, RemoteError::NotFound(_)));
}
#[tokio::test]
async fn writing_over_a_placeholder_is_refused() {
// Writing `a.drsc` beside `a.drsc.stub` makes two files for one document
// and hands the sync client a conflict it resolves arbitrarily.
let t = Tmp::new("vfs-write");
t.file("a.drsc.stub", &[0u8]);
let b = with_stubs(&t);
let e = b
.put(&RemotePath::new("a.drsc"), b"<new/>".to_vec(), None)
.await
.unwrap_err();
assert!(matches!(e, RemoteError::NotMaterialised(_)), "{e:?}");
assert!(!t.0.join("a.drsc").exists(), "no rival file created");
}
#[tokio::test]
async fn trashing_a_photograph_that_is_not_downloaded_moves_the_placeholder() {
// Culling without downloading is the ordinary way to use a VFS library.
// The stub has to move, and has to stay a stub — leaving it behind means
// the next scan re-lists the image and undoes the delete.
let t = Tmp::new("vfs-trash");
t.file("a.CR2.stub", &[0u8]);
let b = with_stubs(&t);
b.move_to(
&RemoteId::Path(RemotePath::new("a.CR2")),
&RemotePath::new(".darkroom-trash/a.CR2"),
)
.await
.unwrap();
assert!(!t.0.join("a.CR2.stub").exists());
assert!(
t.0.join(".darkroom-trash/a.CR2.stub").is_file(),
"still a stub"
);
}
#[tokio::test]
async fn a_folder_without_a_client_still_lists_and_reads_what_is_there() {
// No hydration available is a degraded mode, not a broken one: the
// materialised half of the library works completely.
let t = Tmp::new("vfs-degraded");
t.file("here.CR2", b"real").file("gone.CR2.stub", &[0u8]);
let b = with_stubs(&t);
assert_eq!(
b.capabilities().materialisation,
dr_sync::Materialisation::Placeholders,
"stubs exist and nothing can fetch them"
);
assert!(!b.capabilities().materialisation.can_materialise());
let got = b
.get(&RemoteId::Path(RemotePath::new("here.CR2")), None)
.await
.unwrap();
assert_eq!(got, b"real");
}
#[test]
fn a_plain_folder_reports_that_everything_it_lists_is_readable() {
let t = Tmp::new("vfs-plain");
assert_eq!(
t.backend().capabilities().materialisation,
dr_sync::Materialisation::Always
);
}