Treat a placeholder as the photograph, not as a one-byte file
The folder connector was pointed at a Nextcloud VFS tree and got three things wrong, the first of which loses work. **A dehydrated sidecar read as absent.** `a.drsc` does not exist when the client has dehydrated it — only `a.drsc.nextcloud` does — so `get` missed, `.ok()` swallowed the `NotFound`, and the sidecar writer took that for "there is no sidecar yet" and wrote a fresh document over the existing one. Every edit another device had put there went with it. That function's own doc comment calls this the exact loss the format's unknown-key preservation exists to prevent. **A stub was catalogued as a 1-byte image**, and ARCH §9.0 measured this machine at 121,785 placeholders against 10,267 real files — so a folder library on a synced tree was ~92% broken rows. **Identity changed on hydration**, so downloading a photograph looked like a delete and an add, orphaning its thumbnail and its face rows. Entries now carry the photograph's own name and a `materialised` flag; `get` on a stub returns the new `RemoteError::NotMaterialised`, which is distinct from `NotFound` precisely because the sidecar writer must treat them differently — it fetches the sidecar and merges, or leaves the entry queued. Hydration is a **borrow**. `BorrowPool` records what was on disk before it asked, so `release_all` dehydrates only what a pass brought and leaves what the user already had. Reference counted: the thumbnail pass and the face pass meet on the same RAW, and without counting the first to finish dehydrates the file the second is reading. A borrow against a plain folder or a server does nothing, so a pass written for VFS runs everywhere. Releasing means asking the client to dehydrate and never deleting: a deletion inside a synced tree propagates to the server and removes the photograph from every device. Not a second backend — the capability is per *connection*, not per type, since the same folder hydrates only while the client runs. The convention arrives through a detector the registry supplies, so `dr-sync-folder` still knows nothing about any client's protocol. ARCH §9.0a records this as an amendment: finding 3 rejected hydration because it costs 100× a range read, and that comparison assumed a connector was available. A folder library has none.
This commit is contained in:
@@ -482,7 +482,7 @@ async fn an_upload_lands_where_the_engine_places_it() {
|
||||
#[test]
|
||||
fn an_endpoint_is_checked_before_an_account_is_written_for_it() {
|
||||
let t = Tmp::new("provider");
|
||||
let p = FolderProvider;
|
||||
let p = FolderProvider::new();
|
||||
|
||||
assert!(p.normalise_endpoint(" ").is_err(), "empty");
|
||||
assert!(p.normalise_endpoint("Pictures").is_err(), "relative");
|
||||
@@ -504,7 +504,7 @@ fn two_spellings_of_one_folder_become_one_account() {
|
||||
// Otherwise the same photographs are indexed twice, into two catalogs.
|
||||
let t = Tmp::new("canonical");
|
||||
t.file("sub/a.CR2", b"x");
|
||||
let p = FolderProvider;
|
||||
let p = FolderProvider::new();
|
||||
let direct = p
|
||||
.normalise_endpoint(&t.0.join("sub").to_string_lossy())
|
||||
.unwrap();
|
||||
@@ -516,7 +516,7 @@ fn two_spellings_of_one_folder_become_one_account() {
|
||||
|
||||
#[test]
|
||||
fn a_folder_account_needs_no_credential() {
|
||||
let p = FolderProvider;
|
||||
let p = FolderProvider::new();
|
||||
assert_eq!(p.sign_in(), SignIn::EndpointOnly);
|
||||
assert!(!p.sign_in().needs_secret());
|
||||
|
||||
@@ -532,9 +532,169 @@ fn the_registry_opens_a_folder_account() {
|
||||
// with nothing in between naming this crate.
|
||||
let t = Tmp::new("registry");
|
||||
let mut registry = dr_sync::BackendRegistry::new();
|
||||
registry.register(std::sync::Arc::new(FolderProvider));
|
||||
registry.register(std::sync::Arc::new(FolderProvider::new()));
|
||||
|
||||
let account = Account::new(BACKEND_ID, t.0.to_string_lossy());
|
||||
let backend = registry.connect(&Connection::new(account, None)).unwrap();
|
||||
assert_eq!(backend.name(), "Folder");
|
||||
}
|
||||
|
||||
// --- virtual filesystems --------------------------------------------------
|
||||
//
|
||||
// A suffix-mode convention, matching the only one Linux supports. The
|
||||
// behaviour under test is what the *backend* does with it; the borrow cycle
|
||||
// has its own tests beside the pool.
|
||||
|
||||
struct SuffixVfs;
|
||||
|
||||
impl Vfs for SuffixVfs {
|
||||
fn name(&self) -> &'static str {
|
||||
"suffix"
|
||||
}
|
||||
fn is_placeholder(&self, on_disk: &str) -> bool {
|
||||
on_disk.ends_with(".stub")
|
||||
}
|
||||
fn real_name<'a>(&self, on_disk: &'a str) -> &'a str {
|
||||
on_disk.strip_suffix(".stub").unwrap_or(on_disk)
|
||||
}
|
||||
fn placeholder_name(&self, name: &str) -> std::borrow::Cow<'_, str> {
|
||||
std::borrow::Cow::Owned(format!("{name}.stub"))
|
||||
}
|
||||
}
|
||||
|
||||
fn with_stubs(t: &Tmp) -> FolderBackend {
|
||||
FolderBackend::with_vfs(&t.0, std::sync::Arc::new(SuffixVfs)).unwrap()
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_placeholder_is_listed_under_the_photographs_own_name() {
|
||||
// The catalog records this as `source_ref`, and identity is derived from
|
||||
// it. Reporting the stub's name gives the same photograph two identities
|
||||
// and a name no other device recognises.
|
||||
let t = Tmp::new("vfs-name");
|
||||
t.file("shoot/IMG_0001.CR2.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
let entries = b.list(&RemotePath::new("shoot"), None).await.unwrap();
|
||||
assert_eq!(entries[0].path.as_str(), "shoot/IMG_0001.CR2");
|
||||
assert!(!entries[0].materialised, "the content is not here");
|
||||
// One byte is not the photograph's size, and putting it in the catalog
|
||||
// would claim a 30 MB RAW is a single byte.
|
||||
assert_eq!(entries[0].size, 0, "unknown, not one");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn identity_survives_a_download() {
|
||||
// The failure this prevents: downloading a photograph looked like a
|
||||
// delete and an add, which orphaned its thumbnail and its face rows.
|
||||
let t = Tmp::new("vfs-identity");
|
||||
t.file("a.CR2.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
let before = b.list(&RemotePath::root(), None).await.unwrap()[0]
|
||||
.id
|
||||
.clone();
|
||||
std::fs::remove_file(t.0.join("a.CR2.stub")).unwrap();
|
||||
std::fs::write(t.0.join("a.CR2"), vec![3u8; 4096]).unwrap();
|
||||
let after = b.list(&RemotePath::root(), None).await.unwrap()[0]
|
||||
.id
|
||||
.clone();
|
||||
|
||||
assert_eq!(before, after, "the same photograph throughout");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reading_a_placeholder_is_distinguishable_from_a_missing_file() {
|
||||
// The distinction the sidecar writer depends on: "not here" is fetchable
|
||||
// and "not found" means create a new one. Conflating them overwrites an
|
||||
// existing sidecar with a fresh document.
|
||||
let t = Tmp::new("vfs-read");
|
||||
t.file("a.drsc.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
let stub = b
|
||||
.get(&RemoteId::Path(RemotePath::new("a.drsc")), None)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(stub, RemoteError::NotMaterialised(_)), "{stub:?}");
|
||||
|
||||
let absent = b
|
||||
.get(&RemoteId::Path(RemotePath::new("nothing.drsc")), None)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(absent, RemoteError::NotFound(_)), "{absent:?}");
|
||||
|
||||
// And emphatically not the stub's one byte, which is what made a
|
||||
// dehydrated sidecar parse as an empty document.
|
||||
assert!(!matches!(stub, RemoteError::NotFound(_)));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn writing_over_a_placeholder_is_refused() {
|
||||
// Writing `a.drsc` beside `a.drsc.stub` makes two files for one document
|
||||
// and hands the sync client a conflict it resolves arbitrarily.
|
||||
let t = Tmp::new("vfs-write");
|
||||
t.file("a.drsc.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
let e = b
|
||||
.put(&RemotePath::new("a.drsc"), b"<new/>".to_vec(), None)
|
||||
.await
|
||||
.unwrap_err();
|
||||
assert!(matches!(e, RemoteError::NotMaterialised(_)), "{e:?}");
|
||||
assert!(!t.0.join("a.drsc").exists(), "no rival file created");
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn trashing_a_photograph_that_is_not_downloaded_moves_the_placeholder() {
|
||||
// Culling without downloading is the ordinary way to use a VFS library.
|
||||
// The stub has to move, and has to stay a stub — leaving it behind means
|
||||
// the next scan re-lists the image and undoes the delete.
|
||||
let t = Tmp::new("vfs-trash");
|
||||
t.file("a.CR2.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
b.move_to(
|
||||
&RemoteId::Path(RemotePath::new("a.CR2")),
|
||||
&RemotePath::new(".darkroom-trash/a.CR2"),
|
||||
)
|
||||
.await
|
||||
.unwrap();
|
||||
|
||||
assert!(!t.0.join("a.CR2.stub").exists());
|
||||
assert!(
|
||||
t.0.join(".darkroom-trash/a.CR2.stub").is_file(),
|
||||
"still a stub"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn a_folder_without_a_client_still_lists_and_reads_what_is_there() {
|
||||
// No hydration available is a degraded mode, not a broken one: the
|
||||
// materialised half of the library works completely.
|
||||
let t = Tmp::new("vfs-degraded");
|
||||
t.file("here.CR2", b"real").file("gone.CR2.stub", &[0u8]);
|
||||
let b = with_stubs(&t);
|
||||
|
||||
assert_eq!(
|
||||
b.capabilities().materialisation,
|
||||
dr_sync::Materialisation::Placeholders,
|
||||
"stubs exist and nothing can fetch them"
|
||||
);
|
||||
assert!(!b.capabilities().materialisation.can_materialise());
|
||||
|
||||
let got = b
|
||||
.get(&RemoteId::Path(RemotePath::new("here.CR2")), None)
|
||||
.await
|
||||
.unwrap();
|
||||
assert_eq!(got, b"real");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_plain_folder_reports_that_everything_it_lists_is_readable() {
|
||||
let t = Tmp::new("vfs-plain");
|
||||
assert_eq!(
|
||||
t.backend().capabilities().materialisation,
|
||||
dr_sync::Materialisation::Always
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user