Describe albums, the folder pickers and download progress in the designs
storage.md's trait listing stopped at get; it now has get_reporting, what the default and the Nextcloud override do, and where develop reads the figures. A new §5.3 says how folders are chosen — the portal or Windows dialogue, the server browser whose New folder is create_dir, SAF on Android — and where an album's files go: a server folder relative to the account root, with the outbox's third .dest line, or a device folder that never syncs. catalog.md §8.2 said only collections merge, which had not been true since keywords, people and capture metadata joined them, and is less true with albums; it now lists what merges and why album_folders does not. §2 records that the album tables, like dedup_probes, are made on first use rather than by a migration. outstanding.md said there was no SAF code on Android. There is now, for album folders only, and it carries TRACES: FR-PLAT-AND-1, which the entry says overstates a requirement about the library; FR-PLAT-AND-2 and S10's row follow from that.
This commit is contained in:
+35
-7
@@ -129,6 +129,17 @@ CREATE INDEX members_image ON collection_members(image_id);
|
||||
partial index over the non-NULL subset is both smaller and what FR-CAT-9's reconnection-by-hash
|
||||
and FR-CAT-11's duplicate detection actually query.
|
||||
|
||||
**Made on first use, not by a migration.** A new `user_version` makes every older build refuse this
|
||||
catalog's snapshot at sync (`sync::remote_is_mergeable` compares it and nothing else), and a tablet
|
||||
a release behind would stop merging collections, keywords and people for a feature it does not
|
||||
have. So what later releases added without needing old rows rewritten is created with `IF NOT
|
||||
EXISTS` where it is first used, and an older build that meets it ignores it:
|
||||
|
||||
- `dedup_probes` (FR-CAT-11a, §3.5);
|
||||
- the albums (FR-EXP-10, `core/dr-catalog/src/albums.rs`): `albums`, `album_exports` — one row per
|
||||
file written into an album, keyed on the file name, since two crops of one photograph are two
|
||||
files — and `album_folders`, this device's folder for each (§8.2).
|
||||
|
||||
---
|
||||
|
||||
## 3. Incremental scan
|
||||
@@ -686,13 +697,29 @@ mechanism.
|
||||
|
||||
### 8.2 What the file sync does and does not carry
|
||||
|
||||
Only **collections and their membership** merge. The rest of a catalog describes *local* state —
|
||||
folder mtimes, cache file paths, job rows, `tier_actual` — and importing another device's version
|
||||
of those would be actively wrong. The downloaded remote is read for its collections and discarded.
|
||||
Collections were the first thing merged, and the rules in §8.4 were written for them. What else
|
||||
merges reuses those rules or keys on the same identities, and each has no other home:
|
||||
|
||||
This is what keeps §6.12 substantially intact: nothing here makes the local database authoritative
|
||||
for anything a rebuild could not recover. The catalog is still deletable. What syncs is one table
|
||||
pair that had no other home.
|
||||
- **Collections and their membership** — by uuid and revision, membership as a set union.
|
||||
- **Keywords** — the vocabulary by the same verdict, the assignments as a union.
|
||||
- **People and identity judgements** — people by uuid and revision, and the confirmed and rejected
|
||||
face assignments matched to local faces by box (`merge::match_faces`).
|
||||
- **Albums** (FR-EXP-10, 0.17.0) — by uuid and revision with tombstones, and what went into each as
|
||||
a set union keyed on the server's file id (a content hash on a folder library). An album's
|
||||
server folder is a column of its row and travels with it; a folder on *this device* is in
|
||||
`album_folders`, which the merge never reads and the upload snapshot drops (§8.3), because a
|
||||
path or a SAF grant on one device means nothing on another.
|
||||
- **Capture metadata** — the one exception inside `images`: a date, a camera, a lens and an ISO are
|
||||
facts about the file's bytes, so a row this device has not yet read takes them from a peer that
|
||||
has (`merge_metadata`), matched by `oc:fileid`.
|
||||
|
||||
The rest of a catalog describes *local* state — folder mtimes, cache file paths, job rows,
|
||||
`tier_actual` — and importing another device's version of those would be actively wrong; the
|
||||
downloaded remote is read for the tables above and discarded.
|
||||
|
||||
This is what keeps §6.12 substantially intact. The catalog is still deletable; what a rebuild from
|
||||
sidecars cannot recover — collections and albums, which nothing in the filesystem records — is what
|
||||
the sync exists to carry.
|
||||
|
||||
### 8.3 Two hazards the implementation must handle
|
||||
|
||||
@@ -714,7 +741,8 @@ crop. They were first stripped (2026-08) by copying the whole file with the back
|
||||
`crop` to NULL and `VACUUM`ing. That wrote the file about three times to upload 50 MB. Since #71
|
||||
the snapshot is built without them. Its header is kept as it was (`user_version`, page size and
|
||||
the WAL flag), so every earlier build merges it unchanged. NFR-R2 backups still use the backup API
|
||||
and keep the crops, because a backup is a file the user may have to live on.
|
||||
and keep the crops, because a backup is a file the user may have to live on. `album_folders` is
|
||||
dropped from the built file for the reason §8.2 gives.
|
||||
|
||||
**Integer primary keys are not identities.** Two devices each allocate `collections.id = 1` for
|
||||
different collections, so a row-level merge keyed on the integer id would collide them. Collections
|
||||
|
||||
+27
-15
@@ -34,6 +34,10 @@ beside re-import detection; the accessibility and localisation counts in §6 had
|
||||
against the tree since 2026-08-30 and are replaced; and §4a records what the develop and keyboard
|
||||
work of 0.15.0 and 0.16.0 left open.
|
||||
|
||||
**And again for 0.17.0.** Folders are chosen through the platform's dialogue now, so FR-PLAT-LIN-3
|
||||
in §5 is rewritten around what the Flatpak has still not shown; albums brought the first SAF code,
|
||||
which FR-PLAT-AND-1's entry now describes, along with why its new tag overstates it.
|
||||
|
||||
---
|
||||
|
||||
## 1. Plugins — post-v1 since 2026-09-19
|
||||
@@ -234,22 +238,30 @@ models, has been measured on a tablet ([faces.md §12.1](faces.md)), and since 0
|
||||
develop view zero-copy as the desktop does ([technical-debt.md TD-1](technical-debt.md), paid off).
|
||||
It carries the manual and opens it in a WebView. What is missing is the platform contract around it.
|
||||
|
||||
**FR-PLAT-AND-1 is untagged, and what it was tagged for was intent rather than code.**
|
||||
The requirement demands that library access be obtained *exclusively* through the Storage Access
|
||||
Framework. There is no SAF code: no `ACTION_OPEN_DOCUMENT_TREE`, no `takePersistableUriPermission`,
|
||||
no `DocumentsContract`. Its two tags rested on a `SourceRef::Document` variant constructed only
|
||||
inside `#[cfg(test)]` — `LocalStorage::open` refuses it, and the test that proves so is named
|
||||
`a_reference_of_the_wrong_kind_is_refused_rather_than_guessed_at` — and on
|
||||
`dr_plat::imports_supported`, which *returns false on Android* and whose own documentation says it
|
||||
"stops being false when a SAF implementation lands". The second tag documented the absence of the
|
||||
thing it was counted as evidence for. Both have been removed; this is the "plumbing a future feature
|
||||
would use" case [CONTRIBUTING.md](../../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both
|
||||
warn about. Android reaches a library through a Nextcloud account or a folder, over paths, like the
|
||||
desktop.
|
||||
**FR-PLAT-AND-1 — SAF is built for export folders, not for the library.** The requirement demands
|
||||
that library access be obtained *exclusively* through the Storage Access Framework. Until 0.17.0
|
||||
there was no SAF code at all, and the requirement's two tags rested on a `SourceRef::Document`
|
||||
variant constructed only inside `#[cfg(test)]` and on `dr_plat::imports_supported`, which *returns
|
||||
false on Android* and whose own documentation says it "stops being false when a SAF implementation
|
||||
lands". Both were removed as intent rather than code — the "plumbing a future feature would use"
|
||||
case [CONTRIBUTING.md](../../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both warn
|
||||
about.
|
||||
|
||||
0.17.0 brought the first real SAF code, for albums (FR-EXP-10): `FolderPicker.java` starts
|
||||
`ACTION_OPEN_DOCUMENT_TREE` from a translucent activity of its own (the main activity is
|
||||
`NativeActivity`, whose results are not ours) and takes a persistable grant; `Saf.java` writes each
|
||||
export through `DocumentsContract`; `ui/dr-ui/src/saf.rs` is the JNI bridge. `saf.rs` and the export
|
||||
path now carry `TRACES: FR-PLAT-AND-1`, and the matrix counts the requirement as covered. **That
|
||||
overstates it.** The mechanism is the one the requirement names, but its subject is the library, and
|
||||
Android still reaches a library through a Nextcloud account or a folder, over paths, like the
|
||||
desktop. Either the tags narrow to FR-EXP-10 or the requirement is met for the library too; until
|
||||
one of those, read the coverage figure with this one subtracted.
|
||||
|
||||
That has a consequence for the rest of the cluster: **FR-PLAT-AND-2** — detecting the loss of a
|
||||
granted tree permission and marking images offline rather than deleting rows — cannot be built until
|
||||
there is a permission to lose. It is listed here as unbuilt, but it is blocked, not skipped.
|
||||
granted tree permission and marking images offline rather than deleting rows — is still blocked for
|
||||
the library, because there is no library grant to lose. An album's folder has a grant now, and
|
||||
nothing checks for its loss: an export into a folder whose grant has gone fails with whatever the
|
||||
write raises, rather than the album saying beforehand that it needs a folder again.
|
||||
|
||||
**FR-PLAT-AND-4 — half built.** The runner is done (`core/dr-catalog/src/runner.rs`): the
|
||||
queue that `jobs.rs` always had is now claimed from, completed, failed and recovered after a
|
||||
@@ -501,7 +513,7 @@ requirement text that asks for it:
|
||||
|---|---|---|
|
||||
| S6 | FR-DSP-2, NFR-RES-2 — tiling and images larger than GPU memory | Nothing; needs a device and a large image |
|
||||
| S9 | R1's tolerance threshold, and therefore R1 | Nothing; the threshold is defined *by* running it |
|
||||
| S10 | Whether SAF at 10k files meets NFR-P1/P3 | §5 — there is no SAF code to measure |
|
||||
| S10 | Whether SAF at 10k files meets NFR-P1/P3 | §5 — SAF reaches album folders only, never a library to enumerate |
|
||||
| S11 | NFR-COMPAT-2, and whether Play makes SAF binding | Nothing |
|
||||
| S13 | NFR-A11Y-2 on Android | §6 — there is almost nothing to test with |
|
||||
|
||||
|
||||
+56
-2
@@ -85,6 +85,9 @@ pub trait RemoteBackend: Send + Sync {
|
||||
// transfer
|
||||
async fn get(&self, id: &RemoteId, range: Option<Range<u64>>)
|
||||
-> Result<Vec<u8>, RemoteError>;
|
||||
async fn get_reporting(&self, id: &RemoteId, // defaulted
|
||||
progress: &(dyn Fn(u64, Option<u64>) + Send + Sync))
|
||||
-> Result<Vec<u8>, RemoteError>;
|
||||
async fn put(&self, path: &RemotePath, body: Vec<u8>, precond: Option<Precondition>)
|
||||
-> Result<Validator, RemoteError>;
|
||||
async fn put_many(&self, items: Vec<(RemotePath, Vec<u8>)>) // defaulted
|
||||
@@ -111,6 +114,16 @@ Rules that are not obvious from the signatures:
|
||||
- **`get` takes an optional range, and it is a hint.** A backend without cheap
|
||||
ranges may return the whole object; the caller slices. Correctness holds
|
||||
either way and `Capabilities::range_reads` says whether it was cheap.
|
||||
- **`get_reporting` is for the one transfer somebody is watching.** An original
|
||||
opened in develop is tens of megabytes, and "downloading" alone for that long
|
||||
reads as stuck. It reports the bytes received and the length the server
|
||||
declared, if it declared one. The default is `get` whole and one report at the
|
||||
end, which is right for a backend whose read is local; Nextcloud overrides it
|
||||
to read the body chunk by chunk. The develop view reads the figures by path
|
||||
from the in-flight registry in `ui/dr-ui/src/library/thumbnails_fetch.rs`,
|
||||
because a step along the roll usually lands on a frame the prefetcher is
|
||||
already fetching, and falls back on the catalog's file length when no length
|
||||
was declared.
|
||||
- **Chunked upload is not in the trait.** It is an implementation detail of
|
||||
`put`, chosen by body size. Exposing it would leak one server's protocol.
|
||||
- **`move_to` must preserve identity where the backend has stable ids.** This
|
||||
@@ -118,7 +131,9 @@ Rules that are not obvious from the signatures:
|
||||
allocates a new id, orphaning the thumbnail shard and turning a restore into a
|
||||
full re-download.
|
||||
- **`create_dir` makes parents and succeeds if the directory exists.** Callers
|
||||
use it to guarantee a destination, not to claim they created one.
|
||||
use it to guarantee a destination, not to claim they created one. It is also
|
||||
the server browser's `New folder` (§5.3), which then lists the parent again
|
||||
rather than inserting the name it asked for: the server may have normalised it.
|
||||
|
||||
### 3.2 `Capabilities` — what is cheap
|
||||
|
||||
@@ -294,6 +309,7 @@ for a second backend:
|
||||
| `bulk_upload` | yes, `POST /remote.php/dav/bulk` |
|
||||
| `conditional_write` | yes, `If-Match` |
|
||||
| `server_previews` | `CommonFormatsOnly` — stock Nextcloud renders no RAW |
|
||||
| `get_reporting` | overridden: the body is read chunk by chunk, against `Content-Length` |
|
||||
| sign-in | `SignIn::Browser`, Login Flow v2, system browser, app password |
|
||||
|
||||
Also kept: the `oc:permissions` probe on a refused `PUT`, which is what
|
||||
@@ -315,7 +331,8 @@ which makes it the route that works on a machine with no secrets daemon at all.
|
||||
| `bulk_upload` | no |
|
||||
| `conditional_write` | yes, with a documented residual race |
|
||||
| `server_previews` | `None` |
|
||||
| sign-in | `SignIn::EndpointOnly` |
|
||||
| `get_reporting` | the default: a local read, reported once when it is done |
|
||||
| sign-in | `SignIn::EndpointOnly`, the folder chosen in the platform's dialogue (§5.3) |
|
||||
|
||||
**Why `LocalEtags` and not `PropagatingEtags`.** A POSIX directory's mtime
|
||||
changes when its own entry list changes and at no other time — not when a
|
||||
@@ -377,6 +394,43 @@ behind a network banner. An endpoint that is not a directory at all maps to
|
||||
`RemoteError::Configuration`: nothing was unreachable and no credential was
|
||||
wrong, so neither of the other two would send the user anywhere useful.
|
||||
|
||||
### 5.3 Choosing folders, and the folders exports go to
|
||||
|
||||
**Folders are pointed at, never typed** (FR-EXP-6). On the desktop,
|
||||
`ui/dr-ui/src/folder_dialog.rs` asks the platform: the XDG desktop portal's
|
||||
FileChooser on Linux, through `rfd`, and the common item dialogue on Windows.
|
||||
That is how the folder connector's endpoint is chosen on the launch screen, and
|
||||
the path it returns still goes through `normalise_endpoint` as a typed one did.
|
||||
A folder on the server is chosen in the in-app browser, which lists with `list`
|
||||
and makes a folder with `create_dir`; the launch screen and the album sheet
|
||||
share it through `ui/dr-ui/src/remote_folders.rs`, which keeps both round trips
|
||||
off the interface thread. Android has no filesystem dialogue, so its library
|
||||
folder is still typed, and an album's folder there comes from the Storage
|
||||
Access Framework's tree picker.
|
||||
|
||||
**An album is where exports go** (FR-EXP-10), and never inside the library: a
|
||||
JPEG written into the tree a scan catalogues comes back as a photograph beside
|
||||
the RAW it was made from. Its folder is one of two kinds
|
||||
(`dr_catalog::albums::Place`):
|
||||
|
||||
- **On the server,** relative to the *account* root, not the library root. The
|
||||
browser refuses a folder inside the library and says why; on a server whose
|
||||
whole account is the library, every folder is inside it, and it says that
|
||||
instead. Exports reach it through the outbox like any upload, and a queued
|
||||
file's `.dest` record gains a third line saying its folder is relative to
|
||||
the account — a third line rather than a leading slash, because a record
|
||||
written before albums may carry a stray slash and has to keep the meaning it
|
||||
was written with.
|
||||
- **On this device,** a path, or on Android a SAF tree URI with a persisted
|
||||
grant, written through `DocumentsContract` (`ui/dr-ui/src/saf.rs`). A
|
||||
provider renames on a collision by itself, so the album records the name it
|
||||
was given rather than the one asked for.
|
||||
|
||||
A server folder lives on the album row and syncs with it. A device folder lives
|
||||
in `album_folders`, which the merge never reads and the upload snapshot drops,
|
||||
because a path or a grant on one device means nothing on another: an album made
|
||||
on the desktop arrives on the tablet with no folder until one is chosen there.
|
||||
|
||||
---
|
||||
|
||||
## 6. Virtual filesystems
|
||||
|
||||
Reference in New Issue
Block a user