diff --git a/.gitignore b/.gitignore index ba2478f..69321fb 100644 --- a/.gitignore +++ b/.gitignore @@ -16,3 +16,10 @@ Cargo.lock.bak # tools/film-profiles/convert.py --fetch. Not source: the converted # profiles in core/dr-film/profiles are. tools/film-profiles/upstream/ + +# flatpak-builder's cache and its output tree. `packaging/flatpak/` holds the +# manifest, which is source; everything a build derives from it is not — and +# `.flatpak-builder/` in particular caches an unpacked copy of the whole +# checkout, so it is larger than the repository it sits in. +/.flatpak-builder/ +/build/ diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b6e1e06..7c554e9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -151,6 +151,7 @@ One commit per change. If you fixed two things, that is two commits. | [`docs/architecture.md`](docs/architecture.md) | Anything touching the render path, catalog or sync | | [`docs/code-health.md`](docs/code-health.md) | Deciding what to work on; grades each seam by what it costs | | [`docs/technical-debt.md`](docs/technical-debt.md) | Something looks wrong — check it was not chosen | +| [`docs/distribution.md`](docs/distribution.md) | Packaging a build, or adding a permission to one | | [`docs/requirements.md`](docs/requirements.md) | Reference, not reading | `technical-debt.md` is the one to check before "fixing" anything surprising. diff --git a/docs/distribution.md b/docs/distribution.md new file mode 100644 index 0000000..90e3075 --- /dev/null +++ b/docs/distribution.md @@ -0,0 +1,251 @@ +# DarkRoom — Distribution + +**Satisfies:** NFR-COMPAT-2 (v1 channels) · FR-PLAT-LIN-3 (sandboxed distribution) +**Companion to:** [requirements.md](requirements.md) §3.8, §4.8 · [storage.md](storage.md) + +NFR-COMPAT-2 asks for the v1 channels to be *stated*, and says why in its own +second sentence: the channel decision and the storage design are coupled. A +channel is not a build target. It is a set of constraints that reach back into +the code — what the application is allowed to see, what it may ask for, and +what it must be able to do without asking. This document records which channels +v1 targets and what each one costs, and it is where to look before adding a +permission to a package rather than after. + +--- + +## 1. The channels + +| Platform | Channel | State | What it constrains | +|---|---|---|---| +| Linux | Arch source package — [`packaging/PKGBUILD`](../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon | +| Linux | Flatpak — [`packaging/flatpak/`](../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths | +| Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all | +| Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs | +| Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering | + +Three of these five exist as recipes and two do not. That is stated rather than +smoothed over, because the value of writing the channels down is knowing which +constraints are already being met and which are promises. + +### What every channel has to get right + +Independent of packaging format, and each of these has bitten a package +somewhere: + +- **One identifier, four places.** `paris.tourolle.darkroom` is the AppStream + component id, the `.desktop` basename, the Flatpak application id, and the + string `dr_ui::run` sets as the Wayland `app_id` and X11 `WM_CLASS`. A rename + that misses one of them costs the icon in the shell or the association in the + software centre, and neither failure announces itself. +- **The metainfo, not just the desktop entry.** + [`packaging/paris.tourolle.darkroom.metainfo.xml`](../packaging/paris.tourolle.darkroom.metainfo.xml) + is the single description of the application, installed by every channel that + has somewhere to put it. Its `metadata_license` is CC0-1.0 and its + `project_license` is GPL-3.0-or-later; those differ on purpose — see the + comment in the file. +- **Vulkan is a requirement, not a preference.** The develop pipeline is + compute shaders through wgpu, and NFR-R8 — how far a CPU fallback goes — is + still open, so today there is nothing behind it. A package that installs onto + a machine with no working ICD produces an application that starts and cannot + develop. +- **A Secret Service implementation, or an honest degraded mode.** FR-NC-2 is + explicit that the absence of a secrets daemon is a stated degraded mode and + never a silent fall back to plaintext. Packages express this as an optional + dependency (the PKGBUILD) or a talk hole (the Flatpak manifest), never as a + hard dependency — a headless or minimal-WM install is a supported way to run. +- **The face models are Git LFS objects.** A checkout without `git lfs pull` + has ~130-byte pointers where 11 MB models should be. Both the PKGBUILD and + the Flatpak manifest check the file size and refuse, because the alternative + is a package whose face indexing fails inside the graph loader on a user's + machine rather than on the packager's. + +--- + +## 2. Why Flatpak is the channel that matters most + +Not because it is expected to be the most used. Because it is the only one that +tests anything. + +The Arch package and an AppImage both hand the application the same +unrestricted process the developer runs it in, so neither can discover that a +design assumed unrestricted access. Flatpak takes that assumption away, and +FR-PLAT-LIN-3 exists to make the discovery happen deliberately rather than in a +bug report. §4 is what it discovered. + +The same argument runs the other way on Android, where SAF has been the only +option since before the first line was written (ARCH §6.9) and `SourceRef` +exists because of it. Linux got the abstraction — `LocalStorage::grant` is the +one place a `Path` enters — and never got the constraint that would have proved +it worked. + +--- + +## 3. What already works inside the sandbox, unchanged + +Worth listing, because it is the part FR-PLAT-LIN-1 quietly paid for in +advance: + +- **XDG directories.** Flatpak redirects `XDG_CONFIG_HOME`, `XDG_DATA_HOME` and + `XDG_CACHE_HOME` into `~/.var/app/paris.tourolle.darkroom/`. Settings + (`settings_store.rs`), accounts (`dr_sync::account`), the catalog and the + thumbnail store all read those variables, so every one of them lands in the + application's own directory with no code change and no permission. +- **The face models.** `system_face_models_dirs()` reads `$XDG_DATA_DIRS` + rather than hard-coding `/usr/share`, which is exactly why `/app/share` + inside a Flatpak is found by the same lookup that finds the Arch package's + copy. +- **Opening a photograph from a file manager.** The `.desktop` entry declares + the RAW MIME types and `Exec=darkroom-desktop %F`; under Flatpak the file is + exported through the document portal and arrives in `argv` as a path under + `/run/user/$UID/doc/`, which is mounted in every sandbox. `main.rs` takes + paths from `argv` and `collect()` handles a file or a directory. This is + genuine portal-mediated access and it needs nothing new. +- **The Nextcloud sign-in browser.** `open_in_browser` spawns `xdg-open`; the + freedesktop runtime's `xdg-open` forwards to the OpenURI portal, and portal + calls need no `--talk-name` because Flatpak always permits them. FR-NC-1's + "system browser, never an embedded webview" therefore holds inside the + sandbox for the same reason it holds outside it. +- **Credentials.** The keyring crate speaks the Secret Service D-Bus interface, + reached through the session-bus proxy with one talk hole. The app password + stays visible to `secret-tool` and Seahorse, which is what keeps it + individually revocable by the user. + +--- + +## 4. What does not work: choosing a library + +**FR-PLAT-LIN-3 is not satisfied today, and the manifest does not pretend +otherwise.** + +A folder library is chosen by typing an absolute path. `dr-sync-folder`'s +provider declares `SignIn::EndpointOnly` with the placeholder +`/home/you/Pictures`, and `normalise_endpoint` expands `~`, requires the path +to be absolute, and checks it with `std::fs`. Nothing in the tree calls the +FileChooser portal — there is no `ashpd`, no `rfd`, and no toolkit file dialog +anywhere in `ui/`, `platform/` or `core/`. + +Inside a sandbox with no `--filesystem=`, `$HOME` still resolves to the real +home *path* but that directory holds only the application's own +`.var/app/…` tree. So a typed `~/Pictures` fails the `exists()` check and the +launch screen says `No folder at /home/you/Pictures.` — a truthful message +about a situation the user cannot fix from inside the application. + +Import is blocked one step earlier. `dr_plat::volumes()` finds a camera card by +reading `/proc/self/mountinfo` and the `removable` flag under `/sys`. A +sandboxed process is in its own mount namespace, so the table it reads +describes the sandbox; a card mounted at `/run/media/…` on the host is not in +it. `volumes()` correctly returns an empty list, which the interface presents +as "no card found" — right for the code, wrong for the user, who is looking at +a card. + +### The permission that would hide this, and why it is not in the manifest + +`--filesystem=host` makes both work immediately and is the thing FR-PLAT-LIN-3 +names as the alternative to portals. Granting it would mean the sandboxed build +never exercises the sandbox, which removes the entire reason for shipping one +(§2). `--filesystem=xdg-pictures` is narrower and would be tempting, but it is +still a static grant that lets a typed path resolve — it makes the same design +work by not testing it, only in a smaller directory. + +So the manifest grants no filesystem access at all. The consequence is stated +plainly: **a Flatpak built from this manifest can open photographs handed to it +and cannot yet be pointed at a library.** + +### What closes it + +Two changes, in this order: + +1. **A portal file chooser behind a platform seam.** `ashpd`'s + `OpenFileRequest` with `directory(true)` returns a URI the document portal + has exported, which the sandbox can read and which stays valid across + restarts. It resolves to a real path under `/run/user/$UID/doc/`, so + `normalise_endpoint` accepts it as it stands — `canonicalize()` on a fuse + path returns the path itself. The seam matters more than the crate: this + belongs beside `LocalStorage::grant` in `dr-plat`, which is already the one + place a `Path` enters the application, and must not become a second way for + `ui/` to learn about paths. +2. **Removable volumes through the same door.** There is no portal for "list + the mounted cards". The honest answer is that under a sandbox + `imports_supported()` should report the same `false` it reports on Android, + for the same reason it gives there — the operation cannot be performed + however hard the user tries — and the import flow should offer the folder + chooser instead of a volume list. + +**Done when:** a Flatpak built from +[`packaging/flatpak/paris.tourolle.darkroom.yml`](../packaging/flatpak/paris.tourolle.darkroom.yml), +with its `finish-args` unchanged and no `flatpak override` applied, can select a +library root, scan it, and write a sidecar back into it. + +### Running a Flatpak build before then + +For testing the rest of the application inside the sandbox, grant the access +per-installation rather than in the manifest, so the file that describes the +application keeps telling the truth: + +```bash +flatpak override --user --filesystem=~/Pictures paris.tourolle.darkroom +``` + +--- + +## 5. AppImage + +A v1 channel, and the recipe is outstanding work rather than a decision to be +made. What it will have to account for, none of which is a surprise: + +- **glibc.** An AppImage links against the oldest glibc it must run on, so it + is built in a container with an old base rather than on a rolling-release + developer machine. A release binary built on a current rolling-release host carries + `GLIBC_2.44` references and would run on almost nothing else. +- **What to bundle and what not to.** The binary links fontconfig, freetype, + expat, libpng, zlib, brotli and bzip2 — bundle those. It does *not* link + Vulkan, libxkbcommon or either display-server library: wgpu `dlopen`s + `libvulkan.so.1`, and `x11rb` and `wayland-client` speak the wire protocols + in Rust. The Vulkan loader and the ICD must come from the host, and bundling + a loader is the classic way to break an AppImage on a driver it did not + expect. +- **The models.** ~15 MB of ONNX weights inside the image, or a first-run + download. In-tree is consistent with how the Lensfun database ships and with + NFR-SEC-5's local-first posture; the licence question (D13) is the same one + it is everywhere else and is not made easier or harder by this channel. +- **No sandbox.** An AppImage tests nothing about FR-PLAT-LIN-3. It is a + convenience channel for distributions the PKGBUILD does not serve, and should + never be the channel a portal problem is discovered on. + +--- + +## 6. Android: F-Droid in v1, Play deferred + +NFR-COMPAT-2 says Play distribution is what makes ARCH §6.9's constraints +binding, and that is worth reading precisely, because the constraint is already +met and would be met whatever the channel. + +§6.9 is *verified*, not assumed: `MANAGE_EXTERNAL_STORAGE` is not grantable +under Play policy, and `READ_MEDIA_IMAGES` would not help because proprietary +RAW is not typed `image/*` by the platform scanner and does not appear in +`MediaStore.Images`. SAF is the only route that works, so FR-PLAT-AND-1 asks +for it unconditionally and `SourceRef` (ARCH §3.1) exists to make it possible. +A sideloaded or F-Droid build *could* ask for broader permissions; it would +gain nothing by doing so. + +So the coupling runs the opposite way from how it is usually described. Play is +deferred for a reason that has nothing to do with storage: GPLv3 distribution +through Play is generally workable but has not been confirmed for this project +(ARCH §14), and F-Droid has no such question. Confirming it is a licence-reading +exercise; nothing in the storage design waits on the answer. + +--- + +## 7. Where the recipes live + +``` +packaging/ + PKGBUILD Arch source package + paris.tourolle.darkroom.desktop the desktop entry, installed by every channel + paris.tourolle.darkroom.metainfo.xml AppStream, installed by every channel + flatpak/ + paris.tourolle.darkroom.yml the manifest, and where the permissions are argued +``` + +`packaging/` also accumulates built `.pkg.tar.zst` artefacts from local +`makepkg` runs. Those are not part of any channel and should not be committed. diff --git a/packaging/PKGBUILD b/packaging/PKGBUILD index b75f03d..f5fde21 100644 --- a/packaging/PKGBUILD +++ b/packaging/PKGBUILD @@ -37,6 +37,14 @@ package() { install -Dm644 "packaging/paris.tourolle.darkroom.desktop" \ "${pkgdir}/usr/share/applications/paris.tourolle.darkroom.desktop" + # The same AppStream file the Flatpak installs, so a software centre + # describes the two packages identically instead of falling back to the + # desktop entry's one-line Comment for this one. Installed here rather than + # written twice: the description, the licence fields and the OARS rating + # are facts about the application, not about how it was packaged. + install -Dm644 "packaging/paris.tourolle.darkroom.metainfo.xml" \ + "${pkgdir}/usr/share/metainfo/paris.tourolle.darkroom.metainfo.xml" + # The icon's *name* is the contract, not its path: the desktop entry says # `Icon=paris.tourolle.darkroom` and the compositor resolves that through # the hicolor theme. Installed under 256x256 because that is the source's diff --git a/packaging/flatpak/paris.tourolle.darkroom.yml b/packaging/flatpak/paris.tourolle.darkroom.yml new file mode 100644 index 0000000..c6b9c27 --- /dev/null +++ b/packaging/flatpak/paris.tourolle.darkroom.yml @@ -0,0 +1,186 @@ +# Flatpak manifest — FR-PLAT-LIN-3 (sandboxed distribution), NFR-COMPAT-2. +# +# YAML rather than JSON because this file has more to explain than to declare, +# and JSON cannot hold a comment. flatpak-builder reads both. +# +# Build it, from the repository root: +# +# flatpak-builder --user --install --force-clean \ +# build/flatpak packaging/flatpak/paris.tourolle.darkroom.yml +# +# Read docs/distribution.md before changing any permission below. Every line in +# `finish-args` is a hole in the sandbox, and the one that is conspicuously +# absent — `--filesystem=` — is absent on purpose and is explained there. +id: paris.tourolle.darkroom + +# 25.08 is the current freedesktop runtime, and the choice is made by what the +# binary needs rather than by what is newest. `ldd` on a release build names +# fontconfig, freetype, expat, libpng, zlib, brotli and bzip2 — all in the +# Platform — and nothing else: Vulkan, libxkbcommon and the two display-server +# protocols are reached without a link-time dependency (wgpu dlopens +# libvulkan.so.1, and x11rb and wayland-client speak the wire protocols in Rust +# rather than binding libxcb or libwayland). So the runtime has to supply a +# Vulkan loader and an ICD at *runtime*, which is the GL extension's job, and +# not much else. +runtime: org.freedesktop.Platform +runtime-version: '25.08' +sdk: org.freedesktop.Sdk + +# The Rust toolchain is an SDK extension rather than something this manifest +# installs, so the build is offline-capable in the part that matters and the +# compiler is the one freedesktop tested against its own glibc. +# +# Note what this quietly overrides: `rust-toolchain.toml` pins 1.92.0, and that +# pin is honoured by *rustup*, which is not what the extension provides. The +# extension's cargo therefore ignores the file and builds with its own stable +# (1.98.0 on 25.08). That is fine here and deliberately different from +# CONTRIBUTING.md's "do not override the toolchain": the pin exists so `cargo +# fmt --check` and `clippy -D warnings` agree between a laptop and CI, and +# neither runs in this build. A *release binary* only needs a compiler at or +# above the workspace's `rust-version`. +sdk-extensions: + - org.freedesktop.Sdk.Extension.rust-stable + +command: darkroom-desktop + +finish-args: + # FR-PLAT-LIN-2 asks for both display servers. `fallback-x11` rather than + # `x11`: it grants the X socket only when Wayland is unavailable, so a + # Wayland session does not leave an X11 hole open beside the socket actually + # in use. `--share=ipc` goes with it — without it X11 cannot use shared + # memory and every frame is pushed through the socket instead. + - --socket=wayland + - --socket=fallback-x11 + - --share=ipc + + # The GPU. The develop pipeline is compute shaders through wgpu and there is + # no CPU renderer behind it, so this is not an optimisation: without + # /dev/dri the application starts and cannot develop anything. + # + # `dri` rather than `all`: it covers the render nodes and the NVIDIA device + # nodes, which is the whole of what a Vulkan ICD opens. `all` would add every + # other device on the machine for no gain. + - --device=dri + + # Nextcloud (FR-NC-*). Nothing else here reaches the network — face grouping, + # segmentation and lens correction are all local and stay local (NFR-SEC-5). + - --share=network + + # Credential storage (FR-NC-2). The keyring crate speaks the Secret Service + # D-Bus interface directly, which GNOME Keyring and KWallet's `ksecretd` both + # implement, so what it needs is a talk hole to that well-known name. + # + # Worth being precise, because FR-PLAT-LIN-3 says "Secret Service portal" and + # these are two different things: xdg-desktop-portal's `org.freedesktop. + # portal.Secret` hands an application a master key for a store it keeps + # itself, whereas this talks to the session's secret daemon. Only the latter + # puts the app password where `secret-tool` and Seahorse can see it, which is + # what makes a credential individually revocable by the user rather than + # opaque inside our own data directory. If no daemon answers, FR-NC-2's + # degraded mode is what the user gets — the same behaviour as outside a + # sandbox, which is the point. + - --talk-name=org.freedesktop.secrets + + # No `--filesystem=` line of any kind, and this is the substance of + # FR-PLAT-LIN-3 rather than an omission. + # + # What that leaves working: /run/user/$UID/doc is mounted in every sandbox, so + # a photograph opened from a file manager — the .desktop entry declares the + # RAW MIME types and `Exec=darkroom-desktop %F` — arrives as a document-portal + # path in argv and opens. That path is genuinely portal-mediated and needs no + # code change. + # + # What that leaves broken: choosing a *library root*. The folder connector + # takes a typed absolute path (`SignIn::EndpointOnly`, placeholder + # `/home/you/Pictures`) and checks it with `std::fs`, and nothing in the tree + # calls the FileChooser portal — there is no ashpd, no rfd, no toolkit dialog. + # A path typed into that field does not exist in this sandbox, so the launch + # screen refuses it with "that folder does not exist", which is at least an + # honest error. + # + # `--filesystem=host` would make that work today and is exactly what the + # requirement forbids, so it is not here. docs/distribution.md §4 records what + # closes the gap and how to run a Flatpak build in the meantime. + +modules: + - name: darkroom + buildsystem: simple + + build-options: + append-path: /usr/lib/sdk/rust-stable/bin + env: + # Inside the build sandbox rather than in $HOME, so a rebuild starts + # from the state flatpak-builder is managing and not from whatever the + # host's cargo cache happens to hold. + CARGO_HOME: /run/build/darkroom/cargo + # Cargo fetches 826 crates, and Flathub's builders forbid this — a + # submission there needs `cargo-sources.json` generated by + # flatpak-builder-tools' `flatpak-cargo-generator.py` from Cargo.lock, + # listing every crate as its own source, plus a vendored-registry + # `.cargo/config.toml`. That file is ~30k lines, has to be regenerated on + # every dependency change, and buys nothing for a build from a local + # checkout, which is what this manifest is for and what packaging/PKGBUILD + # is for as well. Add it when there is a Flathub submission, not before. + build-args: + - --share=network + + build-commands: + # `--locked` for the reason CI uses it: a lockfile that resolves + # differently in the packaging build than in the tree is a release whose + # dependency versions nobody chose. + - cargo build --release --locked -p darkroom-desktop + + - install -Dm755 target/release/darkroom-desktop /app/bin/darkroom-desktop + + - install -Dm644 packaging/paris.tourolle.darkroom.desktop + /app/share/applications/paris.tourolle.darkroom.desktop + + - install -Dm644 packaging/paris.tourolle.darkroom.metainfo.xml + /app/share/metainfo/paris.tourolle.darkroom.metainfo.xml + + # The icon's name is the contract, not its path — the desktop entry says + # `Icon=paris.tourolle.darkroom` and the shell resolves that through the + # hicolor theme. 256x256 because that is the source's actual size; + # installing it under a size it is not makes scaled icons look wrong. + - install -Dm644 ui/dr-ui/ui/app-icon.png + /app/share/icons/hicolor/256x256/apps/paris.tourolle.darkroom.png + + # The face models, where `system_face_models_dirs()` looks: it reads + # $XDG_DATA_DIRS, which includes /app/share inside a Flatpak, so this is + # the same lookup that finds /usr/share/darkroom/models from the Arch + # package. Last in the search order, so a pair the user dropped in their + # own data directory still outranks these. + # + # These live in Git LFS. A checkout made without `git lfs pull` has + # ~130-byte pointers here, and `type: dir` below would copy the pointers + # in without complaint — producing a Flatpak whose face indexing fails + # inside the graph loader on the user's machine. Refuse instead, with the + # command that fixes it. + - | + for m in scrfd_500m_640.onnx arcface_mbf_b1.onnx; do + if [ "$(stat -c%s "models/face/$m")" -lt 100000 ]; then + echo "error: $m is an LFS pointer, not a model — run: git lfs pull" >&2 + exit 1 + fi + install -Dm644 "models/face/$m" "/app/share/darkroom/models/$m" + done + + - install -Dm644 README.md /app/share/doc/darkroom/README.md + + sources: + # The local checkout, for the same reason packaging/PKGBUILD builds from + # one: this makes a Flatpak of what you are actually working on. Swap it + # for an `archive` or `git` source with a tag when there is a release to + # point at. + # + # `skip` is not tidiness. `target/` is tens of gigabytes and `.git` with + # LFS objects is not small; flatpak-builder copies a `dir` source + # wholesale, so without these two lines the copy is the slowest part of + # the build by a wide margin. + - type: dir + path: ../.. + skip: + - target + - target-android + - .git + - build diff --git a/packaging/paris.tourolle.darkroom.metainfo.xml b/packaging/paris.tourolle.darkroom.metainfo.xml new file mode 100644 index 0000000..864c662 --- /dev/null +++ b/packaging/paris.tourolle.darkroom.metainfo.xml @@ -0,0 +1,97 @@ + + + + + paris.tourolle.darkroom + + + CC0-1.0 + GPL-3.0-or-later + + DarkRoom + Non-destructive RAW photo library and editor + + +

+ DarkRoom catalogues, culls and develops RAW photographs. Edits are stored + as a graph of operations beside the original rather than baked into it, + so every change stays reversible and the file the camera wrote is never + rewritten. +

+

+ The library can live in a plain directory — a local disk, an external + drive, an NFS or SMB mount — or on a Nextcloud server, browsed and edited + without downloading whole RAW files first. +

+

Where it differs from the tools it sits beside:

+ + + + paris.tourolle.darkroom.desktop + + darkroom-desktop + + + https://gitea.tourolle.paris/dtourolle/DarkRoom + https://gitea.tourolle.paris/dtourolle/DarkRoom/issues + https://gitea.tourolle.paris/dtourolle/DarkRoom + + + Duncan Tourolle + + + + Graphics + Photography + + + + RAW + photography + develop + darkroom + catalog + + + + + 768 + + + pointing + keyboard + touch + + + + + + + +