Compare commits
194
Commits
v0.14.0
...
e22128c16b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e22128c16b | ||
|
|
81ea9359bc | ||
|
|
be37218696 | ||
|
|
5bff453d37 | ||
|
|
61cd226719 | ||
|
|
87d758bd81 | ||
|
|
427a572aad | ||
|
|
01197ca37d | ||
|
|
f9154ad12e | ||
|
|
1fdfb5990c | ||
|
|
7558053932 | ||
|
|
6bccc2db13 | ||
|
|
209ae36163 | ||
|
|
8071f7101a | ||
|
|
502023c0f4 | ||
|
|
adbb9ac9e6 | ||
|
|
2cde287440 | ||
|
|
6e68f1fb3d | ||
|
|
8df3dda9aa | ||
|
|
3912bc0d1d | ||
|
|
b1d1c47261 | ||
|
|
7be1efff32 | ||
|
|
c50d96e949 | ||
|
|
6b99f67f47 | ||
|
|
48c5e74fa8 | ||
|
|
fc54523093 | ||
|
|
8392cf772e | ||
|
|
515d4eb59e | ||
|
|
b2f3936a53 | ||
|
|
ec7a8c07ee | ||
|
|
78df4211b0 | ||
|
|
c78b798cf0 | ||
|
|
6450f54199 | ||
|
|
1479e45637 | ||
|
|
b5b30e3750 | ||
|
|
884b681c21 | ||
|
|
23abfd1827 | ||
|
|
94ea2569ee | ||
|
|
7a56d16df1 | ||
|
|
5baaf9bac2 | ||
|
|
7428f6f845 | ||
|
|
9f95d23ff4 | ||
|
|
1b8d0e740f | ||
|
|
d2b99bb8c1 | ||
|
|
f39b88b005 | ||
|
|
2f5f2041ab | ||
|
|
0f7ea741d8 | ||
|
|
dee509c6ef | ||
|
|
caaae11d98 | ||
|
|
e57c5b8182 | ||
|
|
02ddce8d80 | ||
|
|
faf52f6dbd | ||
|
|
ffdd640170 | ||
|
|
2226d543f9 | ||
|
|
bee5c5866f | ||
|
|
9b580c3720 | ||
|
|
1abb18d972 | ||
|
|
92d4b23bed | ||
|
|
7cbcacc02e | ||
|
|
2eb06b1064 | ||
|
|
6683c14b40 | ||
|
|
94542371f6 | ||
|
|
715fcf8512 | ||
|
|
49b7bc2f9d | ||
|
|
a59f14c797 | ||
|
|
e82190fdf2 | ||
|
|
a7b090cf36 | ||
|
|
90c0695c05 | ||
|
|
c6d4c1ba62 | ||
|
|
ce6705be89 | ||
|
|
ae0281fedd | ||
|
|
981022ab1d | ||
|
|
87badb6f99 | ||
|
|
d537e4a965 | ||
|
|
fe6e523443 | ||
|
|
73059f2656 | ||
|
|
81118728f4 | ||
|
|
408f189019 | ||
|
|
fabc1c5b56 | ||
|
|
441f6f1404 | ||
|
|
b3dbf4a039 | ||
|
|
6e67ef4467 | ||
|
|
5fcd3752d7 | ||
|
|
5da28584a4 | ||
|
|
8a1d9c8642 | ||
|
|
c3b10ed372 | ||
|
|
4bec01eaf1 | ||
|
|
3b97195b37 | ||
|
|
5794f8c6ea | ||
|
|
b1d36b9143 | ||
|
|
048d48f532 | ||
|
|
35d0696b66 | ||
|
|
9e099a07ab | ||
|
|
a76e3bdd02 | ||
|
|
0103200fc5 | ||
|
|
7c838691e9 | ||
|
|
6979b1a2b8 | ||
|
|
9253a16fed | ||
|
|
9b1f74e6d5 | ||
|
|
e31990550f | ||
|
|
79c051e8a6 | ||
|
|
ea44aba110 | ||
|
|
cc905fa2ed | ||
|
|
f7df276295 | ||
|
|
005dc3a835 | ||
|
|
825015a58e | ||
|
|
2ef971bf37 | ||
|
|
e957d483fc | ||
|
|
80938b0527 | ||
|
|
6640ce0ca8 | ||
|
|
7c9a4ee358 | ||
|
|
54aee50539 | ||
|
|
220e9af222 | ||
|
|
8d4ecb75c1 | ||
|
|
3c2eacbf3f | ||
|
|
d430ec9528 | ||
|
|
1dc7b45cfe | ||
|
|
284fc4a456 | ||
|
|
9de37bed81 | ||
|
|
380cfda695 | ||
|
|
d9f259656a | ||
|
|
19dd3257e3 | ||
|
|
a030bfd239 | ||
|
|
ce72fe49a0 | ||
|
|
4583048595 | ||
|
|
32b2a5e317 | ||
|
|
9e786532a1 | ||
|
|
1f26e1e627 | ||
|
|
2fd1b0b8ce | ||
|
|
9cff677392 | ||
|
|
454375243c | ||
|
|
c559d31dad | ||
|
|
058286750b | ||
|
|
548caa733c | ||
|
|
cf84cec96f | ||
|
|
f0e7b8e11c | ||
|
|
90c7cb65d3 | ||
|
|
70583b9b2f | ||
|
|
f426bb903a | ||
|
|
c41f99ea52 | ||
|
|
a6ea6ba83f | ||
|
|
b480de5bff | ||
|
|
a8043e6827 | ||
|
|
4b4c9e2e6d | ||
|
|
dc9da52651 | ||
|
|
dc1add9dbb | ||
|
|
b5ae5c2be1 | ||
|
|
aa2a88f655 | ||
|
|
f8737e3fda | ||
|
|
d489a34190 | ||
|
|
9d1e31ffbb | ||
|
|
150e53e878 | ||
|
|
7591738c73 | ||
|
|
352e59498b | ||
|
|
d8f26fb5cd | ||
|
|
10216355c1 | ||
|
|
d8e031888e | ||
|
|
114d979397 | ||
|
|
5a500118ae | ||
|
|
ff89a4fa21 | ||
|
|
04495afbfd | ||
|
|
96f1d5c896 | ||
|
|
f1db919b9d | ||
|
|
46f5b95828 | ||
|
|
d748527a4c | ||
|
|
89859d39d1 | ||
|
|
ade627a5d0 | ||
|
|
4642c77e18 | ||
|
|
7d0870c3fb | ||
|
|
c3d1f83b96 | ||
|
|
fc0ea8824d | ||
|
|
9772785f81 | ||
|
|
733a033274 | ||
|
|
414094bd38 | ||
|
|
d8fb382ce9 | ||
|
|
e8f68a92f8 | ||
|
|
8f3df7b68d | ||
|
|
5f0b7ac799 | ||
|
|
8cdad3863d | ||
|
|
229def0afc | ||
|
|
5569a066ff | ||
|
|
ea31791388 | ||
|
|
adade27de4 | ||
|
|
a3f3e188e1 | ||
|
|
031315bdb6 | ||
|
|
00c028c8c8 | ||
|
|
bddf3250c5 | ||
|
|
41486bd59b | ||
|
|
d6d27fb062 | ||
|
|
317a2f40bd | ||
|
|
949fe40d5b | ||
|
|
2be80d4203 | ||
|
|
9ebaa15099 | ||
|
|
aee355fada |
+5
-2
@@ -23,6 +23,9 @@ fixtures/** filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
# The manual's pictures live in LFS for the same reason the models do: a
|
||||
# screenshot or a GIF changes wholesale when the interface it shows changes,
|
||||
# and every re-recording would otherwise stay in every clone for good. CI's
|
||||
# pulls exclude the directory; nothing built or tested reads it.
|
||||
# and every re-recording would otherwise stay in every clone for good. The
|
||||
# desktop and benchmark legs exclude the directory, since nothing they build
|
||||
# or test reads it; the Android and Windows legs fetch it, because the APK
|
||||
# and the installer carry the manual (docs/manual/index.html) with its
|
||||
# pictures, and their packagers refuse a pointer.
|
||||
docs/manual/media/** filter=lfs diff=lfs merge=lfs -text
|
||||
|
||||
@@ -7,6 +7,10 @@ name: Build and test
|
||||
on:
|
||||
push:
|
||||
branches: [main, master, develop]
|
||||
# A release tag builds again and publishes what it built (the `release`
|
||||
# job at the end). The master push of the same commit has usually filled
|
||||
# the caches, so the second run is the warm one.
|
||||
tags: ['v*']
|
||||
pull_request:
|
||||
branches: [main, master, develop]
|
||||
|
||||
@@ -96,7 +100,9 @@ jobs:
|
||||
| while read -r key; do git config --local --unset-all "$key"; done || true
|
||||
git config --local lfs.url \
|
||||
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
|
||||
git lfs pull --exclude="fixtures/**,docs/manual/media/**"
|
||||
# The manual's pictures too: the APK carries the manual, and
|
||||
# assemble-apk.sh refuses a pointer where a picture should be.
|
||||
git lfs pull --exclude="fixtures/**"
|
||||
ls -lR models/
|
||||
|
||||
- name: Cache cargo
|
||||
@@ -151,9 +157,32 @@ jobs:
|
||||
- name: Test
|
||||
run: cargo test --workspace
|
||||
|
||||
# The runner has one 99 GB disk shared with its images, and this job
|
||||
# ended at 92 GB (2.1 GB free) on v0.18.0's run; v0.18.1's release
|
||||
# build then died with "No space left on device". The test executables
|
||||
# and examples in target/debug are the largest things in it, are never
|
||||
# reused (a changed source relinks them) and are not what the cache is
|
||||
# for — the dependency rlibs are — so they go before the release build
|
||||
# rather than competing with it.
|
||||
- name: Free the test binaries before the release build
|
||||
run: |
|
||||
find target/debug/deps -maxdepth 1 -type f -executable -delete
|
||||
rm -rf target/debug/examples target/debug/incremental
|
||||
df -h /workspace 2>/dev/null || df -h .
|
||||
|
||||
- name: Build
|
||||
run: cargo build --workspace --release
|
||||
|
||||
# Only on a release tag: the binary is 150 MB and nothing but the
|
||||
# release job wants it.
|
||||
- name: Upload the desktop binary
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: darkroom-desktop-x86_64-linux
|
||||
path: target/release/darkroom-desktop
|
||||
if-no-files-found: error
|
||||
|
||||
- name: Disk after
|
||||
if: always()
|
||||
run: df -h /workspace 2>/dev/null || df -h .
|
||||
@@ -213,7 +242,9 @@ jobs:
|
||||
| while read -r key; do git config --local --unset-all "$key"; done || true
|
||||
git config --local lfs.url \
|
||||
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
|
||||
git lfs pull --exclude="fixtures/**,docs/manual/media/**"
|
||||
# The manual's pictures too: the APK carries the manual, and
|
||||
# assemble-apk.sh refuses a pointer where a picture should be.
|
||||
git lfs pull --exclude="fixtures/**"
|
||||
ls -lR models/
|
||||
|
||||
- name: Cache cargo
|
||||
@@ -406,7 +437,9 @@ jobs:
|
||||
| while read -r key; do git config --local --unset-all "$key"; done || true
|
||||
git config --local lfs.url \
|
||||
"https://x-access-token:${LFS_TOKEN}@gitea.tourolle.paris/dtourolle/DarkRoom.git/info/lfs"
|
||||
git lfs pull --exclude="fixtures/**,docs/manual/media/**"
|
||||
# The manual's pictures too: the installer carries the manual, and
|
||||
# package.sh refuses a pointer where a picture should be.
|
||||
git lfs pull --exclude="fixtures/**"
|
||||
ls -l models/face models/scene
|
||||
|
||||
- name: Cache cargo
|
||||
@@ -460,6 +493,11 @@ jobs:
|
||||
WANT=$(find models/face models/scene models/inpaint -maxdepth 1 -type f ! -name README.md | wc -l)
|
||||
GOT=$(ls "$INST/models" | wc -l)
|
||||
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT model files, installed $GOT"; exit 1; }
|
||||
# The manual, and every picture it shows, counted the same way.
|
||||
[ -f "$INST/manual/index.html" ] || { echo "FAIL: no manual installed"; exit 1; }
|
||||
WANT=$(ls docs/manual/media | wc -l)
|
||||
GOT=$(ls "$INST/manual/media" | wc -l)
|
||||
[ "$GOT" = "$WANT" ] || { echo "FAIL: expected $WANT manual pictures, installed $GOT"; exit 1; }
|
||||
wine reg query 'HKCU\Software\Microsoft\Windows\CurrentVersion\Uninstall\DarkRoom' 2>/dev/null \
|
||||
| grep -q DisplayVersion || { echo "FAIL: no uninstall registry key"; exit 1; }
|
||||
wine "$INST/darkroom.exe" --version 2>/dev/null | grep -q '^darkroom-desktop ' \
|
||||
@@ -519,3 +557,47 @@ jobs:
|
||||
fi
|
||||
done
|
||||
exit $FAILED
|
||||
|
||||
# A v* tag becomes a Gitea Release carrying the three builds and their
|
||||
# SHA256SUMS, titled and described by the tag's message. Until this job
|
||||
# existed every release was made by hand, and most tags never got one.
|
||||
#
|
||||
# It needs all three platform jobs, so a tag whose tests fail publishes
|
||||
# nothing; re-run the failed job and this one follows. The work is
|
||||
# tools/publish-release.sh, which is also how a release is finished by hand.
|
||||
release:
|
||||
if: startsWith(github.ref, 'refs/tags/v')
|
||||
needs: [desktop, android, windows]
|
||||
runs-on: linux/amd64
|
||||
name: Publish the release
|
||||
container:
|
||||
image: catthehacker/ubuntu:act-latest
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Fetch the builds
|
||||
uses: actions/download-artifact@v3
|
||||
with:
|
||||
path: dist
|
||||
|
||||
# Named for the download page, with the version in each name the way
|
||||
# the hand-made releases had them. The installer already carries its
|
||||
# version from package.sh.
|
||||
- name: Publish
|
||||
env:
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN || github.token }}
|
||||
TAG: ${{ github.ref_name }}
|
||||
run: |
|
||||
set -e
|
||||
V="${TAG#v}"
|
||||
ls -lR dist
|
||||
mkdir -p out
|
||||
cp dist/darkroom-arm64-v8a-apk/darkroom.apk "out/darkroom-${V}-arm64-v8a.apk"
|
||||
cp dist/darkroom-desktop-x86_64-linux/darkroom-desktop "out/darkroom-desktop-${V}-x86_64-linux"
|
||||
chmod +x "out/darkroom-desktop-${V}-x86_64-linux"
|
||||
cp dist/darkroom-windows-x86_64-setup/DarkRoom-${V}-x86_64-setup.exe out/
|
||||
bash tools/publish-release.sh "$TAG" out/*
|
||||
|
||||
@@ -67,6 +67,12 @@ jobs:
|
||||
# threshold: zero requirements parsed, zero files scanned, a ratio above
|
||||
# 100%, or any orphan tag all fail the build. A misconfigured run must not
|
||||
# report a plausible-looking 0%.
|
||||
# Every picture the manual shows is made by a scene in
|
||||
# tools/manual/scenes.py, and every picture a scene makes is shown.
|
||||
# Two files read; no app, no display.
|
||||
- name: Manual pictures have scenes
|
||||
run: tools/manual/record.sh --check
|
||||
|
||||
- name: Traceability gate
|
||||
run: cargo run -q -p traceability -- check
|
||||
|
||||
@@ -91,10 +97,19 @@ jobs:
|
||||
# they will conclude the application is broken rather than the page.
|
||||
#
|
||||
# This also fails on a malformed tag, so a typo costs a gesture its
|
||||
# desktop half loudly rather than silently.
|
||||
# desktop half loudly rather than silently — and on a key a Slint
|
||||
# handler binds that no tag names, or a key a tag names that no handler
|
||||
# binds (tools/traceability/src/keymap.rs).
|
||||
- name: Regenerate the gesture vocabulary and check it is committed
|
||||
run: cargo run -q -p traceability -- gestures-check
|
||||
|
||||
# The manual's page, which the packages carry and the help sheet links
|
||||
# into. Blocking for the gesture book's reason: it is shown to the user,
|
||||
# and a page that disagrees with the README is a manual describing an
|
||||
# application that no longer exists.
|
||||
- name: Regenerate the manual page and check it is committed
|
||||
run: cargo run -q -p traceability -- manual-check
|
||||
|
||||
# Advisory, not blocking: not every file implements a requirement, and a
|
||||
# tag on every function is noise that rots faster than it helps. Tag the
|
||||
# unit that decides.
|
||||
|
||||
+15
-1
@@ -26,7 +26,7 @@ fi
|
||||
# The artefacts are generated from the tree, so regenerating them because one
|
||||
# was itself edited would be circular.
|
||||
case "$(tr -d '[:space:]' <<< "${staged}")" in
|
||||
docs/dev/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs)
|
||||
docs/dev/traceability.md | docs/gestures.md | ui/dr-ui/src/gesture_book.rs | docs/manual/index.html)
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
@@ -65,3 +65,17 @@ for f in docs/gestures.md ui/dr-ui/src/gesture_book.rs; do
|
||||
echo "pre-commit: regenerated ${f} and staged it"
|
||||
fi
|
||||
done
|
||||
|
||||
# The manual's page, when its source is part of the commit. Rendered from
|
||||
# nothing but the README, so there is no reason to pay for it otherwise.
|
||||
if grep -qx 'docs/manual/README.md' <<< "${staged}"; then
|
||||
if ! out="$(cargo run -q -p traceability -- manual 2>&1)"; then
|
||||
echo "pre-commit: the manual would not render" >&2
|
||||
echo "${out}" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! git diff --quiet -- docs/manual/index.html; then
|
||||
git add docs/manual/index.html
|
||||
echo "pre-commit: regenerated docs/manual/index.html and staged it"
|
||||
fi
|
||||
fi
|
||||
|
||||
@@ -31,6 +31,30 @@ over a result set — including `deep_count` per sidebar row, which is fine
|
||||
at sidebar scale and would not be at grid scale. Aggregate in one
|
||||
statement and look up in memory.
|
||||
|
||||
**SQL text in a loop is a prepare in a loop.** rusqlite's `execute` and
|
||||
`query_row` compile their statement on every call. A loop that calls them
|
||||
per row pays a prepare per row even when each query is a primary-key seek:
|
||||
the merge of a synced catalog prepared four statements for each of 13,000
|
||||
incoming faces (450 ms of a pass that changed nothing), `persist` four per
|
||||
photograph a scan listed (1.5 s for a first scan), the shard sync one per
|
||||
image each way. Hoist the statement, use `prepare_cached`, or — better, when
|
||||
the loop asks the same table about every row — read that table once into a
|
||||
map. And do not rewrite a row with what it already holds: an upsert of
|
||||
identical values still dirties the page.
|
||||
|
||||
**A `LIKE` is case-insensitive, and no index here serves that.**
|
||||
`source_ref LIKE 'stem.%'` read every name of the root per sidecar a pull
|
||||
took in. When the check that decides is exact, spell the prefix as a range
|
||||
(`>= 'stem.' AND < 'stem/'`, `/` being the byte after `.`), which the
|
||||
`(root_id, source_ref)` key answers with a seek.
|
||||
|
||||
**`Catalog::open` is not free, and every worker thread calls it.** The
|
||||
backfill runs on every open, and the develop view opens a catalog to fetch
|
||||
each original and again for each neighbour it prefetches. Keep each
|
||||
backfill step's no-op case to a read of the small side — the unpaired
|
||||
JPEGs, not every RAW; the distinct keywords, not every assignment — and
|
||||
measure an open with `catalog_bench` after adding one.
|
||||
|
||||
**Filter and aggregate in SQL, and aggregate the small side first.**
|
||||
`faces::people` read 19,000 rows, grouped, sorted them by name, and the
|
||||
screen threw 17,000 away (empty unnamed groups). `people_in_use` filters in
|
||||
@@ -124,6 +148,15 @@ other builds are running on the machine — the wall clock doubles under
|
||||
load, the CPU figure does not. Keep the binary from before the change and
|
||||
run both back to back rather than trusting numbers taken an hour apart.
|
||||
|
||||
`cargo run --release -p dr-catalog --example catalog_bench -- CATALOG
|
||||
[FACES_DIR]` does the same for opening the catalog (the backfill step by
|
||||
step), the upload snapshot, a merge, and the face shard export and import;
|
||||
`persist_bench`, an ignored test in `dr-ui`'s scan module, replays a scan's
|
||||
`persist` and a sidecar pull (`DR_BENCH_CATALOG=copy.sqlite cargo test
|
||||
--release -p dr-ui --lib persist_bench -- --ignored --nocapture
|
||||
--test-threads=1`). Both take copies; hand `catalog_bench` a copy of the
|
||||
face store directory too.
|
||||
|
||||
Reference figures from the 2026-09-19 fixes, largest person (754 faces),
|
||||
24k images, 19k faces, before → after. What one click read: `load_people`
|
||||
22 ms → 12 ms, `load_faces` 316 ms → 2.4 ms, `audit` 190 ms → not run
|
||||
|
||||
+32
-4
@@ -1,6 +1,6 @@
|
||||
# Contributing to DarkRoom
|
||||
|
||||
There is a lot of documentation here — 14 documents and 177 numbered
|
||||
There is a lot of documentation here — twenty-odd documents and 192 numbered
|
||||
requirements — and almost all of it is written for someone who has already
|
||||
decided to work on this. This file is the other thing: how to get a first
|
||||
change landed without reading any of it.
|
||||
@@ -62,7 +62,7 @@ sudo apt-get install pkg-config libfontconfig1-dev libxkbcommon-dev
|
||||
cargo run -p darkroom-desktop
|
||||
```
|
||||
|
||||
The first build resolves 826 crates and takes a while — on a laptop, long
|
||||
The first build resolves some 850 crates and takes a while — on a laptop, long
|
||||
enough to look like a hang. It is not one.
|
||||
|
||||
Android is a containerised toolchain and is not needed for most work; see
|
||||
@@ -130,9 +130,27 @@ strength of plumbing a future feature would use. So: **close a requirement
|
||||
with a test that would fail if the behaviour were removed.** Coverage that
|
||||
moves slowly and means something beats coverage that moves quickly.
|
||||
|
||||
## Two invariants the build defends
|
||||
**Keys and gestures are held the same way.** A key handler in Slint compares
|
||||
one canonical chord, `Keys.chord(event) == "Ctrl+Z"`, under a `// KEYMAP:`
|
||||
comment naming its section of the gesture book, and every key it binds must be
|
||||
named by a `GESTURE:` block beside it. `cargo run -p traceability -- gestures`
|
||||
regenerates [`docs/gestures.md`](docs/gestures.md) and the in-app help sheet
|
||||
from those blocks; `-- gestures-check` fails when a handler binds a key no tag
|
||||
names, or a tag names a key no handler binds. A `manual:` field in a block
|
||||
links the gesture to a section of the manual, and a heading that is not there
|
||||
fails the scan.
|
||||
|
||||
Worth knowing before you trip one, because both failures name a requirement
|
||||
**The manual is checked too.** `docs/manual/index.html` is rendered from
|
||||
`docs/manual/README.md` by `-- manual` and `-- manual-check` fails when they
|
||||
differ; `tools/manual/record.sh --check` fails when the manual shows a picture
|
||||
no scene in `tools/manual/scenes.py` makes. If you change what a pictured
|
||||
screen looks like, [`tools/manual`](tools/manual/README.md) says how to record
|
||||
it again. The pre-commit hook regenerates the matrix, the gesture book and the
|
||||
page; CI runs all three checks.
|
||||
|
||||
## Three invariants the build defends
|
||||
|
||||
Worth knowing before you trip one, because each failure names a requirement
|
||||
rather than a line:
|
||||
|
||||
- **No operation may be named in `ui/`** (FR-DEV-3a). Special-casing one
|
||||
@@ -144,6 +162,16 @@ rather than a line:
|
||||
`order:`, a filename disagreeing with its `id:`, a default outside its own
|
||||
range, an expression naming something that is not a parameter. Each error
|
||||
names the key you got wrong and exits rather than panicking.
|
||||
- **No verdict is written without a user action** (FR-CULL-13). A rating,
|
||||
flag, colour label or trash membership is the photographer's to set, never a
|
||||
signal's. `tools/traceability/src/verdicts.rs` finds every write of one in
|
||||
the shipped code — the catalog setters, SQL that assigns those columns, the
|
||||
sidecar's judgement amendment — and holds each to a reviewed list with its
|
||||
reason: inside a Slint `on_*` callback, writing for callers that are checked
|
||||
in turn, or carrying a verdict made elsewhere, such as a sidecar pull or the
|
||||
sync merge. A new write fails `cargo test` (the `traceability` crate's tests,
|
||||
part of the workspace run) until it is listed, and so does a listed one that
|
||||
has gone; `cargo run -p traceability -- verdicts` prints the list.
|
||||
|
||||
## Commit messages
|
||||
|
||||
|
||||
Generated
+199
-30
@@ -347,6 +347,28 @@ dependencies = [
|
||||
"libloading",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ashpd"
|
||||
version = "0.11.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d2f3f79755c74fd155000314eb349864caa787c6592eace6c6882dad873d9c39"
|
||||
dependencies = [
|
||||
"async-fs",
|
||||
"async-net",
|
||||
"enumflags2",
|
||||
"futures-channel",
|
||||
"futures-util",
|
||||
"rand 0.9.5",
|
||||
"raw-window-handle",
|
||||
"serde",
|
||||
"serde_repr",
|
||||
"url",
|
||||
"wayland-backend",
|
||||
"wayland-client",
|
||||
"wayland-protocols",
|
||||
"zbus",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-broadcast"
|
||||
version = "0.7.2"
|
||||
@@ -385,6 +407,17 @@ dependencies = [
|
||||
"slab",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-fs"
|
||||
version = "2.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8034a681df4aed8b8edbd7fbe472401ecf009251c8b40556b304567052e294c5"
|
||||
dependencies = [
|
||||
"async-lock",
|
||||
"blocking",
|
||||
"futures-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-io"
|
||||
version = "2.6.0"
|
||||
@@ -414,6 +447,17 @@ dependencies = [
|
||||
"pin-project-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-net"
|
||||
version = "2.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b948000fad4873c1c9339d60f2623323a0cfd3816e5181033c6a5cb68b2accf7"
|
||||
dependencies = [
|
||||
"async-io",
|
||||
"blocking",
|
||||
"futures-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "async-process"
|
||||
version = "2.5.0"
|
||||
@@ -1221,7 +1265,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
|
||||
|
||||
[[package]]
|
||||
name = "darkroom-android"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"android_logger",
|
||||
"dr-plat",
|
||||
@@ -1234,7 +1278,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "darkroom-desktop"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"dr-plat",
|
||||
@@ -1356,6 +1400,8 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e0e367e4e7da84520dedcac1901e4da967309406d1e51017ae1abfb97adbd38"
|
||||
dependencies = [
|
||||
"bitflags 2.13.1",
|
||||
"block2 0.6.2",
|
||||
"libc",
|
||||
"objc2 0.6.4",
|
||||
]
|
||||
|
||||
@@ -1408,7 +1454,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
|
||||
|
||||
[[package]]
|
||||
name = "dr-bench"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"dr-catalog",
|
||||
@@ -1425,7 +1471,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-catalog"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-face",
|
||||
"dr-plat",
|
||||
@@ -1440,7 +1486,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-decode"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-types",
|
||||
"env_logger",
|
||||
@@ -1454,7 +1500,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-export"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-decode",
|
||||
"dr-gpu",
|
||||
@@ -1473,7 +1519,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-face"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-inference-engine",
|
||||
"env_logger",
|
||||
@@ -1486,7 +1532,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-film"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"log",
|
||||
"serde",
|
||||
@@ -1495,7 +1541,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-gpu"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"bytemuck",
|
||||
"dr-decode",
|
||||
@@ -1513,7 +1559,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-inference-engine"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"env_logger",
|
||||
"libloading",
|
||||
@@ -1528,7 +1574,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-ingest"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-plat",
|
||||
"dr-types",
|
||||
@@ -1540,7 +1586,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-lens"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"lensfun",
|
||||
"log",
|
||||
@@ -1548,7 +1594,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-pano"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-decode",
|
||||
"dr-inference-engine",
|
||||
@@ -1562,7 +1608,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-pipeline"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-types",
|
||||
"log",
|
||||
@@ -1571,7 +1617,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-plat"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"android-native-keyring-store",
|
||||
"dr-types",
|
||||
@@ -1587,7 +1633,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-preset-xmp"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-pipeline",
|
||||
"log",
|
||||
@@ -1597,7 +1643,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-segment"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-inference-engine",
|
||||
"env_logger",
|
||||
@@ -1610,7 +1656,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-sync"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"dr-plat",
|
||||
@@ -1624,7 +1670,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-sync-folder"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"dr-sync",
|
||||
@@ -1636,7 +1682,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-sync-nextcloud"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"async-trait",
|
||||
"dr-decode",
|
||||
@@ -1658,7 +1704,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-thumbs"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-types",
|
||||
"jpeg-encoder",
|
||||
@@ -1670,7 +1716,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-types"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"serde",
|
||||
"serde_json",
|
||||
@@ -1679,7 +1725,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "dr-ui"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"async-trait",
|
||||
@@ -1704,25 +1750,30 @@ dependencies = [
|
||||
"dr-types",
|
||||
"dr-xmp",
|
||||
"env_logger",
|
||||
"i-slint-backend-testing",
|
||||
"jni 0.22.4",
|
||||
"log",
|
||||
"ndk-context",
|
||||
"png",
|
||||
"pollster",
|
||||
"raw-window-handle",
|
||||
"reqwest",
|
||||
"rfd",
|
||||
"rusqlite",
|
||||
"serde_json",
|
||||
"serde_norway",
|
||||
"sha2",
|
||||
"slint",
|
||||
"slint-build",
|
||||
"thiserror 2.0.20",
|
||||
"tokio",
|
||||
"url",
|
||||
"wgpu",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dr-xmp"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"dr-types",
|
||||
"log",
|
||||
@@ -2780,6 +2831,18 @@ dependencies = [
|
||||
"i-slint-renderer-skia",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "i-slint-backend-testing"
|
||||
version = "1.17.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "521e901e3d47ab829c0ef500c63155776208707cd93259e6a7803ed627fa2786"
|
||||
dependencies = [
|
||||
"cfg_aliases",
|
||||
"i-slint-common",
|
||||
"i-slint-core",
|
||||
"vtable",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "i-slint-backend-winit"
|
||||
version = "1.17.1"
|
||||
@@ -2960,8 +3023,6 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "i-slint-renderer-skia"
|
||||
version = "1.17.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7b6eed7f3f0a9a3d3ca6e8b9d4ca233371d989351fdb2a7ab88ec368b99e7b57"
|
||||
dependencies = [
|
||||
"ash",
|
||||
"bytemuck",
|
||||
@@ -5655,6 +5716,30 @@ dependencies = [
|
||||
"zune-jpeg 0.5.15",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rfd"
|
||||
version = "0.16.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a15ad77d9e70a92437d8f74c35d99b4e4691128df018833e99f90bcd36152672"
|
||||
dependencies = [
|
||||
"ashpd",
|
||||
"block2 0.6.2",
|
||||
"dispatch2",
|
||||
"js-sys",
|
||||
"log",
|
||||
"objc2 0.6.4",
|
||||
"objc2-app-kit 0.3.2",
|
||||
"objc2-core-foundation",
|
||||
"objc2-foundation 0.3.2",
|
||||
"pollster",
|
||||
"raw-window-handle",
|
||||
"urlencoding",
|
||||
"wasm-bindgen",
|
||||
"wasm-bindgen-futures",
|
||||
"web-sys",
|
||||
"windows-sys 0.60.2",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "rgb"
|
||||
version = "0.8.53"
|
||||
@@ -6273,6 +6358,7 @@ dependencies = [
|
||||
"num-traits",
|
||||
"once_cell",
|
||||
"pin-weak",
|
||||
"raw-window-handle",
|
||||
"slint-macros",
|
||||
"unicode-segmentation",
|
||||
"vtable",
|
||||
@@ -7023,11 +7109,14 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
|
||||
|
||||
[[package]]
|
||||
name = "traceability"
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"proc-macro2",
|
||||
"pulldown-cmark",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"syn 2.0.119",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -7433,8 +7522,15 @@ dependencies = [
|
||||
"idna",
|
||||
"percent-encoding",
|
||||
"serde",
|
||||
"serde_derive",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "urlencoding"
|
||||
version = "2.1.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "daf8dba3b7eb870caf1ddeed7bc9d2a049f3cfdfae7cb521b087cc33ae4c49da"
|
||||
|
||||
[[package]]
|
||||
name = "usvg"
|
||||
version = "0.47.0"
|
||||
@@ -7923,8 +8019,6 @@ dependencies = [
|
||||
[[package]]
|
||||
name = "wgpu-hal"
|
||||
version = "29.0.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "97ace1c17727311c22a46e4e3faf56ea6de81af99dcc839bdfb54857b94d448d"
|
||||
dependencies = [
|
||||
"android_system_properties",
|
||||
"arrayvec",
|
||||
@@ -8157,6 +8251,15 @@ dependencies = [
|
||||
"windows-targets 0.52.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.60.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb"
|
||||
dependencies = [
|
||||
"windows-targets 0.53.5",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-sys"
|
||||
version = "0.61.2"
|
||||
@@ -8205,13 +8308,30 @@ dependencies = [
|
||||
"windows_aarch64_gnullvm 0.52.6",
|
||||
"windows_aarch64_msvc 0.52.6",
|
||||
"windows_i686_gnu 0.52.6",
|
||||
"windows_i686_gnullvm",
|
||||
"windows_i686_gnullvm 0.52.6",
|
||||
"windows_i686_msvc 0.52.6",
|
||||
"windows_x86_64_gnu 0.52.6",
|
||||
"windows_x86_64_gnullvm 0.52.6",
|
||||
"windows_x86_64_msvc 0.52.6",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-targets"
|
||||
version = "0.53.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3"
|
||||
dependencies = [
|
||||
"windows-link",
|
||||
"windows_aarch64_gnullvm 0.53.1",
|
||||
"windows_aarch64_msvc 0.53.1",
|
||||
"windows_i686_gnu 0.53.1",
|
||||
"windows_i686_gnullvm 0.53.1",
|
||||
"windows_i686_msvc 0.53.1",
|
||||
"windows_x86_64_gnu 0.53.1",
|
||||
"windows_x86_64_gnullvm 0.53.1",
|
||||
"windows_x86_64_msvc 0.53.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "windows-threading"
|
||||
version = "0.2.1"
|
||||
@@ -8239,6 +8359,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_gnullvm"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_msvc"
|
||||
version = "0.42.2"
|
||||
@@ -8257,6 +8383,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469"
|
||||
|
||||
[[package]]
|
||||
name = "windows_aarch64_msvc"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnu"
|
||||
version = "0.42.2"
|
||||
@@ -8275,12 +8407,24 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnu"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnullvm"
|
||||
version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_gnullvm"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_msvc"
|
||||
version = "0.42.2"
|
||||
@@ -8299,6 +8443,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66"
|
||||
|
||||
[[package]]
|
||||
name = "windows_i686_msvc"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnu"
|
||||
version = "0.42.2"
|
||||
@@ -8317,6 +8467,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnu"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnullvm"
|
||||
version = "0.42.2"
|
||||
@@ -8335,6 +8491,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_gnullvm"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_msvc"
|
||||
version = "0.42.2"
|
||||
@@ -8353,6 +8515,12 @@ version = "0.52.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec"
|
||||
|
||||
[[package]]
|
||||
name = "windows_x86_64_msvc"
|
||||
version = "0.53.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650"
|
||||
|
||||
[[package]]
|
||||
name = "winit"
|
||||
version = "0.30.13"
|
||||
@@ -8818,6 +8986,7 @@ dependencies = [
|
||||
"endi",
|
||||
"enumflags2",
|
||||
"serde",
|
||||
"url",
|
||||
"winnow 1.0.4",
|
||||
"zvariant_derive",
|
||||
"zvariant_utils",
|
||||
|
||||
+23
-1
@@ -27,9 +27,12 @@ members = [
|
||||
"tools/bench",
|
||||
"tools/traceability",
|
||||
]
|
||||
# Patched copies of upstream crates, not our code: see third_party/README.md.
|
||||
# Excluded so `--workspace` does not test, lint or format them as ours.
|
||||
exclude = ["third_party"]
|
||||
|
||||
[workspace.package]
|
||||
version = "0.14.0"
|
||||
version = "0.18.2"
|
||||
edition = "2021"
|
||||
rust-version = "1.92"
|
||||
license = "GPL-3.0-or-later"
|
||||
@@ -127,6 +130,16 @@ url = "2.5"
|
||||
async-trait = "0.1"
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
# The manual's HTML rendering (tools/traceability). Already in the tree as
|
||||
# Slint's Markdown parser, so this adds a dependency edge and no crate; only
|
||||
# the HTML writer is needed, not the command-line front end.
|
||||
pulldown-cmark = { version = "0.13", default-features = false, features = ["html"] }
|
||||
# The verdict-writer check (tools/traceability, FR-CULL-13) reads Rust as Rust:
|
||||
# a text scan cannot tell a call from a comment, a test module from shipped
|
||||
# code, or which callback closure a call sits in. Both already in the tree as
|
||||
# every proc macro's parser; `span-locations` gives a problem its line.
|
||||
syn = { version = "2", default-features = false, features = ["full", "parsing", "visit", "printing"] }
|
||||
proc-macro2 = { version = "1", default-features = false, features = ["span-locations"] }
|
||||
base64 = "0.23"
|
||||
|
||||
# Display-server clients, for FR-DSP-8's per-display profile acquisition.
|
||||
@@ -262,3 +275,12 @@ opt-level = 0
|
||||
[profile.release]
|
||||
lto = "thin"
|
||||
codegen-units = 1
|
||||
|
||||
# Two upstream crates carry a local patch so that the Android build can draw
|
||||
# with wgpu on a rotated display (technical-debt.md TD-1). Both are exact
|
||||
# copies of the version the lockfile already resolves, plus that patch;
|
||||
# third_party/README.md says what was changed and how to carry it forward
|
||||
# when Slint or wgpu moves.
|
||||
[patch.crates-io]
|
||||
wgpu-hal = { path = "third_party/wgpu-hal-29.0.4" }
|
||||
i-slint-renderer-skia = { path = "third_party/i-slint-renderer-skia-1.17.1" }
|
||||
|
||||
@@ -15,19 +15,30 @@ is still missing.
|
||||
**A library.** Point it at a folder — on this machine, on a network mount,
|
||||
or one a Nextcloud client keeps in virtual-files mode, where a placeholder
|
||||
is treated as the photograph rather than as a one-byte file — or at a
|
||||
Nextcloud account directly. The grid is virtualised, ordered by capture
|
||||
time with a timeline beside it, and filtered by rating, flag, person and
|
||||
whether the file is here. Ratings, keywords, collections and a
|
||||
trash that survives a crash mid-operation. Card ingest. Bursts fold. Face
|
||||
detection and identity, with the index syncing between devices.
|
||||
Nextcloud account directly; a photograph that is only on the server opens on
|
||||
its thumbnail with the download's progress over it. The grid is virtualised,
|
||||
ordered by capture time with a timeline beside it, and filtered by rating,
|
||||
flag, colour label, person and whether the file is here. Ratings, colour
|
||||
labels, keywords, collections and a trash that survives a crash
|
||||
mid-operation. Card ingest. Bursts fold. The same RAW catalogued twice — a
|
||||
dated folder and a backup beside it — is found, proved the same, and folded
|
||||
onto one copy with the spares in the trash. Face detection and identity,
|
||||
with the index syncing between devices.
|
||||
|
||||
**Developing.** Eighteen declared operations fused into one compute
|
||||
dispatch, plus the neighbourhood work that cannot be: clarity, texture,
|
||||
capture sharpening, noise reduction, lens correction, spectral film
|
||||
simulation. Crop and straighten, spot repair, and local adjustments over
|
||||
masks the model draws — click a subject or a category, then paint, subtract
|
||||
a gradient, grow or shrink the edge. Focus peaking and a raw histogram for
|
||||
judging what is recoverable. Named presets; XMP sidecars other editors read.
|
||||
simulation. Crop, straighten and correct converging verticals, spot repair,
|
||||
and local adjustments over masks the model draws — click a subject or a
|
||||
category, then paint, subtract a gradient or keep only where two selections
|
||||
agree, grow or shrink the edge. A mask's sliders add to the photograph's, the
|
||||
film's among them, so a sky can be burned in on the print as a darkroom
|
||||
printer would. Hot and dead photosites are mended before the demosaic, with
|
||||
nothing to set. Focus peaking and a raw histogram for judging
|
||||
what is recoverable. Presets, with a collection shipped in the application —
|
||||
everyday corrections, and a look for each measured colour, cinema and
|
||||
black-and-white stock — and Lightroom presets imported as looks that leave a
|
||||
photograph's own corrections alone. XMP sidecars other editors read.
|
||||
|
||||
[](docs/manual/README.md#local-adjustments)
|
||||
|
||||
@@ -37,13 +48,22 @@ sources as a DNG, with a sidecar recording what it was merged from.
|
||||
|
||||
[](docs/manual/README.md#merging-a-panorama)
|
||||
|
||||
**From the keyboard, and with its manual.** Rating, flagging and labelling
|
||||
have keys in the grid and in develop, as do zoom, undo and stepping through a
|
||||
shoot in develop, and none of them is keyboard-only. The help sheet (`F1`, or
|
||||
`?` in develop) lists every key and gesture, generated from the code that
|
||||
binds it, and links them to the sections of the manual that show them — the
|
||||
manual ships with the application and opens offline.
|
||||
|
||||
**Export.** JPEG, PNG, AVIF, JPEG XL, 8- and 16-bit TIFF, with resize, output
|
||||
sharpening, a naming template and a colour space — to a folder here or back
|
||||
into the library.
|
||||
sharpening, a naming template and a colour space — into albums: named export
|
||||
folders on this machine or on the server, never inside the library, which
|
||||
remember the photograph behind each file and sync between devices as
|
||||
collections do.
|
||||
|
||||
**On both platforms.** The same core runs on a desktop and a 12-inch
|
||||
tablet; the interface is one layout, tuned for a wide viewport with touch
|
||||
targets throughout. On desktop the develop view draws the compute pass's
|
||||
targets throughout. On both, the develop view draws the compute pass's
|
||||
texture directly — no readback between the GPU and the screen.
|
||||
|
||||
## Getting it
|
||||
@@ -53,7 +73,7 @@ texture directly — no readback between the GPU and the screen.
|
||||
| Arch Linux | [`packaging/PKGBUILD`](packaging/PKGBUILD) — `makepkg -si` | Built from every release |
|
||||
| Android | The APK from each CI run, or `./docker/android/package.sh --install` | Runs on a tablet; F-Droid not yet submitted |
|
||||
| Windows | `DarkRoom-<version>-x86_64-setup.exe`, cross-built by CI ([windows.md](docs/dev/windows.md)) | Verified under Wine only; unsigned |
|
||||
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; choosing a library does not yet work in the sandbox |
|
||||
| Flatpak | [`packaging/flatpak/`](packaging/flatpak/) | Manifest in tree; folders are chosen through the portal, but no Flatpak has been built to prove it |
|
||||
|
||||
Or build it. Git LFS is required for the model weights, and the toolchain
|
||||
pins itself to 1.92.0:
|
||||
@@ -76,27 +96,22 @@ controls, its place in the chain and its tests.
|
||||
|
||||
## Where it stands
|
||||
|
||||
**0.14.0**, twenty-one tagged releases in. 188 numbered requirements in
|
||||
scope, 82% of them claimed by code and [traced to it](docs/dev/traceability.md);
|
||||
**0.18.2**, twenty-eight tagged releases in. 192 numbered requirements in
|
||||
scope, 84% of them claimed by code and [traced to it](docs/dev/traceability.md);
|
||||
the rest are written down rather than merely absent.
|
||||
|
||||
**Not built:** plugins (post-v1, [D12](docs/dev/requirements.md)), compare and
|
||||
survey culling, AI denoise, tiled and progressive rendering, HDR merge and
|
||||
focus stacking, most of the Android platform integration beyond running,
|
||||
and the Flatpak's library chooser. The performance targets are half
|
||||
verified: the per-commit benchmark suite §8 requires exists for everything
|
||||
that does not need a frame — the catalog, the scan, the thumbnails — and
|
||||
not yet for the render path, so a regression there fails nothing.
|
||||
survey culling, AI denoise, tiled rendering, HDR merge and
|
||||
focus stacking, importing a Lightroom or darktable catalog, translations
|
||||
beyond the launch screen, most of the Android platform integration beyond
|
||||
running, and a Flatpak actually built and run in its sandbox. The
|
||||
performance targets are half verified: the per-commit benchmark suite §8
|
||||
requires exists for everything that does not need a frame — the catalog,
|
||||
the scan, the thumbnails — and not yet for the render path, so a regression
|
||||
there fails nothing.
|
||||
[outstanding.md](docs/dev/outstanding.md) is the list, with the reasoning for
|
||||
each.
|
||||
|
||||
**The one deliberate compromise worth knowing about before reading
|
||||
anything else:** the Android develop view reads its frame back through the
|
||||
CPU, because zero-copy there needs wgpu's Vulkan swapchain and that tears a
|
||||
portrait window on a tablet whose panel is mounted landscape. It is debt,
|
||||
not a revision of the rule — [technical-debt.md TD-1](docs/dev/technical-debt.md)
|
||||
has the measurements and the three things any one of which would remove it.
|
||||
|
||||
## Documentation
|
||||
|
||||
[docs/README.md](docs/README.md) is the index. The short version, for someone using it:
|
||||
|
||||
@@ -141,6 +141,35 @@
|
||||
</intent-filter>
|
||||
</activity>
|
||||
|
||||
<!-- The manual (dr_ui::manual): a WebView over the copy the APK
|
||||
carries in assets/manual. See ManualActivity.java for why it is
|
||||
not the browser.
|
||||
|
||||
Not exported: nothing outside this app has a reason to start it,
|
||||
and dr_ui starts it by class name, which needs no intent filter.
|
||||
Its own task entry is not wanted either — it is a page over the
|
||||
app, and Back returns to the photograph it was opened from.
|
||||
configChanges so a rotation reflows the page rather than
|
||||
reloading it at the top. -->
|
||||
<activity
|
||||
android:name="paris.tourolle.darkroom.ManualActivity"
|
||||
android:exported="false"
|
||||
android:label="DarkRoom manual"
|
||||
android:theme="@style/ManualTheme"
|
||||
android:configChanges="orientation|keyboardHidden|screenSize|screenLayout|uiMode" />
|
||||
|
||||
<!-- FR-EXP-10: the system's folder picker, for an album's folder on
|
||||
this device. NativeActivity's onActivityResult is not ours, so
|
||||
this activity exists only to ask and hand the answer back (see
|
||||
FolderPicker.java). Translucent and without a title so nothing
|
||||
of it shows but the system chooser; not exported, and started by
|
||||
class name from dr_ui::saf. -->
|
||||
<activity
|
||||
android:name="paris.tourolle.darkroom.FolderPicker"
|
||||
android:exported="false"
|
||||
android:theme="@android:style/Theme.Translucent.NoTitleBar"
|
||||
android:configChanges="orientation|keyboardHidden|screenSize|screenLayout|uiMode" />
|
||||
|
||||
<!-- FR-PLAT-AND-6, outbound. Android has refused file:// URIs
|
||||
between apps since API 24 — handing one out raises
|
||||
FileUriExposedException in *this* process — so an exported JPEG
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
package paris.tourolle.darkroom;
|
||||
|
||||
import android.app.Activity;
|
||||
import android.content.ActivityNotFoundException;
|
||||
import android.content.Context;
|
||||
import android.content.Intent;
|
||||
import android.net.Uri;
|
||||
import android.os.Bundle;
|
||||
import android.util.Log;
|
||||
|
||||
/**
|
||||
* The system's folder picker, for an album's folder on this device (FR-EXP-10).
|
||||
*
|
||||
* <h2>Why an activity of its own</h2>
|
||||
*
|
||||
* <p>{@code ACTION_OPEN_DOCUMENT_TREE} answers through
|
||||
* {@code onActivityResult}, and the main activity is {@code NativeActivity},
|
||||
* whose result callback is not ours to override. So this one exists only to
|
||||
* ask: it starts the picker, takes the answer, and finishes — no layout, a
|
||||
* translucent theme, nothing on screen but the system's own chooser, which has
|
||||
* its own "New folder".
|
||||
*
|
||||
* <p>The answer is left in a static for Rust to poll ({@link #poll}), rather
|
||||
* than called back into native code: a callback would need a registered
|
||||
* native method and a thread to deliver on, and a poll from the Slint timer
|
||||
* that is already running is one static call.
|
||||
*
|
||||
* <h2>The grant</h2>
|
||||
*
|
||||
* <p>A tree URI is usable only while its permission is held, and a plain
|
||||
* result grants it until the process dies. {@code takePersistableUriPermission}
|
||||
* keeps it across restarts — an album's folder is chosen once and exported to
|
||||
* for months.
|
||||
*/
|
||||
public final class FolderPicker extends Activity {
|
||||
private static final String TAG = "DarkRoom";
|
||||
private static final int REQUEST = 0x5AF;
|
||||
|
||||
/** The last answer: a tree URI, "" for a cancel, null while none has come. */
|
||||
private static volatile String answer = null;
|
||||
|
||||
/**
|
||||
* Start asking. Clears any answer left from before.
|
||||
*
|
||||
* <p>Takes a {@code Context} rather than an {@code Activity}, because what
|
||||
* native code holds (ndk_context's handle) is the application context,
|
||||
* and starting an activity from one that is not an activity needs
|
||||
* {@code FLAG_ACTIVITY_NEW_TASK} — without it the call throws. The picker
|
||||
* shares the app's task affinity, so it still opens over the app and Back
|
||||
* still returns to it.
|
||||
*/
|
||||
public static void start(Context from) {
|
||||
answer = null;
|
||||
Intent intent = new Intent(from, FolderPicker.class);
|
||||
if (!(from instanceof Activity)) {
|
||||
intent.addFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
|
||||
}
|
||||
from.startActivity(intent);
|
||||
}
|
||||
|
||||
/**
|
||||
* The answer, once: a tree URI, "" if the user backed out, or null while
|
||||
* the picker is still open. Reading it clears it, so a second poll after a
|
||||
* cancel does not see the cancel again.
|
||||
*/
|
||||
public static String poll() {
|
||||
String a = answer;
|
||||
if (a != null) {
|
||||
answer = null;
|
||||
}
|
||||
return a;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void onCreate(Bundle state) {
|
||||
super.onCreate(state);
|
||||
// Recreated after a rotation with the picker already up: asking again
|
||||
// would stack a second chooser over the first.
|
||||
if (state != null) {
|
||||
return;
|
||||
}
|
||||
Intent pick = new Intent(Intent.ACTION_OPEN_DOCUMENT_TREE);
|
||||
pick.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION
|
||||
| Intent.FLAG_GRANT_WRITE_URI_PERMISSION
|
||||
| Intent.FLAG_GRANT_PERSISTABLE_URI_PERMISSION);
|
||||
try {
|
||||
startActivityForResult(pick, REQUEST);
|
||||
} catch (ActivityNotFoundException e) {
|
||||
Log.w(TAG, "no folder picker on this device", e);
|
||||
answer = "";
|
||||
finish();
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void onActivityResult(int request, int result, Intent data) {
|
||||
if (request != REQUEST) {
|
||||
return;
|
||||
}
|
||||
Uri tree = (result == RESULT_OK && data != null) ? data.getData() : null;
|
||||
if (tree == null) {
|
||||
answer = "";
|
||||
} else {
|
||||
try {
|
||||
getContentResolver().takePersistableUriPermission(tree,
|
||||
Intent.FLAG_GRANT_READ_URI_PERMISSION
|
||||
| Intent.FLAG_GRANT_WRITE_URI_PERMISSION);
|
||||
} catch (SecurityException e) {
|
||||
// Still usable this session; said in the log so a folder that
|
||||
// stops working after a restart has an explanation.
|
||||
Log.w(TAG, "the folder grant could not be kept: " + tree, e);
|
||||
}
|
||||
answer = tree.toString();
|
||||
}
|
||||
finish();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
package paris.tourolle.darkroom;
|
||||
|
||||
import android.app.Activity;
|
||||
import android.content.ActivityNotFoundException;
|
||||
import android.content.Intent;
|
||||
import android.net.Uri;
|
||||
import android.os.Bundle;
|
||||
import android.webkit.WebResourceRequest;
|
||||
import android.webkit.WebSettings;
|
||||
import android.webkit.WebView;
|
||||
import android.webkit.WebViewClient;
|
||||
|
||||
/**
|
||||
* The manual that ships in the APK, shown in a WebView.
|
||||
*
|
||||
* <h2>Why an activity of our own rather than the browser</h2>
|
||||
*
|
||||
* <p>The desktop hands the manual to the system browser. Android leaves no
|
||||
* way to do the same: the page is an asset inside the APK, which is not a
|
||||
* file; an unpacked copy in app-private storage is a file no browser may
|
||||
* read; a {@code file:} URI handed to another app is refused since API 24;
|
||||
* and a {@code content:} URI serves the page but leaves the browser to fetch
|
||||
* every picture by a relative URL against the provider, which browsers do not
|
||||
* reliably do. A WebView reads {@code file:///android_asset/} straight from
|
||||
* the APK, pictures and section anchor included, and nothing is unpacked.
|
||||
*
|
||||
* <h2>What it is not</h2>
|
||||
*
|
||||
* <p>A browser. JavaScript stays off (the page has none), and a link that
|
||||
* leaves the manual — the design documents are on the forge — goes to the
|
||||
* user's browser rather than opening inside this view, so the only thing ever
|
||||
* shown here is the page the APK carries.
|
||||
*
|
||||
* <p>Started by {@code dr_ui::manual} with {@code Intent.setClassName}, so the
|
||||
* name here and there must agree; a test in lib.rs checks the manifest
|
||||
* declares it.
|
||||
*/
|
||||
public final class ManualActivity extends Activity {
|
||||
/** The section to open at, a heading's anchor. Absent opens the top. */
|
||||
public static final String EXTRA_ANCHOR = "anchor";
|
||||
|
||||
private static final String PAGE = "file:///android_asset/manual/index.html";
|
||||
|
||||
private WebView web;
|
||||
|
||||
@Override
|
||||
protected void onCreate(Bundle saved) {
|
||||
super.onCreate(saved);
|
||||
setTitle("DarkRoom manual");
|
||||
|
||||
web = new WebView(this);
|
||||
WebSettings settings = web.getSettings();
|
||||
settings.setJavaScriptEnabled(false);
|
||||
// Pinch to zoom into a screenshot, which is 1600 pixels wide and drawn
|
||||
// at the width of a phone.
|
||||
settings.setBuiltInZoomControls(true);
|
||||
settings.setDisplayZoomControls(false);
|
||||
web.setWebViewClient(new WebViewClient() {
|
||||
@Override
|
||||
public boolean shouldOverrideUrlLoading(WebView view, WebResourceRequest request) {
|
||||
Uri uri = request.getUrl();
|
||||
if ("file".equals(uri.getScheme())) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
startActivity(new Intent(Intent.ACTION_VIEW, uri));
|
||||
} catch (ActivityNotFoundException e) {
|
||||
// No browser on the device: the link does nothing, which
|
||||
// is all it could do.
|
||||
}
|
||||
return true;
|
||||
}
|
||||
});
|
||||
setContentView(web);
|
||||
|
||||
if (saved != null) {
|
||||
web.restoreState(saved);
|
||||
} else {
|
||||
String anchor = getIntent().getStringExtra(EXTRA_ANCHOR);
|
||||
web.loadUrl(anchor == null || anchor.isEmpty() ? PAGE : PAGE + "#" + anchor);
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void onSaveInstanceState(Bundle out) {
|
||||
super.onSaveInstanceState(out);
|
||||
web.saveState(out);
|
||||
}
|
||||
|
||||
/** Back walks back through the sections visited, then leaves. */
|
||||
@Override
|
||||
public void onBackPressed() {
|
||||
if (web.canGoBack()) {
|
||||
web.goBack();
|
||||
} else {
|
||||
super.onBackPressed();
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void onDestroy() {
|
||||
web.destroy();
|
||||
super.onDestroy();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package paris.tourolle.darkroom;
|
||||
|
||||
import android.content.ContentResolver;
|
||||
import android.content.Context;
|
||||
import android.database.Cursor;
|
||||
import android.net.Uri;
|
||||
import android.provider.DocumentsContract;
|
||||
import android.util.Log;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.io.OutputStream;
|
||||
|
||||
/**
|
||||
* Writing an export into a folder the user granted through
|
||||
* {@link FolderPicker} — the Storage Access Framework, which is the only way
|
||||
* this app reaches a folder on the device (FR-PLAT-AND-1).
|
||||
*
|
||||
* <p>A tree URI is not a path: a child is found by listing the folder and
|
||||
* matching its display name, and created through the provider, which may
|
||||
* rename it on a collision. So the name that was actually written is handed
|
||||
* back, and the album records that one.
|
||||
*
|
||||
* <p>Two static calls, strings and a byte array in, a string out, for the
|
||||
* reason {@link Intents} gives: every call here would be a signature typed as
|
||||
* a string on the Rust side, and the fewer of those the better.
|
||||
*/
|
||||
public final class Saf {
|
||||
private static final String TAG = "DarkRoom";
|
||||
|
||||
private Saf() {
|
||||
}
|
||||
|
||||
/** Whether {@code name} already exists in the folder. False on any error. */
|
||||
public static boolean exists(Context context, String tree, String name) {
|
||||
try {
|
||||
return find(context.getContentResolver(), Uri.parse(tree), name) != null;
|
||||
} catch (RuntimeException e) {
|
||||
Log.w(TAG, "checking " + name + " in " + tree, e);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Write {@code bytes} as {@code name} in the folder, replacing a file of
|
||||
* that name when {@code replace} is set.
|
||||
*
|
||||
* @return the name the file has in the folder — the provider may have
|
||||
* added " (1)" — or null on failure, with the reason in the log.
|
||||
*/
|
||||
public static String write(Context context, String tree, String name, String mime,
|
||||
byte[] bytes, boolean replace) {
|
||||
ContentResolver resolver = context.getContentResolver();
|
||||
Uri treeUri = Uri.parse(tree);
|
||||
try {
|
||||
Uri target = replace ? find(resolver, treeUri, name) : null;
|
||||
if (target == null) {
|
||||
Uri folder = DocumentsContract.buildDocumentUriUsingTree(treeUri,
|
||||
DocumentsContract.getTreeDocumentId(treeUri));
|
||||
target = DocumentsContract.createDocument(resolver, folder, mime, name);
|
||||
}
|
||||
if (target == null) {
|
||||
Log.w(TAG, "the folder refused to create " + name + " in " + tree);
|
||||
return null;
|
||||
}
|
||||
// "wt": truncate. A replacement shorter than what it replaces
|
||||
// must not keep the old file's tail.
|
||||
try (OutputStream out = resolver.openOutputStream(target, "wt")) {
|
||||
if (out == null) {
|
||||
Log.w(TAG, "no stream for " + target);
|
||||
return null;
|
||||
}
|
||||
out.write(bytes);
|
||||
}
|
||||
String written = displayName(resolver, target);
|
||||
return written != null ? written : name;
|
||||
} catch (IOException | RuntimeException e) {
|
||||
Log.w(TAG, "writing " + name + " to " + tree, e);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** The document for {@code name} directly in the tree's folder, or null. */
|
||||
private static Uri find(ContentResolver resolver, Uri tree, String name) {
|
||||
String folderId = DocumentsContract.getTreeDocumentId(tree);
|
||||
Uri children = DocumentsContract.buildChildDocumentsUriUsingTree(tree, folderId);
|
||||
String[] columns = {
|
||||
DocumentsContract.Document.COLUMN_DOCUMENT_ID,
|
||||
DocumentsContract.Document.COLUMN_DISPLAY_NAME,
|
||||
};
|
||||
try (Cursor c = resolver.query(children, columns, null, null, null)) {
|
||||
if (c == null) {
|
||||
return null;
|
||||
}
|
||||
while (c.moveToNext()) {
|
||||
if (name.equals(c.getString(1))) {
|
||||
return DocumentsContract.buildDocumentUriUsingTree(tree, c.getString(0));
|
||||
}
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static String displayName(ContentResolver resolver, Uri document) {
|
||||
String[] columns = {DocumentsContract.Document.COLUMN_DISPLAY_NAME};
|
||||
try (Cursor c = resolver.query(document, columns, null, null, null)) {
|
||||
if (c != null && c.moveToFirst()) {
|
||||
return c.getString(0);
|
||||
}
|
||||
} catch (RuntimeException e) {
|
||||
Log.w(TAG, "reading the name of " + document, e);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Day or night as the system is; see values/themes.xml. -->
|
||||
<resources>
|
||||
<style name="ManualTheme" parent="@android:style/Theme.DeviceDefault.DayNight" />
|
||||
</resources>
|
||||
@@ -0,0 +1,10 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!--
|
||||
The manual's theme (ManualActivity). Light below API 29, which has no
|
||||
day-night theme in the platform; values-v29 follows the system from there.
|
||||
The WebView takes prefers-color-scheme from whether this theme is light, and
|
||||
the manual's stylesheet takes its colours from that.
|
||||
-->
|
||||
<resources>
|
||||
<style name="ManualTheme" parent="@android:style/Theme.DeviceDefault.Light" />
|
||||
</resources>
|
||||
@@ -300,7 +300,9 @@ fn install_bundled_models(app: slint::android::AndroidApp) {
|
||||
// on the worker because `AAssetManager` is thread-safe by contract and
|
||||
// reading the pointer takes only the app's read lock, which `poll_events`
|
||||
// also only ever holds shared.
|
||||
std::thread::spawn(move || unpack_bundled_models(&app));
|
||||
dr_ui::executors::spawn(dr_ui::executors::Executor::Io, "models", move || {
|
||||
unpack_bundled_models(&app)
|
||||
});
|
||||
}
|
||||
|
||||
/// The copy itself, on the worker [`install_bundled_models`] starts.
|
||||
@@ -581,6 +583,37 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// `dr_ui::manual` starts the manual by class name. A name the manifest
|
||||
/// does not declare is an `ActivityNotFoundException` on the device and a
|
||||
/// Manual button that does nothing, so the three spellings — dr_ui's, the
|
||||
/// manifest's and the Java file's — are checked to be one.
|
||||
#[test]
|
||||
fn the_manual_activity_dr_ui_starts_is_declared() {
|
||||
let manifest = manifest();
|
||||
let wanted = dr_ui::manual::ANDROID_ACTIVITY;
|
||||
let element = manifest
|
||||
.split("<activity")
|
||||
.skip(1)
|
||||
.find(|a| attribute(a, "android:name").as_deref() == Some(wanted))
|
||||
.unwrap_or_else(|| panic!("the manifest declares no activity {wanted}"));
|
||||
assert_eq!(
|
||||
attribute(element, "android:exported").as_deref(),
|
||||
Some("false"),
|
||||
"the manual activity has no reason to be startable by another app"
|
||||
);
|
||||
let java = include_str!("../android/java/paris/tourolle/darkroom/ManualActivity.java");
|
||||
let (package, class) = wanted.rsplit_once('.').expect("unqualified class name");
|
||||
assert!(java.contains(&format!("package {package};")));
|
||||
assert!(java.contains(&format!("class {class} ")));
|
||||
assert!(
|
||||
java.contains(&format!(
|
||||
"EXTRA_ANCHOR = \"{}\"",
|
||||
dr_ui::manual::ANDROID_EXTRA_ANCHOR
|
||||
)),
|
||||
"ManualActivity reads the section from a different extra than dr_ui writes"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_provider_hands_out_one_file_at_a_time_and_nothing_by_itself() {
|
||||
let manifest = manifest();
|
||||
|
||||
@@ -25,3 +25,6 @@ winresource = "0.1"
|
||||
|
||||
[features]
|
||||
default = []
|
||||
# The manual's recording hook (dr-ui's `automation`); tools/manual/record.sh
|
||||
# builds with it, nothing else does.
|
||||
automation = ["dr-ui/automation"]
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
//! What the catalog's routine reads cost on a real library, off the GUI.
|
||||
//!
|
||||
//! cargo run --release -p dr-catalog --example catalog_bench -- CATALOG.sqlite [FACES_DIR] [--remote PEER.sqlite]
|
||||
//!
|
||||
//! Times `Catalog::open` — which every worker thread pays, including the
|
||||
//! develop view's fetch of each original and each neighbour it prefetches —
|
||||
//! and the backfill that runs inside it, step by step. Run it against a
|
||||
//! *copy* of a real catalog: opening migrates and backfills, which write.
|
||||
//!
|
||||
//! Then what the library screen reads on every keystroke and scroll: the
|
||||
//! filter chips' counts, the keyword panel, and the grid's total. Two of
|
||||
//! those live in `dr-ui` (`library::local_original_count` and the grid
|
||||
//! count), whose `library` module is private; their SQL is spelled here as
|
||||
//! it is spelled there, and has to be kept in step by hand.
|
||||
//!
|
||||
//! `--remote` also times a merge with another device's catalog — the
|
||||
//! server snapshot — which is the pass where the two disagree: faces one
|
||||
//! side found and the other did not, boxes that moved. The merge with a copy
|
||||
//! of itself matches every face by its box and never reaches that work. The
|
||||
//! first of its runs writes what the peer brought; the rest are the steady
|
||||
//! state, so compare two builds from two fresh copies of one catalog.
|
||||
//!
|
||||
//! The figures are for reading side by side before and after a change; they
|
||||
//! are not a gate. Compare the `cpu` column when the machine is busy. The
|
||||
//! answers are printed too, so two builds can be checked for agreeing.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use dr_catalog::{keywords, rating, schema, Catalog};
|
||||
|
||||
fn main() {
|
||||
let mut args: Vec<String> = std::env::args().skip(1).collect();
|
||||
let peer = args.iter().position(|a| a == "--remote").map(|at| {
|
||||
let path = args.get(at + 1).map(PathBuf::from).unwrap_or_else(|| {
|
||||
eprintln!("--remote needs a catalog");
|
||||
std::process::exit(2);
|
||||
});
|
||||
args.drain(at..at + 2);
|
||||
path
|
||||
});
|
||||
let Some(path) = args.first().map(PathBuf::from) else {
|
||||
eprintln!("usage: catalog_bench CATALOG.sqlite");
|
||||
std::process::exit(2);
|
||||
};
|
||||
|
||||
// Once untimed, so a migration or a first backfill is not in the figures.
|
||||
drop(Catalog::open(&path).expect("catalog"));
|
||||
|
||||
time("Catalog::open", 20, || {
|
||||
drop(Catalog::open(&path).unwrap());
|
||||
});
|
||||
|
||||
// One develop landing, in the two shapes the app has had. Five opens is
|
||||
// what `fetch_original` and a `holds_original` per prefetched neighbour
|
||||
// cost when each asked on a connection of its own; two is the fetch plus
|
||||
// one connection the prefetch worker keeps for its batch's row checks.
|
||||
// Five images from the library, against an empty cache: the question is
|
||||
// asked the same way whatever the answer.
|
||||
let images: Vec<dr_types::ImageId> = {
|
||||
let c = Catalog::open(&path).unwrap();
|
||||
let mut stmt = c
|
||||
.connection()
|
||||
.prepare("SELECT id FROM images ORDER BY id LIMIT 5 OFFSET 1000")
|
||||
.unwrap();
|
||||
let ids = stmt
|
||||
.query_map([], |r| r.get::<_, i64>(0))
|
||||
.unwrap()
|
||||
.map(|id| dr_types::ImageId(id.unwrap() as u64))
|
||||
.collect();
|
||||
ids
|
||||
};
|
||||
let cache_dir = path.with_extension("bench-cache");
|
||||
let budget = dr_catalog::Budget::default();
|
||||
time("landing: 5 opens (fetch + 4 row checks)", 20, || {
|
||||
let store = dr_catalog::Cache::open(&cache_dir, budget).unwrap();
|
||||
let c = Catalog::open(&path).unwrap();
|
||||
let _ = store.load(c.connection(), images[0], 0).unwrap();
|
||||
for &image in &images[1..] {
|
||||
let store = dr_catalog::Cache::open(&cache_dir, budget).unwrap();
|
||||
let c = Catalog::open(&path).unwrap();
|
||||
let _ = store.holds_original(c.connection(), image);
|
||||
}
|
||||
});
|
||||
time("landing: 2 opens (fetch + held row checks)", 20, || {
|
||||
let store = dr_catalog::Cache::open(&cache_dir, budget).unwrap();
|
||||
let c = Catalog::open(&path).unwrap();
|
||||
let _ = store.load(c.connection(), images[0], 0).unwrap();
|
||||
let held = Catalog::open(&path).unwrap();
|
||||
for &image in &images[1..] {
|
||||
let store = dr_catalog::Cache::open(&cache_dir, budget).unwrap();
|
||||
let _ = store.holds_original(held.connection(), image);
|
||||
}
|
||||
});
|
||||
let _ = std::fs::remove_dir_all(&cache_dir);
|
||||
|
||||
let catalog = Catalog::open(&path).unwrap();
|
||||
let conn = catalog.connection();
|
||||
time("schema::backfill (all steps)", 20, || {
|
||||
schema::backfill(conn).unwrap();
|
||||
});
|
||||
time(" rating::ensure_default_versions", 20, || {
|
||||
rating::ensure_default_versions(conn).unwrap();
|
||||
});
|
||||
time(" rating::align_default_version_uuids", 20, || {
|
||||
rating::align_default_version_uuids(conn).unwrap();
|
||||
});
|
||||
time(" keywords::adopt_orphan_terms", 20, || {
|
||||
keywords::adopt_orphan_terms(conn).unwrap();
|
||||
});
|
||||
|
||||
interactive(conn);
|
||||
|
||||
// A sync pass: the upload snapshot, then a merge of the catalog with a
|
||||
// copy of itself — every row a match, which is the steady state.
|
||||
let scratch = path.with_extension("bench-snapshot");
|
||||
time("snapshot_for_upload", 3, || {
|
||||
let _ = std::fs::remove_file(&scratch);
|
||||
catalog.snapshot_for_upload(&scratch).unwrap();
|
||||
});
|
||||
println!(
|
||||
" snapshot size {:.1} MB",
|
||||
std::fs::metadata(&scratch).map(|m| m.len()).unwrap_or(0) as f64 / 1e6
|
||||
);
|
||||
let remote = path.with_extension("bench-remote");
|
||||
let _ = std::fs::remove_file(&remote);
|
||||
conn.execute("VACUUM INTO ?1", [remote.to_string_lossy().as_ref()])
|
||||
.unwrap();
|
||||
time("merge_remote_catalog (self)", 5, || {
|
||||
catalog.merge_remote_catalog(&remote).unwrap();
|
||||
});
|
||||
let _ = std::fs::remove_file(&scratch);
|
||||
let _ = std::fs::remove_file(&remote);
|
||||
|
||||
if let Some(peer) = &peer {
|
||||
// A copy, so nothing the merge does to its input reaches the file
|
||||
// the caller named.
|
||||
std::fs::copy(peer, &remote).unwrap();
|
||||
let mut first = None;
|
||||
time("merge_remote_catalog (--remote)", 5, || {
|
||||
let report = catalog.merge_remote_catalog(&remote).unwrap();
|
||||
first.get_or_insert(report);
|
||||
});
|
||||
println!(" first pass: {first:?}");
|
||||
let _ = std::fs::remove_file(&remote);
|
||||
}
|
||||
|
||||
// The face half of a sync pass, against a copy of the face store: both
|
||||
// directions in the steady state, where nothing is new either way.
|
||||
if let Some(faces) = args.get(1).map(PathBuf::from) {
|
||||
let model = "scrfd_10g+w600k_mbf";
|
||||
let mut store = dr_catalog::FaceShardStore::open(&faces).unwrap();
|
||||
println!(
|
||||
" first export sent {}, first import adopted {}",
|
||||
dr_catalog::face_shard::export_to_shards(conn, &mut store, model).unwrap(),
|
||||
dr_catalog::face_shard::import_from_shards(conn, &store, model).unwrap()
|
||||
);
|
||||
time("face_shard::export_to_shards (steady)", 5, || {
|
||||
dr_catalog::face_shard::export_to_shards(conn, &mut store, model).unwrap();
|
||||
});
|
||||
time("face_shard::import_from_shards (steady)", 5, || {
|
||||
dr_catalog::face_shard::import_from_shards(conn, &store, model).unwrap();
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// What one click in the library reads: a rating or label keystroke
|
||||
/// refreshes the chips, a selection change redraws the keyword panel, and
|
||||
/// every scroll reload counts the grid.
|
||||
fn interactive(conn: &rusqlite::Connection) {
|
||||
println!(
|
||||
" rating_histogram {:?}, label_histogram {:?}, local originals {}, grid {} / rated {}",
|
||||
rating::rating_histogram(conn).unwrap(),
|
||||
rating::label_histogram(conn).unwrap(),
|
||||
local_original_count(conn),
|
||||
grid_count(conn, ""),
|
||||
grid_count(conn, RATED_AT_LEAST_ONE),
|
||||
);
|
||||
let words = keywords::list(conn).unwrap();
|
||||
println!(
|
||||
" keywords::list {} terms, digest {:016x}",
|
||||
words.len(),
|
||||
digest(&format!("{words:?}"))
|
||||
);
|
||||
// A selection the size of a grid window, from the start of the library.
|
||||
let selection: Vec<dr_types::ImageId> = conn
|
||||
.prepare("SELECT id FROM images ORDER BY id LIMIT 120")
|
||||
.unwrap()
|
||||
.query_map([], |r| Ok(dr_types::ImageId(r.get::<_, i64>(0)? as u64)))
|
||||
.unwrap()
|
||||
.collect::<Result<_, _>>()
|
||||
.unwrap();
|
||||
println!(
|
||||
" keywords::for_images digest {:016x}",
|
||||
digest(&format!(
|
||||
"{:?}",
|
||||
keywords::for_images(conn, &selection).unwrap()
|
||||
))
|
||||
);
|
||||
|
||||
time("rating::rating_histogram", 50, || {
|
||||
rating::rating_histogram(conn).unwrap();
|
||||
});
|
||||
time("library::local_original_count", 50, || {
|
||||
local_original_count(conn);
|
||||
});
|
||||
time("rating::label_histogram", 50, || {
|
||||
rating::label_histogram(conn).unwrap();
|
||||
});
|
||||
time("keywords::list", 50, || {
|
||||
keywords::list(conn).unwrap();
|
||||
});
|
||||
time("keywords::for_images (120)", 50, || {
|
||||
keywords::for_images(conn, &selection).unwrap();
|
||||
});
|
||||
time("grid count", 50, || {
|
||||
grid_count(conn, "");
|
||||
});
|
||||
time("grid count, rated >= 1", 50, || {
|
||||
grid_count(conn, RATED_AT_LEAST_ONE);
|
||||
});
|
||||
}
|
||||
|
||||
/// `dr_ui::library::local_original_count`, spelled as it is there.
|
||||
fn local_original_count(conn: &rusqlite::Connection) -> i64 {
|
||||
conn.query_row(
|
||||
"SELECT count(*) FROM images i
|
||||
WHERE i.shadowed_by IS NULL AND i.trashed_at IS NULL
|
||||
AND i.id IN (SELECT ic.image_id FROM image_cache ic
|
||||
WHERE ic.tier_actual >= 2)",
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// `RatingFilter::sql` for one star and up.
|
||||
const RATED_AT_LEAST_ONE: &str = " AND coalesce((SELECT dv.rating FROM versions dv
|
||||
WHERE dv.image_id = i.id AND dv.is_default = 1
|
||||
LIMIT 1), 0) >= 1";
|
||||
|
||||
/// `dr_ui::library::total_images_filtered`, spelled as it is there.
|
||||
fn grid_count(conn: &rusqlite::Connection, rated: &str) -> i64 {
|
||||
let visible = "i.shadowed_by IS NULL AND i.trashed_at IS NULL";
|
||||
let hidden = dr_catalog::bursts::collapsed_away_frames("i");
|
||||
conn.query_row(
|
||||
&format!(
|
||||
"SELECT (SELECT count(*) FROM images i WHERE {visible}{rated})
|
||||
- (SELECT count(*) FROM {hidden} AND {visible}{rated})"
|
||||
),
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// FNV-1a, to print a long answer as something two runs can compare.
|
||||
fn digest(s: &str) -> u64 {
|
||||
s.bytes().fold(0xcbf29ce484222325, |h, b| {
|
||||
(h ^ u64::from(b)).wrapping_mul(0x100000001b3)
|
||||
})
|
||||
}
|
||||
|
||||
/// Run `f` a few times and print the best wall-clock, the median, and the
|
||||
/// best CPU time — the figure to compare across runs on a busy machine.
|
||||
fn time(label: &str, runs: usize, mut f: impl FnMut()) {
|
||||
let mut wall: Vec<Duration> = Vec::with_capacity(runs);
|
||||
let mut cpu: Vec<Duration> = Vec::with_capacity(runs);
|
||||
for _ in 0..runs {
|
||||
let c = cpu_now();
|
||||
let t = Instant::now();
|
||||
f();
|
||||
wall.push(t.elapsed());
|
||||
cpu.push(cpu_now().saturating_sub(c));
|
||||
}
|
||||
wall.sort();
|
||||
cpu.sort();
|
||||
println!(
|
||||
"{label:42} best {:8.2} ms median {:8.2} ms cpu {:8.2} ms",
|
||||
wall[0].as_secs_f64() * 1e3,
|
||||
wall[runs / 2].as_secs_f64() * 1e3,
|
||||
cpu[0].as_secs_f64() * 1e3
|
||||
);
|
||||
}
|
||||
|
||||
/// This thread's time on a CPU so far, from `/proc/self/schedstat`; zero where
|
||||
/// the file is missing, which only makes the CPU column useless.
|
||||
fn cpu_now() -> Duration {
|
||||
std::fs::read_to_string("/proc/self/schedstat")
|
||||
.ok()
|
||||
.and_then(|s| s.split_whitespace().next()?.parse::<u64>().ok())
|
||||
.map(Duration::from_nanos)
|
||||
.unwrap_or_default()
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
//! Run the people and face deduplication (#78) on a copy of a real catalog.
|
||||
//!
|
||||
//! cargo run --release -p dr-catalog --example dedup_people -- COPY.sqlite [--peer PEER_COPY.sqlite]
|
||||
//!
|
||||
//! It writes: run it against a *copy* (`sqlite3 catalog.sqlite ".backup
|
||||
//! copy.sqlite"`), never the library's own file. Prints the live people and
|
||||
//! faces before and after, what the first run merged and kept apart, and
|
||||
//! how long the first and a second run took -- the second is the cost the
|
||||
//! job adds to every sync once a catalog is clean.
|
||||
//!
|
||||
//! `--peer` then plays a sync round trip with another device's catalog (a
|
||||
//! copy of the server snapshot, which it also writes): the peer merges this
|
||||
//! one as the previous release would, with no job after it, then this one
|
||||
//! merges the peer back through `sync::merge_remote`, twice. The named
|
||||
//! people each side lists are printed after each step; they should agree.
|
||||
|
||||
use std::path::PathBuf;
|
||||
use std::time::Instant;
|
||||
|
||||
use dr_catalog::{dedup_people, merge, schema, sync};
|
||||
use rusqlite::Connection;
|
||||
|
||||
fn open(path: &std::path::Path) -> Connection {
|
||||
let conn = Connection::open(path).expect("open the catalog copy");
|
||||
schema::configure(&conn).expect("configure");
|
||||
schema::migrate(&conn).expect("migrate");
|
||||
conn
|
||||
}
|
||||
|
||||
/// The named people a device lists, as `name (uuid prefix)`, sorted.
|
||||
fn named(conn: &Connection) -> Vec<String> {
|
||||
let mut v: Vec<String> = conn
|
||||
.prepare(
|
||||
"SELECT name, substr(uuid, 1, 8) FROM people
|
||||
WHERE merged_into IS NULL AND trim(name) <> ''",
|
||||
)
|
||||
.unwrap()
|
||||
.query_map([], |r| {
|
||||
Ok(format!(
|
||||
"{} ({})",
|
||||
r.get::<_, String>(0)?,
|
||||
r.get::<_, String>(1)?
|
||||
))
|
||||
})
|
||||
.unwrap()
|
||||
.collect::<Result<_, _>>()
|
||||
.unwrap();
|
||||
v.sort();
|
||||
v
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let mut args: Vec<String> = std::env::args().skip(1).collect();
|
||||
let peer = args.iter().position(|a| a == "--peer").map(|at| {
|
||||
let p = PathBuf::from(&args[at + 1]);
|
||||
args.drain(at..at + 2);
|
||||
p
|
||||
});
|
||||
let Some(path) = args.first().map(PathBuf::from) else {
|
||||
eprintln!("usage: dedup_people COPY.sqlite [--peer PEER_COPY.sqlite]");
|
||||
std::process::exit(2);
|
||||
};
|
||||
let conn = open(&path);
|
||||
|
||||
let counts = |label: &str| {
|
||||
let q = |sql: &str| -> i64 { conn.query_row(sql, [], |r| r.get(0)).unwrap() };
|
||||
println!(
|
||||
"{label}: {} people listed ({} named), {} redirects, {} faces, {} confirmed",
|
||||
q("SELECT COUNT(*) FROM people WHERE merged_into IS NULL"),
|
||||
q("SELECT COUNT(*) FROM people WHERE merged_into IS NULL AND trim(name) <> ''"),
|
||||
q("SELECT COUNT(*) FROM people WHERE merged_into IS NOT NULL"),
|
||||
q("SELECT COUNT(*) FROM faces"),
|
||||
q("SELECT COUNT(*) FROM face_person WHERE confirmed = 1"),
|
||||
);
|
||||
};
|
||||
|
||||
counts("before");
|
||||
for pass in ["first", "second", "third"] {
|
||||
let started = Instant::now();
|
||||
let report = dedup_people::run(&conn).expect("dedup");
|
||||
let took = started.elapsed();
|
||||
println!("{pass} run: {took:?}, changed: {}", report.changed());
|
||||
if pass == "first" {
|
||||
println!(" merged: {:?}", report.merged);
|
||||
for k in &report.kept_apart {
|
||||
println!(
|
||||
" kept apart: {:?} ({}) from {}: {:?}",
|
||||
k.name, k.uuid, k.survivor, k.why
|
||||
);
|
||||
}
|
||||
println!(
|
||||
" redirects followed {}, cycles broken {}, faces fused {}, faces confirmed apart {}",
|
||||
report.redirects_followed,
|
||||
report.cycles_broken,
|
||||
report.faces_fused,
|
||||
report.faces_confirmed_apart
|
||||
);
|
||||
}
|
||||
}
|
||||
counts("after");
|
||||
|
||||
let Some(peer_path) = peer else { return };
|
||||
let peer = open(&peer_path);
|
||||
let show = |step: &str| {
|
||||
let (ours, theirs) = (named(&conn), named(&peer));
|
||||
println!(
|
||||
"{step}: this device lists {} named, the peer {}; {}",
|
||||
ours.len(),
|
||||
theirs.len(),
|
||||
if ours == theirs {
|
||||
"the same".to_string()
|
||||
} else {
|
||||
format!("differ:\n here {ours:?}\n peer {theirs:?}")
|
||||
}
|
||||
);
|
||||
};
|
||||
show("before the round trip");
|
||||
for round in 1..=2 {
|
||||
peer.execute(
|
||||
"ATTACH DATABASE ?1 AS remote_cat",
|
||||
[path.to_string_lossy().as_ref()],
|
||||
)
|
||||
.unwrap();
|
||||
let theirs = merge::merge_all(&peer).expect("the peer's merge");
|
||||
peer.execute("DETACH DATABASE remote_cat", []).unwrap();
|
||||
println!(
|
||||
"round {round}: the peer took {} people updated, {} inserted",
|
||||
theirs.people_updated, theirs.people_inserted
|
||||
);
|
||||
show(&format!("round {round}, after the peer's merge"));
|
||||
let started = Instant::now();
|
||||
let ours = sync::merge_remote(&conn, &peer_path).expect("our merge");
|
||||
println!(
|
||||
"round {round}: merge_remote with the job took {:?}; {} people updated, {} inserted",
|
||||
started.elapsed(),
|
||||
ours.people_updated,
|
||||
ours.people_inserted
|
||||
);
|
||||
show(&format!("round {round}, after ours"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,516 @@
|
||||
//! TRACES: FR-EXP-10 | FR-EXP-6 | FR-CAT-7
|
||||
//! Albums: named export folders, and which photographs went into each.
|
||||
//!
|
||||
//! An album is where finished pictures go — a folder of JPEGs somebody else
|
||||
//! looks at — as opposed to a collection, which is a set of originals the
|
||||
//! photographer works on. The folder holds only the exported files. What the
|
||||
//! catalog adds is the link back: each export is recorded against the image
|
||||
//! it was rendered from, so opening an album in the library shows the RAWs
|
||||
//! behind its JPEGs, and re-exporting after an edit is one selection away.
|
||||
//!
|
||||
//! # Where the folder is, and why that is two tables
|
||||
//!
|
||||
//! An album's folder is either on the library's server or on this device.
|
||||
//!
|
||||
//! A **server folder** is one path on the account, the same from every device
|
||||
//! signed in to it, so it lives on the album row and syncs with it.
|
||||
//!
|
||||
//! A **local folder** — a filesystem path on a desktop, a Storage Access
|
||||
//! Framework tree on Android — means nothing on any other device. It lives in
|
||||
//! `album_folders`, which the merge never reads and the upload snapshot drops
|
||||
//! ([`crate::sync::snapshot_for_upload`]). An album made on the desktop with a
|
||||
//! local folder therefore reaches the tablet as an album with no folder there
|
||||
//! yet, which is true, and which the tablet can fix by choosing one.
|
||||
//!
|
||||
//! # Created on first use, not by a migration
|
||||
//!
|
||||
//! A new schema version makes every older build refuse this catalog's
|
||||
//! snapshot at sync (`crate::sync::remote_is_mergeable`), so the tablet would
|
||||
//! stop merging collections, keywords and people until it was updated — for
|
||||
//! a feature it does not have. The tables are created by [`ensure_tables`]
|
||||
//! instead, the way `dedup_probes` is; an older build that meets them ignores
|
||||
//! them, and its merge keeps working.
|
||||
//!
|
||||
//! # Sync
|
||||
//!
|
||||
//! Albums merge by uuid and revision with tombstones, and their exports as a
|
||||
//! set union keyed on the image's server file id — the rules
|
||||
//! [`crate::merge`] applies to collections, for the same reasons.
|
||||
|
||||
use rusqlite::{Connection, OptionalExtension};
|
||||
|
||||
use dr_types::ImageId;
|
||||
|
||||
use crate::error::CatalogError;
|
||||
|
||||
/// Identifies an album within one catalog. Local, like every integer id here;
|
||||
/// the uuid is what crosses devices.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, PartialOrd, Ord)]
|
||||
pub struct AlbumId(pub u64);
|
||||
|
||||
/// Where an album's files go.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Place {
|
||||
/// A folder on the library's server, relative to the account root, with no
|
||||
/// leading slash. The same on every device.
|
||||
Server(String),
|
||||
/// A folder on this device: a filesystem path, or on Android a SAF tree
|
||||
/// URI. Never synced.
|
||||
Local(String),
|
||||
}
|
||||
|
||||
/// One album, as the sidebar and the export sheet show it.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Album {
|
||||
pub id: AlbumId,
|
||||
pub uuid: String,
|
||||
pub name: String,
|
||||
/// Where exports go from this device, or `None` for an album whose folder
|
||||
/// is local to another device and has not been chosen here.
|
||||
pub place: Option<Place>,
|
||||
/// Distinct photographs exported into it — what the grid shows when the
|
||||
/// album is opened.
|
||||
pub sources: usize,
|
||||
}
|
||||
|
||||
/// Create the album tables if this catalog does not have them yet.
|
||||
///
|
||||
/// Cheap when they exist: `IF NOT EXISTS` is answered from the schema, and
|
||||
/// every function below calls this first so no caller has to remember to.
|
||||
pub fn ensure_tables(conn: &Connection) -> Result<(), CatalogError> {
|
||||
conn.execute_batch(
|
||||
"CREATE TABLE IF NOT EXISTS albums (
|
||||
id INTEGER PRIMARY KEY,
|
||||
-- The merge identity; the integer id is local.
|
||||
uuid TEXT NOT NULL UNIQUE,
|
||||
name TEXT NOT NULL,
|
||||
-- A folder on the server, relative to the account root. NULL for
|
||||
-- an album whose folder is local to some device.
|
||||
server_path TEXT,
|
||||
created INTEGER NOT NULL,
|
||||
revision INTEGER NOT NULL DEFAULT 1,
|
||||
modified INTEGER NOT NULL,
|
||||
deleted INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
-- One row per file written into an album. Keyed on the file, not the
|
||||
-- image: a photograph exported twice — two crops, or once before an
|
||||
-- edit and once after — is two files in the folder and two rows here.
|
||||
CREATE TABLE IF NOT EXISTS album_exports (
|
||||
album_id INTEGER NOT NULL REFERENCES albums(id) ON DELETE CASCADE,
|
||||
file_name TEXT NOT NULL,
|
||||
image_id INTEGER NOT NULL REFERENCES images(id) ON DELETE CASCADE,
|
||||
exported_at INTEGER NOT NULL,
|
||||
PRIMARY KEY (album_id, file_name)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS album_exports_image ON album_exports(image_id);
|
||||
-- This device's folder for an album. Never merged, never uploaded.
|
||||
CREATE TABLE IF NOT EXISTS album_folders (
|
||||
album_id INTEGER PRIMARY KEY REFERENCES albums(id) ON DELETE CASCADE,
|
||||
folder TEXT NOT NULL
|
||||
);",
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Make an album.
|
||||
///
|
||||
/// The name is trimmed and must not be empty; two albums may share one, as
|
||||
/// two collections may, because the uuid is the identity and refusing a
|
||||
/// duplicate name here would refuse it on one device and not another.
|
||||
pub fn create(conn: &Connection, name: &str, place: &Place) -> Result<AlbumId, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let name = name.trim();
|
||||
if name.is_empty() {
|
||||
return Err(CatalogError::EmptyName);
|
||||
}
|
||||
let now = now_secs();
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
tx.execute(
|
||||
"INSERT INTO albums(uuid, name, server_path, created, revision, modified)
|
||||
VALUES (?1, ?2, ?3, ?4, 1, ?4)",
|
||||
rusqlite::params![
|
||||
crate::collections::new_uuid(),
|
||||
name,
|
||||
server_path(place),
|
||||
now
|
||||
],
|
||||
)?;
|
||||
let id = AlbumId(tx.last_insert_rowid() as u64);
|
||||
if let Place::Local(folder) = place {
|
||||
tx.execute(
|
||||
"INSERT INTO album_folders(album_id, folder) VALUES (?1, ?2)",
|
||||
rusqlite::params![id.0 as i64, folder],
|
||||
)?;
|
||||
}
|
||||
tx.commit()?;
|
||||
Ok(id)
|
||||
}
|
||||
|
||||
/// Rename an album. The folder keeps its name: the album is what the
|
||||
/// photographer calls it, the folder is what is already out there.
|
||||
pub fn rename(conn: &Connection, id: AlbumId, name: &str) -> Result<(), CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let name = name.trim();
|
||||
if name.is_empty() {
|
||||
return Err(CatalogError::EmptyName);
|
||||
}
|
||||
let n = conn.execute(
|
||||
"UPDATE albums SET name = ?2, revision = revision + 1, modified = ?3
|
||||
WHERE id = ?1 AND deleted = 0",
|
||||
rusqlite::params![id.0 as i64, name, now_secs()],
|
||||
)?;
|
||||
if n == 0 {
|
||||
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Point an album at a different folder, from this device.
|
||||
///
|
||||
/// A server folder replaces the synced path, and bumps the revision so the
|
||||
/// move reaches every device. A local folder is recorded for this device
|
||||
/// only; it also clears a server path, because an album goes to one place and
|
||||
/// the photographer has just said which.
|
||||
pub fn set_place(conn: &Connection, id: AlbumId, place: &Place) -> Result<(), CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let n = tx.execute(
|
||||
"UPDATE albums SET server_path = ?2, revision = revision + 1, modified = ?3
|
||||
WHERE id = ?1 AND deleted = 0",
|
||||
rusqlite::params![id.0 as i64, server_path(place), now_secs()],
|
||||
)?;
|
||||
if n == 0 {
|
||||
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||
}
|
||||
match place {
|
||||
Place::Local(folder) => tx.execute(
|
||||
"INSERT INTO album_folders(album_id, folder) VALUES (?1, ?2)
|
||||
ON CONFLICT(album_id) DO UPDATE SET folder = excluded.folder",
|
||||
rusqlite::params![id.0 as i64, folder],
|
||||
)?,
|
||||
Place::Server(_) => tx.execute(
|
||||
"DELETE FROM album_folders WHERE album_id = ?1",
|
||||
[id.0 as i64],
|
||||
)?,
|
||||
};
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Delete an album, leaving a tombstone. The files in its folder are not
|
||||
/// touched: they are finished work somebody may already have been sent a
|
||||
/// link to, and the album was only ever this catalog's note of them.
|
||||
pub fn delete(conn: &Connection, id: AlbumId) -> Result<(), CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let n = tx.execute(
|
||||
"UPDATE albums SET deleted = 1, revision = revision + 1, modified = ?2
|
||||
WHERE id = ?1 AND deleted = 0",
|
||||
rusqlite::params![id.0 as i64, now_secs()],
|
||||
)?;
|
||||
if n == 0 {
|
||||
return Err(CatalogError::NoSuchAlbum(id.0));
|
||||
}
|
||||
tx.execute(
|
||||
"DELETE FROM album_exports WHERE album_id = ?1",
|
||||
[id.0 as i64],
|
||||
)?;
|
||||
tx.execute(
|
||||
"DELETE FROM album_folders WHERE album_id = ?1",
|
||||
[id.0 as i64],
|
||||
)?;
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Every live album, by name, with how many photographs each holds.
|
||||
///
|
||||
/// One statement: the counts are aggregated from `album_exports` first and
|
||||
/// joined to the (few) albums, not counted per row.
|
||||
pub fn list(conn: &Connection) -> Result<Vec<Album>, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT a.id, a.uuid, a.name, a.server_path, f.folder, coalesce(e.n, 0)
|
||||
FROM albums a
|
||||
LEFT JOIN album_folders f ON f.album_id = a.id
|
||||
LEFT JOIN (SELECT album_id, count(DISTINCT image_id) AS n
|
||||
FROM album_exports GROUP BY album_id) e
|
||||
ON e.album_id = a.id
|
||||
WHERE a.deleted = 0
|
||||
ORDER BY a.name COLLATE NOCASE, a.id",
|
||||
)?;
|
||||
let rows = stmt
|
||||
.query_map([], album_from_row)?
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// One album, or `None` if it is gone.
|
||||
pub fn get(conn: &Connection, id: AlbumId) -> Result<Option<Album>, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT a.id, a.uuid, a.name, a.server_path, f.folder,
|
||||
(SELECT count(DISTINCT image_id) FROM album_exports WHERE album_id = a.id)
|
||||
FROM albums a
|
||||
LEFT JOIN album_folders f ON f.album_id = a.id
|
||||
WHERE a.id = ?1 AND a.deleted = 0",
|
||||
[id.0 as i64],
|
||||
album_from_row,
|
||||
)
|
||||
.optional()?)
|
||||
}
|
||||
|
||||
/// The album with this uuid, if this catalog holds it live.
|
||||
pub fn id_for_uuid(conn: &Connection, uuid: &str) -> Result<Option<AlbumId>, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
Ok(conn
|
||||
.query_row(
|
||||
"SELECT id FROM albums WHERE uuid = ?1 AND deleted = 0",
|
||||
[uuid],
|
||||
|r| r.get::<_, i64>(0),
|
||||
)
|
||||
.optional()?
|
||||
.map(|id| AlbumId(id as u64)))
|
||||
}
|
||||
|
||||
/// Record the files one export wrote into an album, and which image each
|
||||
/// came from. One transaction for the batch, however many files it placed.
|
||||
///
|
||||
/// A file name already recorded is re-pointed at the image that wrote it
|
||||
/// last: an export that overwrote `IMG_0001.jpg` replaced the picture in the
|
||||
/// folder, and the link must say what is there now.
|
||||
pub fn record_exports(
|
||||
conn: &Connection,
|
||||
id: AlbumId,
|
||||
files: &[(ImageId, String)],
|
||||
) -> Result<(), CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
if files.is_empty() {
|
||||
return Ok(());
|
||||
}
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let now = now_secs();
|
||||
{
|
||||
let mut insert = tx.prepare(
|
||||
"INSERT INTO album_exports(album_id, file_name, image_id, exported_at)
|
||||
VALUES (?1, ?2, ?3, ?4)
|
||||
ON CONFLICT(album_id, file_name) DO UPDATE SET
|
||||
image_id = excluded.image_id, exported_at = excluded.exported_at",
|
||||
)?;
|
||||
for (image, name) in files {
|
||||
insert.execute(rusqlite::params![id.0 as i64, name, image.0 as i64, now])?;
|
||||
}
|
||||
}
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// The photographs behind an album's files, most recently exported first —
|
||||
/// what the grid shows when the album is opened.
|
||||
pub fn sources(conn: &Connection, id: AlbumId) -> Result<Vec<ImageId>, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT image_id FROM album_exports
|
||||
WHERE album_id = ?1
|
||||
GROUP BY image_id
|
||||
ORDER BY max(exported_at) DESC, image_id",
|
||||
)?;
|
||||
let rows = stmt
|
||||
.query_map([id.0 as i64], |r| r.get::<_, i64>(0))?
|
||||
.map(|r| r.map(|i| ImageId(i as u64)))
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// The names of the files an image left in an album — the "which JPEG is
|
||||
/// this" half of the link.
|
||||
pub fn files_of(
|
||||
conn: &Connection,
|
||||
id: AlbumId,
|
||||
image: ImageId,
|
||||
) -> Result<Vec<String>, CatalogError> {
|
||||
ensure_tables(conn)?;
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT file_name FROM album_exports
|
||||
WHERE album_id = ?1 AND image_id = ?2
|
||||
ORDER BY exported_at DESC, file_name",
|
||||
)?;
|
||||
let rows = stmt
|
||||
.query_map(rusqlite::params![id.0 as i64, image.0 as i64], |r| r.get(0))?
|
||||
.collect::<Result<Vec<_>, _>>()?;
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
fn album_from_row(r: &rusqlite::Row<'_>) -> rusqlite::Result<Album> {
|
||||
let server: Option<String> = r.get(3)?;
|
||||
let local: Option<String> = r.get(4)?;
|
||||
Ok(Album {
|
||||
id: AlbumId(r.get::<_, i64>(0)? as u64),
|
||||
uuid: r.get(1)?,
|
||||
name: r.get(2)?,
|
||||
// A server path wins: `set_place` clears the local folder when it
|
||||
// sets one, so both being present means a merge brought a server
|
||||
// path in over a local choice — and the newer revision decided that.
|
||||
place: server.map(Place::Server).or(local.map(Place::Local)),
|
||||
sources: r.get::<_, i64>(5)? as usize,
|
||||
})
|
||||
}
|
||||
|
||||
fn server_path(place: &Place) -> Option<&str> {
|
||||
match place {
|
||||
Place::Server(p) => Some(p.trim_matches('/')),
|
||||
Place::Local(_) => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn now_secs() -> i64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
.map(|d| d.as_secs() as i64)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A catalog, and the connection to it. The `Catalog` has to outlive the
|
||||
/// connection it hands out, so tests hold both.
|
||||
fn catalog() -> crate::Catalog {
|
||||
let cat = crate::Catalog::in_memory().unwrap();
|
||||
cat.connection()
|
||||
.execute(
|
||||
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'lib')",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
cat
|
||||
}
|
||||
|
||||
fn image(conn: &Connection, path: &str) -> ImageId {
|
||||
conn.execute(
|
||||
"INSERT INTO images(root_id, source_ref, added_at) VALUES (1, ?1, 0)",
|
||||
[path],
|
||||
)
|
||||
.unwrap();
|
||||
ImageId(conn.last_insert_rowid() as u64)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_album_lists_with_its_place_and_no_photographs() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let web = create(conn, " Web ", &Place::Server("Shared/Web/".into())).unwrap();
|
||||
let print = create(conn, "Print", &Place::Local("/mnt/print".into())).unwrap();
|
||||
|
||||
let all = list(conn).unwrap();
|
||||
assert_eq!(all.len(), 2);
|
||||
assert_eq!(all[0].id, print, "sorted by name");
|
||||
assert_eq!(all[0].place, Some(Place::Local("/mnt/print".into())));
|
||||
assert_eq!(all[1].id, web);
|
||||
assert_eq!(all[1].name, "Web", "trimmed");
|
||||
assert_eq!(all[1].place, Some(Place::Server("Shared/Web".into())));
|
||||
assert_eq!(all[1].sources, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_empty_name_is_refused() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
assert!(matches!(
|
||||
create(conn, " ", &Place::Local("/x".into())),
|
||||
Err(CatalogError::EmptyName)
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn exports_link_files_back_to_their_images() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||
let a = image(conn, "a.cr3");
|
||||
let b = image(conn, "b.cr3");
|
||||
|
||||
record_exports(
|
||||
conn,
|
||||
album,
|
||||
&[
|
||||
(a, "a.jpg".into()),
|
||||
(a, "a (1).jpg".into()),
|
||||
(b, "b.jpg".into()),
|
||||
],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let got = get(conn, album).unwrap().unwrap();
|
||||
assert_eq!(got.sources, 2, "two photographs, three files");
|
||||
let mut s = sources(conn, album).unwrap();
|
||||
s.sort();
|
||||
assert_eq!(s, vec![a, b]);
|
||||
assert_eq!(files_of(conn, album, a).unwrap().len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_overwritten_file_points_at_what_wrote_it_last() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||
let a = image(conn, "a.cr3");
|
||||
let b = image(conn, "b.cr3");
|
||||
record_exports(conn, album, &[(a, "x.jpg".into())]).unwrap();
|
||||
record_exports(conn, album, &[(b, "x.jpg".into())]).unwrap();
|
||||
assert_eq!(sources(conn, album).unwrap(), vec![b]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn moving_to_the_server_forgets_the_local_folder() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||
set_place(conn, album, &Place::Server("Web".into())).unwrap();
|
||||
assert_eq!(
|
||||
get(conn, album).unwrap().unwrap().place,
|
||||
Some(Place::Server("Web".into()))
|
||||
);
|
||||
set_place(conn, album, &Place::Local("/again".into())).unwrap();
|
||||
assert_eq!(
|
||||
get(conn, album).unwrap().unwrap().place,
|
||||
Some(Place::Local("/again".into()))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_deleted_album_is_gone_and_its_uuid_no_longer_resolves() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||
let uuid = get(conn, album).unwrap().unwrap().uuid;
|
||||
let a = image(conn, "a.cr3");
|
||||
record_exports(conn, album, &[(a, "a.jpg".into())]).unwrap();
|
||||
|
||||
delete(conn, album).unwrap();
|
||||
assert!(list(conn).unwrap().is_empty());
|
||||
assert_eq!(id_for_uuid(conn, &uuid).unwrap(), None);
|
||||
assert!(matches!(
|
||||
rename(conn, album, "Again"),
|
||||
Err(CatalogError::NoSuchAlbum(_))
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rename_bumps_the_revision_the_merge_compares() {
|
||||
let cat = catalog();
|
||||
let conn = cat.connection();
|
||||
let album = create(conn, "Web", &Place::Local("/out".into())).unwrap();
|
||||
rename(conn, album, "Website").unwrap();
|
||||
let rev: i64 = conn
|
||||
.query_row(
|
||||
"SELECT revision FROM albums WHERE id = ?1",
|
||||
[album.0 as i64],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(rev, 2);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,417 @@
|
||||
//! TRACES: NFR-P9
|
||||
//! Which catalog files this process has already backfilled, and as of what.
|
||||
//!
|
||||
//! # Why this exists
|
||||
//!
|
||||
//! [`crate::schema::backfill`] used to run inside every [`crate::Catalog::open`],
|
||||
//! and every worker thread opens its own connection. Landing on a photograph
|
||||
//! in develop opened the catalog five times — the fetch of the original and a
|
||||
//! cache check per prefetched neighbour — and each open paid the whole
|
||||
//! backfill: an anti-join of every image against its versions, a pass over
|
||||
//! every default version's uuid, the unpaired JPEGs and the keyword
|
||||
//! vocabulary. On the reference library that was ~12 ms an open and ~60 ms of
|
||||
//! CPU a landing, spent confirming that nothing had changed since the open
|
||||
//! before.
|
||||
//!
|
||||
//! # What makes skipping it safe
|
||||
//!
|
||||
//! Everything the backfill repairs is a row some write *added*: an image
|
||||
//! inserted by a scan or an import has no default version and may be the RAW
|
||||
//! beside an unpaired JPEG; a version merged or restored from an older build
|
||||
//! may carry a minted uuid; a keyword assignment merged from a remote may name
|
||||
//! a word with no term. So the question "is any work owed?" is answered by
|
||||
//! whether those tables have gained rows since the last backfill, and that is
|
||||
//! a read of each table's last row — the last page of its b-tree — rather than
|
||||
//! a scan.
|
||||
//!
|
||||
//! # Why the last row, and not only its id
|
||||
//!
|
||||
//! None of these tables is `AUTOINCREMENT`, so SQLite hands out the largest
|
||||
//! rowid plus one, and an id freed by deleting the newest row is handed out
|
||||
//! again. That is an ordinary sequence, not a contrived one: emptying the
|
||||
//! trash of the newest photograph and then scanning a new one, or a local
|
||||
//! folder's walk removing a renamed file's row and inserting the new name in
|
||||
//! the same pass. `max(id)` does not move, and neither does `count(*)`. And
|
||||
//! the row that took the id is exactly one that needs the backfill, because
|
||||
//! neither scan creates default versions — `persist` and the walk insert the
|
||||
//! image and leave the version, the pairing and the keyword terms to the next
|
||||
//! open. Skipped, it would go without them until the app restarted: a rating
|
||||
//! or a keyword with nowhere to land, a JPEG beside its RAW shown twice.
|
||||
//!
|
||||
//! So the stamp carries the last row's content as well as its id: the newest
|
||||
//! image's path, when it was added, and **whether it has a version**; the
|
||||
//! newest version's image; the newest assignment's word and version. Whether
|
||||
//! the newest image has a version is the part that cannot be fooled: once the
|
||||
//! backfill has run, every image has one, and a row that has just taken a
|
||||
//! freed id has none, so the two stamps differ whatever the path and the time
|
||||
//! say. The others make the newest version or assignment a different row
|
||||
//! whenever a different one took its id; one that is the same content at the
|
||||
//! same id is the same row as far as the backfill is concerned.
|
||||
//!
|
||||
//! The [`Stamp`] is those, the schema version, and the file's identity.
|
||||
//! An open whose stamp matches the one recorded at the last backfill of the
|
||||
//! same path skips it; anything else runs it. That covers the cases that must
|
||||
//! run it:
|
||||
//!
|
||||
//! - **The first open in a process.** Nothing is recorded yet.
|
||||
//! - **A migration.** `user_version` is in the stamp, and [`crate::Catalog::open`]
|
||||
//! also runs the backfill unconditionally whenever `migrate` moved the
|
||||
//! schema, because that is what the backfill was written for.
|
||||
//! - **A pulled catalog.** The merge inserts assignments, which moves the
|
||||
//! stamp; and [`crate::sync::merge_remote`] [`forget`]s the path as well, so
|
||||
//! the next open backfills even when every incoming row collided.
|
||||
//! - **A file replaced underneath the path** — a restore from backup, a
|
||||
//! rebuild, a catalog copied in. On unix the device and inode are in the
|
||||
//! stamp, and a replacement is a new inode; [`crate::recovery::set_aside`],
|
||||
//! the first step of both a restore and a rebuild, forgets the path too.
|
||||
//! - **Another process writing.** The stamp is read from the file, not from
|
||||
//! anything this process did, so a scan in a second instance moves it just
|
||||
//! the same.
|
||||
//!
|
||||
//! # Why the stamp is taken before the backfill
|
||||
//!
|
||||
//! The backfill adds versions and terms itself, so a stamp read afterwards
|
||||
//! would describe its own writes. Read afterwards it could also describe an
|
||||
//! image another connection inserted between the backfill's read and the
|
||||
//! stamp's — and record that image as covered when it was not. Read before,
|
||||
//! the worst case is the reverse: the backfill's own inserts move the stamp,
|
||||
//! and the next open runs one more backfill that finds nothing. That costs one
|
||||
//! redundant pass after a backfill that did real work, and never misses a row.
|
||||
//!
|
||||
//! # What it does not see
|
||||
//!
|
||||
//! An `UPDATE` that creates work without adding a row. None of this build's
|
||||
//! writers does: a scan's move of a file is a new `source_ref` and so a new
|
||||
//! image, and uuids are only rewritten by the backfill itself. Should one
|
||||
//! appear, the cost is that its repair waits for the next insert or the next
|
||||
//! start of the app — which is exactly where the backfill ran before it ran on
|
||||
//! every open.
|
||||
//!
|
||||
//! Kept in memory rather than in the catalog on purpose: a row in the file
|
||||
//! would travel in the sync snapshot and would need a table an older build
|
||||
//! does not have, and a flag that another device's catalog carried in would
|
||||
//! say nothing about this one.
|
||||
|
||||
use std::collections::HashMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::{Mutex, OnceLock};
|
||||
|
||||
use rusqlite::Connection;
|
||||
|
||||
use crate::error::CatalogError;
|
||||
|
||||
/// What a catalog looked like, as far as the backfill cares.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub(crate) struct Stamp {
|
||||
/// Device and inode, so a file swapped in under the same name is a new
|
||||
/// catalog. `None` where the platform has no such thing.
|
||||
file: Option<(u64, u64)>,
|
||||
user_version: i64,
|
||||
/// The newest image: id, path, when added, and whether it has a version.
|
||||
last_image: Option<String>,
|
||||
/// The newest version: id and the image it belongs to.
|
||||
last_version: Option<String>,
|
||||
/// The newest keyword assignment: rowid, version and word.
|
||||
last_keyword: Option<String>,
|
||||
}
|
||||
|
||||
/// The stamp recorded at the last backfill, per catalog file.
|
||||
fn done() -> &'static Mutex<HashMap<PathBuf, Stamp>> {
|
||||
static DONE: OnceLock<Mutex<HashMap<PathBuf, Stamp>>> = OnceLock::new();
|
||||
DONE.get_or_init(Default::default)
|
||||
}
|
||||
|
||||
/// One name per file, whichever spelling of its path the caller used.
|
||||
fn key(path: &Path) -> PathBuf {
|
||||
std::fs::canonicalize(path).unwrap_or_else(|_| path.to_path_buf())
|
||||
}
|
||||
|
||||
/// Read the stamp of the catalog behind `conn`, which was opened from `path`.
|
||||
///
|
||||
/// One statement: the last row of each of three tables, each found by
|
||||
/// descending its rowid b-tree to the last page, plus one probe of
|
||||
/// `versions_image` for the newest image — and a `stat` of the file.
|
||||
pub(crate) fn stamp(conn: &Connection, path: &Path) -> Result<Stamp, CatalogError> {
|
||||
let (user_version, last_image, last_version, last_keyword) = conn.query_row(
|
||||
"SELECT (SELECT user_version FROM pragma_user_version),
|
||||
(SELECT printf('%d|%d|%d|%s', i.id, i.added_at,
|
||||
EXISTS (SELECT 1 FROM versions v WHERE v.image_id = i.id),
|
||||
i.source_ref)
|
||||
FROM images i ORDER BY i.id DESC LIMIT 1),
|
||||
(SELECT printf('%d|%d', id, image_id)
|
||||
FROM versions ORDER BY id DESC LIMIT 1),
|
||||
(SELECT printf('%d|%d|%s', rowid, version_id, keyword)
|
||||
FROM keywords ORDER BY rowid DESC LIMIT 1)",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)),
|
||||
)?;
|
||||
Ok(Stamp {
|
||||
file: file_identity(path),
|
||||
user_version,
|
||||
last_image,
|
||||
last_version,
|
||||
last_keyword,
|
||||
})
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
fn file_identity(path: &Path) -> Option<(u64, u64)> {
|
||||
use std::os::unix::fs::MetadataExt;
|
||||
std::fs::metadata(path).ok().map(|m| (m.dev(), m.ino()))
|
||||
}
|
||||
|
||||
#[cfg(not(unix))]
|
||||
fn file_identity(_path: &Path) -> Option<(u64, u64)> {
|
||||
None
|
||||
}
|
||||
|
||||
/// Whether the catalog at `path` was last backfilled at exactly `stamp`.
|
||||
pub(crate) fn is_current(path: &Path, stamp: &Stamp) -> bool {
|
||||
done()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.get(&key(path))
|
||||
== Some(stamp)
|
||||
}
|
||||
|
||||
/// Record that the catalog at `path` has been backfilled as of `stamp`.
|
||||
pub(crate) fn record(path: &Path, stamp: Stamp) {
|
||||
done()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.insert(key(path), stamp);
|
||||
}
|
||||
|
||||
/// Make the next open of `path` backfill, whatever its stamp says.
|
||||
///
|
||||
/// For the writers that know they have changed the catalog wholesale — a
|
||||
/// merge of a pulled catalog, a restore from backup — so their correctness
|
||||
/// does not rest on the stamp happening to move.
|
||||
pub(crate) fn forget(path: &Path) {
|
||||
done()
|
||||
.lock()
|
||||
.unwrap_or_else(|e| e.into_inner())
|
||||
.remove(&key(path));
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::rating::derived_version_uuid;
|
||||
use crate::Catalog;
|
||||
use std::path::PathBuf;
|
||||
|
||||
/// A catalog file of its own, holding one image the server has named,
|
||||
/// backfilled and settled.
|
||||
///
|
||||
/// Opened three times on the way: to create it; after the image went in,
|
||||
/// which gives the image its default version; and once more, because that
|
||||
/// version moved the stamp and the next open runs the one redundant pass
|
||||
/// the module header describes. After that the stamp stands still.
|
||||
fn catalog(tag: &str) -> PathBuf {
|
||||
let dir = std::env::temp_dir().join(format!(
|
||||
"dr-backfilled-{tag}-{}-{:?}",
|
||||
std::process::id(),
|
||||
std::thread::current().id()
|
||||
));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
let path = dir.join("catalog.sqlite");
|
||||
{
|
||||
let cat = Catalog::open(&path).unwrap();
|
||||
let c = cat.connection();
|
||||
c.execute_batch(
|
||||
"INSERT INTO roots(id, kind, label) VALUES (1, 'remote', 'Photos');
|
||||
INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (1, 1, 'Photos/a.CR3', 0);
|
||||
INSERT INTO remote(image_id, file_id) VALUES (1, 77);",
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
path
|
||||
}
|
||||
|
||||
/// The default version's uuid for `image`, read through an ordinary open.
|
||||
fn uuid(path: &std::path::Path, image: i64) -> Option<String> {
|
||||
let cat = Catalog::open(path).unwrap();
|
||||
cat.connection()
|
||||
.query_row(
|
||||
"SELECT uuid FROM versions WHERE image_id = ?1 AND is_default = 1",
|
||||
[image],
|
||||
|r| r.get(0),
|
||||
)
|
||||
.ok()
|
||||
}
|
||||
|
||||
/// Put the one row back into the state the backfill repairs, with an
|
||||
/// `UPDATE` — which moves none of the stamp's maxima, so only the stamp's
|
||||
/// other parts or an explicit `forget` can bring the backfill back.
|
||||
fn unalign(path: &std::path::Path) {
|
||||
rusqlite::Connection::open(path)
|
||||
.unwrap()
|
||||
.execute("UPDATE versions SET uuid = 'minted' WHERE image_id = 1", [])
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unchanged_catalog_is_not_backfilled_again() {
|
||||
let path = catalog("unchanged");
|
||||
unalign(&path);
|
||||
assert_eq!(
|
||||
uuid(&path, 1).as_deref(),
|
||||
Some("minted"),
|
||||
"nothing was added since the last backfill, so the open skipped it"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_image_a_scan_added_is_backfilled_on_the_next_open() {
|
||||
let path = catalog("scanned");
|
||||
rusqlite::Connection::open(&path)
|
||||
.unwrap()
|
||||
.execute(
|
||||
"INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (2, 1, 'Photos/b.CR3', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
assert!(uuid(&path, 2).is_some(), "the new image got its version");
|
||||
}
|
||||
|
||||
/// The id of a deleted newest row is handed out again, so `max(id)` is
|
||||
/// the same before and after — the sequence emptying the trash and then
|
||||
/// scanning makes. The image that took the id still needs its version.
|
||||
///
|
||||
/// A virtual copy on the older image holds the newest version id, so
|
||||
/// the deletion does not move `max(versions.id)` either: nothing the old
|
||||
/// stamp read changes, which is the case that went unrepaired.
|
||||
#[test]
|
||||
fn an_image_that_reuses_a_deleted_id_is_backfilled_on_the_next_open() {
|
||||
let path = catalog("reused");
|
||||
rusqlite::Connection::open(&path)
|
||||
.unwrap()
|
||||
.execute(
|
||||
"INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (2, 1, 'Photos/b.CR3', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
assert!(uuid(&path, 2).is_some());
|
||||
rusqlite::Connection::open(&path)
|
||||
.unwrap()
|
||||
.execute(
|
||||
"INSERT INTO versions(image_id, uuid, name, is_default)
|
||||
VALUES (1, 'copy', 'Crop', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
// Settle: one open backfills after the new version, one more runs
|
||||
// the redundant pass and records the stamp that stands.
|
||||
assert!(uuid(&path, 2).is_some());
|
||||
assert!(uuid(&path, 2).is_some());
|
||||
|
||||
let c = rusqlite::Connection::open(&path).unwrap();
|
||||
let before: (i64, i64) = c
|
||||
.query_row(
|
||||
"SELECT (SELECT max(id) FROM images), (SELECT max(id) FROM versions)",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
.unwrap();
|
||||
c.execute("DELETE FROM images WHERE id = 2", []).unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO images(root_id, source_ref, added_at)
|
||||
VALUES (1, 'Photos/c.CR3', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
let after: (i64, i64) = c
|
||||
.query_row(
|
||||
"SELECT (SELECT max(id) FROM images), (SELECT max(id) FROM versions)",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(before, after, "SQLite handed the freed id out again");
|
||||
drop(c);
|
||||
assert!(uuid(&path, 2).is_some(), "the new image got its version");
|
||||
}
|
||||
|
||||
/// The same, with the same file coming back at the same id in the same
|
||||
/// second: path and time match, and only the missing version tells.
|
||||
#[test]
|
||||
fn the_same_file_back_at_the_same_id_is_backfilled_on_the_next_open() {
|
||||
let path = catalog("returned");
|
||||
let c = rusqlite::Connection::open(&path).unwrap();
|
||||
// As above: a newer version on another image keeps the deletion
|
||||
// from moving `max(versions.id)`.
|
||||
c.execute_batch(
|
||||
"INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (0, 1, 'Photos/0.CR3', 0);
|
||||
INSERT INTO versions(image_id, uuid, name, is_default)
|
||||
VALUES (0, 'copy', 'Crop', 0);",
|
||||
)
|
||||
.unwrap();
|
||||
drop(c);
|
||||
assert!(uuid(&path, 1).is_some());
|
||||
assert!(uuid(&path, 1).is_some());
|
||||
let c = rusqlite::Connection::open(&path).unwrap();
|
||||
c.execute_batch(
|
||||
"DELETE FROM images WHERE id = 1;
|
||||
INSERT INTO images(id, root_id, source_ref, added_at)
|
||||
VALUES (1, 1, 'Photos/a.CR3', 0);
|
||||
INSERT INTO remote(image_id, file_id) VALUES (1, 77);",
|
||||
)
|
||||
.unwrap();
|
||||
drop(c);
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_first_open_after_a_migration_backfills() {
|
||||
let path = catalog("migrated");
|
||||
unalign(&path);
|
||||
rusqlite::Connection::open(&path)
|
||||
.unwrap()
|
||||
.pragma_update(None, "user_version", crate::schema::SCHEMA_VERSION - 1)
|
||||
.unwrap();
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_first_open_after_a_pulled_catalog_backfills() {
|
||||
let path = catalog("pulled");
|
||||
// The remote is this catalog as it stands, so every row the merge
|
||||
// offers collides and nothing in the stamp moves: only the merge
|
||||
// saying so can make the next open backfill.
|
||||
let remote = path.with_file_name("remote.sqlite");
|
||||
rusqlite::Connection::open(&path)
|
||||
.unwrap()
|
||||
.execute("VACUUM INTO ?1", [remote.to_string_lossy().as_ref()])
|
||||
.unwrap();
|
||||
unalign(&path);
|
||||
assert_eq!(uuid(&path, 1).as_deref(), Some("minted"));
|
||||
|
||||
Catalog::open(&path)
|
||||
.unwrap()
|
||||
.merge_remote_catalog(&remote)
|
||||
.unwrap();
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
}
|
||||
|
||||
#[cfg(unix)]
|
||||
#[test]
|
||||
fn a_catalog_replaced_under_the_same_name_backfills() {
|
||||
let path = catalog("replaced");
|
||||
unalign(&path);
|
||||
assert_eq!(uuid(&path, 1).as_deref(), Some("minted"));
|
||||
// Every connection is closed, so the WAL is folded in and the main
|
||||
// file is the whole catalog. A copy renamed over it is the same rows
|
||||
// in a new file — which is what a restore or a copied-in catalog is.
|
||||
let copy = path.with_file_name("copy.sqlite");
|
||||
std::fs::copy(&path, ©).unwrap();
|
||||
std::fs::rename(©, &path).unwrap();
|
||||
assert_eq!(uuid(&path, 1), Some(derived_version_uuid(77)));
|
||||
}
|
||||
}
|
||||
@@ -722,6 +722,31 @@ pub fn not_collapsed_away(image: &str) -> String {
|
||||
)
|
||||
}
|
||||
|
||||
/// SQL for the rows [`not_collapsed_away`] drops, as a `FROM ... WHERE`
|
||||
/// joining each such frame to its image under the alias `image`.
|
||||
///
|
||||
/// For counting. A count that applies [`not_collapsed_away`] to every row
|
||||
/// pays two primary-key probes per image to find the handful a collapsed
|
||||
/// burst hides; counting everything and subtracting what this lists walks
|
||||
/// only `burst_members`, which is empty on a library without bursts. The
|
||||
/// caller appends its own conditions on `image` with `AND`, the same ones
|
||||
/// it counted the whole with, so the subtraction takes away only rows the
|
||||
/// whole included. `image_id` is `burst_members`' key, so no image is
|
||||
/// listed twice.
|
||||
///
|
||||
/// The two must describe the same rows: change one, change both, and
|
||||
/// `the_collapsed_frames_are_what_the_predicate_drops` will say if they drift.
|
||||
///
|
||||
/// Never interpolate anything user-supplied as `image`.
|
||||
pub fn collapsed_away_frames(image: &str) -> String {
|
||||
format!(
|
||||
"burst_members bm CROSS JOIN images {image} ON {image}.id = bm.image_id
|
||||
WHERE bm.representative = 0
|
||||
AND NOT EXISTS (SELECT 1 FROM burst_expanded be
|
||||
WHERE be.burst_id = bm.burst_id)"
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
@@ -1235,6 +1260,46 @@ mod tests {
|
||||
assert_eq!(visible(cat.connection()), vec![1, 2, 3, 4]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_collapsed_frames_are_what_the_predicate_drops() {
|
||||
// `collapsed_away_frames` is `not_collapsed_away` turned inside out
|
||||
// for counting; the two must name the same rows, open or closed.
|
||||
let cat = seeded(&[
|
||||
(1, 1000, Some(0xFF00)),
|
||||
(2, 1001, Some(0xFF00)),
|
||||
(3, 1002, Some(0xFF00)),
|
||||
(4, 9000, Some(0xAA00)),
|
||||
(5, 9001, Some(0xAA00)),
|
||||
(6, 20000, Some(0xFF00)),
|
||||
]);
|
||||
regroup(cat.connection(), Rules::default()).unwrap();
|
||||
let ids = |sql: String| -> Vec<i64> {
|
||||
let c = cat.connection();
|
||||
let mut stmt = c.prepare(&sql).unwrap();
|
||||
let rows = stmt.query_map([], |r| r.get::<_, i64>(0)).unwrap();
|
||||
rows.collect::<Result<Vec<_>, _>>().unwrap()
|
||||
};
|
||||
let dropped = || {
|
||||
ids(format!(
|
||||
"SELECT id FROM images i WHERE NOT {} ORDER BY id",
|
||||
not_collapsed_away("i")
|
||||
))
|
||||
};
|
||||
let listed = || {
|
||||
ids(format!(
|
||||
"SELECT i.id FROM {} ORDER BY i.id",
|
||||
collapsed_away_frames("i")
|
||||
))
|
||||
};
|
||||
|
||||
assert_eq!(listed(), dropped());
|
||||
set_expanded(cat.connection(), ImageId(1), false).unwrap();
|
||||
assert_eq!(dropped(), vec![2, 3]);
|
||||
assert_eq!(listed(), dropped());
|
||||
set_expanded(cat.connection(), ImageId(4), false).unwrap();
|
||||
assert_eq!(listed(), dropped());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_library_with_no_bursts_hides_nothing() {
|
||||
// The predicate is in every grid query, so its cost and its effect on a
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -65,6 +65,16 @@ pub enum CatalogError {
|
||||
#[error("no such collection: {0}")]
|
||||
NoSuchCollection(u64),
|
||||
|
||||
/// An album the caller named is gone — deleted here, or by a merge while
|
||||
/// its id sat in a UI model.
|
||||
#[error("no such album: {0}")]
|
||||
NoSuchAlbum(u64),
|
||||
|
||||
/// A name that is empty once trimmed. Refused rather than stored, because
|
||||
/// a row with no name is one the sidebar cannot draw and nobody can pick.
|
||||
#[error("a name is required")]
|
||||
EmptyName,
|
||||
|
||||
/// A keyword the caller named is gone — deleted, or fused into another by a
|
||||
/// merge while its id sat in a UI model.
|
||||
///
|
||||
@@ -96,6 +106,13 @@ pub enum CatalogError {
|
||||
|
||||
#[error("io: {0}")]
|
||||
Io(String),
|
||||
|
||||
/// TRACES: FR-CAT-11a
|
||||
/// A duplicate group planned earlier no longer holds: a copy was trashed,
|
||||
/// rescanned or changed since the review was drawn. The group is left
|
||||
/// untouched rather than consolidated on a stale plan.
|
||||
#[error("no longer a duplicate: {0}")]
|
||||
StaleDuplicate(String),
|
||||
}
|
||||
|
||||
impl From<rusqlite::Error> for CatalogError {
|
||||
|
||||
@@ -30,6 +30,7 @@
|
||||
//! identity every client agrees on (FR-NC-5), and it survives a server-side
|
||||
//! move, so a shard written before a reorganisation still applies after it.
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use rusqlite::{Connection, OptionalExtension};
|
||||
@@ -155,6 +156,45 @@ impl FaceShardStore {
|
||||
.flatten()
|
||||
}
|
||||
|
||||
/// [`indexed_at`](Self::indexed_at) for every entry at once.
|
||||
///
|
||||
/// What a pass over the whole library asks instead of one lookup per image:
|
||||
/// the export compares every marker in the catalog with this, and a lookup
|
||||
/// each was 19,000 statements prepared and run on every sync pass that had
|
||||
/// nothing to send.
|
||||
fn all_indexed_at(&self) -> Result<HashMap<(u64, String), Option<i64>>, CatalogError> {
|
||||
let mut q = self
|
||||
.index
|
||||
.prepare("SELECT file_id, model_id, indexed_at FROM entries")?;
|
||||
let rows = q.query_map([], |r| {
|
||||
Ok((
|
||||
(r.get::<_, i64>(0)? as u64, r.get::<_, String>(1)?),
|
||||
r.get::<_, Option<i64>>(2)?,
|
||||
))
|
||||
})?;
|
||||
Ok(rows.collect::<Result<_, _>>()?)
|
||||
}
|
||||
|
||||
/// [`held_model`](Self::held_model) for every file at once: one statement,
|
||||
/// ordered exactly as that one is, keeping the first row per file.
|
||||
fn all_held_models(&self, model_id: &str) -> Result<HashMap<u64, String>, CatalogError> {
|
||||
let mut q = self.index.prepare(&format!(
|
||||
"SELECT file_id, model_id FROM entries
|
||||
WHERE {} = ?1
|
||||
ORDER BY file_id, indexed_at DESC NULLS LAST, model_id",
|
||||
crate::faces::embedder_sql("model_id")
|
||||
))?;
|
||||
let rows = q.query_map([crate::faces::embedder_of(model_id)], |r| {
|
||||
Ok((r.get::<_, i64>(0)? as u64, r.get::<_, String>(1)?))
|
||||
})?;
|
||||
let mut out = HashMap::new();
|
||||
for row in rows {
|
||||
let (file, model) = row?;
|
||||
out.entry(file).or_insert(model);
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// The pipeline this store holds an image under, among those sharing
|
||||
/// `model_id`'s embedder — the most recently indexed where a peer has
|
||||
/// sent more than one.
|
||||
@@ -784,6 +824,18 @@ pub fn export_to_shards_reporting(
|
||||
/// enough that the reporting is lost in the write it accompanies.
|
||||
const REPORT_EVERY: usize = 25;
|
||||
|
||||
// What the store holds, read once. A put below rewrites only its own
|
||||
// file's entries -- its generation, and siblings it supersedes -- so a
|
||||
// file already written in this pass is asked of the store again and every
|
||||
// other answer is the one a lookup would have given.
|
||||
//
|
||||
// An index that cannot be read answers as each lookup did: nothing held.
|
||||
let held = store.all_indexed_at().unwrap_or_else(|e| {
|
||||
log::debug!("reading the shard index: {e}");
|
||||
HashMap::new()
|
||||
});
|
||||
let mut written: HashSet<u64> = HashSet::new();
|
||||
|
||||
let total = rows.len();
|
||||
let mut exported = 0;
|
||||
for (seen, (file_id, image_id, edge, indexed_at, model_id)) in rows.into_iter().enumerate() {
|
||||
@@ -800,10 +852,14 @@ pub fn export_to_shards_reporting(
|
||||
//
|
||||
// The comparison is against when the *catalog* indexed it, so a
|
||||
// re-index is visible and an unchanged image still costs nothing.
|
||||
if store
|
||||
.indexed_at(file_id as u64, model_id)
|
||||
.is_some_and(|was| was >= indexed_at)
|
||||
{
|
||||
let was = if written.contains(&(file_id as u64)) {
|
||||
store.indexed_at(file_id as u64, model_id)
|
||||
} else {
|
||||
held.get(&(file_id as u64, model_id.to_string()))
|
||||
.copied()
|
||||
.flatten()
|
||||
};
|
||||
if was.is_some_and(|was| was >= indexed_at) {
|
||||
continue;
|
||||
}
|
||||
let mut fq = conn.prepare(
|
||||
@@ -840,6 +896,7 @@ pub fn export_to_shards_reporting(
|
||||
&faces,
|
||||
Some(indexed_at),
|
||||
)?;
|
||||
written.insert(file_id as u64);
|
||||
exported += 1;
|
||||
}
|
||||
progress(total, total);
|
||||
@@ -902,6 +959,18 @@ pub fn import_from_shards(
|
||||
/// the lock, waits a fraction of a second and not the whole import.
|
||||
const CHUNK: usize = 100;
|
||||
|
||||
// Every file's held pipeline, read once rather than asked per candidate --
|
||||
// 23,000 prepared lookups on every pass, nearly all of them for images
|
||||
// this device already holds. Nothing below changes which pipeline the
|
||||
// store holds a file under (`set_indexed_at` touches only a file already
|
||||
// decided), and each candidate is a different file, so these are the
|
||||
// answers the lookups gave.
|
||||
// An index that cannot be read answers as each lookup did: nothing held.
|
||||
let held_models = store.all_held_models(model_id).unwrap_or_else(|e| {
|
||||
log::debug!("reading the shard index: {e}");
|
||||
HashMap::new()
|
||||
});
|
||||
|
||||
let mut adopted = 0;
|
||||
let mut tx = conn.unchecked_transaction()?;
|
||||
let mut in_chunk = 0;
|
||||
@@ -911,7 +980,7 @@ pub fn import_from_shards(
|
||||
tx = conn.unchecked_transaction()?;
|
||||
in_chunk = 0;
|
||||
}
|
||||
let Some(held) = store.held_model(file_id as u64, model_id) else {
|
||||
let Some(held) = held_models.get(&(file_id as u64)).cloned() else {
|
||||
continue;
|
||||
};
|
||||
if let Some(local) = local {
|
||||
|
||||
@@ -1029,7 +1029,8 @@ fn people_where(conn: &Connection, in_use: bool) -> Result<Vec<Person>, CatalogE
|
||||
///
|
||||
/// Confirmations survive the move: a face the user confirmed as the source
|
||||
/// person is now a confirmed face of the target, which is what the user meant
|
||||
/// by saying they are the same person.
|
||||
/// by saying they are the same person. So do rejections — see
|
||||
/// [`merge_people_within`].
|
||||
pub fn merge_people(
|
||||
conn: &Connection,
|
||||
target: PersonId,
|
||||
@@ -1039,7 +1040,53 @@ pub fn merge_people(
|
||||
return Ok(0);
|
||||
}
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let moved = merge_people_within(&tx, target, source)?;
|
||||
tx.commit()?;
|
||||
Ok(moved)
|
||||
}
|
||||
|
||||
/// [`merge_people`] inside a transaction the caller holds, so a job that
|
||||
/// merges several pairs commits once (`crate::dedup_people`).
|
||||
///
|
||||
/// **Rejections move with the faces.** "This face is not Annie" is a
|
||||
/// judgement about the person, and once Annie is Anna it is one about Anna.
|
||||
/// Left on the redirect it binds nothing, and the next grouping pass
|
||||
/// suggests the face the user pushed away to the person it now belongs to. Where the
|
||||
/// two halves disagree about one face — confirmed as one, rejected as the
|
||||
/// other — the confirmation stands, which is the rule [`confirm`] applies
|
||||
/// to one face; and a moved rejection takes a suggestion of the same face
|
||||
/// with it, the rule [`reject`] applies.
|
||||
pub(crate) fn merge_people_within(
|
||||
tx: &Connection,
|
||||
target: PersonId,
|
||||
source: PersonId,
|
||||
) -> Result<u64, CatalogError> {
|
||||
if target == source {
|
||||
return Ok(0);
|
||||
}
|
||||
let moved = move_judgements(tx, target, source)?;
|
||||
tx.execute(
|
||||
"UPDATE people SET merged_into = ?1, revision = revision + 1, modified = ?3
|
||||
WHERE id = ?2",
|
||||
rusqlite::params![target.0 as i64, source.0 as i64, now_secs()],
|
||||
)?;
|
||||
Ok(moved)
|
||||
}
|
||||
|
||||
/// The half of [`merge_people_within`] that moves faces and rejections,
|
||||
/// without touching either person's row.
|
||||
///
|
||||
/// Also what follows a redirect that arrived by sync
|
||||
/// (`crate::dedup_people`): the other device merged the people, and this
|
||||
/// one still holds judgements on the person merged away. Bumping the
|
||||
/// person's revision there would be an edit of this device's own, sent back
|
||||
/// on every pass, so the row is left as the merge wrote it.
|
||||
pub(crate) fn move_judgements(
|
||||
tx: &Connection,
|
||||
target: PersonId,
|
||||
source: PersonId,
|
||||
) -> Result<u64, CatalogError> {
|
||||
let (t, s) = (target.0 as i64, source.0 as i64);
|
||||
// A face already assigned to the target must not gain a second row —
|
||||
// `face_person` is keyed by face. Where both hold the same face, the
|
||||
// target's row wins and the source's is dropped.
|
||||
@@ -1047,18 +1094,31 @@ pub fn merge_people(
|
||||
"DELETE FROM face_person
|
||||
WHERE person_id = ?2
|
||||
AND face_id IN (SELECT face_id FROM face_person WHERE person_id = ?1)",
|
||||
rusqlite::params![target.0 as i64, source.0 as i64],
|
||||
rusqlite::params![t, s],
|
||||
)?;
|
||||
let moved = tx.execute(
|
||||
"UPDATE face_person SET person_id = ?1 WHERE person_id = ?2",
|
||||
rusqlite::params![target.0 as i64, source.0 as i64],
|
||||
rusqlite::params![t, s],
|
||||
)?;
|
||||
tx.execute(
|
||||
"UPDATE people SET merged_into = ?1, revision = revision + 1, modified = ?3
|
||||
WHERE id = ?2",
|
||||
rusqlite::params![target.0 as i64, source.0 as i64, now_secs()],
|
||||
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
|
||||
SELECT face_id, ?1 FROM face_person_rejected WHERE person_id = ?2",
|
||||
rusqlite::params![t, s],
|
||||
)?;
|
||||
tx.execute("DELETE FROM face_person_rejected WHERE person_id = ?1", [s])?;
|
||||
tx.execute(
|
||||
"DELETE FROM face_person_rejected
|
||||
WHERE person_id = ?1
|
||||
AND face_id IN (SELECT face_id FROM face_person
|
||||
WHERE person_id = ?1 AND confirmed = 1)",
|
||||
[t],
|
||||
)?;
|
||||
tx.execute(
|
||||
"DELETE FROM face_person
|
||||
WHERE person_id = ?1 AND confirmed = 0
|
||||
AND face_id IN (SELECT face_id FROM face_person_rejected WHERE person_id = ?1)",
|
||||
[t],
|
||||
)?;
|
||||
tx.commit()?;
|
||||
Ok(moved as u64)
|
||||
}
|
||||
|
||||
@@ -1599,6 +1659,170 @@ fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-11a | FR-CULL-10
|
||||
/// What [`carry_onto_copy`] did with one byte-identical copy's faces.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub struct FaceCarry {
|
||||
/// Faces moved onto the survivor outright, because it had none.
|
||||
pub moved: usize,
|
||||
/// Names and suggestions put on a survivor's face that lacked one.
|
||||
pub named: usize,
|
||||
/// Rejections added to a survivor's face.
|
||||
pub rejections: usize,
|
||||
/// Faces named one person on the copy and another on the survivor. The
|
||||
/// survivor's name is kept; the copy keeps its own, in the trash.
|
||||
pub conflicts: usize,
|
||||
/// Named faces on the copy that match nothing on the survivor. Left with
|
||||
/// the copy rather than mixed into another pipeline's faces.
|
||||
pub unmatched_named: usize,
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-11a | FR-CULL-10
|
||||
/// Bring what one copy of a photograph knows about its faces onto another
|
||||
/// copy of the same bytes, inside the caller's transaction.
|
||||
///
|
||||
/// Faces are per image, so two copies of one file indexed separately hold
|
||||
/// two sets of the same boxes, and a name confirmed on one is invisible on
|
||||
/// the other. The rule is the one [`record_detections_within`] keeps: an
|
||||
/// image holds one pipeline's faces at a time, and the user's judgements
|
||||
/// are what must survive.
|
||||
///
|
||||
/// - **The survivor has no faces at all.** The copy's faces and its run
|
||||
/// markers move over wholesale — nothing is duplicated, and the survivor
|
||||
/// is spared a detection pass it would only repeat. Its own markers go
|
||||
/// first, because a marker saying "examined, nothing found" over an image
|
||||
/// that now holds faces is the V12 state.
|
||||
/// - **The survivor has faces.** Each of the copy's faces is paired with the
|
||||
/// survivor face its box overlaps most (IoU above one half — the bytes are
|
||||
/// the same, so the boxes coincide). A name or suggestion is carried onto
|
||||
/// a survivor face that has none, a confirmation outranks a suggestion,
|
||||
/// and two different confirmed names are a conflict the survivor wins.
|
||||
/// Rejections are unioned. The copy's faces stay where they are, with the
|
||||
/// copy.
|
||||
pub fn carry_onto_copy(
|
||||
tx: &Connection,
|
||||
from: ImageId,
|
||||
to: ImageId,
|
||||
) -> Result<FaceCarry, CatalogError> {
|
||||
let mut out = FaceCarry::default();
|
||||
let boxes = |image: ImageId| -> Result<Vec<CopyFace>, CatalogError> {
|
||||
let mut q = tx.prepare(
|
||||
"SELECT f.id, f.x, f.y, f.w, f.h, fp.person_id, fp.probability, fp.confirmed
|
||||
FROM faces f
|
||||
LEFT JOIN face_person fp ON fp.face_id = f.id
|
||||
WHERE f.image_id = ?1
|
||||
ORDER BY f.id",
|
||||
)?;
|
||||
let rows = q.query_map([image.0 as i64], |r| {
|
||||
let person: Option<i64> = r.get(5)?;
|
||||
Ok(CopyFace {
|
||||
id: r.get(0)?,
|
||||
rect: (
|
||||
r.get::<_, f64>(1)? as f32,
|
||||
r.get::<_, f64>(2)? as f32,
|
||||
r.get::<_, f64>(3)? as f32,
|
||||
r.get::<_, f64>(4)? as f32,
|
||||
),
|
||||
assignment: match person {
|
||||
Some(p) => Some((p, r.get::<_, f64>(6)?, r.get::<_, i64>(7)? != 0)),
|
||||
None => None,
|
||||
},
|
||||
})
|
||||
})?;
|
||||
Ok(rows.collect::<Result<Vec<_>, _>>()?)
|
||||
};
|
||||
|
||||
let theirs = boxes(from)?;
|
||||
if theirs.is_empty() {
|
||||
return Ok(out);
|
||||
}
|
||||
let ours = boxes(to)?;
|
||||
|
||||
if ours.is_empty() {
|
||||
tx.execute("DELETE FROM face_index WHERE image_id = ?1", [to.0 as i64])?;
|
||||
tx.execute(
|
||||
"UPDATE face_index SET image_id = ?2 WHERE image_id = ?1",
|
||||
rusqlite::params![from.0 as i64, to.0 as i64],
|
||||
)?;
|
||||
out.moved = tx.execute(
|
||||
"UPDATE faces SET image_id = ?2 WHERE image_id = ?1",
|
||||
rusqlite::params![from.0 as i64, to.0 as i64],
|
||||
)?;
|
||||
return Ok(out);
|
||||
}
|
||||
|
||||
let mut taken = vec![false; ours.len()];
|
||||
for face in &theirs {
|
||||
let best = ours
|
||||
.iter()
|
||||
.enumerate()
|
||||
.filter(|(i, _)| !taken[*i])
|
||||
.map(|(i, o)| (i, iou(face.rect, o.rect)))
|
||||
.filter(|(_, overlap)| *overlap > 0.5)
|
||||
.max_by(|a, b| a.1.total_cmp(&b.1));
|
||||
let Some((at, _)) = best else {
|
||||
if face.assignment.is_some_and(|(_, _, confirmed)| confirmed) {
|
||||
out.unmatched_named += 1;
|
||||
}
|
||||
continue;
|
||||
};
|
||||
taken[at] = true;
|
||||
let target = &ours[at];
|
||||
|
||||
out.rejections += tx.execute(
|
||||
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
|
||||
SELECT ?2, person_id FROM face_person_rejected WHERE face_id = ?1",
|
||||
rusqlite::params![face.id, target.id],
|
||||
)?;
|
||||
|
||||
let Some((person, probability, confirmed)) = face.assignment else {
|
||||
continue;
|
||||
};
|
||||
let carry = match target.assignment {
|
||||
None => true,
|
||||
// The same person: only a confirmation upgrades a suggestion.
|
||||
Some((p, _, theirs_confirmed)) if p == person => confirmed && !theirs_confirmed,
|
||||
// Another person, only suggested there: the user's word wins.
|
||||
Some((_, _, false)) => confirmed,
|
||||
// Another person, confirmed there: the survivor keeps its name.
|
||||
Some((_, _, true)) => {
|
||||
if confirmed {
|
||||
out.conflicts += 1;
|
||||
}
|
||||
false
|
||||
}
|
||||
};
|
||||
if carry {
|
||||
tx.execute(
|
||||
"INSERT INTO face_person (face_id, person_id, probability, confirmed)
|
||||
VALUES (?1, ?2, ?3, ?4)
|
||||
ON CONFLICT(face_id) DO UPDATE SET
|
||||
person_id = excluded.person_id,
|
||||
probability = excluded.probability,
|
||||
confirmed = excluded.confirmed",
|
||||
rusqlite::params![target.id, person, probability, confirmed],
|
||||
)?;
|
||||
// A name the user gave outranks a rejection of the same pair made
|
||||
// on the survivor — the same order `confirm` applies.
|
||||
if confirmed {
|
||||
tx.execute(
|
||||
"DELETE FROM face_person_rejected WHERE face_id = ?1 AND person_id = ?2",
|
||||
rusqlite::params![target.id, person],
|
||||
)?;
|
||||
}
|
||||
out.named += 1;
|
||||
}
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// One face as [`carry_onto_copy`] pairs it.
|
||||
struct CopyFace {
|
||||
id: i64,
|
||||
rect: (f32, f32, f32, f32),
|
||||
assignment: Option<(i64, f64, bool)>,
|
||||
}
|
||||
|
||||
pub(crate) fn now_secs() -> i64 {
|
||||
std::time::SystemTime::now()
|
||||
.duration_since(std::time::UNIX_EPOCH)
|
||||
@@ -2354,6 +2578,48 @@ mod tests {
|
||||
assert_eq!(people(&c).unwrap().len(), 1);
|
||||
}
|
||||
|
||||
/// A rejection left on the redirect bound nothing: the next grouping
|
||||
/// pass suggested the face to the merged person, whom the user had told
|
||||
/// it was somebody else.
|
||||
#[test]
|
||||
fn merging_moves_the_rejections_too() {
|
||||
let c = db();
|
||||
let ids: Vec<FaceId> = (1..=3)
|
||||
.map(|n| {
|
||||
let img = image(&c, n);
|
||||
record_detections(&c, img, "w600k_mbf", 1024, &[face(n as u8)]).unwrap()[0]
|
||||
})
|
||||
.collect();
|
||||
let anna = create_person(&c, "Anna").unwrap();
|
||||
let annie = create_person(&c, "Annie").unwrap();
|
||||
// Rejected as Annie, and nothing said about Anna.
|
||||
reject(&c, ids[0], annie).unwrap();
|
||||
// Rejected as Annie, suggested as Anna: the rejection now covers it.
|
||||
suggest(&c, ids[1], anna, 0.8).unwrap();
|
||||
reject(&c, ids[1], annie).unwrap();
|
||||
// Rejected as Annie, confirmed as Anna: the confirmation stands.
|
||||
confirm(&c, ids[2], anna).unwrap();
|
||||
reject(&c, ids[2], annie).unwrap();
|
||||
|
||||
merge_people(&c, anna, annie).unwrap();
|
||||
|
||||
let rejected: Vec<(i64, i64)> = c
|
||||
.prepare("SELECT face_id, person_id FROM face_person_rejected ORDER BY face_id")
|
||||
.unwrap()
|
||||
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
|
||||
.unwrap()
|
||||
.collect::<Result<_, _>>()
|
||||
.unwrap();
|
||||
let anna_id = anna.0 as i64;
|
||||
assert_eq!(
|
||||
rejected,
|
||||
[(ids[0].0 as i64, anna_id), (ids[1].0 as i64, anna_id)]
|
||||
);
|
||||
assert_eq!(for_image(&c, ImageId(2)).unwrap()[0].person, None);
|
||||
let kept = &for_image(&c, ImageId(3)).unwrap()[0];
|
||||
assert_eq!((kept.person, kept.confirmed), (Some(anna), true));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn merging_does_not_duplicate_a_face_both_people_hold() {
|
||||
let c = db();
|
||||
|
||||
@@ -29,7 +29,11 @@ pub enum JobKind {
|
||||
ScanFolder = 0,
|
||||
/// Promote an image from stat-only to full EXIF.
|
||||
ExtractMetadata = 1,
|
||||
/// Build or rebuild a thumbnail.
|
||||
/// Build or rebuild a thumbnail. **Retired** — see [`JobKind::RETIRED`].
|
||||
///
|
||||
/// Kept so the number stays taken: a catalog written by 0.16.0 or earlier
|
||||
/// holds rows of kind 2, and reusing it would hand them to whatever took
|
||||
/// its place.
|
||||
Thumbnail = 2,
|
||||
/// A sidecar on disk is newer than what the catalog read.
|
||||
ReadSidecar = 3,
|
||||
@@ -69,6 +73,18 @@ impl JobKind {
|
||||
JobKind::DetectFaces,
|
||||
];
|
||||
|
||||
/// Kinds that are no longer queued by anything, whose rows are deleted on
|
||||
/// sight by [`drop_retired`].
|
||||
///
|
||||
/// `Thumbnail` is here because thumbnails are owed by the store, not by
|
||||
/// the queue. The grid's worker and the thumbnail sweep both find their
|
||||
/// work by asking `ThumbStore` what it lacks, and the store is shared
|
||||
/// between devices, so it is the only thing that can say another device
|
||||
/// already made one. Up to 0.16.0 every scan enqueued a job per
|
||||
/// photograph anyway and no handler ever claimed one: the reference
|
||||
/// catalog held 23,582 of them (#73; catalog.md §6.1).
|
||||
pub const RETIRED: [JobKind; 1] = [JobKind::Thumbnail];
|
||||
|
||||
fn from_i64(v: i64) -> Option<Self> {
|
||||
Some(match v {
|
||||
0 => JobKind::ScanFolder,
|
||||
@@ -185,7 +201,8 @@ pub fn enqueue(
|
||||
priority: Priority,
|
||||
payload: Option<&str>,
|
||||
) -> Result<(), CatalogError> {
|
||||
conn.execute(
|
||||
// Cached: a scan enqueues one per photograph it lists.
|
||||
conn.prepare_cached(
|
||||
"INSERT INTO jobs(kind, subject_id, priority, state, payload)
|
||||
VALUES (?1, ?2, ?3, 0, ?4)
|
||||
ON CONFLICT(kind, subject_id) DO UPDATE SET
|
||||
@@ -195,8 +212,13 @@ pub fn enqueue(
|
||||
state = CASE WHEN jobs.state = 2 THEN 0 ELSE jobs.state END,
|
||||
attempts = CASE WHEN jobs.state = 2 THEN 0 ELSE jobs.attempts END,
|
||||
not_before = CASE WHEN jobs.state = 2 THEN 0 ELSE jobs.not_before END",
|
||||
rusqlite::params![kind as i64, subject_id, priority as i64, payload],
|
||||
)?;
|
||||
)?
|
||||
.execute(rusqlite::params![
|
||||
kind as i64,
|
||||
subject_id,
|
||||
priority as i64,
|
||||
payload
|
||||
])?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
@@ -393,7 +415,7 @@ pub fn recover_orphaned(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
///
|
||||
/// Coalescing keeps the table one row per unit of work, but nothing shrinks it
|
||||
/// when the work stops existing: a library that has been culled carries a
|
||||
/// thumbnail job for every photograph deleted since the last time anything
|
||||
/// job for every photograph deleted since the last time anything
|
||||
/// looked. Each one would be claimed, run, and failed five times.
|
||||
///
|
||||
/// Only kinds whose subject really is an image ([`JobKind::subject_is_image`])
|
||||
@@ -425,6 +447,32 @@ pub fn reap_orphan_subjects(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// Delete every row of a [`JobKind::RETIRED`] kind.
|
||||
///
|
||||
/// Not a migration, deliberately. A schema bump makes an older build refuse
|
||||
/// the synced catalog snapshot, and a device still on 0.16.0 would lose the
|
||||
/// catalog to save a megabyte. So this runs where the queue is readied —
|
||||
/// [`crate::runner::recover`], at every open — and has to be cheap when there
|
||||
/// is nothing to do: `kind` leads the `UNIQUE(kind, subject_id)` index, so an
|
||||
/// empty answer is one index probe, not a table scan.
|
||||
///
|
||||
/// Every open rather than once, because once is not enough: an older build
|
||||
/// opening the same catalog enqueues them again on its next scan.
|
||||
///
|
||||
/// Rows in any state go. Nothing claims these kinds, so none can be running,
|
||||
/// and a failed one would be a report about work nobody was going to do.
|
||||
pub fn drop_retired(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
let kinds: Vec<i64> = JobKind::RETIRED.iter().map(|k| *k as i64).collect();
|
||||
let placeholders = std::iter::repeat_n("?", kinds.len())
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let n = conn.execute(
|
||||
&format!("DELETE FROM jobs WHERE kind IN ({placeholders})"),
|
||||
rusqlite::params_from_iter(kinds.iter()),
|
||||
)?;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// How much is left, by state.
|
||||
///
|
||||
/// One query rather than a listing, because the caller is a progress line: a
|
||||
|
||||
@@ -244,7 +244,8 @@ pub fn delete(conn: &Connection, id: KeywordId) -> Result<usize, CatalogError> {
|
||||
/// every assignment, and a query per keyword would be one statement per word
|
||||
/// in the library.
|
||||
pub fn list(conn: &Connection) -> Result<Vec<Keyword>, CatalogError> {
|
||||
let mut stmt = conn.prepare(
|
||||
ensure_term_index(conn);
|
||||
let mut stmt = conn.prepare_cached(
|
||||
// DISTINCT image, not row: a word on two versions of one frame is one
|
||||
// photograph, and reporting two is the kind of small lie that makes a
|
||||
// user stop trusting the counts.
|
||||
@@ -269,6 +270,27 @@ pub fn list(conn: &Connection) -> Result<Vec<Keyword>, CatalogError> {
|
||||
Ok(rows)
|
||||
}
|
||||
|
||||
/// The index [`list`]'s per-word count is served from: `keywords_term`
|
||||
/// with the version beside the word, so the count reads no `keywords` row.
|
||||
///
|
||||
/// `keywords_term` alone gave the row id, and each of the 10,800 assignments
|
||||
/// on the reference library cost a probe of the table for its version --
|
||||
/// 4 ms of the keyword panel's redraw, on every selection change.
|
||||
///
|
||||
/// Created on first use rather than by a migration, for the reason
|
||||
/// `duplicates::ensure_probe_table` gives: a new schema version makes every
|
||||
/// older build refuse this catalog's snapshot at sync, and an older build
|
||||
/// that meets an extra index ignores it. Once it exists, the statement is a
|
||||
/// lookup in the schema (microseconds). A failure to create it is logged and
|
||||
/// the list read without it: the index is a speed-up, never an answer.
|
||||
fn ensure_term_index(conn: &Connection) {
|
||||
if let Err(e) = conn.execute_batch(
|
||||
"CREATE INDEX IF NOT EXISTS keywords_term_version ON keywords(keyword, version_id);",
|
||||
) {
|
||||
log::warn!("keywords: could not create keywords_term_version: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
/// Assign a keyword to images, creating the keyword if it is new.
|
||||
///
|
||||
/// The bulk form is the *only* form, because keywording a selection is the
|
||||
@@ -491,7 +513,12 @@ pub fn adopt_orphan_terms(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
// quietly readmitted to the vocabulary; it stays visible as an
|
||||
// orphan in [`for_images`] instead, which is a state someone can
|
||||
// see and act on rather than one that silently undoes a deletion.
|
||||
"SELECT DISTINCT k.keyword FROM keywords k
|
||||
//
|
||||
// The distinct words first, then the check: the vocabulary has no
|
||||
// index a tombstone-inclusive lookup can use, so checking once per
|
||||
// assignment scanned it 10,000 times on every open. Once per word
|
||||
// is a few dozen scans of a few dozen rows.
|
||||
"SELECT k.keyword FROM (SELECT DISTINCT keyword FROM keywords) k
|
||||
WHERE NOT EXISTS (SELECT 1 FROM keyword_terms t
|
||||
WHERE t.name = k.keyword)",
|
||||
)?;
|
||||
|
||||
@@ -36,10 +36,14 @@ use std::path::Path;
|
||||
use dr_types::{Availability, ImageId};
|
||||
use rusqlite::Connection;
|
||||
|
||||
pub mod albums;
|
||||
mod backfilled;
|
||||
pub mod bursts;
|
||||
pub mod cache;
|
||||
pub mod collections;
|
||||
pub mod dedup;
|
||||
pub mod dedup_people;
|
||||
pub mod duplicates;
|
||||
pub mod error;
|
||||
pub mod face_shard;
|
||||
pub mod faces;
|
||||
@@ -56,6 +60,7 @@ pub mod sync;
|
||||
pub mod trash;
|
||||
pub mod walk;
|
||||
|
||||
pub use albums::{Album, AlbumId, Place};
|
||||
pub use cache::{Budget, Cache, DEFAULT_BUDGET_BYTES};
|
||||
pub use collections::{Collection, CollectionKind, TreeRow};
|
||||
pub use dedup::{seen_by_content, seen_by_metadata, set_content_hash};
|
||||
@@ -246,8 +251,18 @@ impl Catalog {
|
||||
// A migration adds a column; it cannot know what the value should be
|
||||
// for rows that already existed. Backfilling on open is what stops
|
||||
// those rows being silently partial.
|
||||
for (what, n) in schema::backfill(&conn)? {
|
||||
log::info!("backfilled {what} for {n} row(s) (schema was v{from})");
|
||||
//
|
||||
// Once per catalog state rather than once per open (NFR-P9): every
|
||||
// worker thread opens its own connection, and a develop landing made
|
||||
// five, each paying the whole backfill to confirm nothing had changed.
|
||||
// [`backfilled`] says what "changed" means and why it is enough. A
|
||||
// migration always backfills, stamp or no stamp.
|
||||
let stamp = backfilled::stamp(&conn, path)?;
|
||||
if from < schema::SCHEMA_VERSION || !backfilled::is_current(path, &stamp) {
|
||||
for (what, n) in schema::backfill(&conn)? {
|
||||
log::info!("backfilled {what} for {n} row(s) (schema was v{from})");
|
||||
}
|
||||
backfilled::record(path, stamp);
|
||||
}
|
||||
Ok(Catalog { conn })
|
||||
}
|
||||
|
||||
+792
-105
File diff suppressed because it is too large
Load Diff
+318
-22
@@ -49,6 +49,12 @@ pub struct Judgement {
|
||||
/// 0..=5. Zero means *unrated*, which is a state in its own right.
|
||||
pub rating: u8,
|
||||
pub flag: FlagState,
|
||||
/// TRACES: FR-CAT-5
|
||||
/// The colour label, or `None`. Not part of [`Judgement::is_judged`]:
|
||||
/// a label sorts photographs into piles of the photographer's own
|
||||
/// meaning — "to print", "send to Anna" — and says nothing about whether
|
||||
/// a frame has been culled, which is the question "unjudged" asks.
|
||||
pub label: Option<ColourLabel>,
|
||||
}
|
||||
|
||||
impl Judgement {
|
||||
@@ -267,14 +273,6 @@ pub fn align_default_version_uuids(conn: &Connection) -> Result<usize, CatalogEr
|
||||
Ok(moved)
|
||||
}
|
||||
|
||||
/// The default version's row id for an image, creating one if it has none.
|
||||
///
|
||||
/// Every write path goes through this rather than assuming a version exists.
|
||||
/// An image can arrive without one in two ways that are not worth trying to
|
||||
/// prevent: a row inserted by a build predating this module, and a scan whose
|
||||
/// version pass was interrupted between the image insert and the commit.
|
||||
/// Failing a rating because of either would be the wrong answer — the user
|
||||
/// pressed a key and expects a star.
|
||||
/// TRACES: FR-CAT-13
|
||||
/// How `versions.label` encodes a colour label, and back.
|
||||
///
|
||||
@@ -304,6 +302,14 @@ pub fn label_from_code(code: Option<i64>) -> Option<ColourLabel> {
|
||||
})
|
||||
}
|
||||
|
||||
/// The default version's row id for an image, creating one if it has none.
|
||||
///
|
||||
/// Every write path goes through this rather than assuming a version exists.
|
||||
/// An image can arrive without one in two ways that are not worth trying to
|
||||
/// prevent: a row inserted by a build predating this module, and a scan whose
|
||||
/// version pass was interrupted between the image insert and the commit.
|
||||
/// Failing a rating because of either would be the wrong answer — the user
|
||||
/// pressed a key and expects a star.
|
||||
pub fn default_version_id(conn: &Connection, image: ImageId) -> Result<i64, CatalogError> {
|
||||
let existing: Option<i64> = conn
|
||||
.query_row(
|
||||
@@ -387,6 +393,97 @@ pub fn set_flag_many(
|
||||
apply_many(conn, images, |conn, id| set_flag(conn, id, flag))
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-5
|
||||
/// Set or clear the colour label for one image.
|
||||
pub fn set_label(
|
||||
conn: &Connection,
|
||||
image: ImageId,
|
||||
label: Option<ColourLabel>,
|
||||
) -> Result<(), CatalogError> {
|
||||
let version = default_version_id(conn, image)?;
|
||||
conn.execute(
|
||||
"UPDATE versions SET label = ?2 WHERE id = ?1",
|
||||
rusqlite::params![version, label.map(label_code)],
|
||||
)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-5
|
||||
/// Set or clear a label on many images in one transaction — one keystroke
|
||||
/// over a selection is one commit, as for [`set_rating_many`].
|
||||
pub fn set_label_many(
|
||||
conn: &Connection,
|
||||
images: &[ImageId],
|
||||
label: Option<ColourLabel>,
|
||||
) -> Result<usize, CatalogError> {
|
||||
apply_many(conn, images, |conn, id| set_label(conn, id, label))
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-5
|
||||
/// What a label key does to a set of images: Lightroom's toggle.
|
||||
///
|
||||
/// Pressing the key for the label every one of them already carries takes it
|
||||
/// off; otherwise every one of them gets it. Decided over the whole set
|
||||
/// rather than per image, so a selection that was half red comes out all red
|
||||
/// rather than inverted — the photographer pressed "red", and a key that
|
||||
/// turned half of them red and the other half plain would be two answers to
|
||||
/// one question.
|
||||
pub fn toggled_label(
|
||||
current: impl IntoIterator<Item = Option<ColourLabel>>,
|
||||
pressed: ColourLabel,
|
||||
) -> Option<ColourLabel> {
|
||||
let mut any = false;
|
||||
for label in current {
|
||||
any = true;
|
||||
if label != Some(pressed) {
|
||||
return Some(pressed);
|
||||
}
|
||||
}
|
||||
if any {
|
||||
None
|
||||
} else {
|
||||
Some(pressed)
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-5 | FR-CAT-6
|
||||
/// How the library divides by colour label, for the filter chips' counts.
|
||||
///
|
||||
/// Index 0 is unlabelled and index `n` the label whose code is `n`. The
|
||||
/// same shape as [`rating_histogram`], and for the same reason the
|
||||
/// unlabelled slot is what is left of [`judged_rows`]: an image without a
|
||||
/// version row is unlabelled, not missing.
|
||||
///
|
||||
/// Only labelled rows are grouped. The join this replaced (2026-09-26)
|
||||
/// probed `versions_judgement` per image and then read each version's row
|
||||
/// for `label`, which the index does not carry -- 10 ms on the reference
|
||||
/// library, on every label keystroke, to find that none of 23,500 images
|
||||
/// had one. This walks the default versions in the index's order, which
|
||||
/// is close to the table's, and groups the few that are labelled.
|
||||
pub fn label_histogram(conn: &Connection) -> Result<[usize; 6], CatalogError> {
|
||||
let mut out = [0usize; 6];
|
||||
let mut stmt = conn.prepare_cached(
|
||||
"SELECT label, count(*) FROM versions
|
||||
WHERE is_default = 1 AND label IS NOT NULL
|
||||
GROUP BY label",
|
||||
)?;
|
||||
let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?;
|
||||
let mut counted = 0usize;
|
||||
for (code, count) in rows.flatten() {
|
||||
// A code this build does not know counts as unlabelled, which is how
|
||||
// `label_from_code` reads it everywhere else.
|
||||
let slot = if label_from_code(Some(code)).is_some() {
|
||||
code as usize
|
||||
} else {
|
||||
0
|
||||
};
|
||||
out[slot] += count as usize;
|
||||
counted += count as usize;
|
||||
}
|
||||
out[0] += judged_rows(conn)?.saturating_sub(counted);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Shared bulk wrapper, so the two axes cannot drift in their commit
|
||||
/// behaviour — a partially-committed rating and a fully-committed flag from
|
||||
/// the same keystroke would be hard to explain and harder to notice.
|
||||
@@ -411,21 +508,22 @@ fn apply_many(
|
||||
/// An image with no version reads as unrated and unflagged rather than as an
|
||||
/// error: that is exactly what it is.
|
||||
pub fn judgement(conn: &Connection, image: ImageId) -> Result<Judgement, CatalogError> {
|
||||
let row: Option<(i64, i64)> = conn
|
||||
let row: Option<(i64, i64, Option<i64>)> = conn
|
||||
.query_row(
|
||||
"SELECT rating, flag FROM versions
|
||||
"SELECT rating, flag, label FROM versions
|
||||
WHERE image_id = ?1
|
||||
ORDER BY is_default DESC, id ASC
|
||||
LIMIT 1",
|
||||
[image.0 as i64],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
|r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)),
|
||||
)
|
||||
.optional()?;
|
||||
|
||||
Ok(match row {
|
||||
Some((rating, flag)) => Judgement {
|
||||
Some((rating, flag, label)) => Judgement {
|
||||
rating: rating.clamp(0, MAX_RATING as i64) as u8,
|
||||
flag: flag_from_code(flag),
|
||||
label: label_from_code(label),
|
||||
},
|
||||
None => Judgement::default(),
|
||||
})
|
||||
@@ -452,7 +550,7 @@ pub fn judgements(
|
||||
.collect::<Vec<_>>()
|
||||
.join(",");
|
||||
let sql = format!(
|
||||
"SELECT image_id, rating, flag FROM versions
|
||||
"SELECT image_id, rating, flag, label FROM versions
|
||||
WHERE image_id IN ({placeholders}) AND is_default = 1"
|
||||
);
|
||||
|
||||
@@ -467,15 +565,17 @@ pub fn judgements(
|
||||
r.get::<_, i64>(0)?,
|
||||
r.get::<_, i64>(1)?,
|
||||
r.get::<_, i64>(2)?,
|
||||
r.get::<_, Option<i64>>(3)?,
|
||||
))
|
||||
})?;
|
||||
|
||||
for (image, rating, flag) in rows.flatten() {
|
||||
for (image, rating, flag, label) in rows.flatten() {
|
||||
out.insert(
|
||||
ImageId(image as u64),
|
||||
Judgement {
|
||||
rating: rating.clamp(0, MAX_RATING as i64) as u8,
|
||||
flag: flag_from_code(flag),
|
||||
label: label_from_code(label),
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -491,25 +591,61 @@ pub fn judgements(
|
||||
pub fn rating_histogram(conn: &Connection) -> Result<[usize; 6], CatalogError> {
|
||||
let mut out = [0usize; 6];
|
||||
|
||||
// LEFT JOIN, so an image whose version row is missing still counts as
|
||||
// unrated rather than vanishing from the totals. The histogram has to sum
|
||||
// to the library size or it is not believable.
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT coalesce(v.rating, 0) AS r, count(*)
|
||||
FROM images i
|
||||
LEFT JOIN versions v ON v.image_id = i.id AND v.is_default = 1
|
||||
GROUP BY r",
|
||||
// Only the rated rows are grouped; the unrated slot is what is left of
|
||||
// [`judged_rows`]. So an image whose version row is missing still counts
|
||||
// as unrated rather than vanishing from the totals -- the histogram has
|
||||
// to sum to the library size or it is not believable.
|
||||
//
|
||||
// It was one `images LEFT JOIN versions ... GROUP BY` until 2026-09-26:
|
||||
// a probe of `versions_judgement` per image and a sort of every row, to
|
||||
// put 22,000 of 23,500 in slot zero. 9 ms on every star keystroke on the
|
||||
// reference library; this is a pass over the index that sorts only the
|
||||
// rated few, and [`judged_rows`] is three index-only counts.
|
||||
let mut stmt = conn.prepare_cached(
|
||||
"SELECT rating, count(*) FROM versions
|
||||
WHERE is_default = 1 AND rating != 0
|
||||
GROUP BY rating",
|
||||
)?;
|
||||
let rows = stmt.query_map([], |r| Ok((r.get::<_, i64>(0)?, r.get::<_, i64>(1)?)))?;
|
||||
|
||||
let mut counted = 0usize;
|
||||
for (rating, count) in rows.flatten() {
|
||||
if let Some(slot) = out.get_mut(rating.clamp(0, MAX_RATING as i64) as usize) {
|
||||
*slot += count as usize;
|
||||
counted += count as usize;
|
||||
}
|
||||
}
|
||||
out[0] += judged_rows(conn)?.saturating_sub(counted);
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// How many rows `images LEFT JOIN versions ON ... AND is_default = 1` has:
|
||||
/// one per image with no default version, and one per default version for
|
||||
/// the rest. The total both histograms divide up, and the unjudged slot is
|
||||
/// what is left of it once the judged rows are counted.
|
||||
///
|
||||
/// Spelled as three counts rather than as that join because the join probes
|
||||
/// `versions_judgement` once per image, where each count here is one pass
|
||||
/// over an index without reading a row: the library size, the default
|
||||
/// versions, and the images holding one. An image with two default versions
|
||||
/// -- nothing prevents it -- is two rows of the join and one image of the
|
||||
/// third count, so it adds one here exactly as it did there. A version
|
||||
/// always belongs to an image; `foreign_keys` is on and deletes cascade.
|
||||
///
|
||||
/// `count(DISTINCT image_id)` alone in its statement: that is what lets
|
||||
/// SQLite read the distinct values off the index's order instead of
|
||||
/// building a temporary b-tree of them.
|
||||
fn judged_rows(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
let n: i64 = conn
|
||||
.prepare_cached(
|
||||
"SELECT (SELECT count(*) FROM images)
|
||||
+ (SELECT count(*) FROM versions WHERE is_default = 1)
|
||||
- (SELECT count(DISTINCT image_id) FROM versions WHERE is_default = 1)",
|
||||
)?
|
||||
.query_row([], |r| r.get(0))?;
|
||||
Ok(n.max(0) as usize)
|
||||
}
|
||||
|
||||
/// How many images carry each flag: `(picks, rejects)`.
|
||||
pub fn flag_counts(conn: &Connection) -> Result<(usize, usize), CatalogError> {
|
||||
let picks: i64 = conn.query_row(
|
||||
@@ -686,6 +822,72 @@ mod tests {
|
||||
assert_eq!(distinct, 200);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_label_round_trips_and_clears() {
|
||||
// TRACES: FR-CAT-5
|
||||
let cat = with_images(1);
|
||||
let id = ids(&cat)[0];
|
||||
set_label(cat.connection(), id, Some(ColourLabel::Green)).unwrap();
|
||||
assert_eq!(
|
||||
judgement(cat.connection(), id).unwrap().label,
|
||||
Some(ColourLabel::Green)
|
||||
);
|
||||
set_label(cat.connection(), id, None).unwrap();
|
||||
assert_eq!(judgement(cat.connection(), id).unwrap().label, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_label_is_not_a_judgement() {
|
||||
// "Unjudged" is the cull's resume point; a label is a pile of the
|
||||
// photographer's own, and labelling a frame must not hide it there.
|
||||
let cat = with_images(1);
|
||||
let id = ids(&cat)[0];
|
||||
set_label(cat.connection(), id, Some(ColourLabel::Red)).unwrap();
|
||||
assert!(!judgement(cat.connection(), id).unwrap().is_judged());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn labelling_a_selection_is_one_commit_and_reaches_every_image() {
|
||||
// TRACES: FR-CAT-5
|
||||
let cat = with_images(4);
|
||||
let all = ids(&cat);
|
||||
assert_eq!(
|
||||
set_label_many(cat.connection(), &all, Some(ColourLabel::Blue)).unwrap(),
|
||||
4
|
||||
);
|
||||
let found = judgements(cat.connection(), &all).unwrap();
|
||||
assert!(all
|
||||
.iter()
|
||||
.all(|id| found[id].label == Some(ColourLabel::Blue)));
|
||||
assert_eq!(
|
||||
label_histogram(cat.connection()).unwrap(),
|
||||
[0, 0, 0, 0, 4, 0]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_label_key_toggles_only_when_every_image_already_has_it() {
|
||||
// TRACES: FR-CAT-5
|
||||
use ColourLabel::*;
|
||||
assert_eq!(toggled_label([Some(Red), Some(Red)], Red), None);
|
||||
assert_eq!(toggled_label([Some(Red), None], Red), Some(Red));
|
||||
assert_eq!(toggled_label([Some(Blue)], Red), Some(Red));
|
||||
assert_eq!(toggled_label([], Red), Some(Red));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_label_histogram_sums_to_the_library() {
|
||||
// TRACES: FR-CAT-6
|
||||
// Images without a version row count as unlabelled rather than
|
||||
// vanishing, as the rating histogram's do.
|
||||
let cat = with_images(3);
|
||||
let first = ids(&cat)[0];
|
||||
set_label(cat.connection(), first, Some(ColourLabel::Purple)).unwrap();
|
||||
let h = label_histogram(cat.connection()).unwrap();
|
||||
assert_eq!(h, [2, 0, 0, 0, 0, 1]);
|
||||
assert_eq!(h.iter().sum::<usize>(), 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_rating_round_trips() {
|
||||
let cat = with_images(1);
|
||||
@@ -830,6 +1032,100 @@ mod tests {
|
||||
assert_eq!(h.iter().sum::<usize>(), 4);
|
||||
}
|
||||
|
||||
/// The rows of the join the histograms used to be spelled as, grouped the
|
||||
/// way `rating_histogram` groups them. What the counts must still agree
|
||||
/// with, in the states nothing in the schema prevents.
|
||||
fn by_join(cat: &Catalog, column: &str) -> Vec<(i64, i64)> {
|
||||
cat.connection()
|
||||
.prepare(&format!(
|
||||
"SELECT coalesce(v.{column}, 0) AS c, count(*)
|
||||
FROM images i
|
||||
LEFT JOIN versions v ON v.image_id = i.id AND v.is_default = 1
|
||||
GROUP BY c ORDER BY c"
|
||||
))
|
||||
.unwrap()
|
||||
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
|
||||
.unwrap()
|
||||
.map(Result::unwrap)
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// A library in every awkward state at once: an image with no version,
|
||||
/// one with only a virtual copy, one with two default versions, and
|
||||
/// values out of range on both axes.
|
||||
fn awkward() -> Catalog {
|
||||
let cat = with_images(8);
|
||||
ensure_default_versions(cat.connection()).unwrap();
|
||||
let all = ids(&cat);
|
||||
let c = cat.connection();
|
||||
set_rating(c, all[0], 5).unwrap();
|
||||
set_rating(c, all[1], 2).unwrap();
|
||||
set_label(c, all[1], Some(ColourLabel::Blue)).unwrap();
|
||||
c.execute(
|
||||
"DELETE FROM versions WHERE image_id = ?1",
|
||||
[all[2].0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"UPDATE versions SET is_default = 0 WHERE image_id = ?1",
|
||||
[all[3].0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO versions(image_id, uuid, name, is_default, rating, label)
|
||||
VALUES (?1, 'second-default', 'Copy', 1, 4, 3)",
|
||||
[all[4].0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"UPDATE versions SET rating = -1, label = 9 WHERE image_id = ?1",
|
||||
[all[5].0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"UPDATE versions SET rating = 7, label = 0 WHERE image_id = ?1",
|
||||
[all[6].0 as i64],
|
||||
)
|
||||
.unwrap();
|
||||
cat
|
||||
}
|
||||
|
||||
/// Fold the join's rows into slots the way the old code did.
|
||||
fn folded(rows: &[(i64, i64)], slot: impl Fn(i64) -> usize) -> [usize; 6] {
|
||||
let mut out = [0usize; 6];
|
||||
for &(code, n) in rows {
|
||||
out[slot(code)] += n as usize;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_rating_histogram_agrees_with_the_join_it_replaced() {
|
||||
let cat = awkward();
|
||||
let expected = folded(&by_join(&cat, "rating"), |r| {
|
||||
r.clamp(0, MAX_RATING as i64) as usize
|
||||
});
|
||||
assert_eq!(rating_histogram(cat.connection()).unwrap(), expected);
|
||||
// Nine rows for eight images: the doubled default counts twice, as
|
||||
// it always has.
|
||||
assert_eq!(expected.iter().sum::<usize>(), 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_label_histogram_agrees_with_the_join_it_replaced() {
|
||||
let cat = awkward();
|
||||
let expected = folded(&by_join(&cat, "label"), |code| {
|
||||
if label_from_code(Some(code)).is_some() {
|
||||
code as usize
|
||||
} else {
|
||||
0
|
||||
}
|
||||
});
|
||||
assert_eq!(label_histogram(cat.connection()).unwrap(), expected);
|
||||
assert_eq!(expected[3], 1, "the second default's label is counted");
|
||||
assert_eq!(expected.iter().sum::<usize>(), 9);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn flag_counts_separate_picks_from_rejects() {
|
||||
let cat = with_images(5);
|
||||
|
||||
@@ -322,6 +322,10 @@ pub fn restore(catalog: &Path, backup: &Path) -> Result<(), CatalogError> {
|
||||
/// to move — a caller may be recovering from a file SQLite could not open
|
||||
/// because it was never created.
|
||||
pub fn set_aside(catalog: &Path) -> Result<Option<PathBuf>, CatalogError> {
|
||||
// Whatever takes this name next — a rebuild or a restored backup — is not
|
||||
// the file this process last backfilled. Forgotten while the path still
|
||||
// resolves, so it is the same key the open recorded.
|
||||
crate::backfilled::forget(catalog);
|
||||
let moved = if catalog.exists() {
|
||||
let dest = with_suffix(catalog, DAMAGED_SUFFIX);
|
||||
// An earlier damaged copy is replaced rather than accumulating: two of
|
||||
|
||||
@@ -77,7 +77,7 @@ pub enum Outcome {
|
||||
|
||||
/// Something that can actually do the work a job describes.
|
||||
///
|
||||
/// The catalog knows what needs doing and nothing about how — a thumbnail
|
||||
/// The catalog knows what needs doing and nothing about how — face detection
|
||||
/// needs a decoder, a fetch needs a network stack, and neither belongs under
|
||||
/// `core/dr-catalog` (ARCH §4.1: calls go downward). So the queue lives here
|
||||
/// and the handlers are supplied from above.
|
||||
@@ -187,17 +187,19 @@ pub struct Recovered {
|
||||
pub reclaimed: usize,
|
||||
/// Jobs deleted because the photograph they name no longer exists.
|
||||
pub reaped: usize,
|
||||
/// Jobs deleted because their kind is retired ([`JobKind::RETIRED`]).
|
||||
pub retired: usize,
|
||||
}
|
||||
|
||||
impl Recovered {
|
||||
pub fn did_anything(&self) -> bool {
|
||||
self.reclaimed > 0 || self.reaped > 0
|
||||
self.reclaimed > 0 || self.reaped > 0 || self.retired > 0
|
||||
}
|
||||
}
|
||||
|
||||
/// Ready the queue for a fresh run, before any worker touches it.
|
||||
///
|
||||
/// Two distinct cleanups, and both are startup-only:
|
||||
/// Three distinct cleanups, and all are startup-only:
|
||||
///
|
||||
/// - **Reclaim.** A `Running` row has no owner; the process that claimed it is
|
||||
/// gone. On Android that is a routine morning, not a crash (FR-PLAT-AND-3).
|
||||
@@ -205,19 +207,27 @@ impl Recovered {
|
||||
/// process down with it three times running should not be retried forever,
|
||||
/// and the attempt counter is the only evidence of that we have.
|
||||
/// - **Reap.** Jobs naming an image the catalog no longer has. A library that
|
||||
/// has been culled leaves thumbnail jobs for photographs that were deleted
|
||||
/// has been culled leaves jobs for photographs that were deleted
|
||||
/// months ago, and every one of them would be claimed, run and failed.
|
||||
/// - **Retire.** Rows of a kind nothing enqueues or claims any more
|
||||
/// ([`jobs::drop_retired`]). Here rather than in a migration so that no
|
||||
/// schema bump locks an older device out of the synced catalog, and every
|
||||
/// time rather than once because an older build sharing the catalog will
|
||||
/// queue them again.
|
||||
///
|
||||
/// Reclaim runs first so its count is the honest number of interrupted jobs,
|
||||
/// Retiring runs first, so the other two never touch rows about to go.
|
||||
/// Reclaim runs next so its count is the honest number of interrupted jobs,
|
||||
/// before reaping removes whichever of them pointed at nothing.
|
||||
///
|
||||
/// **Call this exactly once per catalog, at startup.** It cannot distinguish a
|
||||
/// job a dead process was holding from one a live runner is holding right now,
|
||||
/// because there is no owner column — the queue is durable, not distributed.
|
||||
pub fn recover(conn: &Connection) -> Result<Recovered, CatalogError> {
|
||||
let retired = jobs::drop_retired(conn)?;
|
||||
Ok(Recovered {
|
||||
reclaimed: jobs::recover_orphaned(conn)?,
|
||||
reaped: jobs::reap_orphan_subjects(conn)?,
|
||||
retired,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -729,7 +739,7 @@ mod tests {
|
||||
// which is the window a durable queue exists to survive: no `complete`,
|
||||
// no `fail`, just a row marked `Running` with nobody holding it.
|
||||
let c = db();
|
||||
queued(&c, JobKind::Thumbnail, 1);
|
||||
queued(&c, JobKind::ContentHash, 1);
|
||||
|
||||
// The dead process. It claimed the job and never came back.
|
||||
let claimed = jobs::claim_next(&c, 0).unwrap().expect("claimable");
|
||||
@@ -738,7 +748,7 @@ mod tests {
|
||||
// A fresh runner, before it starts, finds the queue empty — the row is
|
||||
// `Running` and no claim will touch it.
|
||||
let seen = Arc::new(Mutex::new(Vec::new()));
|
||||
let mut runner = Runner::new(&c).with(recording(vec![JobKind::Thumbnail], seen.clone()));
|
||||
let mut runner = Runner::new(&c).with(recording(vec![JobKind::ContentHash], seen.clone()));
|
||||
assert_eq!(
|
||||
runner.drain_all(0).unwrap().ran(),
|
||||
0,
|
||||
@@ -766,7 +776,7 @@ mod tests {
|
||||
// the only evidence we keep across a death. Without this a poison-pill
|
||||
// job would be reclaimed and re-run forever.
|
||||
let c = db();
|
||||
queued(&c, JobKind::Thumbnail, 1);
|
||||
queued(&c, JobKind::ContentHash, 1);
|
||||
|
||||
for _ in 0..MAX_ATTEMPTS {
|
||||
jobs::claim_next(&c, 0).unwrap().expect("claimable");
|
||||
@@ -782,13 +792,49 @@ mod tests {
|
||||
assert_eq!(state, JobState::Failed as i64);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recovery_drops_retired_kinds_every_time_and_nothing_else() {
|
||||
// What 0.16.0 left behind: a thumbnail job per photograph that nothing
|
||||
// would ever claim, beside live work that must survive.
|
||||
let c = db();
|
||||
queued(&c, JobKind::Thumbnail, 1);
|
||||
queued(&c, JobKind::Thumbnail, 2);
|
||||
enqueue(
|
||||
&c,
|
||||
JobKind::DetectFaces,
|
||||
Some(1),
|
||||
Priority::Background,
|
||||
None,
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
let first = recover(&c).unwrap();
|
||||
assert_eq!(first.retired, 2);
|
||||
assert!(first.did_anything());
|
||||
|
||||
let kinds: Vec<i64> = c
|
||||
.prepare("SELECT kind FROM jobs")
|
||||
.unwrap()
|
||||
.query_map([], |r| r.get(0))
|
||||
.unwrap()
|
||||
.map(Result::unwrap)
|
||||
.collect();
|
||||
assert_eq!(kinds, vec![JobKind::DetectFaces as i64]);
|
||||
|
||||
// An older build opening the same catalog queues them again on its
|
||||
// next scan. The next open by this one clears them again.
|
||||
enqueue(&c, JobKind::Thumbnail, Some(1), Priority::Background, None).unwrap();
|
||||
assert_eq!(recover(&c).unwrap().retired, 1);
|
||||
assert_eq!(recover(&c).unwrap(), Recovered::default());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recovery_drops_jobs_whose_photograph_is_gone() {
|
||||
// A culled library leaves thumbnail jobs for images deleted months
|
||||
// ago. Every one would be claimed, run and failed.
|
||||
let c = db();
|
||||
queued(&c, JobKind::Thumbnail, 1);
|
||||
queued(&c, JobKind::Thumbnail, 2);
|
||||
queued(&c, JobKind::ContentHash, 1);
|
||||
queued(&c, JobKind::ContentHash, 2);
|
||||
c.execute("DELETE FROM images WHERE id = 2", []).unwrap();
|
||||
|
||||
let recovered = recover(&c).unwrap();
|
||||
@@ -804,7 +850,7 @@ mod tests {
|
||||
#[test]
|
||||
fn a_quiet_startup_recovers_nothing() {
|
||||
let c = db();
|
||||
queued(&c, JobKind::Thumbnail, 1);
|
||||
queued(&c, JobKind::ContentHash, 1);
|
||||
assert_eq!(recover(&c).unwrap(), Recovered::default());
|
||||
assert!(!recover(&c).unwrap().did_anything());
|
||||
}
|
||||
|
||||
@@ -301,8 +301,11 @@ pub const EYE_COLUMNS: [&str; 7] = [
|
||||
/// silently partial — present, queryable, and wrong — which is worse than
|
||||
/// missing, because nothing signals that they need attention.
|
||||
///
|
||||
/// Cheap enough to run on every open: each pass is one indexed UPDATE, and
|
||||
/// re-running it is a no-op once the values are already right.
|
||||
/// Idempotent: re-running it is a no-op once the values are already right.
|
||||
/// [`crate::Catalog::open`] runs it once per catalog state rather than on
|
||||
/// every open — each pass scans a whole table, and together they were most of
|
||||
/// what an open cost — and `backfilled` in this crate says what counts as a
|
||||
/// new state.
|
||||
///
|
||||
/// Returns how many rows each backfill touched, for logging.
|
||||
pub fn backfill(conn: &Connection) -> Result<Vec<(&'static str, usize)>, CatalogError> {
|
||||
@@ -494,15 +497,51 @@ fn rewrite_for_attached(sql: &str, schema_name: &str) -> String {
|
||||
fn pair_raw_and_jpeg(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
use std::collections::HashMap;
|
||||
|
||||
// (folder, lowercase stem) -> RAW id, built in one pass over the RAWs.
|
||||
// The small side first: the JPEGs not yet paired. On a settled library
|
||||
// these are the ones with no RAW beside them -- 1,900 of 24,000 on the
|
||||
// reference library -- and this runs on every open, including the ones
|
||||
// the develop view makes for each photograph it fetches. Reading every
|
||||
// RAW to find the handful that share a folder with one of them was most
|
||||
// of what opening the catalog cost.
|
||||
let jpegs: Vec<(i64, Option<i64>, String)> = {
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT id, folder_id, source_ref FROM images
|
||||
WHERE lower(format) IN ('jpg','jpeg') AND shadowed_by IS NULL",
|
||||
)?;
|
||||
let rows = stmt.query_map([], |r| {
|
||||
Ok((
|
||||
r.get::<_, i64>(0)?,
|
||||
r.get::<_, Option<i64>>(1)?,
|
||||
r.get::<_, String>(2)?,
|
||||
))
|
||||
})?;
|
||||
rows.filter_map(Result::ok).collect()
|
||||
};
|
||||
if jpegs.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
|
||||
// (folder, lowercase stem) -> RAW id, over the folders those JPEGs are in
|
||||
// and no others: a pair is same-folder by definition. Ordered by id so
|
||||
// that where two RAWs share a stem the later one wins, as it did when this
|
||||
// read every RAW in table order.
|
||||
let mut folders: Vec<i64> = jpegs.iter().filter_map(|(_, f, _)| *f).collect();
|
||||
folders.sort_unstable();
|
||||
folders.dedup();
|
||||
let unfiled = jpegs.iter().any(|(_, f, _)| f.is_none());
|
||||
let folders = serde_json::to_string(&folders).unwrap_or_else(|_| "[]".to_string());
|
||||
|
||||
let mut raws: HashMap<(Option<i64>, String), i64> = HashMap::new();
|
||||
{
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT id, folder_id, source_ref FROM images
|
||||
WHERE lower(format) IN
|
||||
('cr2','cr3','nef','arw','raf','rw2','orf','dng')",
|
||||
('cr2','cr3','nef','arw','raf','rw2','orf','dng')
|
||||
AND (folder_id IN (SELECT value FROM json_each(?1))
|
||||
OR (?2 AND folder_id IS NULL))
|
||||
ORDER BY id",
|
||||
)?;
|
||||
let rows = stmt.query_map([], |r| {
|
||||
let rows = stmt.query_map(rusqlite::params![folders, unfiled], |r| {
|
||||
Ok((
|
||||
r.get::<_, i64>(0)?,
|
||||
r.get::<_, Option<i64>>(1)?,
|
||||
@@ -518,25 +557,16 @@ fn pair_raw_and_jpeg(conn: &Connection) -> Result<usize, CatalogError> {
|
||||
return Ok(0);
|
||||
}
|
||||
|
||||
let pairs: Vec<(i64, i64)> = {
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT id, folder_id, source_ref FROM images
|
||||
WHERE lower(format) IN ('jpg','jpeg') AND shadowed_by IS NULL",
|
||||
)?;
|
||||
let rows = stmt.query_map([], |r| {
|
||||
Ok((
|
||||
r.get::<_, i64>(0)?,
|
||||
r.get::<_, Option<i64>>(1)?,
|
||||
r.get::<_, String>(2)?,
|
||||
))
|
||||
})?;
|
||||
rows.filter_map(|row| {
|
||||
let (id, folder, path) = row.ok()?;
|
||||
let raw = raws.get(&(folder, stem_of(&path).to_ascii_lowercase()))?;
|
||||
Some((id, *raw))
|
||||
let pairs: Vec<(i64, i64)> = jpegs
|
||||
.iter()
|
||||
.filter_map(|(id, folder, path)| {
|
||||
let raw = raws.get(&(*folder, stem_of(path).to_ascii_lowercase()))?;
|
||||
Some((*id, *raw))
|
||||
})
|
||||
.collect()
|
||||
};
|
||||
.collect();
|
||||
if pairs.is_empty() {
|
||||
return Ok(0);
|
||||
}
|
||||
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
for (jpeg, raw) in &pairs {
|
||||
@@ -1582,6 +1612,36 @@ mod tests {
|
||||
assert_eq!(backfilled(&c, "shadowed_by"), 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_pair_is_found_among_other_folders_and_unfiled_images() {
|
||||
// The RAWs are read only from the folders an unpaired JPEG is in,
|
||||
// plus the unfiled ones when an unfiled JPEG is waiting: each JPEG
|
||||
// must still find its own sibling, and only its own.
|
||||
let c = with_root();
|
||||
let raw_a = image(&c, Some(1), "a/IMG_7.CR2", "cr2");
|
||||
image(&c, Some(2), "b/IMG_7.CR2", "cr2");
|
||||
image(&c, Some(2), "b/IMG_8.CR2", "cr2");
|
||||
let raw_unfiled = image(&c, None, "IMG_9.DNG", "dng");
|
||||
let jpeg_a = image(&c, Some(1), "a/IMG_7.JPG", "jpg");
|
||||
let jpeg_unfiled = image(&c, None, "IMG_9.jpg", "jpg");
|
||||
image(&c, Some(1), "a/IMG_9.jpg", "jpg");
|
||||
|
||||
assert_eq!(backfilled(&c, "shadowed_by"), 2);
|
||||
let of = |id: i64| -> Option<i64> {
|
||||
c.query_row("SELECT shadowed_by FROM images WHERE id = ?1", [id], |r| {
|
||||
r.get(0)
|
||||
})
|
||||
.unwrap()
|
||||
};
|
||||
assert_eq!(of(jpeg_a), Some(raw_a));
|
||||
assert_eq!(of(jpeg_unfiled), Some(raw_unfiled));
|
||||
assert_eq!(
|
||||
backfilled(&c, "shadowed_by"),
|
||||
0,
|
||||
"settled on the second pass"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_raw_is_never_shadowed_by_a_jpeg() {
|
||||
// The relationship is one-way: the RAW is the photograph.
|
||||
|
||||
+493
-45
@@ -46,18 +46,210 @@ pub fn checkpoint(conn: &Connection) -> Result<(), CatalogError> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Write a consistent snapshot of the catalog to `dest`, ready to upload.
|
||||
/// Write a consistent snapshot of the catalog to `dest`, ready to upload,
|
||||
/// without the face crops.
|
||||
///
|
||||
/// Uses the backup API rather than a filesystem copy so the snapshot is
|
||||
/// coherent even with writers active. Callers should still prefer a quiet
|
||||
/// moment — this competes with background jobs for the write lock.
|
||||
/// Built rather than copied. The snapshot is the *whole catalog* bar the
|
||||
/// crops, uploaded on every sync and downloaded by every device. The crops are
|
||||
/// most of the file (96 MB of a 158 MB reference catalog), and copying them in
|
||||
/// only to delete them was most of the cost. A backup-API copy followed by
|
||||
/// `UPDATE faces SET crop = NULL` and `VACUUM` wrote the file roughly three
|
||||
/// times over to produce 50 MB (#71). So this creates the schema in an empty
|
||||
/// file and copies every table into it with `crop` left NULL. That is one
|
||||
/// pass, with nothing written that is not uploaded.
|
||||
///
|
||||
/// Consistency comes from doing the whole copy inside one transaction on the
|
||||
/// snapshot's connection, which holds a single read snapshot of the source for
|
||||
/// its duration. A writer committing meanwhile lands in the source's WAL and is
|
||||
/// simply not seen, the same serialisation the backup API gave.
|
||||
///
|
||||
/// Crops are not lost by this: they travel in the face shards
|
||||
/// ([`crate::face_shard::export_to_shards`]), which are written once and
|
||||
/// downloaded once. Nothing reads a crop out of a merged remote catalog. The
|
||||
/// merge reads a remote face's box and model to match it to a local one, and
|
||||
/// no more. So leaving them out costs a receiving device nothing it would
|
||||
/// otherwise have had. A device never adopts a downloaded catalog as its own,
|
||||
/// so a fresh one gets its crops from the shards too.
|
||||
///
|
||||
/// The result must stay what every earlier build already merges: same schema,
|
||||
/// same `user_version`, same page size and the same WAL flag in the header.
|
||||
/// The `the_snapshot_*` tests pin those.
|
||||
pub fn snapshot_for_upload(conn: &Connection, dest: &Path) -> Result<(), CatalogError> {
|
||||
let out = copy_to(conn, dest)?;
|
||||
strip_face_crops(&out)?;
|
||||
// The source is read through a second connection, attached to the
|
||||
// snapshot's, so it needs to be a file. Every catalog is one.
|
||||
let source = conn
|
||||
.path()
|
||||
.filter(|p| !p.is_empty())
|
||||
.map(PathBuf::from)
|
||||
.ok_or_else(|| CatalogError::Io("the catalog to snapshot has no file".into()))?;
|
||||
|
||||
// Not needed for consistency, since the read transaction below sees the
|
||||
// WAL, but it keeps the live WAL from growing across syncs, as before.
|
||||
checkpoint(conn)?;
|
||||
|
||||
// A leftover from a pass that died mid-build would otherwise be built on.
|
||||
for stale in [
|
||||
dest.to_path_buf(),
|
||||
sidecar_of(dest, "-wal"),
|
||||
sidecar_of(dest, "-journal"),
|
||||
sidecar_of(dest, "-shm"),
|
||||
] {
|
||||
match std::fs::remove_file(&stale) {
|
||||
Ok(()) => {}
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {}
|
||||
Err(e) => return Err(CatalogError::Io(format!("{}: {e}", stale.display()))),
|
||||
}
|
||||
}
|
||||
|
||||
let out = Connection::open(dest)?;
|
||||
build_snapshot(conn, &source, &out)?;
|
||||
// This device's local album folders are paths and SAF grants nobody
|
||||
// else can use; the merge never reads them, and the snapshot is what
|
||||
// a fresh device would otherwise adopt whole.
|
||||
out.execute_batch("DROP TABLE IF EXISTS album_folders")?;
|
||||
verify_snapshot(&out)?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `catalog.sqlite` + `-wal` → `catalog.sqlite-wal`.
|
||||
fn sidecar_of(path: &Path, suffix: &str) -> PathBuf {
|
||||
let mut s = path.as_os_str().to_owned();
|
||||
s.push(suffix);
|
||||
PathBuf::from(s)
|
||||
}
|
||||
|
||||
/// Schema name the source catalog is attached under while a snapshot is built.
|
||||
const SOURCE_SCHEMA: &str = "snap_src";
|
||||
|
||||
/// The body of [`snapshot_for_upload`]: fill the empty database `out` from
|
||||
/// the catalog at `source`.
|
||||
fn build_snapshot(conn: &Connection, source: &Path, out: &Connection) -> Result<(), CatalogError> {
|
||||
// Settings that only take on an empty file, copied from the source so the
|
||||
// result is the file a backup would have been.
|
||||
let page_size: i64 = conn.query_row("PRAGMA main.page_size", [], |r| r.get(0))?;
|
||||
let auto_vacuum: i64 = conn.query_row("PRAGMA main.auto_vacuum", [], |r| r.get(0))?;
|
||||
out.pragma_update(None, "page_size", page_size)?;
|
||||
out.pragma_update(None, "auto_vacuum", auto_vacuum)?;
|
||||
// A scratch file, rebuilt whole on every pass and checked before upload:
|
||||
// durability during the build buys nothing. MEMORY rather than OFF keeps
|
||||
// ROLLBACK defined on the failure path.
|
||||
out.pragma_update(None, "journal_mode", "MEMORY")?;
|
||||
out.pragma_update(None, "synchronous", "OFF")?;
|
||||
// The rows were checked when they were written, and the copy has them
|
||||
// all by the end. The bundled SQLite turns foreign keys on by default,
|
||||
// and with them a multi-row INSERT into `images` scans `images` for
|
||||
// children of every row it adds (`shadowed_by` refers to the same table
|
||||
// and has no index): 1.2 s of a 1.7 s snapshot on 24k images.
|
||||
out.pragma_update(None, "foreign_keys", false)?;
|
||||
|
||||
// Bound as a parameter, so a path containing a quote cannot break out.
|
||||
out.execute(
|
||||
&format!("ATTACH DATABASE ?1 AS {SOURCE_SCHEMA}"),
|
||||
[source.to_string_lossy().as_ref()],
|
||||
)?;
|
||||
let result = copy_schema_and_rows(out);
|
||||
if let Err(e) = out.execute(&format!("DETACH DATABASE {SOURCE_SCHEMA}"), []) {
|
||||
log::warn!("failed to detach the catalog from its snapshot: {e}");
|
||||
}
|
||||
result?;
|
||||
|
||||
// Last, and outside any transaction, which is the only place it can be
|
||||
// set: the header says WAL, as every snapshot uploaded so far has.
|
||||
out.pragma_update(None, "journal_mode", "WAL")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn copy_schema_and_rows(out: &Connection) -> Result<(), CatalogError> {
|
||||
let tx = out.unchecked_transaction()?;
|
||||
|
||||
// The first read of the source opens its read snapshot. Everything from
|
||||
// here, schema included, is as of that one moment.
|
||||
let objects: Vec<(String, String, String)> = {
|
||||
let mut stmt = tx.prepare(&format!(
|
||||
"SELECT type, name, sql FROM {SOURCE_SCHEMA}.sqlite_master
|
||||
WHERE sql IS NOT NULL AND name NOT LIKE 'sqlite\\_%' ESCAPE '\\'
|
||||
ORDER BY rowid"
|
||||
))?;
|
||||
let rows = stmt.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?)))?;
|
||||
rows.collect::<Result<_, _>>()?
|
||||
};
|
||||
let user_version: i64 =
|
||||
tx.query_row(&format!("PRAGMA {SOURCE_SCHEMA}.user_version"), [], |r| {
|
||||
r.get(0)
|
||||
})?;
|
||||
let application_id: i64 =
|
||||
tx.query_row(&format!("PRAGMA {SOURCE_SCHEMA}.application_id"), [], |r| {
|
||||
r.get(0)
|
||||
})?;
|
||||
|
||||
// Tables and their rows first, then indexes, triggers and views, so that
|
||||
// an index is built once over the data rather than maintained per row,
|
||||
// and no trigger fires on the copy. Foreign keys are off on this
|
||||
// connection, so the order tables are filled in does not matter.
|
||||
for (_, name, sql) in objects.iter().filter(|(k, _, _)| k == "table") {
|
||||
// Verbatim: an unqualified CREATE lands in `main`, the snapshot.
|
||||
tx.execute_batch(sql)?;
|
||||
let columns: Vec<String> = {
|
||||
let mut stmt = tx.prepare("SELECT name FROM pragma_table_info(?1, 'main')")?;
|
||||
let rows = stmt.query_map([name], |r| r.get::<_, String>(0))?;
|
||||
rows.collect::<Result<_, _>>()?
|
||||
};
|
||||
let select = columns
|
||||
.iter()
|
||||
.map(|c| {
|
||||
if name == "faces" && c == "crop" {
|
||||
"NULL".to_string()
|
||||
} else {
|
||||
quote_ident(c)
|
||||
}
|
||||
})
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let insert = columns
|
||||
.iter()
|
||||
.map(|c| quote_ident(c))
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let table = quote_ident(name);
|
||||
tx.execute(
|
||||
&format!(
|
||||
"INSERT INTO main.{table} ({insert})
|
||||
SELECT {select} FROM {SOURCE_SCHEMA}.{table}"
|
||||
),
|
||||
[],
|
||||
)?;
|
||||
}
|
||||
// AUTOINCREMENT's counters live in a table the filter above skips; the
|
||||
// CREATE of such a table makes an empty one here.
|
||||
let has_sequence: bool = tx.query_row(
|
||||
&format!(
|
||||
"SELECT EXISTS(SELECT 1 FROM {SOURCE_SCHEMA}.sqlite_master
|
||||
WHERE name = 'sqlite_sequence')"
|
||||
),
|
||||
[],
|
||||
|r| r.get(0),
|
||||
)?;
|
||||
if has_sequence {
|
||||
tx.execute_batch(&format!(
|
||||
"DELETE FROM main.sqlite_sequence;
|
||||
INSERT INTO main.sqlite_sequence SELECT * FROM {SOURCE_SCHEMA}.sqlite_sequence;"
|
||||
))?;
|
||||
}
|
||||
for (_, _, sql) in objects.iter().filter(|(k, _, _)| k != "table") {
|
||||
tx.execute_batch(sql)?;
|
||||
}
|
||||
|
||||
tx.pragma_update(None, "user_version", user_version)?;
|
||||
tx.pragma_update(None, "application_id", application_id)?;
|
||||
tx.commit()?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// `name` as an SQL identifier, whatever it contains.
|
||||
fn quote_ident(name: &str) -> String {
|
||||
format!("\"{}\"", name.replace('"', "\"\""))
|
||||
}
|
||||
|
||||
/// TRACES: NFR-R2
|
||||
/// Refuse to hand over a snapshot that will not pass `quick_check`.
|
||||
///
|
||||
@@ -82,12 +274,12 @@ fn verify_snapshot(snapshot: &Connection) -> Result<(), CatalogError> {
|
||||
/// Checkpoint, then copy the whole database to `dest`, and hand back the
|
||||
/// connection to the copy.
|
||||
///
|
||||
/// Split out from [`snapshot_for_upload`] because [`crate::recovery`] wants
|
||||
/// exactly this and none of what follows it there: an NFR-R2 backup is the
|
||||
/// file the user may have to *live on*, so it keeps the face crops that an
|
||||
/// upload strips. Sharing the copy rather than reimplementing it is what keeps
|
||||
/// the WAL discipline in one place — a backup taken with `fs::copy` would be
|
||||
/// the torn snapshot this module's header exists to warn about.
|
||||
/// What [`crate::recovery`] takes its NFR-R2 backups with. A backup is the
|
||||
/// file the user may have to *live on*, so it keeps the face crops that
|
||||
/// [`snapshot_for_upload`] leaves out, and a byte-for-byte page copy is the
|
||||
/// right tool. Keeping it here beside the upload keeps the WAL discipline in
|
||||
/// one place: a backup taken with `fs::copy` would be the torn snapshot this
|
||||
/// module's header exists to warn about.
|
||||
pub(crate) fn copy_to(conn: &Connection, dest: &Path) -> Result<Connection, CatalogError> {
|
||||
checkpoint(conn)?;
|
||||
|
||||
@@ -103,39 +295,6 @@ pub(crate) fn copy_to(conn: &Connection, dest: &Path) -> Result<Connection, Cata
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Drop the stored face crops from a snapshot before it is uploaded.
|
||||
///
|
||||
/// The snapshot is the *whole catalog*, uploaded on every sync and downloaded
|
||||
/// by every device. Face crops are a few KB each and a fully indexed library
|
||||
/// holds tens of thousands of them, so leaving them in would put tens of MB on
|
||||
/// every round trip — the exact cost `face_shard`'s 25 MB cap exists to bound,
|
||||
/// and the reason the bulk per-face data lives in shards in the first place.
|
||||
///
|
||||
/// Crops are not lost by this: they travel in the face shards
|
||||
/// ([`crate::face_shard::export_to_shards`]), which are written once and
|
||||
/// downloaded once. Nothing reads a crop out of a merged remote catalog —
|
||||
/// [`merge_all`] touches collections and keywords only — so removing them here
|
||||
/// costs a receiving device nothing it would otherwise have had.
|
||||
///
|
||||
/// `VACUUM` afterwards because SQLite does not return freed pages to the file
|
||||
/// on its own, and an upload sized by the file rather than by its contents
|
||||
/// would keep paying for bytes that are no longer there.
|
||||
fn strip_face_crops(snapshot: &Connection) -> Result<(), CatalogError> {
|
||||
// A catalog older than the crop column is a legitimate input here — a
|
||||
// snapshot taken mid-migration, or a test fixture built from an earlier
|
||||
// schema — so an absent column is nothing to fail over.
|
||||
let has_crop = snapshot
|
||||
.prepare("SELECT crop FROM faces LIMIT 1")
|
||||
.map(|_| true)
|
||||
.unwrap_or(false);
|
||||
if !has_crop {
|
||||
return Ok(());
|
||||
}
|
||||
snapshot.execute("UPDATE faces SET crop = NULL WHERE crop IS NOT NULL", [])?;
|
||||
snapshot.execute_batch("VACUUM")?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Whether a downloaded remote catalog is worth merging.
|
||||
///
|
||||
/// Cheap guard before attaching: a remote written by a newer build may contain
|
||||
@@ -171,6 +330,13 @@ pub fn merge_remote(conn: &Connection, remote: &Path) -> Result<MergeReport, Cat
|
||||
|
||||
let result = merge::merge_all(conn);
|
||||
|
||||
// The merge brings in rows the backfill exists for — assignments whose
|
||||
// word this device has no term for, from a remote older than v6 — so the
|
||||
// next open must run it, whether or not the stamp happened to move.
|
||||
if let Some(path) = conn.path().filter(|p| !p.is_empty()) {
|
||||
crate::backfilled::forget(Path::new(path));
|
||||
}
|
||||
|
||||
// Detach even if the merge failed, or the next attempt errors with
|
||||
// "database remote_cat is already in use".
|
||||
let detach = conn.execute(&format!("DETACH DATABASE {REMOTE_SCHEMA}"), []);
|
||||
@@ -178,6 +344,18 @@ pub fn merge_remote(conn: &Connection, remote: &Path) -> Result<MergeReport, Cat
|
||||
log::warn!("failed to detach remote catalog: {e}");
|
||||
}
|
||||
|
||||
// After every merge, because a merge is where two devices' people meet:
|
||||
// the same name typed on each, or a redirect one of them made. Its own
|
||||
// transaction, and a failure is logged rather than returned -- what the
|
||||
// merge took is committed and valid whether or not the duplicates were
|
||||
// folded, and the next pass tries again. Runs on the sync worker, never
|
||||
// the UI thread, and costs ~10 ms when there is nothing to do.
|
||||
if result.is_ok() {
|
||||
if let Err(e) = crate::dedup_people::run(conn) {
|
||||
log::warn!("dedup after the catalog merge: {e}");
|
||||
}
|
||||
}
|
||||
|
||||
result
|
||||
}
|
||||
|
||||
@@ -288,6 +466,92 @@ mod tests {
|
||||
assert_eq!(n, 2);
|
||||
}
|
||||
|
||||
/// One image, known to the server by `file_id`, in a catalog.
|
||||
fn with_image(c: &Connection, file_id: i64) -> dr_types::ImageId {
|
||||
c.execute(
|
||||
"INSERT OR IGNORE INTO roots(id, kind, label) VALUES (1, 'remote', 'lib')",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO images(root_id, source_ref, added_at) VALUES (1, ?1, 0)",
|
||||
[format!("IMG_{file_id}.CR3")],
|
||||
)
|
||||
.unwrap();
|
||||
let id = c.last_insert_rowid();
|
||||
c.execute(
|
||||
"INSERT INTO remote(image_id, file_id) VALUES (?1, ?2)",
|
||||
[id, file_id],
|
||||
)
|
||||
.unwrap();
|
||||
dr_types::ImageId(id as u64)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_album_and_its_exports_reach_another_device_but_its_folder_does_not() {
|
||||
use crate::albums::{self, Place};
|
||||
let dir = tempdir();
|
||||
let remote_path = dir.join("remote.sqlite");
|
||||
let snap = dir.join("snap.sqlite");
|
||||
{
|
||||
// The desktop: two albums, one on the server and one on its own
|
||||
// disk, each with an export of the same photograph.
|
||||
let r = seeded(&dir.join("desktop.sqlite"));
|
||||
let img = with_image(&r, 4242);
|
||||
let web = albums::create(&r, "Web", &Place::Server("Shared/Web".into())).unwrap();
|
||||
let print = albums::create(&r, "Print", &Place::Local("/mnt/print".into())).unwrap();
|
||||
albums::record_exports(&r, web, &[(img, "IMG_4242.jpg".into())]).unwrap();
|
||||
albums::record_exports(&r, print, &[(img, "IMG_4242.tif".into())]).unwrap();
|
||||
snapshot_for_upload(&r, &snap).unwrap();
|
||||
std::fs::rename(&snap, &remote_path).unwrap();
|
||||
}
|
||||
|
||||
// The tablet knows the same file under its own image id.
|
||||
let local = seeded(&dir.join("tablet.sqlite"));
|
||||
with_image(&local, 1);
|
||||
let img = with_image(&local, 4242);
|
||||
|
||||
let report = merge_remote(&local, &remote_path).unwrap();
|
||||
assert_eq!(report.albums_taken, 2);
|
||||
assert_eq!(report.album_exports_added, 2);
|
||||
assert!(report.local_changed());
|
||||
|
||||
let all = albums::list(&local).unwrap();
|
||||
let print = all.iter().find(|a| a.name == "Print").unwrap();
|
||||
let web = all.iter().find(|a| a.name == "Web").unwrap();
|
||||
assert_eq!(print.place, None, "the desktop's disk is not the tablet's");
|
||||
assert_eq!(web.place, Some(Place::Server("Shared/Web".into())));
|
||||
assert_eq!(albums::sources(&local, web.id).unwrap(), vec![img]);
|
||||
|
||||
// Nothing changed on either side, so a second pass takes nothing.
|
||||
let again = merge_remote(&local, &remote_path).unwrap();
|
||||
assert_eq!(again.albums_taken, 0);
|
||||
assert_eq!(again.album_exports_added, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_device_that_never_made_an_album_still_uploads_this_ones() {
|
||||
use crate::albums::{self, Place};
|
||||
let dir = tempdir();
|
||||
let remote_path = dir.join("remote.sqlite");
|
||||
{
|
||||
// A snapshot from a build that predates albums altogether.
|
||||
let r = seeded(&remote_path);
|
||||
checkpoint(&r).unwrap();
|
||||
}
|
||||
let local = seeded(&dir.join("local.sqlite"));
|
||||
albums::create(&local, "Web", &Place::Server("Web".into())).unwrap();
|
||||
|
||||
let report = merge_remote(&local, &remote_path).unwrap();
|
||||
assert_eq!(report.albums_taken, 0);
|
||||
// Its albums table is absent, so nothing was compared — and the local
|
||||
// album has still to reach the server.
|
||||
assert!(
|
||||
albums::list(&local).unwrap().len() == 1,
|
||||
"the local album survives a merge with a catalog that has none"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_remote_can_be_merged_twice_without_attach_conflict() {
|
||||
// Detach must happen even on the failure path, or the second attempt
|
||||
@@ -380,4 +644,188 @@ mod tests {
|
||||
.unwrap();
|
||||
assert_eq!(kept, 1, "stripping the snapshot damaged the live catalog");
|
||||
}
|
||||
/// Device-side setup for the snapshot tests: an image both devices know by
|
||||
/// its cross-device file id, and one face on it carrying `crop`.
|
||||
fn with_a_face(c: &Connection, face_id: i64, x: f64, crop: &[u8]) {
|
||||
c.execute(
|
||||
"INSERT OR IGNORE INTO roots(id, kind, label) VALUES (1, 'local', 'lib')",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO images(id, root_id, source_ref, added_at) VALUES (1, 1, 'a.CR3', 0)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
c.execute("INSERT INTO remote(image_id, file_id) VALUES (1, 5000)", [])
|
||||
.unwrap();
|
||||
c.execute(
|
||||
"INSERT INTO faces
|
||||
(id, image_id, x, y, w, h, landmarks, detector_confidence, embedding,
|
||||
crop_px, model_id, detected_at, crop)
|
||||
VALUES (?1, 1, ?2, 0.2, 0.2, 0.2, X'00', 0.9, X'00', 150.0, 'w600k_mbf', 0, ?3)",
|
||||
rusqlite::params![face_id, x, crop],
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
|
||||
fn crop_of(c: &Connection, face_id: i64) -> Option<Vec<u8>> {
|
||||
c.query_row("SELECT crop FROM faces WHERE id = ?1", [face_id], |r| {
|
||||
r.get(0)
|
||||
})
|
||||
.unwrap()
|
||||
}
|
||||
|
||||
/// The snapshot is built table by table rather than copied, so what has to
|
||||
/// hold is that it is still the same database bar the crops: every table,
|
||||
/// index and row, and the header fields an older build checks before it
|
||||
/// will merge (`user_version`) or open it the way it always has (the WAL
|
||||
/// flag and page size a backup-API copy carried).
|
||||
#[test]
|
||||
fn the_snapshot_is_the_catalog_bar_the_crops() {
|
||||
let dir = tempdir();
|
||||
let live = dir.join("catalog.sqlite");
|
||||
let snap = dir.join("snap.sqlite");
|
||||
let c = seeded(&live);
|
||||
with_a_face(&c, 7, 0.3, &[7u8; 4096]);
|
||||
c.execute(
|
||||
"INSERT INTO collections(uuid, name, kind, created, revision, modified)
|
||||
VALUES ('u1', 'Iceland', 0, 0, 1, 1)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
// Created on first use rather than by a migration: the copy must not
|
||||
// depend on the migrations knowing every table.
|
||||
crate::duplicates::ensure_probe_table(&c).unwrap();
|
||||
|
||||
snapshot_for_upload(&c, &snap).unwrap();
|
||||
let out = Connection::open(&snap).unwrap();
|
||||
|
||||
let objects = |conn: &Connection| -> Vec<(String, String)> {
|
||||
let mut stmt = conn
|
||||
.prepare("SELECT type, name FROM sqlite_master ORDER BY type, name")
|
||||
.unwrap();
|
||||
let rows = stmt
|
||||
.query_map([], |r| Ok((r.get(0).unwrap(), r.get(1).unwrap())))
|
||||
.unwrap();
|
||||
rows.map(Result::unwrap).collect()
|
||||
};
|
||||
assert_eq!(objects(&out), objects(&c), "the snapshot's schema differs");
|
||||
|
||||
for (kind, table) in objects(&c) {
|
||||
if kind != "table" {
|
||||
continue;
|
||||
}
|
||||
let count = |conn: &Connection| -> i64 {
|
||||
conn.query_row(&format!("SELECT COUNT(*) FROM \"{table}\""), [], |r| {
|
||||
r.get(0)
|
||||
})
|
||||
.unwrap()
|
||||
};
|
||||
assert_eq!(count(&out), count(&c), "rows differ in {table}");
|
||||
}
|
||||
|
||||
for pragma in [
|
||||
"user_version",
|
||||
"application_id",
|
||||
"page_size",
|
||||
"journal_mode",
|
||||
] {
|
||||
let read = |conn: &Connection| -> String {
|
||||
conn.query_row(&format!("PRAGMA {pragma}"), [], |r| {
|
||||
r.get::<_, rusqlite::types::Value>(0)
|
||||
})
|
||||
.map(|v| format!("{v:?}"))
|
||||
.unwrap()
|
||||
};
|
||||
assert_eq!(read(&out), read(&c), "{pragma} differs");
|
||||
}
|
||||
drop(out);
|
||||
// Bytes 18 and 19 of the header are 2 for a WAL database, which is
|
||||
// what every snapshot uploaded before this one said.
|
||||
let header = std::fs::read(&snap).unwrap();
|
||||
assert_eq!(&header[18..20], &[2, 2], "the snapshot is not WAL-flagged");
|
||||
|
||||
// The face is there; its pixels are not.
|
||||
let out = Connection::open(&snap).unwrap();
|
||||
assert_eq!(crop_of(&out, 7), None);
|
||||
}
|
||||
|
||||
/// A pass that died mid-build leaves a file behind; the next one must
|
||||
/// build afresh rather than on top of it.
|
||||
#[test]
|
||||
fn a_leftover_snapshot_is_replaced_not_built_on() {
|
||||
let dir = tempdir();
|
||||
let live = dir.join("catalog.sqlite");
|
||||
let snap = dir.join("snap.sqlite");
|
||||
let c = seeded(&live);
|
||||
|
||||
std::fs::write(&snap, b"not a database").unwrap();
|
||||
snapshot_for_upload(&c, &snap).unwrap();
|
||||
// And twice over a good one, which is the steady state.
|
||||
snapshot_for_upload(&c, &snap).unwrap();
|
||||
|
||||
let out = Connection::open(&snap).unwrap();
|
||||
let v: i64 = out
|
||||
.query_row("PRAGMA user_version", [], |r| r.get(0))
|
||||
.unwrap();
|
||||
assert_eq!(v, schema::SCHEMA_VERSION);
|
||||
}
|
||||
|
||||
/// The merge side of a crop-less snapshot: the other device's names still
|
||||
/// cross over — the match is by box, not by pixels — and this device's
|
||||
/// own crop is left exactly as it was, never replaced by the snapshot's
|
||||
/// NULL.
|
||||
#[test]
|
||||
fn merging_a_crop_less_snapshot_keeps_local_crops_and_takes_the_names() {
|
||||
let dir = tempdir();
|
||||
let snap = dir.join("snap.sqlite");
|
||||
|
||||
let desktop = seeded(&dir.join("desktop.sqlite"));
|
||||
with_a_face(&desktop, 42, 0.31, &[1u8; 3000]);
|
||||
desktop
|
||||
.execute(
|
||||
"INSERT INTO people(id, uuid, name, ignored, created, revision, modified)
|
||||
VALUES (3, 'u-anna', 'Anna', 0, 0, 1, 1)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
desktop
|
||||
.execute(
|
||||
"INSERT INTO face_person(face_id, person_id, probability, confirmed)
|
||||
VALUES (42, 3, 0.9, 1)",
|
||||
[],
|
||||
)
|
||||
.unwrap();
|
||||
snapshot_for_upload(&desktop, &snap).unwrap();
|
||||
|
||||
// The tablet found the same face itself, under its own row id, and has
|
||||
// its own crop of it — from its own detection or from the shards.
|
||||
let tablet = seeded(&dir.join("tablet.sqlite"));
|
||||
let mine = vec![9u8; 2500];
|
||||
with_a_face(&tablet, 7, 0.30, &mine);
|
||||
|
||||
let report = merge_remote(&tablet, &snap).unwrap();
|
||||
assert_eq!(report.people_inserted, 1);
|
||||
assert_eq!(report.faces_assigned, 1);
|
||||
let named: (String, bool) = tablet
|
||||
.query_row(
|
||||
"SELECT p.name, fp.confirmed
|
||||
FROM face_person fp JOIN people p ON p.id = fp.person_id
|
||||
WHERE fp.face_id = 7",
|
||||
[],
|
||||
|r| Ok((r.get(0)?, r.get(1)?)),
|
||||
)
|
||||
.unwrap();
|
||||
assert_eq!(named, ("Anna".to_string(), true));
|
||||
assert_eq!(
|
||||
crop_of(&tablet, 7),
|
||||
Some(mine),
|
||||
"the merge touched a local crop"
|
||||
);
|
||||
|
||||
// Idempotent over a crop-less remote too.
|
||||
let again = merge_remote(&tablet, &snap).unwrap();
|
||||
assert!(!again.local_changed());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -129,28 +129,40 @@ pub fn record_trashed(
|
||||
return Ok(0);
|
||||
}
|
||||
let tx = conn.unchecked_transaction()?;
|
||||
let mut n = 0;
|
||||
|
||||
{
|
||||
let mut stmt = tx.prepare(
|
||||
"UPDATE images
|
||||
SET trashed_from = CASE
|
||||
WHEN trashed_at IS NULL THEN source_ref
|
||||
ELSE trashed_from
|
||||
END,
|
||||
source_ref = ?2,
|
||||
trashed_at = coalesce(trashed_at, ?3)
|
||||
WHERE id = ?1",
|
||||
)?;
|
||||
for (image, path) in moved {
|
||||
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
|
||||
}
|
||||
}
|
||||
|
||||
let n = record_trashed_within(&tx, moved, now)?;
|
||||
tx.commit()?;
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// [`record_trashed`] inside a transaction the caller owns.
|
||||
///
|
||||
/// For a caller whose trash is one half of a larger write that must land
|
||||
/// whole or not at all — consolidating duplicates (`crate::duplicates`)
|
||||
/// merges a copy's judgements onto the survivor and trashes the copy in one
|
||||
/// commit. `unchecked_transaction` cannot nest, so this is offered here
|
||||
/// rather than wrapped from above.
|
||||
pub fn record_trashed_within(
|
||||
tx: &Connection,
|
||||
moved: &[(ImageId, String)],
|
||||
now: i64,
|
||||
) -> Result<usize, CatalogError> {
|
||||
let mut n = 0;
|
||||
let mut stmt = tx.prepare(
|
||||
"UPDATE images
|
||||
SET trashed_from = CASE
|
||||
WHEN trashed_at IS NULL THEN source_ref
|
||||
ELSE trashed_from
|
||||
END,
|
||||
source_ref = ?2,
|
||||
trashed_at = coalesce(trashed_at, ?3)
|
||||
WHERE id = ?1",
|
||||
)?;
|
||||
for (image, path) in moved {
|
||||
n += stmt.execute(rusqlite::params![image.0 as i64, path, now])?;
|
||||
}
|
||||
Ok(n)
|
||||
}
|
||||
|
||||
/// Record that images have been moved back out of the trash.
|
||||
///
|
||||
/// Call after the move succeeds, for the same reason as [`record_trashed`].
|
||||
|
||||
+32
-53
@@ -48,7 +48,6 @@ use dr_types::{Availability, FormatFilter, RootId, SourceRef};
|
||||
use rusqlite::{Connection, OptionalExtension};
|
||||
|
||||
use crate::error::CatalogError;
|
||||
use crate::jobs::{self, JobKind, Priority};
|
||||
use crate::query::availability_code;
|
||||
use crate::scan::{
|
||||
classify_dir, classify_entry, DirAction, DirState, EntryAction, KnownFile, ScanOutcome,
|
||||
@@ -336,7 +335,7 @@ pub fn scan_root(
|
||||
}
|
||||
}
|
||||
EntryAction::Insert => {
|
||||
let id = insert_image(
|
||||
insert_image(
|
||||
&tx,
|
||||
root,
|
||||
folder_id,
|
||||
@@ -345,11 +344,10 @@ pub fn scan_root(
|
||||
entry.meta.mtime,
|
||||
now,
|
||||
)?;
|
||||
queue_reading_it(&tx, id)?;
|
||||
report.inserted += 1;
|
||||
}
|
||||
EntryAction::Changed => {
|
||||
let id = update_image(
|
||||
update_image(
|
||||
&tx,
|
||||
root,
|
||||
folder_id,
|
||||
@@ -357,7 +355,6 @@ pub fn scan_root(
|
||||
entry.meta.size,
|
||||
entry.meta.mtime,
|
||||
)?;
|
||||
queue_reading_it(&tx, id)?;
|
||||
report.updated += 1;
|
||||
}
|
||||
EntryAction::Ignored => unreachable!("returned above"),
|
||||
@@ -640,7 +637,7 @@ fn insert_image(
|
||||
size: u64,
|
||||
mtime: i64,
|
||||
now: i64,
|
||||
) -> Result<i64, CatalogError> {
|
||||
) -> Result<(), CatalogError> {
|
||||
// `metadata_state = 1`: the scan knows the name, the size and the mtime,
|
||||
// and has read no EXIF. Claiming otherwise would make a date filter
|
||||
// silently wrong on a freshly scanned library.
|
||||
@@ -664,7 +661,7 @@ fn insert_image(
|
||||
now,
|
||||
],
|
||||
)?;
|
||||
image_id(conn, root, src)
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn update_image(
|
||||
@@ -674,7 +671,7 @@ fn update_image(
|
||||
src: &SourceRef,
|
||||
size: u64,
|
||||
mtime: i64,
|
||||
) -> Result<i64, CatalogError> {
|
||||
) -> Result<(), CatalogError> {
|
||||
// The content hash is dropped, not recomputed: it described bytes that no
|
||||
// longer exist, and leaving it would let reconnect-by-hash match this image
|
||||
// to a file it is no longer a copy of. `metadata_state` goes back to 1 for
|
||||
@@ -692,37 +689,7 @@ fn update_image(
|
||||
src.key(),
|
||||
],
|
||||
)?;
|
||||
image_id(conn, root, src)
|
||||
}
|
||||
|
||||
fn image_id(conn: &Connection, root: RootId, src: &SourceRef) -> Result<i64, CatalogError> {
|
||||
Ok(conn.query_row(
|
||||
"SELECT id FROM images WHERE root_id = ?1 AND source_ref = ?2",
|
||||
rusqlite::params![root.0 as i64, src.key()],
|
||||
|r| r.get(0),
|
||||
)?)
|
||||
}
|
||||
|
||||
/// Queue the work that turns a stat-only row into a usable grid cell.
|
||||
///
|
||||
/// Enqueued inside the scan's transaction, so a folder's rows and the jobs that
|
||||
/// finish them land together — a crash between the two would otherwise leave
|
||||
/// images no worker was ever told about.
|
||||
fn queue_reading_it(conn: &Connection, image_id: i64) -> Result<(), CatalogError> {
|
||||
jobs::enqueue(
|
||||
conn,
|
||||
JobKind::ExtractMetadata,
|
||||
Some(image_id),
|
||||
Priority::Background,
|
||||
None,
|
||||
)?;
|
||||
jobs::enqueue(
|
||||
conn,
|
||||
JobKind::Thumbnail,
|
||||
Some(image_id),
|
||||
Priority::Background,
|
||||
None,
|
||||
)
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// TRACES: FR-CAT-9
|
||||
@@ -831,6 +798,17 @@ mod tests {
|
||||
let tmp = target.with_extension("tmp");
|
||||
fs::write(&tmp, bytes).expect("write");
|
||||
fs::rename(&tmp, &target).expect("rename");
|
||||
// A scan tells a changed file by its mtime, at whole-second
|
||||
// resolution; a resave landing in the same second as the scan
|
||||
// before it looks unchanged, and the test fails when the machine
|
||||
// is fast enough. Two seconds ahead, on the file and its folder,
|
||||
// is what a real resave some time later would look like.
|
||||
let later = std::time::SystemTime::now() + std::time::Duration::from_secs(2);
|
||||
for p in [target.as_path(), target.parent().expect("parent")] {
|
||||
fs::File::open(p)
|
||||
.and_then(|f| f.set_modified(later))
|
||||
.expect("set mtime");
|
||||
}
|
||||
self
|
||||
}
|
||||
|
||||
@@ -1096,7 +1074,7 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_resaved_file_is_queued_for_rereading_and_loses_its_stale_hash() {
|
||||
fn a_resaved_file_owes_a_reread_and_loses_its_stale_hash() {
|
||||
let lib = Library::new("resaved");
|
||||
lib.file("IMG.CR3", b"raw");
|
||||
lib.scan();
|
||||
@@ -1121,9 +1099,8 @@ mod tests {
|
||||
"a hash of bytes that no longer exist would match this image to the \
|
||||
wrong file on reconnect"
|
||||
);
|
||||
assert_eq!(lib.count("SELECT metadata_state FROM images"), 1);
|
||||
assert_eq!(
|
||||
lib.count("SELECT count(*) FROM jobs WHERE kind = 1"),
|
||||
lib.count("SELECT metadata_state FROM images"),
|
||||
1,
|
||||
"EXIF must be re-read"
|
||||
);
|
||||
@@ -1137,7 +1114,11 @@ mod tests {
|
||||
let lib = Library::new("no-requeue");
|
||||
lib.file("a.CR3", b"raw");
|
||||
lib.scan();
|
||||
lib.conn().execute("DELETE FROM jobs", []).unwrap();
|
||||
// As if the metadata sweep had read it: what is owed is recorded in
|
||||
// `metadata_state`, and the thumbnail store answers for itself.
|
||||
lib.conn()
|
||||
.execute("UPDATE images SET metadata_state = 2", [])
|
||||
.unwrap();
|
||||
|
||||
lib.file("b.CR3", b"raw");
|
||||
let r = lib.scan();
|
||||
@@ -1145,9 +1126,9 @@ mod tests {
|
||||
assert_eq!(r.inserted, 1);
|
||||
assert_eq!(r.unchanged, 1);
|
||||
assert_eq!(
|
||||
lib.count("SELECT count(*) FROM jobs"),
|
||||
2,
|
||||
"EXIF and a thumbnail for the new image, and nothing for the old one"
|
||||
lib.count("SELECT count(*) FROM images WHERE metadata_state < 2"),
|
||||
1,
|
||||
"EXIF owed for the new image, and nothing for the old one"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1171,17 +1152,15 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_new_image_is_queued_for_a_thumbnail_and_for_exif() {
|
||||
fn a_new_image_owes_its_exif_and_queues_nothing() {
|
||||
// The sweeps find their work from `metadata_state` and the thumbnail
|
||||
// store. A queued job would be a second record of the same debt, and
|
||||
// no handler claims one (#73).
|
||||
let lib = Library::new("queued");
|
||||
lib.file("IMG.CR3", b"raw");
|
||||
lib.scan();
|
||||
|
||||
assert_eq!(
|
||||
lib.count("SELECT count(*) FROM jobs WHERE kind = 2"),
|
||||
1,
|
||||
"no thumbnail job means an empty grid cell forever"
|
||||
);
|
||||
assert_eq!(lib.count("SELECT count(*) FROM jobs WHERE kind = 1"), 1);
|
||||
assert_eq!(lib.count("SELECT count(*) FROM jobs"), 0);
|
||||
assert_eq!(
|
||||
lib.count("SELECT metadata_state FROM images"),
|
||||
1,
|
||||
|
||||
@@ -0,0 +1,341 @@
|
||||
// TRACES: FR-PLAT-AND-3
|
||||
//! No job kind is enqueued without something that claims it.
|
||||
//!
|
||||
//! The queue coalesces, so a producer with no consumer does not fail — it
|
||||
//! just leaves a row per subject for ever. That is how the reference catalog
|
||||
//! came to hold 23,582 `Thumbnail` jobs, one per photograph, re-coalesced on
|
||||
//! every scan, with no handler for the kind anywhere in the tree (#73). Nothing
|
||||
//! at runtime notices: the rows are cheap one at a time and invisible in the
|
||||
//! interface. So the pairing is checked here, over the source, instead.
|
||||
//!
|
||||
//! ## What counts
|
||||
//!
|
||||
//! In shipping code under `core/`, `ui/`, `apps/` and `platform/` — every
|
||||
//! `src/` tree, with `#[cfg(test)]` items dropped:
|
||||
//!
|
||||
//! - **Enqueued**: the `JobKind::X` named in the arguments of a call to
|
||||
//! `enqueue(`. An enqueue whose kind is not spelled there — passed in a
|
||||
//! variable — is refused outright, because this scan could not say what it
|
||||
//! queues.
|
||||
//! - **Claimed**: the `JobKind::X` in the body of a `fn kinds(` (what a
|
||||
//! `JobHandler` declares, and all a `Runner` claims), or named in a call to
|
||||
//! `claim_next_matching(`. A call to `claim_next(` claims every kind.
|
||||
//! `JobKind::ALL` in either place means every kind.
|
||||
//!
|
||||
//! Tests and examples are left out on purpose: a unit test of the queue's
|
||||
//! mechanics enqueues and claims whatever it likes, and proves nothing about
|
||||
//! the app.
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::fs;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// Every `.rs` file under each `src/` of each crate in `group`.
|
||||
fn crate_sources(group: &Path, out: &mut Vec<PathBuf>) {
|
||||
let Ok(crates) = fs::read_dir(group) else {
|
||||
return;
|
||||
};
|
||||
for krate in crates {
|
||||
let src = krate.expect("read dir entry").path().join("src");
|
||||
if src.is_dir() {
|
||||
rust_files(&src, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn rust_files(dir: &Path, out: &mut Vec<PathBuf>) {
|
||||
for entry in fs::read_dir(dir).unwrap_or_else(|e| panic!("cannot read {}: {e}", dir.display()))
|
||||
{
|
||||
let path = entry.expect("read dir entry").path();
|
||||
if path.is_dir() {
|
||||
rust_files(&path, out);
|
||||
} else if path.extension().and_then(|e| e.to_str()) == Some("rs") {
|
||||
out.push(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Blank out string literals and line comments, so neither a brace nor a
|
||||
/// `JobKind::` inside prose is read as code.
|
||||
fn strip_literals_and_comments(line: &str) -> String {
|
||||
let mut out = String::with_capacity(line.len());
|
||||
let mut chars = line.chars().peekable();
|
||||
let mut in_string = false;
|
||||
while let Some(c) = chars.next() {
|
||||
if in_string {
|
||||
match c {
|
||||
'\\' => {
|
||||
chars.next();
|
||||
}
|
||||
'"' => in_string = false,
|
||||
_ => {}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
match c {
|
||||
'"' => in_string = true,
|
||||
'/' if chars.peek() == Some(&'/') => break,
|
||||
_ => out.push(c),
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The shipping code of a file, comments and strings blanked, with every
|
||||
/// `#[cfg(test)]` item dropped. The attribute must be the whole line, so a
|
||||
/// doc comment mentioning it is not mistaken for one.
|
||||
fn shipping_code(text: &str) -> String {
|
||||
let lines: Vec<String> = text.lines().map(strip_literals_and_comments).collect();
|
||||
let mut out = String::new();
|
||||
let mut i = 0;
|
||||
while i < lines.len() {
|
||||
if lines[i].trim() != "#[cfg(test)]" {
|
||||
out.push_str(&lines[i]);
|
||||
out.push('\n');
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
let (mut j, mut depth, mut opened) = (i + 1, 0i32, false);
|
||||
while j < lines.len() {
|
||||
depth += lines[j].matches('{').count() as i32;
|
||||
depth -= lines[j].matches('}').count() as i32;
|
||||
opened |= lines[j].contains('{');
|
||||
if (opened && depth <= 0) || (!opened && lines[j].contains(';')) {
|
||||
break;
|
||||
}
|
||||
j += 1;
|
||||
}
|
||||
i = j + 1;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The text from `start` (just past an opening delimiter) to its matching
|
||||
/// close.
|
||||
fn balanced(code: &str, start: usize, open: char, close: char) -> &str {
|
||||
let mut depth = 1;
|
||||
for (i, c) in code[start..].char_indices() {
|
||||
if c == open {
|
||||
depth += 1;
|
||||
} else if c == close {
|
||||
depth -= 1;
|
||||
if depth == 0 {
|
||||
return &code[start..start + i];
|
||||
}
|
||||
}
|
||||
}
|
||||
&code[start..]
|
||||
}
|
||||
|
||||
/// Each call of `name(` in `code` that is a call rather than the function's
|
||||
/// own definition or a longer name ending in it, as its argument text.
|
||||
fn calls<'a>(code: &'a str, name: &str) -> Vec<&'a str> {
|
||||
let needle = format!("{name}(");
|
||||
let mut found = Vec::new();
|
||||
for (at, _) in code.match_indices(&needle) {
|
||||
let before = &code[..at];
|
||||
let prev = before.chars().next_back();
|
||||
if prev.is_some_and(|c| c.is_alphanumeric() || c == '_') {
|
||||
continue;
|
||||
}
|
||||
if before.trim_end().ends_with("fn") {
|
||||
continue;
|
||||
}
|
||||
found.push(balanced(code, at + needle.len(), '(', ')'));
|
||||
}
|
||||
found
|
||||
}
|
||||
|
||||
/// Bodies of every `fn kinds(` that has one — a trait declaration ending in
|
||||
/// `;` has none.
|
||||
fn kinds_bodies(code: &str) -> Vec<&str> {
|
||||
let mut found = Vec::new();
|
||||
for (at, _) in code.match_indices("fn kinds(") {
|
||||
let rest = &code[at..];
|
||||
let (Some(brace), semi) = (rest.find('{'), rest.find(';')) else {
|
||||
continue;
|
||||
};
|
||||
if semi.is_some_and(|s| s < brace) {
|
||||
continue;
|
||||
}
|
||||
found.push(balanced(code, at + brace + 1, '{', '}'));
|
||||
}
|
||||
found
|
||||
}
|
||||
|
||||
/// The `X` of each `JobKind::X` in `text`.
|
||||
fn kinds_named(text: &str) -> Vec<String> {
|
||||
text.match_indices("JobKind::")
|
||||
.map(|(at, m)| {
|
||||
text[at + m.len()..]
|
||||
.chars()
|
||||
.take_while(|c| c.is_alphanumeric() || *c == '_')
|
||||
.collect()
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[derive(Default, Debug)]
|
||||
struct Ledger {
|
||||
/// Kind → where it is enqueued.
|
||||
enqueued: Vec<(String, String)>,
|
||||
/// Enqueue calls whose kind could not be read.
|
||||
unreadable: Vec<String>,
|
||||
claimed: BTreeSet<String>,
|
||||
claims_everything: bool,
|
||||
}
|
||||
|
||||
fn read(files: &[(String, String)]) -> Ledger {
|
||||
let mut ledger = Ledger::default();
|
||||
for (name, text) in files {
|
||||
let code = shipping_code(text);
|
||||
|
||||
for args in calls(&code, "enqueue") {
|
||||
let kinds = kinds_named(args);
|
||||
if kinds.is_empty() {
|
||||
ledger
|
||||
.unreadable
|
||||
.push(format!("{name}: enqueue({})", args.trim()));
|
||||
}
|
||||
for k in kinds {
|
||||
ledger.enqueued.push((k, name.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
let claimed = kinds_bodies(&code)
|
||||
.into_iter()
|
||||
.chain(calls(&code, "claim_next_matching"));
|
||||
for text in claimed {
|
||||
for k in kinds_named(text) {
|
||||
if k == "ALL" {
|
||||
ledger.claims_everything = true;
|
||||
} else {
|
||||
ledger.claimed.insert(k);
|
||||
}
|
||||
}
|
||||
}
|
||||
if !calls(&code, "claim_next").is_empty() {
|
||||
ledger.claims_everything = true;
|
||||
}
|
||||
}
|
||||
ledger
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_kind_enqueued_is_claimed_by_something() {
|
||||
let repo = Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.and_then(Path::parent)
|
||||
.expect("core/dr-catalog has a grandparent");
|
||||
|
||||
let mut paths = Vec::new();
|
||||
for group in ["core", "ui", "apps", "platform"] {
|
||||
crate_sources(&repo.join(group), &mut paths);
|
||||
}
|
||||
let files: Vec<(String, String)> = paths
|
||||
.iter()
|
||||
.map(|p| {
|
||||
let text = fs::read_to_string(p)
|
||||
.unwrap_or_else(|e| panic!("cannot read {}: {e}", p.display()));
|
||||
let name = p.strip_prefix(repo).unwrap_or(p).display().to_string();
|
||||
(name, text)
|
||||
})
|
||||
.collect();
|
||||
|
||||
// A scan over nothing passes for the wrong reason. The queue's own file
|
||||
// and the scan that used to feed it must both have been read, and the
|
||||
// queue's definitions found in them.
|
||||
for must in [
|
||||
"core/dr-catalog/src/jobs.rs",
|
||||
"core/dr-catalog/src/runner.rs",
|
||||
"ui/dr-ui/src/library/scan.rs",
|
||||
] {
|
||||
assert!(
|
||||
files.iter().any(|(n, _)| n == must),
|
||||
"{must} was not scanned — the source walk is wrong, not the code"
|
||||
);
|
||||
}
|
||||
let jobs = &files
|
||||
.iter()
|
||||
.find(|(n, _)| n == "core/dr-catalog/src/jobs.rs")
|
||||
.unwrap()
|
||||
.1;
|
||||
assert!(shipping_code(jobs).contains("pub fn enqueue("));
|
||||
|
||||
let ledger = read(&files);
|
||||
|
||||
assert!(
|
||||
ledger.unreadable.is_empty(),
|
||||
"\n\nThese enqueue calls do not name their JobKind, so this test cannot \
|
||||
check that anything claims it. Spell the kind at the call:\n {}\n",
|
||||
ledger.unreadable.join("\n ")
|
||||
);
|
||||
|
||||
if ledger.claims_everything {
|
||||
return;
|
||||
}
|
||||
let orphans: Vec<String> = ledger
|
||||
.enqueued
|
||||
.iter()
|
||||
.filter(|(k, _)| !ledger.claimed.contains(k))
|
||||
.map(|(k, at)| format!("JobKind::{k}, enqueued in {at}"))
|
||||
.collect();
|
||||
assert!(
|
||||
orphans.is_empty(),
|
||||
"\n\nEnqueued, and claimed by nothing (claimed: {:?}):\n {}\n\n\
|
||||
A kind nobody claims is a row per subject that stays for ever — the \
|
||||
queue coalesces, so it never fails, it only grows (#73). Register a \
|
||||
JobHandler for the kind, or stop enqueueing it and add it to \
|
||||
JobKind::RETIRED so the rows already queued are dropped.\n",
|
||||
ledger.claimed,
|
||||
orphans.join("\n ")
|
||||
);
|
||||
}
|
||||
|
||||
/// The reader itself, on code whose answer is known — so a parsing bug shows
|
||||
/// up as this failing, rather than the real check quietly finding nothing.
|
||||
#[test]
|
||||
fn the_reader_sees_producers_and_consumers() {
|
||||
let producer = r#"
|
||||
use dr_catalog::jobs;
|
||||
fn persist(tx: &Connection, id: i64) {
|
||||
// jobs::enqueue(tx, JobKind::ContentHash, ...) in a comment is not a call
|
||||
let _ = dr_catalog::jobs::enqueue(
|
||||
tx,
|
||||
JobKind::Thumbnail,
|
||||
Some(id),
|
||||
Priority::Background,
|
||||
None,
|
||||
);
|
||||
jobs::enqueue(tx, kind, Some(id), Priority::Background, None)?;
|
||||
}
|
||||
pub fn enqueue(conn: &Connection, kind: JobKind) {}
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
fn t() { enqueue(&c, JobKind::FetchOriginal, None, P, None); }
|
||||
}
|
||||
"#;
|
||||
let consumer = r#"
|
||||
impl JobHandler for Faces {
|
||||
fn kinds(&self) -> &[JobKind] {
|
||||
&[JobKind::DetectFaces]
|
||||
}
|
||||
fn run(&mut self) {}
|
||||
}
|
||||
trait JobHandler { fn kinds(&self) -> &[JobKind]; }
|
||||
fn pull(c: &Connection) { claim_next_matching(c, 0, &[JobKind::FetchPreview]); }
|
||||
"#;
|
||||
let ledger = read(&[
|
||||
("producer.rs".into(), producer.into()),
|
||||
("consumer.rs".into(), consumer.into()),
|
||||
]);
|
||||
|
||||
let enqueued: Vec<&str> = ledger.enqueued.iter().map(|(k, _)| k.as_str()).collect();
|
||||
assert_eq!(enqueued, vec!["Thumbnail"]);
|
||||
assert_eq!(ledger.unreadable.len(), 1, "{:?}", ledger.unreadable);
|
||||
assert_eq!(
|
||||
ledger.claimed,
|
||||
BTreeSet::from(["DetectFaces".to_string(), "FetchPreview".to_string()])
|
||||
);
|
||||
assert!(!ledger.claims_everything);
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
//! TRACES: FR-RAW-2
|
||||
//! The seam a second decoder plugs into.
|
||||
//!
|
||||
//! D2 keeps LibRaw as the fallback for bodies rawler does not cover. Adding
|
||||
//! it later should be a new `impl Decoder`, not an edit to every caller that
|
||||
//! reads a header, cuts a thumbnail or opens a photograph for export — which
|
||||
//! is what the free functions alone would have made it. So the callers take a
|
||||
//! `&dyn Decoder`, and only the places that start a job name [`default`].
|
||||
//!
|
||||
//! Bytes in, always. Nothing here takes a path or a `SourceRef`: resolving a
|
||||
//! file to bytes is `Storage`'s job at the caller, so the same decoder serves a
|
||||
//! local file, an Android document and a range fetched from Nextcloud. The
|
||||
//! decoder's part in that is to say how much of a file it needs
|
||||
//! ([`Decoder::header_bytes`]) and where its preview sits
|
||||
//! ([`Decoder::locate_preview`]); the storage layer fetches exactly that.
|
||||
//!
|
||||
//! What stays a free function is what is not a decoder's to vary: recognising
|
||||
//! a JPEG ([`crate::probe`]), decoding one ([`crate::decode_jpeg`]) and
|
||||
//! checking one is whole ([`crate::is_complete_jpeg`]). A second RAW decoder
|
||||
//! would not read a JPEG differently.
|
||||
|
||||
use dr_types::Orientation;
|
||||
|
||||
use crate::{DecodeError, Metadata, Preview, PreviewLocation, PreviewSize, RawImage};
|
||||
|
||||
/// TRACES: FR-RAW-2
|
||||
/// A RAW decoder, over bytes.
|
||||
///
|
||||
/// Object-safe so a caller can hold `&dyn Decoder` without becoming generic,
|
||||
/// `Send + Sync` because the callers that need one most — the thumbnail
|
||||
/// lanes, the export worker — run off the UI thread, and `Debug` so a job
|
||||
/// description that carries one can still be printed.
|
||||
pub trait Decoder: Send + Sync + std::fmt::Debug {
|
||||
/// How much of the start of a file [`Self::metadata`] and
|
||||
/// [`Self::locate_preview`] need. A caller reading over a network fetches
|
||||
/// this range and no more.
|
||||
fn header_bytes(&self) -> u64;
|
||||
|
||||
/// Capture metadata, from a header or a whole file, without touching
|
||||
/// sensor data.
|
||||
fn metadata(&self, bytes: &[u8]) -> Result<Metadata, DecodeError>;
|
||||
|
||||
/// How the stored pixels are turned, from a header. `None` where the file
|
||||
/// does not say, which callers take as upright.
|
||||
fn orientation(&self, header: &[u8]) -> Option<Orientation>;
|
||||
|
||||
/// Where the embedded preview best suited to a thumbnail sits in the file,
|
||||
/// from its header, so a remote caller can fetch that range alone.
|
||||
fn locate_preview(&self, header: &[u8], file_len: u64) -> Option<PreviewLocation>;
|
||||
|
||||
/// The embedded preview at the size asked for, falling through the ladder
|
||||
/// to the next size where the file lacks it.
|
||||
fn preview(&self, bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError>;
|
||||
|
||||
/// Sensor data, for develop and export. The expensive path.
|
||||
fn decode(&self, bytes: &[u8]) -> Result<RawImage, DecodeError>;
|
||||
}
|
||||
|
||||
/// TRACES: FR-RAW-2
|
||||
/// The decoder the application ships: rawler for sensor data and the
|
||||
/// previews it knows, DarkRoom's own container walk for headers and ranges.
|
||||
///
|
||||
/// Its methods are the crate's free functions, unchanged. They stay public
|
||||
/// for the tools and examples that read one file and have no caller to keep
|
||||
/// decoder-agnostic.
|
||||
#[derive(Debug, Clone, Copy, Default)]
|
||||
pub struct Rawler;
|
||||
|
||||
impl Decoder for Rawler {
|
||||
fn header_bytes(&self) -> u64 {
|
||||
crate::HEADER_BYTES
|
||||
}
|
||||
|
||||
fn metadata(&self, bytes: &[u8]) -> Result<Metadata, DecodeError> {
|
||||
crate::metadata(bytes)
|
||||
}
|
||||
|
||||
fn orientation(&self, header: &[u8]) -> Option<Orientation> {
|
||||
crate::orientation(header)
|
||||
}
|
||||
|
||||
fn locate_preview(&self, header: &[u8], file_len: u64) -> Option<PreviewLocation> {
|
||||
crate::locate_preview(header, file_len)
|
||||
}
|
||||
|
||||
fn preview(&self, bytes: &[u8], size: PreviewSize) -> Result<Preview, DecodeError> {
|
||||
crate::extract_preview(bytes, size)
|
||||
}
|
||||
|
||||
fn decode(&self, bytes: &[u8]) -> Result<RawImage, DecodeError> {
|
||||
crate::decode(bytes)
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-RAW-2
|
||||
/// The decoder a job uses unless it was handed another.
|
||||
///
|
||||
/// Named by the places that start work — a thread, a UI handler — and by
|
||||
/// nothing below them. Returning `&'static dyn Decoder` rather than `Rawler`
|
||||
/// is the point: a caller that only has this cannot reach past the trait.
|
||||
pub fn default() -> &'static dyn Decoder {
|
||||
static RAWLER: Rawler = Rawler;
|
||||
&RAWLER
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// The default is the shipped decoder, reached through the trait: same
|
||||
/// header budget, and the same answer to bytes neither can read.
|
||||
#[test]
|
||||
fn the_default_is_rawler_behind_the_trait() {
|
||||
let d = default();
|
||||
assert_eq!(d.header_bytes(), crate::HEADER_BYTES);
|
||||
let junk = [0u8; 64];
|
||||
assert_eq!(d.metadata(&junk).is_err(), crate::metadata(&junk).is_err());
|
||||
assert!(d.decode(&junk).is_err());
|
||||
assert_eq!(d.orientation(&junk), crate::orientation(&junk));
|
||||
}
|
||||
}
|
||||
@@ -11,14 +11,20 @@
|
||||
//!
|
||||
//! Fusing them would force a full decode where a header read suffices, which
|
||||
//! is exactly why Lightroom stalls ~2 s per image during culling.
|
||||
//!
|
||||
//! Callers reach these through the [`Decoder`] trait rather than by name, so a
|
||||
//! second decoder can be put behind them without changing any of them
|
||||
//! (FR-RAW-2). [`Rawler`] is the one that ships; [`default`] hands it out.
|
||||
|
||||
pub mod base_curve;
|
||||
mod decoder;
|
||||
mod error;
|
||||
mod locate;
|
||||
mod preview;
|
||||
pub mod profile;
|
||||
|
||||
pub use base_curve::BaseCurve;
|
||||
pub use decoder::{default, Decoder, Rawler};
|
||||
pub use error::DecodeError;
|
||||
pub use locate::{
|
||||
defects, is_complete_jpeg, jpeg_metadata, locate_preview, tiff_metadata, BadLine, BadPixel,
|
||||
|
||||
+22
-6
@@ -41,18 +41,34 @@ matters — see [`src/bake.rs`](src/bake.rs) for the argument:
|
||||
1. **A 3×3 matrix**, linear sRGB to the three layers' exposure. Exact, not an
|
||||
approximation: the reconstructed scene spectrum is linear in the sRGB
|
||||
triple, so the integral collapses into nine numbers.
|
||||
2. **Three 1D curves**, log exposure to density, sampled at 256 points.
|
||||
3. **One 32³ lookup**, density to linear sRGB — dye absorption, the print
|
||||
through the negative, the paper, the viewing illuminant and the chromatic
|
||||
adaptation, all of which take exactly three numbers in.
|
||||
2. **Three 1D curves**, log exposure to density, sampled at 256 points — one
|
||||
row per development time the datasheet measures. Push picks between the
|
||||
rows, and interpolating them is exact, because density is linear in push
|
||||
between two measured processes.
|
||||
3. **One 32³ lookup**, density to linear sRGB — dye absorption, the viewing
|
||||
illuminant and the chromatic adaptation, all of which take exactly three
|
||||
numbers in. A printed negative is two: the film's cube ends at the paper's
|
||||
log exposure through the negative, the enlarger's exposure is added there,
|
||||
and the paper's own curve row and cube take it to linear sRGB.
|
||||
|
||||
Per pixel that is a matrix multiply, three curve taps and one texture fetch.
|
||||
Splitting 2 from 3, rather than baking one LUT over exposure, is measured
|
||||
Per pixel that is a matrix multiply, a handful of curve taps and one texture
|
||||
fetch — two for a print. Splitting 2 from 3, rather than baking one LUT over exposure, is measured
|
||||
rather than assumed: the curve carries all the sharp shape and the dye mixing
|
||||
is smooth, so folding the curve into the 3D lookup would need it three times
|
||||
larger for the same error. At 32³ the worst interpolation error is about 0.003
|
||||
in linear sRGB, below one 8-bit code value, and there is a test that says so.
|
||||
|
||||
**No slider is baked.** Camera exposure is a gain before the matrix, push
|
||||
chooses between curve rows, print exposure is the addition between the two
|
||||
cubes, and format sets the grain; each reaches the shader as a uniform that is
|
||||
linear in what it does. That is what lets a mask layer hold its own film
|
||||
settings, and a pixel under several layers take the weighted average of them.
|
||||
Only the enlarger's filtration is solved at bake time, against the
|
||||
photograph's exposure — an enlarger has one filtration for the whole print —
|
||||
so the film's Exposure, set on the whole photograph, is the one slider that
|
||||
rebakes. The stock and its
|
||||
paper are the photograph's; a layer has no picker.
|
||||
|
||||
## Adding a stock
|
||||
|
||||
If spektrafilm has it, add its name to `STOCKS` in
|
||||
|
||||
+443
-112
@@ -21,14 +21,16 @@
|
||||
//! curves and dyes, the viewing illuminant, the adaptation — all of it takes
|
||||
//! three numbers in and gives three numbers out. So it bakes into one small
|
||||
//! 3D lookup, and the per-pixel cost is a matrix multiply, three curve taps
|
||||
//! and one texture fetch.
|
||||
//! and one texture fetch. A print is two: the film's lookup ends at the
|
||||
//! paper's log exposure, where the enlarger's exposure is an addition, and
|
||||
//! the paper's curve and lookup take it from there — see [`Paper`].
|
||||
//!
|
||||
//! Splitting 2 from 3 rather than baking a single LUT over exposure is
|
||||
//! deliberate and measured: the curve carries all of the sharp shape and the
|
||||
//! dye mixing is smooth, so putting the curve in the 3D LUT would force it
|
||||
//! three times larger for the same error.
|
||||
|
||||
use crate::profile::Profile;
|
||||
use crate::profile::{Profile, CURVE_SAMPLES};
|
||||
use crate::spectrum::{illuminant, Spectrum, Viewing};
|
||||
use crate::tables::{SPECTRUM, SRGB_BASIS};
|
||||
|
||||
@@ -47,7 +49,19 @@ pub const MID_GREY: f32 = 0.184;
|
||||
/// that on: the error is already under what the output can represent.
|
||||
pub const LUT_SIZE: usize = 32;
|
||||
|
||||
/// What to develop, and how.
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// The most development times a stock may measure: one curve row, and one
|
||||
/// push station, each. Every stock shipped measures five; the ceiling is what
|
||||
/// the shader's fixed uniform block can hold.
|
||||
pub const MAX_CURVE_ROWS: usize = 8;
|
||||
|
||||
/// What to develop: the materials, and where the enlarger is balanced.
|
||||
///
|
||||
/// **Not how far, and not how bright.** Push, print exposure and camera
|
||||
/// exposure are [`Settings`], evaluated per pixel against these tables, so
|
||||
/// that a mask layer can hold its own and a pixel under it can take the
|
||||
/// weighted average of everyone's (FR-DEV-3f). What is left here is what a
|
||||
/// photograph has one of.
|
||||
pub struct Recipe<'a> {
|
||||
/// The stock the picture was taken on.
|
||||
pub film: &'a Profile,
|
||||
@@ -55,17 +69,15 @@ pub struct Recipe<'a> {
|
||||
/// what a reversal stock wants and what makes a negative come out orange
|
||||
/// and inverted — that being what a negative actually looks like.
|
||||
pub print: Option<&'a Profile>,
|
||||
/// Camera exposure, in stops.
|
||||
pub exposure_ev: f32,
|
||||
/// Enlarger exposure, in stops. Ignored without a `print`.
|
||||
pub print_exposure_ev: f32,
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// Development, in stops of push. Positive develops longer.
|
||||
/// The camera exposure the enlarger is balanced at, in stops. Ignored
|
||||
/// without a `print`.
|
||||
///
|
||||
/// Ignored by a stock measured at one process, of which there are many —
|
||||
/// see [`crate::profile::Profile::curves_at_push`], which returns the one
|
||||
/// measured curve rather than inventing a pushed one.
|
||||
pub push_stops: f32,
|
||||
/// The *photograph's* exposure, never a region's. An enlarger has one
|
||||
/// filtration for the whole print: a negative exposed a stop brighter in
|
||||
/// one corner prints a stop darker there, and that difference is the
|
||||
/// picture — balancing it away per pixel would erase every local exposure
|
||||
/// change a layer made.
|
||||
pub exposure_ev: f32,
|
||||
}
|
||||
|
||||
impl<'a> Recipe<'a> {
|
||||
@@ -76,12 +88,27 @@ impl<'a> Recipe<'a> {
|
||||
film,
|
||||
print,
|
||||
exposure_ev: 0.0,
|
||||
print_exposure_ev: 0.0,
|
||||
push_stops: 0.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// What a pixel is developed with, against a [`Baked`] stock.
|
||||
///
|
||||
/// The shader's uniforms, as the CPU sees them: every field is linear in what
|
||||
/// the tables are indexed by, which is what lets the composer blend several
|
||||
/// layers' settings into one before the fragment runs.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq)]
|
||||
pub struct Settings {
|
||||
/// Camera exposure, in stops: a gain on the scene.
|
||||
pub exposure_ev: f32,
|
||||
/// Development, in stops of push. Positive develops longer. Nothing for a
|
||||
/// stock measured at one process, of which there are many.
|
||||
pub push_stops: f32,
|
||||
/// Enlarger exposure, in stops. Nothing without a print.
|
||||
pub print_exposure_ev: f32,
|
||||
}
|
||||
|
||||
/// A recipe reduced to three tables.
|
||||
///
|
||||
/// Plain `f32` with a documented layout, and no notion of a texture: what to
|
||||
@@ -89,16 +116,31 @@ impl<'a> Recipe<'a> {
|
||||
/// the whole model be tested on the CPU.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Baked {
|
||||
/// Linear sRGB to the three layers' log₁₀ exposure, before the log — row
|
||||
/// `l`, column `c` is layer `l`'s response to sRGB channel `c`.
|
||||
/// Linear sRGB to the three layers' exposure, before the log — row `l`,
|
||||
/// column `c` is layer `l`'s response to sRGB channel `c`. At unit gain:
|
||||
/// [`Settings::exposure_ev`] is applied per pixel.
|
||||
pub exposure_matrix: [[f32; 3]; 3],
|
||||
/// The characteristic curves, `CURVE_SAMPLES` samples per layer, uniform
|
||||
/// over `[curve_log_min, curve_log_max]`.
|
||||
/// The characteristic curves: `curve_rows` rows of `CURVE_SAMPLES`
|
||||
/// samples, row after row, each uniform over
|
||||
/// `[curve_log_min, curve_log_max]`.
|
||||
///
|
||||
/// Row `r` is the stock as measured at its `r`th development time, which
|
||||
/// is push [`Self::push_stations`]`[r]`. The rows are the measurements
|
||||
/// themselves rather than a resampling: between two, density is linear in
|
||||
/// push (development is interpolated in log time, and push is log time),
|
||||
/// so interpolating the rows by push reproduces
|
||||
/// [`Profile::curves_at_push`] exactly. A stock measured at one process
|
||||
/// has one row.
|
||||
pub curves: Vec<[f32; 3]>,
|
||||
pub curve_rows: usize,
|
||||
/// The push each row was developed to, ascending, one per row.
|
||||
pub push_stations: Vec<f32>,
|
||||
pub curve_log_min: f32,
|
||||
pub curve_log_max: f32,
|
||||
/// Density to linear sRGB, `LUT_SIZE³` entries uniform over
|
||||
/// `[0, density_max]` on each axis.
|
||||
/// Film density to what comes next, `LUT_SIZE³` entries uniform over
|
||||
/// `[0, density_max]` on each axis: linear sRGB when the film is viewed
|
||||
/// directly, and the paper's log₁₀ exposure through it, per layer, when it
|
||||
/// is printed.
|
||||
///
|
||||
/// **The red axis varies fastest**, then green, then blue — that is,
|
||||
/// `lut[(b * size + g) * size + r]`. Stated because it is not the order
|
||||
@@ -108,60 +150,142 @@ pub struct Baked {
|
||||
/// picture with red and blue transposed, which looks like a plausible
|
||||
/// photograph of the wrong colour.
|
||||
pub lut: Vec<[f32; 3]>,
|
||||
/// The paper, when there is one. See [`Paper`].
|
||||
pub paper: Option<Paper>,
|
||||
/// The deepest density any row develops to, so one lookup covers every
|
||||
/// push.
|
||||
pub density_max: f32,
|
||||
pub lut_size: usize,
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// The print half of a baked stock: enlarger to paper to viewing.
|
||||
///
|
||||
/// Split from the film's lookup at the paper's log exposure, for the reason
|
||||
/// the film is split from its own curve. The enlarger's exposure is a shift
|
||||
/// *in that log exposure*, the same stops on all three layers, so a print
|
||||
/// exposure is an addition between the two lookups — exact at any value and
|
||||
/// free per pixel. Baking it into one lookup instead needs a slice per
|
||||
/// setting, and interpolating between slices misses by several code values,
|
||||
/// because the paper's curve is the sharpest thing in the print.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Paper {
|
||||
/// The enlarger's filtration, per layer, in log₁₀ exposure: what makes a
|
||||
/// mid-grey scene print neutral at the photograph's exposure. See
|
||||
/// [`Recipe::exposure_ev`].
|
||||
pub balance: [f32; 3],
|
||||
/// The paper's characteristic curves, `CURVE_SAMPLES` samples uniform
|
||||
/// over `[log_min, log_max]`.
|
||||
pub curves: Vec<[f32; 3]>,
|
||||
pub log_min: f32,
|
||||
pub log_max: f32,
|
||||
/// Paper density to linear sRGB, laid out as [`Baked::lut`] is, uniform
|
||||
/// over `[0, density_max]`.
|
||||
pub lut: Vec<[f32; 3]>,
|
||||
pub density_max: f32,
|
||||
}
|
||||
|
||||
/// Where `push` falls among the rows: the lower row and the fraction toward
|
||||
/// the next. Clamped at both ends, as `curves_at_push` clamps to the first and
|
||||
/// last measured process.
|
||||
fn push_row(stations: &[f32], push: f32) -> (usize, f32) {
|
||||
if stations.len() < 2 {
|
||||
return (0, 0.0);
|
||||
}
|
||||
let last = stations.len() - 1;
|
||||
let hi = stations
|
||||
.iter()
|
||||
.position(|p| *p >= push)
|
||||
.unwrap_or(last)
|
||||
.max(1);
|
||||
let lo = hi - 1;
|
||||
let f = (push - stations[lo]) / (stations[hi] - stations[lo]).max(1e-6);
|
||||
(lo, f.clamp(0.0, 1.0))
|
||||
}
|
||||
|
||||
impl Baked {
|
||||
/// Look a colour up the way the shader will, for tests and for previews.
|
||||
/// Look a colour up the way the shader will, at the stock's own settings.
|
||||
pub fn apply(&self, rgb: [f32; 3]) -> [f32; 3] {
|
||||
self.apply_at(rgb, &Settings::default())
|
||||
}
|
||||
|
||||
/// Look a colour up the way the shader will, for tests and for previews.
|
||||
pub fn apply_at(&self, rgb: [f32; 3], settings: &Settings) -> [f32; 3] {
|
||||
let gain = 2f32.powf(settings.exposure_ev);
|
||||
let mut log_exposure = [0.0f32; 3];
|
||||
for (l, slot) in log_exposure.iter_mut().enumerate() {
|
||||
let m = self.exposure_matrix[l];
|
||||
let e = m[0] * rgb[0] + m[1] * rgb[1] + m[2] * rgb[2];
|
||||
let e = gain * (m[0] * rgb[0] + m[1] * rgb[1] + m[2] * rgb[2]);
|
||||
*slot = (e.max(0.0) + 1e-10).log10();
|
||||
}
|
||||
self.sample_lut(self.sample_curves(log_exposure))
|
||||
let density = self.sample_curves(log_exposure, settings.push_stops);
|
||||
let through = sample_cube(&self.lut, self.lut_size, density, self.density_max);
|
||||
let Some(paper) = &self.paper else {
|
||||
return through;
|
||||
};
|
||||
let shift = settings.print_exposure_ev * 2f32.log10();
|
||||
let paper_log = [0, 1, 2].map(|l| through[l] + paper.balance[l] + shift);
|
||||
let paper_density = sample_curve(&paper.curves, paper.log_min, paper.log_max, paper_log);
|
||||
sample_cube(&paper.lut, self.lut_size, paper_density, paper.density_max)
|
||||
}
|
||||
|
||||
fn sample_curves(&self, log_exposure: [f32; 3]) -> [f32; 3] {
|
||||
let last = self.curves.len() - 1;
|
||||
let span = self.curve_log_max - self.curve_log_min;
|
||||
let mut out = [0.0f32; 3];
|
||||
for (c, slot) in out.iter_mut().enumerate() {
|
||||
let t = ((log_exposure[c] - self.curve_log_min) / span).clamp(0.0, 1.0) * last as f32;
|
||||
let i = (t.floor() as usize).min(last - 1);
|
||||
let f = t - i as f32;
|
||||
*slot = self.curves[i][c] * (1.0 - f) + self.curves[i + 1][c] * f;
|
||||
fn sample_curves(&self, log_exposure: [f32; 3], push_stops: f32) -> [f32; 3] {
|
||||
let (row, g) = push_row(&self.push_stations, push_stops);
|
||||
let lo = self.sample_curve_row(log_exposure, row);
|
||||
if self.curve_rows < 2 {
|
||||
return lo;
|
||||
}
|
||||
out
|
||||
let hi = self.sample_curve_row(log_exposure, row + 1);
|
||||
[0, 1, 2].map(|c| lo[c] * (1.0 - g) + hi[c] * g)
|
||||
}
|
||||
|
||||
fn sample_lut(&self, density: [f32; 3]) -> [f32; 3] {
|
||||
let n = self.lut_size;
|
||||
let mut base = [0usize; 3];
|
||||
let mut frac = [0f32; 3];
|
||||
for c in 0..3 {
|
||||
let t = (density[c] / self.density_max).clamp(0.0, 1.0) * (n - 1) as f32;
|
||||
base[c] = (t.floor() as usize).min(n - 2);
|
||||
frac[c] = t - base[c] as f32;
|
||||
}
|
||||
let mut out = [0.0f32; 3];
|
||||
for dx in 0..2 {
|
||||
for dy in 0..2 {
|
||||
for dz in 0..2 {
|
||||
let w = if dx == 0 { 1.0 - frac[0] } else { frac[0] }
|
||||
* if dy == 0 { 1.0 - frac[1] } else { frac[1] }
|
||||
* if dz == 0 { 1.0 - frac[2] } else { frac[2] };
|
||||
let e = self.lut[((base[2] + dz) * n + base[1] + dy) * n + base[0] + dx];
|
||||
for c in 0..3 {
|
||||
out[c] += w * e[c];
|
||||
}
|
||||
fn sample_curve_row(&self, log_exposure: [f32; 3], row: usize) -> [f32; 3] {
|
||||
let samples = self.curves.len() / self.curve_rows;
|
||||
let curve = &self.curves[row * samples..(row + 1) * samples];
|
||||
sample_curve(curve, self.curve_log_min, self.curve_log_max, log_exposure)
|
||||
}
|
||||
}
|
||||
|
||||
/// Three curves sampled uniformly over `[log_min, log_max]`, read at a log
|
||||
/// exposure per layer. Clamped at both ends, as the shader's is.
|
||||
fn sample_curve(curve: &[[f32; 3]], log_min: f32, log_max: f32, at: [f32; 3]) -> [f32; 3] {
|
||||
let last = curve.len() - 1;
|
||||
let span = log_max - log_min;
|
||||
let mut out = [0.0f32; 3];
|
||||
for (c, slot) in out.iter_mut().enumerate() {
|
||||
let t = ((at[c] - log_min) / span).clamp(0.0, 1.0) * last as f32;
|
||||
let i = (t.floor() as usize).min(last - 1);
|
||||
let f = t - i as f32;
|
||||
*slot = curve[i][c] * (1.0 - f) + curve[i + 1][c] * f;
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// A cube of `n³` triples over `[0, max]` per axis, red fastest, read
|
||||
/// trilinearly.
|
||||
fn sample_cube(lut: &[[f32; 3]], n: usize, density: [f32; 3], max: f32) -> [f32; 3] {
|
||||
let mut base = [0usize; 3];
|
||||
let mut frac = [0f32; 3];
|
||||
for c in 0..3 {
|
||||
let t = (density[c] / max).clamp(0.0, 1.0) * (n - 1) as f32;
|
||||
base[c] = (t.floor() as usize).min(n - 2);
|
||||
frac[c] = t - base[c] as f32;
|
||||
}
|
||||
let mut out = [0.0f32; 3];
|
||||
for dx in 0..2 {
|
||||
for dy in 0..2 {
|
||||
for dz in 0..2 {
|
||||
let w = if dx == 0 { 1.0 - frac[0] } else { frac[0] }
|
||||
* if dy == 0 { 1.0 - frac[1] } else { frac[1] }
|
||||
* if dz == 0 { 1.0 - frac[2] } else { frac[2] };
|
||||
let e = lut[((base[2] + dz) * n + base[1] + dy) * n + base[0] + dx];
|
||||
for c in 0..3 {
|
||||
out[c] += w * e[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Linear sRGB to the three layers' exposure, mid-grey normalised.
|
||||
@@ -204,12 +328,7 @@ pub fn exposure_matrix(film: &Profile) -> [[f32; 3]; 3] {
|
||||
/// goes: the mask is a fixed density, so balancing mid-grey to neutral cancels
|
||||
/// it — which is why a printed negative looks like a photograph while a scanned
|
||||
/// one looks orange.
|
||||
fn print_balance(
|
||||
film: &Profile,
|
||||
paper: &Profile,
|
||||
exposure_ev: f32,
|
||||
print_exposure_ev: f32,
|
||||
) -> [f32; 3] {
|
||||
pub fn print_balance(film: &Profile, paper: &Profile, exposure_ev: f32) -> [f32; 3] {
|
||||
let matrix = exposure_matrix(film);
|
||||
let scene = MID_GREY * 2f32.powf(exposure_ev);
|
||||
let mut log_exposure = [0.0f32; 3];
|
||||
@@ -227,7 +346,7 @@ fn print_balance(
|
||||
|
||||
let mut offsets = [0.0f32; 3];
|
||||
for (l, slot) in offsets.iter_mut().enumerate() {
|
||||
*slot = target - (mid_raw[l] + 1e-10).log10() + print_exposure_ev * 2f32.log10();
|
||||
*slot = target - (mid_raw[l] + 1e-10).log10();
|
||||
}
|
||||
offsets
|
||||
}
|
||||
@@ -257,77 +376,117 @@ fn paper_exposure(film: &Profile, paper: &Profile, density: [f32; 3]) -> [f32; 3
|
||||
/// Bake a recipe into the tables a shader runs.
|
||||
pub fn bake(recipe: &Recipe) -> Baked {
|
||||
let film = recipe.film;
|
||||
let mut matrix = exposure_matrix(film);
|
||||
// Camera exposure rides in the matrix rather than in the shader: it is a
|
||||
// scalar on a linear quantity, and folding it in here costs nothing and
|
||||
// keeps the per-pixel work identical whether or not it has been moved.
|
||||
let gain = 2f32.powf(recipe.exposure_ev);
|
||||
for row in &mut matrix {
|
||||
for v in row.iter_mut() {
|
||||
*v *= gain;
|
||||
}
|
||||
}
|
||||
// At unit gain. Camera exposure is a scalar on a linear quantity, so the
|
||||
// shader applies it for the price of one multiply — and has to, since a
|
||||
// layer may hold its own.
|
||||
let matrix = exposure_matrix(film);
|
||||
|
||||
// TRACES: FR-DEV-3f
|
||||
// Developed to the requested push before anything else reads the curves:
|
||||
// the density ceiling, the print balance and the grain all depend on how
|
||||
// far this film was taken, and a push that only reached one of them would
|
||||
// be a contrast change wearing a push's name.
|
||||
let curves = film.curves_at_push(recipe.push_stops);
|
||||
let density_max = curves
|
||||
.iter()
|
||||
.flat_map(|row| row.iter())
|
||||
.fold(0.0f32, |a, &b| a.max(b))
|
||||
.max(1e-3);
|
||||
|
||||
let viewing = match recipe.print {
|
||||
Some(paper) => Viewing::new(&paper.viewing_illuminant),
|
||||
None => Viewing::new(&film.viewing_illuminant),
|
||||
// Every measured process, not the one the slider is at: the shader
|
||||
// interpolates between rows per pixel, so a layer can push a region.
|
||||
// Resampled to one length because the rows share a texture.
|
||||
let measured = film.development_curves.len() >= 2
|
||||
&& film.development_times.len() == film.development_curves.len();
|
||||
let (curves, push_stations): (Vec<[f32; 3]>, Vec<f32>) = if measured {
|
||||
let rows = film.development_curves.len().min(MAX_CURVE_ROWS);
|
||||
(
|
||||
film.development_curves[..rows]
|
||||
.iter()
|
||||
.flat_map(|c| resample(c))
|
||||
.collect(),
|
||||
film.development_times[..rows]
|
||||
.iter()
|
||||
.map(|t| 2.0 * (t / film.development_normal).log2())
|
||||
.collect(),
|
||||
)
|
||||
} else {
|
||||
(resample(&film.density_curves), vec![0.0])
|
||||
};
|
||||
let balance = recipe
|
||||
.print
|
||||
.map(|paper| print_balance(film, paper, recipe.exposure_ev, recipe.print_exposure_ev));
|
||||
let curve_rows = push_stations.len();
|
||||
// The ceiling of the deepest row, so one lookup covers every push.
|
||||
let density_max = ceiling(&curves);
|
||||
|
||||
let n = LUT_SIZE;
|
||||
let mut lut = Vec::with_capacity(n * n * n);
|
||||
// Blue outermost and red innermost, so the red axis varies fastest. See
|
||||
// `Baked::lut`: this is the layout a 3D texture upload wants, and getting
|
||||
// it backwards transposes red and blue in the finished picture.
|
||||
for b in 0..n {
|
||||
for g in 0..n {
|
||||
for r in 0..n {
|
||||
let density = [
|
||||
density_max * r as f32 / (n - 1) as f32,
|
||||
density_max * g as f32 / (n - 1) as f32,
|
||||
density_max * b as f32 / (n - 1) as f32,
|
||||
];
|
||||
lut.push(match recipe.print.zip(balance) {
|
||||
Some((paper, offsets)) => {
|
||||
let raw = paper_exposure(film, paper, density);
|
||||
let mut log_exposure = [0.0f32; 3];
|
||||
for (l, slot) in log_exposure.iter_mut().enumerate() {
|
||||
*slot = (raw[l] + 1e-10).log10() + offsets[l];
|
||||
}
|
||||
let paper_density = paper.density_at(log_exposure);
|
||||
viewing.to_srgb(&paper.transmittance(paper_density))
|
||||
}
|
||||
None => viewing.to_srgb(&film.transmittance(density)),
|
||||
});
|
||||
let cube = |max: f32, f: &dyn Fn([f32; 3]) -> [f32; 3]| {
|
||||
let mut out = Vec::with_capacity(n * n * n);
|
||||
for b in 0..n {
|
||||
for g in 0..n {
|
||||
for r in 0..n {
|
||||
let step = max / (n - 1) as f32;
|
||||
out.push(f([r as f32 * step, g as f32 * step, b as f32 * step]));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out
|
||||
};
|
||||
|
||||
let (lut, paper) = match recipe.print {
|
||||
None => {
|
||||
let viewing = Viewing::new(&film.viewing_illuminant);
|
||||
(
|
||||
cube(density_max, &|d| viewing.to_srgb(&film.transmittance(d))),
|
||||
None,
|
||||
)
|
||||
}
|
||||
Some(paper) => {
|
||||
let viewing = Viewing::new(&paper.viewing_illuminant);
|
||||
let curves = resample(&paper.density_curves);
|
||||
let paper_max = ceiling(&curves);
|
||||
let lut = cube(density_max, &|d| {
|
||||
paper_exposure(film, paper, d).map(|raw| (raw + 1e-10).log10())
|
||||
});
|
||||
let paper = Paper {
|
||||
balance: print_balance(film, paper, recipe.exposure_ev),
|
||||
log_min: paper.log_exposure_min,
|
||||
log_max: paper.log_exposure_max,
|
||||
lut: cube(paper_max, &|d| viewing.to_srgb(&paper.transmittance(d))),
|
||||
density_max: paper_max,
|
||||
curves,
|
||||
};
|
||||
(lut, Some(paper))
|
||||
}
|
||||
};
|
||||
|
||||
Baked {
|
||||
exposure_matrix: matrix,
|
||||
curves,
|
||||
curve_rows,
|
||||
push_stations,
|
||||
curve_log_min: film.log_exposure_min,
|
||||
curve_log_max: film.log_exposure_max,
|
||||
lut,
|
||||
paper,
|
||||
density_max,
|
||||
lut_size: n,
|
||||
}
|
||||
}
|
||||
|
||||
/// A curve at `CURVE_SAMPLES`, uniform over the same domain it came in on.
|
||||
fn resample(curve: &[[f32; 3]]) -> Vec<[f32; 3]> {
|
||||
if curve.len() == CURVE_SAMPLES {
|
||||
return curve.to_vec();
|
||||
}
|
||||
(0..CURVE_SAMPLES)
|
||||
.map(|i| {
|
||||
let at = i as f32 / (CURVE_SAMPLES - 1) as f32;
|
||||
sample_curve(curve, 0.0, 1.0, [at; 3])
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The deepest density in a set of curves, floored so a lookup over it has
|
||||
/// a width.
|
||||
fn ceiling(curves: &[[f32; 3]]) -> f32 {
|
||||
curves
|
||||
.iter()
|
||||
.flat_map(|row| row.iter())
|
||||
.fold(0.0f32, |a, &b| a.max(b))
|
||||
.max(1e-3)
|
||||
}
|
||||
|
||||
fn mean(s: &Spectrum) -> f32 {
|
||||
s.iter().sum::<f32>() / SPECTRUM as f32
|
||||
}
|
||||
@@ -467,6 +626,10 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn exposure_moves_the_print_the_way_it_moves_a_photograph() {
|
||||
// The photograph's exposure: the enlarger balanced at it, and the
|
||||
// scene brighter by it. Mid-grey stays where the balance puts it —
|
||||
// that is what the balance is for — so what a stop more does to a
|
||||
// print is lift everything either side of it along the paper's curve.
|
||||
let film = portra();
|
||||
let paper = endura();
|
||||
let brighter = bake(&Recipe {
|
||||
@@ -474,7 +637,155 @@ mod tests {
|
||||
..Recipe::new(&film, Some(&paper))
|
||||
});
|
||||
let base = bake(&Recipe::new(&film, Some(&paper)));
|
||||
assert!(brighter.apply([MID_GREY; 3])[1] > base.apply([MID_GREY; 3])[1]);
|
||||
let one_stop = Settings {
|
||||
exposure_ev: 1.0,
|
||||
..Settings::default()
|
||||
};
|
||||
for v in [0.02f32, 0.6] {
|
||||
assert!(
|
||||
brighter.apply_at([v; 3], &one_stop)[1] > base.apply([v; 3])[1],
|
||||
"{v} did not print brighter a stop up"
|
||||
);
|
||||
}
|
||||
let (a, b) = (
|
||||
brighter.apply_at([MID_GREY; 3], &one_stop)[1],
|
||||
base.apply([MID_GREY; 3])[1],
|
||||
);
|
||||
assert!(
|
||||
(a - b).abs() < 1.0 / 255.0,
|
||||
"the balance let mid-grey move: {a} vs {b}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_region_exposed_brighter_prints_brighter_than_the_enlarger_expects() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// A layer's exposure is the scene's, not the enlarger's: the balance
|
||||
// stays where the photograph put it, so the region prints lighter by
|
||||
// more than the whole photograph would, which is what dodging at the
|
||||
// camera is.
|
||||
let film = portra();
|
||||
let paper = endura();
|
||||
let base = bake(&Recipe::new(&film, Some(&paper)));
|
||||
let rebalanced = bake(&Recipe {
|
||||
exposure_ev: 1.0,
|
||||
..Recipe::new(&film, Some(&paper))
|
||||
});
|
||||
let one_stop = Settings {
|
||||
exposure_ev: 1.0,
|
||||
..Settings::default()
|
||||
};
|
||||
let local = base.apply_at([MID_GREY; 3], &one_stop)[1];
|
||||
let global = rebalanced.apply_at([MID_GREY; 3], &one_stop)[1];
|
||||
assert!(local > base.apply([MID_GREY; 3])[1], "not brighter at all");
|
||||
assert!(
|
||||
local > global,
|
||||
"a region was rebalanced as though it were the whole print: {local} vs {global}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn more_light_through_the_enlarger_darkens_the_print() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// Paper is negative-working. Opening the enlarger a stop is burning
|
||||
// in, and a slider that brightened would be the wrong way round for
|
||||
// anyone who has printed.
|
||||
let film = portra();
|
||||
let paper = endura();
|
||||
let baked = bake(&Recipe::new(&film, Some(&paper)));
|
||||
let at = |stops: f32| {
|
||||
baked.apply_at(
|
||||
[MID_GREY; 3],
|
||||
&Settings {
|
||||
print_exposure_ev: stops,
|
||||
..Settings::default()
|
||||
},
|
||||
)[1]
|
||||
};
|
||||
assert!(at(1.0) < at(0.0) && at(0.0) < at(-1.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_push_on_a_row_is_the_measured_curve() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// The rows are the measured processes, so at a row the table must be
|
||||
// that curve exactly, and between rows — density being linear in push
|
||||
// there — it must be `curves_at_push` to rounding.
|
||||
let film = profile(include_str!("../profiles/kodak_doublex.yaml"));
|
||||
let baked = bake(&Recipe::new(&film, None));
|
||||
assert_eq!(
|
||||
baked.curve_rows, 5,
|
||||
"Double-X measures five development times"
|
||||
);
|
||||
|
||||
let span = film.log_exposure_max - film.log_exposure_min;
|
||||
let mut worst = 0.0f32;
|
||||
let stations = baked.push_stations.clone();
|
||||
let mut pushes: Vec<(f32, bool)> = stations.iter().map(|p| (*p, true)).collect();
|
||||
for k in 0..=16 {
|
||||
pushes.push((-1.0 + 4.0 * k as f32 / 16.0, false));
|
||||
}
|
||||
for (push, on_row) in pushes {
|
||||
let exact = film.curves_at_push(push);
|
||||
for i in (0..exact.len()).step_by(7) {
|
||||
let log = film.log_exposure_min + span * i as f32 / (exact.len() - 1) as f32;
|
||||
let got = baked.sample_curves([log; 3], push);
|
||||
for c in 0..3 {
|
||||
let err = (got[c] - exact[i][c]).abs();
|
||||
if on_row {
|
||||
assert!(err < 1e-4, "push {push} is a row but misses it by {err}");
|
||||
}
|
||||
worst = worst.max(err);
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(worst < 1e-3, "between rows the density is off by {worst}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_print_exposure_is_exact_at_any_setting() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// The enlarger's exposure is added between the two lookups rather than
|
||||
// baked into either, so no setting is nearer the tables than another.
|
||||
// Compared against the chain evaluated spectrally, end to end, at
|
||||
// settings chosen off every half and whole stop.
|
||||
let film = portra();
|
||||
let paper = endura();
|
||||
let baked = bake(&Recipe::new(&film, Some(&paper)));
|
||||
let offsets = print_balance(&film, &paper, 0.0);
|
||||
let viewing = Viewing::new(&paper.viewing_illuminant);
|
||||
|
||||
let mut worst = 0.0f32;
|
||||
for stops in [-2.3f32, -0.6, 0.0, 0.35, 1.7] {
|
||||
for i in 0..14 {
|
||||
let v = 0.004 * 2f32.powf(i as f32 * 0.6);
|
||||
let rgb = [v, v * 0.8, v * 1.1];
|
||||
let mut log_exposure = [0.0f32; 3];
|
||||
for (l, slot) in log_exposure.iter_mut().enumerate() {
|
||||
let m = baked.exposure_matrix[l];
|
||||
*slot =
|
||||
((m[0] * rgb[0] + m[1] * rgb[1] + m[2] * rgb[2]).max(0.0) + 1e-10).log10();
|
||||
}
|
||||
let raw = paper_exposure(&film, &paper, film.density_at(log_exposure));
|
||||
let paper_log =
|
||||
[0, 1, 2].map(|l| (raw[l] + 1e-10).log10() + offsets[l] + stops * 2f32.log10());
|
||||
let exact = viewing.to_srgb(&paper.transmittance(paper.density_at(paper_log)));
|
||||
let approx = baked.apply_at(
|
||||
rgb,
|
||||
&Settings {
|
||||
print_exposure_ev: stops,
|
||||
..Settings::default()
|
||||
},
|
||||
);
|
||||
for c in 0..3 {
|
||||
worst = worst.max((exact[c] - approx[c]).abs());
|
||||
}
|
||||
}
|
||||
}
|
||||
assert!(
|
||||
worst < 1.0 / 255.0,
|
||||
"the print misses the spectral chain by {worst}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -550,5 +861,25 @@ mod tests {
|
||||
let baked = bake(&Recipe::new(&film, None));
|
||||
assert_eq!(baked.lut.len(), LUT_SIZE * LUT_SIZE * LUT_SIZE);
|
||||
assert_eq!(baked.curves.len(), CURVE_SAMPLES);
|
||||
assert_eq!(baked.curve_rows, 1);
|
||||
assert!(baked.paper.is_none());
|
||||
|
||||
// A print has a second lookup and a curve of its own; a development
|
||||
// series a row per push. Neither is inferred from the other.
|
||||
let negative = portra();
|
||||
let paper = endura();
|
||||
let printed = bake(&Recipe::new(&negative, Some(&paper)));
|
||||
let print = printed
|
||||
.paper
|
||||
.as_ref()
|
||||
.expect("a printed negative has a paper");
|
||||
assert_eq!(print.lut.len(), LUT_SIZE.pow(3));
|
||||
assert_eq!(print.curves.len(), CURVE_SAMPLES);
|
||||
|
||||
let pushable = profile(include_str!("../profiles/kodak_doublex.yaml"));
|
||||
let rows = bake(&Recipe::new(&pushable, None));
|
||||
assert_eq!(rows.curve_rows, pushable.development_times.len());
|
||||
assert_eq!(rows.push_stations.len(), rows.curve_rows);
|
||||
assert_eq!(rows.curves.len(), rows.curve_rows * CURVE_SAMPLES);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -680,15 +680,18 @@ fn film_tables() -> FilmTables {
|
||||
FilmTables {
|
||||
exposure_matrix: baked.exposure_matrix,
|
||||
curves: baked.curves.clone(),
|
||||
push_stations: baked.push_stations.clone(),
|
||||
curve_log_min: baked.curve_log_min,
|
||||
curve_log_max: baked.curve_log_max,
|
||||
lut: baked.lut.clone(),
|
||||
density_max: baked.density_max,
|
||||
lut_size: baked.lut_size,
|
||||
// Viewed directly: `STOCK` is baked without a paper above.
|
||||
paper: None,
|
||||
// Grain off. It is a per-pixel hash and would be measured; it is also
|
||||
// not part of every edit, and the chain being measured here is "every
|
||||
// operation active", not "every option of every operation".
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; dr_pipeline::ops::film_sim::FORMAT_COUNT],
|
||||
grain_density_max: [baked.density_max; 3],
|
||||
grain_uniformity: 0.97,
|
||||
}
|
||||
|
||||
+486
-9
@@ -124,6 +124,9 @@ pub struct AdjustPass {
|
||||
/// Cleared by any render that does not write it, so a stale intermediate
|
||||
/// cannot survive a change of image and be handed to a later detail chain.
|
||||
colour_key: Option<(u64, u32, u32)>,
|
||||
/// The source texels the fused pass read on an earlier frame, kept so a
|
||||
/// slider drag at fit reads them contiguously. See [`SampleCache`].
|
||||
sample: SampleCache,
|
||||
/// Fused dispatches actually encoded. Exposed so a test can see the reuse
|
||||
/// above happening rather than take it on trust.
|
||||
colour_dispatches: usize,
|
||||
@@ -147,6 +150,205 @@ struct Target {
|
||||
const FILM_FORMAT: wgpu::TextureFormat = wgpu::TextureFormat::Rgba32Float;
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// What a cached sample is valid for: the composer's
|
||||
/// [`ComposedShader::sample_key`], the source image, and the render size.
|
||||
type SampleKey = (u64, u64, u32, u32);
|
||||
|
||||
/// The largest render the sample cache is kept for, in pixels.
|
||||
///
|
||||
/// 4K and a little over, which is every develop view there is. An export
|
||||
/// renders a whole sensor once and gains nothing from a cache it will not
|
||||
/// read again; without a ceiling, two exports of the same frame in a row
|
||||
/// would park a full-resolution copy of it on the device.
|
||||
const SAMPLE_CACHE_MAX_PIXELS: u64 = 3840 * 2400;
|
||||
|
||||
/// How the fused dispatch gets its source colour this frame.
|
||||
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
|
||||
enum SampleUse {
|
||||
/// From the source, as it always did.
|
||||
Direct,
|
||||
/// From the source, and stored in the cache for the frames after.
|
||||
Write,
|
||||
/// From the cache.
|
||||
Read,
|
||||
}
|
||||
|
||||
/// TRACES: NFR-P5
|
||||
/// The fused pass's gather from the source, remembered across frames.
|
||||
///
|
||||
/// At fit, each output pixel of the fused pass reads one texel of a source
|
||||
/// three or four times its width, on a stride, and the memory system fetches
|
||||
/// the neighbours it skips along with it. On a 60 MP source that gather was
|
||||
/// most of the fused pass: 10.9 ms of a 2560 x 1600 frame against 3.8 ms for
|
||||
/// the same work reading contiguously (RTX 3050, clocks held down). But which
|
||||
/// texel a pixel reads depends on the framing and nothing else, and a slider
|
||||
/// drag does not move the framing. So the shader writes what it gathered to a
|
||||
/// render-sized texture on one frame and reads it back contiguously on every
|
||||
/// frame after, until the framing, the image or the size changes.
|
||||
///
|
||||
/// Bit-for-bit the same picture: the source is `rgba16float` and so is the
|
||||
/// cache, so the stored texel is the texel. Only the whole-texel sampling path
|
||||
/// takes part — [`ComposedShader::sample_key`] is `None` when the sample is
|
||||
/// interpolated.
|
||||
///
|
||||
/// **Written on the second frame with a key, not the first.** A drag of the
|
||||
/// crop or of a zoom changes the key on every frame, and a cache written then
|
||||
/// is never read — it would add a write per frame to exactly the gestures that
|
||||
/// can least afford one. Waiting for the key to repeat once costs a slider drag
|
||||
/// one uncached frame and costs a crop drag nothing.
|
||||
struct SampleCache {
|
||||
/// The cache itself, `rgba16float`, sampled and written as storage.
|
||||
target: Option<Target>,
|
||||
/// What `target` holds, once a dispatch has written it.
|
||||
holds: Option<SampleKey>,
|
||||
/// The key the previous fused dispatch had, cached or not.
|
||||
last: Option<SampleKey>,
|
||||
/// What the most recent plan decided. Read by the tests, which have no
|
||||
/// other way to tell a cached frame from an uncached one — the point being
|
||||
/// that the pictures are identical.
|
||||
last_use: SampleUse,
|
||||
/// Bound at `@binding(6)` when the cache is not being read.
|
||||
no_sampled: wgpu::TextureView,
|
||||
/// Bound at `@binding(7)` when the cache is not being written.
|
||||
no_sample_out: wgpu::TextureView,
|
||||
}
|
||||
|
||||
impl SampleCache {
|
||||
const FORMAT: wgpu::TextureFormat = DemosaicedImage::FORMAT;
|
||||
|
||||
fn new(ctx: &GpuContext) -> Self {
|
||||
let placeholder = |label, usage| {
|
||||
ctx.device
|
||||
.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some(label),
|
||||
size: wgpu::Extent3d {
|
||||
width: 1,
|
||||
height: 1,
|
||||
depth_or_array_layers: 1,
|
||||
},
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format: Self::FORMAT,
|
||||
usage,
|
||||
view_formats: &[],
|
||||
})
|
||||
.create_view(&Default::default())
|
||||
};
|
||||
Self {
|
||||
target: None,
|
||||
holds: None,
|
||||
last: None,
|
||||
last_use: SampleUse::Direct,
|
||||
no_sampled: placeholder("adjust-no-sampled", wgpu::TextureUsages::TEXTURE_BINDING),
|
||||
no_sample_out: placeholder(
|
||||
"adjust-no-sample-out",
|
||||
wgpu::TextureUsages::STORAGE_BINDING,
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
/// Decide how this dispatch samples, on the assumption that it will be
|
||||
/// submitted — call it after anything that can still fail.
|
||||
fn plan(
|
||||
&mut self,
|
||||
ctx: &GpuContext,
|
||||
source: &DemosaicedImage,
|
||||
shader: &ComposedShader,
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> SampleUse {
|
||||
self.last_use = self.decide(ctx, source, shader, width, height);
|
||||
self.last_use
|
||||
}
|
||||
|
||||
fn decide(
|
||||
&mut self,
|
||||
ctx: &GpuContext,
|
||||
source: &DemosaicedImage,
|
||||
shader: &ComposedShader,
|
||||
width: u32,
|
||||
height: u32,
|
||||
) -> SampleUse {
|
||||
let key = shader
|
||||
.sample_key
|
||||
.filter(|_| u64::from(width) * u64::from(height) <= SAMPLE_CACHE_MAX_PIXELS)
|
||||
.map(|k| (k, source.id(), width, height));
|
||||
let last = std::mem::replace(&mut self.last, key);
|
||||
let Some(key) = key else {
|
||||
return SampleUse::Direct;
|
||||
};
|
||||
if self.holds == Some(key) {
|
||||
return SampleUse::Read;
|
||||
}
|
||||
if last != Some(key) {
|
||||
return SampleUse::Direct;
|
||||
}
|
||||
|
||||
if !self
|
||||
.target
|
||||
.as_ref()
|
||||
.is_some_and(|t| t.width == width && t.height == height)
|
||||
{
|
||||
let texture = ctx.device.create_texture(&wgpu::TextureDescriptor {
|
||||
label: Some("adjust-sample-cache"),
|
||||
size: wgpu::Extent3d {
|
||||
width,
|
||||
height,
|
||||
depth_or_array_layers: 1,
|
||||
},
|
||||
mip_level_count: 1,
|
||||
sample_count: 1,
|
||||
dimension: wgpu::TextureDimension::D2,
|
||||
format: Self::FORMAT,
|
||||
usage: wgpu::TextureUsages::STORAGE_BINDING | wgpu::TextureUsages::TEXTURE_BINDING,
|
||||
view_formats: &[],
|
||||
});
|
||||
let view = texture.create_view(&Default::default());
|
||||
self.target = Some(Target {
|
||||
texture,
|
||||
view,
|
||||
width,
|
||||
height,
|
||||
});
|
||||
}
|
||||
// Marked as held now: the dispatch that writes it is submitted before
|
||||
// any that could read it, and one queue orders the two.
|
||||
self.holds = Some(key);
|
||||
SampleUse::Write
|
||||
}
|
||||
|
||||
/// Write the flags for `usage` into a fused uniform block.
|
||||
fn flag(usage: SampleUse, uniforms: &mut [f32]) {
|
||||
let o = dr_pipeline::SAMPLE_CACHE_UNIFORM_OFFSET;
|
||||
uniforms[o] = if usage == SampleUse::Read { 1.0 } else { 0.0 };
|
||||
uniforms[o + 1] = if usage == SampleUse::Write { 1.0 } else { 0.0 };
|
||||
}
|
||||
|
||||
/// The views for `@binding(6)` and `@binding(7)`.
|
||||
fn views(&self, usage: SampleUse) -> (wgpu::TextureView, wgpu::TextureView) {
|
||||
let cache = || {
|
||||
self.target
|
||||
.as_ref()
|
||||
.expect("planned with a target")
|
||||
.view
|
||||
.clone()
|
||||
};
|
||||
match usage {
|
||||
SampleUse::Direct => (self.no_sampled.clone(), self.no_sample_out.clone()),
|
||||
SampleUse::Write => (self.no_sampled.clone(), cache()),
|
||||
SampleUse::Read => (cache(), self.no_sample_out.clone()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Forget everything; the next render starts over.
|
||||
fn release(&mut self) {
|
||||
self.target = None;
|
||||
self.holds = None;
|
||||
self.last = None;
|
||||
}
|
||||
}
|
||||
|
||||
/// A baked film stock, resident on the GPU.
|
||||
struct FilmTextures {
|
||||
curves: wgpu::TextureView,
|
||||
@@ -180,9 +382,23 @@ fn film_key(t: &dr_pipeline::ops::FilmTables) -> u64 {
|
||||
t.curve_log_max,
|
||||
t.density_max,
|
||||
t.lut_size as f32,
|
||||
t.curves.len() as f32,
|
||||
t.lut.len() as f32,
|
||||
] {
|
||||
mix(v.to_bits());
|
||||
}
|
||||
for v in &t.push_stations {
|
||||
mix(v.to_bits());
|
||||
}
|
||||
// The paper's balance is left out on purpose: it reaches the shader as
|
||||
// uniforms, not texels, and it is what moves when the photograph's
|
||||
// exposure does — keying on it would re-upload a megabyte per tick of a
|
||||
// slider that changes three floats.
|
||||
if let Some(p) = &t.paper {
|
||||
for v in [p.log_min, p.log_max, p.density_max] {
|
||||
mix(v.to_bits());
|
||||
}
|
||||
}
|
||||
for e in t.lut.iter().step_by(8).chain(t.curves.iter().step_by(8)) {
|
||||
mix(e[0].to_bits() ^ e[1].to_bits().rotate_left(11) ^ e[2].to_bits().rotate_left(22));
|
||||
}
|
||||
@@ -229,12 +445,16 @@ impl AdjustPass {
|
||||
return;
|
||||
}
|
||||
|
||||
// One row per curve — the film's at each measured push, then the
|
||||
// paper's — and one cube per stage stacked in depth. The shader reads
|
||||
// the layout from `FilmTables`' uniforms, not from these sizes.
|
||||
let samples = dr_pipeline::ops::film_sim::CURVE_SAMPLES as u32;
|
||||
let curves = self.upload_film(
|
||||
"adjust-film-curves",
|
||||
wgpu::TextureDimension::D2,
|
||||
wgpu::Extent3d {
|
||||
width: t.curves.len() as u32,
|
||||
height: 1,
|
||||
width: samples,
|
||||
height: t.curves.len() as u32 / samples,
|
||||
depth_or_array_layers: 1,
|
||||
},
|
||||
&to_rgba(&t.curves),
|
||||
@@ -246,7 +466,7 @@ impl AdjustPass {
|
||||
wgpu::Extent3d {
|
||||
width: n,
|
||||
height: n,
|
||||
depth_or_array_layers: n,
|
||||
depth_or_array_layers: t.lut.len() as u32 / (n * n),
|
||||
},
|
||||
&to_rgba(&t.lut),
|
||||
);
|
||||
@@ -440,6 +660,7 @@ impl AdjustPass {
|
||||
camera_pipeline_layout,
|
||||
camera_target: None,
|
||||
colour_key: None,
|
||||
sample: SampleCache::new(ctx),
|
||||
colour_dispatches: 0,
|
||||
detail_dispatches: 0,
|
||||
}
|
||||
@@ -537,6 +758,30 @@ impl AdjustPass {
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
// The sample cache, read and written. Present in every
|
||||
// layout for the reason the masks are, and bound to
|
||||
// placeholders whenever the flags leave it alone. See
|
||||
// `SampleCache`.
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
binding: 6,
|
||||
visibility: wgpu::ShaderStages::COMPUTE,
|
||||
ty: wgpu::BindingType::Texture {
|
||||
sample_type: wgpu::TextureSampleType::Float { filterable: false },
|
||||
view_dimension: wgpu::TextureViewDimension::D2,
|
||||
multisampled: false,
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
binding: 7,
|
||||
visibility: wgpu::ShaderStages::COMPUTE,
|
||||
ty: wgpu::BindingType::StorageTexture {
|
||||
access: wgpu::StorageTextureAccess::WriteOnly,
|
||||
format: SampleCache::FORMAT,
|
||||
view_dimension: wgpu::TextureViewDimension::D2,
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
],
|
||||
})
|
||||
}
|
||||
@@ -716,7 +961,15 @@ impl AdjustPass {
|
||||
let (width, height) = (width.max(1), height.max(1));
|
||||
self.ensure_target(width, height);
|
||||
|
||||
let uniforms = Self::fused_uniforms(source, shader);
|
||||
// Compile first: `pipeline` takes &mut self, and the sample cache's
|
||||
// plan below assumes the dispatch it plans for is submitted, so nothing
|
||||
// after it may fail.
|
||||
let _ = self.pipeline(shader)?;
|
||||
|
||||
let mut uniforms = Self::fused_uniforms(source, shader);
|
||||
let sampling = self.sample.plan(&self.ctx, source, shader, width, height);
|
||||
SampleCache::flag(sampling, &mut uniforms);
|
||||
let (sampled, sample_out) = self.sample.views(sampling);
|
||||
|
||||
let params_buf = self
|
||||
.ctx
|
||||
@@ -727,8 +980,6 @@ impl AdjustPass {
|
||||
usage: wgpu::BufferUsages::UNIFORM,
|
||||
});
|
||||
|
||||
// Borrow order: compile first, since `pipeline` takes &mut self.
|
||||
let _ = self.pipeline(shader)?;
|
||||
let pipeline = self
|
||||
.cache
|
||||
.get(&shader.structure_hash)
|
||||
@@ -768,6 +1019,14 @@ impl AdjustPass {
|
||||
binding: 5,
|
||||
resource: wgpu::BindingResource::TextureView(self.film_lut_view()),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 6,
|
||||
resource: wgpu::BindingResource::TextureView(&sampled),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 7,
|
||||
resource: wgpu::BindingResource::TextureView(&sample_out),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
@@ -885,7 +1144,12 @@ impl AdjustPass {
|
||||
label: Some("adjust-detail-encoder"),
|
||||
});
|
||||
|
||||
let mut sampling = SampleUse::Direct;
|
||||
if !reuse {
|
||||
let mut uniforms = uniforms;
|
||||
sampling = self.sample.plan(&self.ctx, source, shader, width, height);
|
||||
SampleCache::flag(sampling, &mut uniforms);
|
||||
let (sampled, sample_out) = self.sample.views(sampling);
|
||||
let params_buf =
|
||||
self.ctx
|
||||
.device
|
||||
@@ -927,6 +1191,14 @@ impl AdjustPass {
|
||||
binding: 5,
|
||||
resource: wgpu::BindingResource::TextureView(&film_lut),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 6,
|
||||
resource: wgpu::BindingResource::TextureView(&sampled),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 7,
|
||||
resource: wgpu::BindingResource::TextureView(&sample_out),
|
||||
},
|
||||
],
|
||||
});
|
||||
let pipeline = self
|
||||
@@ -954,9 +1226,20 @@ impl AdjustPass {
|
||||
.expect("ensured above")
|
||||
.view
|
||||
.clone();
|
||||
let ran = self
|
||||
let ran = match self
|
||||
.detail
|
||||
.encode(&mut enc, detail, &target_view, width, height)?;
|
||||
.encode(&mut enc, detail, &target_view, width, height)
|
||||
{
|
||||
Ok(ran) => ran,
|
||||
Err(e) => {
|
||||
// Nothing is submitted, so a cache this frame was to write
|
||||
// holds nothing, and must not be read as though it did.
|
||||
if sampling == SampleUse::Write {
|
||||
self.sample.release();
|
||||
}
|
||||
return Err(e);
|
||||
}
|
||||
};
|
||||
self.ctx.queue.submit(Some(enc.finish()));
|
||||
self.detail_dispatches += ran;
|
||||
self.colour_key = Some((key, width, height));
|
||||
@@ -1090,6 +1373,7 @@ impl AdjustPass {
|
||||
self.detail.release_caches();
|
||||
self.targets = [None, None];
|
||||
self.colour_key = None;
|
||||
self.sample.release();
|
||||
}
|
||||
|
||||
/// How many distinct pipelines are compiled. Exposed for tests asserting
|
||||
@@ -1244,6 +1528,16 @@ impl AdjustPass {
|
||||
binding: 5,
|
||||
resource: wgpu::BindingResource::TextureView(self.film_lut_view()),
|
||||
},
|
||||
// The sample cache is the display's; a camera-space tap
|
||||
// reads its source directly (its flags are zero).
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 6,
|
||||
resource: wgpu::BindingResource::TextureView(&self.sample.no_sampled),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 7,
|
||||
resource: wgpu::BindingResource::TextureView(&self.sample.no_sample_out),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
@@ -1898,6 +2192,62 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-20
|
||||
#[test]
|
||||
fn a_keystone_reshapes_the_frame_without_exposing_a_corner() {
|
||||
// The shader half of perspective correction, end to end. A top-bright
|
||||
// frame with a full vertical keystone spreads its top across the
|
||||
// output, so the bright half reaches further down than the middle;
|
||||
// and since the frame is mapped onto a trapezoid *inside* the source,
|
||||
// no corner is left without a pixel behind it.
|
||||
let Some(ctx) = ctx() else { return };
|
||||
let mut pass = AdjustPass::new(&ctx);
|
||||
let img = split_image(&ctx, true);
|
||||
|
||||
let plain = EditGraph::default_chain().compose();
|
||||
let tex = pass.render(&img, &plain, 32, 32).expect("render");
|
||||
let below_middle = read_pixel(&ctx, tex, 16, 19)[0];
|
||||
assert!(
|
||||
below_middle < 90,
|
||||
"unkeyed, row 19 is the dark half: {below_middle}"
|
||||
);
|
||||
|
||||
let mut g = EditGraph::default_chain();
|
||||
g.set_param(
|
||||
dr_pipeline::framing::ID,
|
||||
dr_pipeline::framing::KEYSTONE_V,
|
||||
100.0,
|
||||
);
|
||||
g.set_param(
|
||||
dr_pipeline::framing::ID,
|
||||
dr_pipeline::framing::KEYSTONE_H,
|
||||
100.0,
|
||||
);
|
||||
let shader = g.compose();
|
||||
let tex = pass.render(&img, &shader, 32, 32).expect("render");
|
||||
|
||||
for (x, y) in [(0, 0), (31, 0), (0, 31), (31, 31)] {
|
||||
assert_ne!(
|
||||
read_pixel(&ctx, tex, x, y),
|
||||
[0, 0, 0, 255],
|
||||
"corner ({x},{y}) has no source pixel behind it"
|
||||
);
|
||||
}
|
||||
|
||||
let mut g = EditGraph::default_chain();
|
||||
g.set_param(
|
||||
dr_pipeline::framing::ID,
|
||||
dr_pipeline::framing::KEYSTONE_V,
|
||||
100.0,
|
||||
);
|
||||
let tex = pass.render(&img, &g.compose(), 32, 32).expect("render");
|
||||
let keyed = read_pixel(&ctx, tex, 16, 19)[0];
|
||||
assert!(
|
||||
keyed > 128,
|
||||
"the spread top half must reach row 19: {keyed}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn dragging_the_crop_does_not_recompile() {
|
||||
// The cache contract for framing, which is what makes an interactive
|
||||
@@ -1971,7 +2321,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; N * N * N],
|
||||
density_max: 3.0,
|
||||
lut_size: N,
|
||||
grain_particles: [0.0; 3],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_particles: [[0.0; 3]; dr_pipeline::ops::film_sim::FORMAT_COUNT],
|
||||
grain_density_max: [3.0; 3],
|
||||
grain_uniformity: 0.97,
|
||||
}
|
||||
@@ -2568,6 +2920,131 @@ mod tests {
|
||||
|
||||
/// A flat RGBA8 image on the JPEG path — already gamma-encoded, as a
|
||||
/// decoded JPEG is.
|
||||
/// A frame with a different value at every pixel, several times the size
|
||||
/// of the renders below, so that a fit view reads it on a stride and a
|
||||
/// texel read from the wrong place cannot go unnoticed.
|
||||
fn busy_image(ctx: &GpuContext) -> DemosaicedImage {
|
||||
let (w, h) = (97u32, 61u32);
|
||||
let mut data = Vec::with_capacity((w * h * 4) as usize);
|
||||
for y in 0..h {
|
||||
for x in 0..w {
|
||||
let n = (x.wrapping_mul(2_654_435_761) ^ y.wrapping_mul(1_640_531_527)) >> 7;
|
||||
data.extend_from_slice(&[n as u8, (n >> 8) as u8, (x * 2 + y) as u8, 255]);
|
||||
}
|
||||
}
|
||||
DemosaicedImage::from_rgba8(ctx, &data, w, h).expect("upload")
|
||||
}
|
||||
|
||||
/// Render `g` with a pass that has never seen it, which reads the source
|
||||
/// directly by construction: the reference a cached frame must equal.
|
||||
fn fresh(ctx: &GpuContext, img: &DemosaicedImage, g: &EditGraph) -> Vec<u8> {
|
||||
let mut pass = AdjustPass::new(ctx);
|
||||
let detail = g.compose_detail(img.size(), (23, 15));
|
||||
pass.render_detailed(img, &g.compose(), 23, 15, None, &detail, 1)
|
||||
.expect("render");
|
||||
assert_eq!(pass.sample.last_use, SampleUse::Direct);
|
||||
pass.export_pixels().expect("read").0
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cached_sample_is_the_same_picture() {
|
||||
// TRACES: NFR-P5
|
||||
// The sample cache's whole claim: a frame that reads the source through
|
||||
// it is bit-for-bit the frame that reads the source directly. Walked
|
||||
// through each state — direct, writing, reading, reading after a
|
||||
// slider moved, and direct again once the framing moves — against a
|
||||
// fresh pass each time.
|
||||
let Some(ctx) = ctx() else { return };
|
||||
let img = busy_image(&ctx);
|
||||
let mut pass = AdjustPass::new(&ctx);
|
||||
let mut g = EditGraph::default_chain();
|
||||
|
||||
let frame = |pass: &mut AdjustPass, g: &EditGraph, key: u64| {
|
||||
let detail = g.compose_detail(img.size(), (23, 15));
|
||||
pass.render_detailed(&img, &g.compose(), 23, 15, None, &detail, key)
|
||||
.expect("render");
|
||||
(pass.sample.last_use, pass.export_pixels().expect("read").0)
|
||||
};
|
||||
|
||||
for (i, (expected, value)) in [
|
||||
(SampleUse::Direct, 0.3),
|
||||
(SampleUse::Write, 0.4),
|
||||
(SampleUse::Read, 0.5),
|
||||
(SampleUse::Read, -0.7),
|
||||
]
|
||||
.into_iter()
|
||||
.enumerate()
|
||||
{
|
||||
g.set_param(exposure::ID, exposure::EXPOSURE, value);
|
||||
let (used, pixels) = frame(&mut pass, &g, i as u64);
|
||||
assert_eq!(used, expected, "frame {i}");
|
||||
assert_eq!(pixels, fresh(&ctx, &img, &g), "frame {i} ({used:?})");
|
||||
}
|
||||
|
||||
// A neighbourhood operation: the fused pass writes the linear
|
||||
// intermediate instead, through the same sampling.
|
||||
g.set_param(
|
||||
dr_pipeline::ops::noise_reduction::ID,
|
||||
dr_pipeline::ops::noise_reduction::CHROMA,
|
||||
60.0,
|
||||
);
|
||||
for i in 10..13 {
|
||||
g.set_param(exposure::ID, exposure::EXPOSURE, i as f32 * 0.01);
|
||||
let (used, pixels) = frame(&mut pass, &g, i);
|
||||
assert_eq!(used, SampleUse::Read, "detail frame {i}");
|
||||
assert_eq!(pixels, fresh(&ctx, &img, &g), "detail frame {i}");
|
||||
}
|
||||
|
||||
// The framing moves: what was cached is for the old framing.
|
||||
g.set_param(dr_pipeline::framing::ID, dr_pipeline::framing::CROP_W, 0.6);
|
||||
let (used, pixels) = frame(&mut pass, &g, 20);
|
||||
assert_eq!(used, SampleUse::Direct, "a new framing reads directly");
|
||||
assert_eq!(pixels, fresh(&ctx, &img, &g));
|
||||
let (used, _) = frame(&mut pass, &g, 21);
|
||||
assert_eq!(used, SampleUse::Write, "and caches once it holds still");
|
||||
let (used, pixels) = frame(&mut pass, &g, 22);
|
||||
assert_eq!(used, SampleUse::Read);
|
||||
assert_eq!(pixels, fresh(&ctx, &img, &g));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_cache_is_not_read_for_another_image_or_size() {
|
||||
// The key is the composer's half plus the two things only this side
|
||||
// knows. A second photograph with the same edit and the same framing
|
||||
// must not be shown the first one's texels.
|
||||
let Some(ctx) = ctx() else { return };
|
||||
let (a, b) = (busy_image(&ctx), grey_image(&ctx, 8000));
|
||||
let mut pass = AdjustPass::new(&ctx);
|
||||
let shader = EditGraph::default_chain().compose();
|
||||
for _ in 0..3 {
|
||||
pass.render(&a, &shader, 23, 15).expect("render");
|
||||
}
|
||||
assert_eq!(pass.sample.last_use, SampleUse::Read);
|
||||
|
||||
pass.render(&b, &shader, 23, 15).expect("render");
|
||||
assert_eq!(pass.sample.last_use, SampleUse::Direct, "another image");
|
||||
pass.render(&b, &shader, 23, 15).expect("render");
|
||||
pass.render(&b, &shader, 24, 15).expect("render");
|
||||
assert_eq!(pass.sample.last_use, SampleUse::Direct, "another size");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_straightened_frame_samples_directly() {
|
||||
// Interpolated: the sample is a blend of four texels, which the cache's
|
||||
// format could not hold exactly, so the composer offers no key.
|
||||
let Some(ctx) = ctx() else { return };
|
||||
let img = busy_image(&ctx);
|
||||
let mut pass = AdjustPass::new(&ctx);
|
||||
let mut g = EditGraph::default_chain();
|
||||
g.set_param(dr_pipeline::framing::ID, dr_pipeline::framing::ANGLE, 3.0);
|
||||
let shader = g.compose();
|
||||
assert!(shader.sample_key.is_none());
|
||||
for _ in 0..3 {
|
||||
pass.render(&img, &shader, 23, 15).expect("render");
|
||||
assert_eq!(pass.sample.last_use, SampleUse::Direct);
|
||||
}
|
||||
}
|
||||
|
||||
fn jpeg_image(ctx: &GpuContext, rgb: [u8; 3]) -> DemosaicedImage {
|
||||
let size = 16u32;
|
||||
let mut data = Vec::with_capacity((size * size) as usize * 4);
|
||||
|
||||
+279
-1
@@ -57,6 +57,32 @@ struct XTransParams {
|
||||
tile: [u32; 4],
|
||||
}
|
||||
|
||||
/// Uniform block for the hot-pixel repair. Layout must match
|
||||
/// `hot_pixels.wgsl`.
|
||||
///
|
||||
/// One block for both colour filter arrays: the repair asks only "which
|
||||
/// photosites share this one's colour", and a 6×6 tile answers that for a
|
||||
/// Bayer cell as well as for X-Trans.
|
||||
#[repr(C)]
|
||||
#[derive(Copy, Clone, Debug, bytemuck::Pod, bytemuck::Zeroable)]
|
||||
struct HotPixelParams {
|
||||
crop_x: u32,
|
||||
crop_y: u32,
|
||||
width: u32,
|
||||
height: u32,
|
||||
stride: u32,
|
||||
words: u32,
|
||||
row_invocations: u32,
|
||||
samples: u32,
|
||||
black: [f32; 4],
|
||||
inv_range: [f32; 4],
|
||||
tile: [u32; 4],
|
||||
}
|
||||
|
||||
/// The repair's workgroup width. Must match `@workgroup_size` in
|
||||
/// `hot_pixels.wgsl`.
|
||||
const HOT_PIXEL_GROUP: u32 = 64;
|
||||
|
||||
/// A demosaiced image living on the GPU.
|
||||
///
|
||||
/// RGBA16Float, scene-referred, camera colour space. This is the input every
|
||||
@@ -95,6 +121,15 @@ pub struct DemosaicedImage {
|
||||
base_curve: BaseCurve,
|
||||
/// Whether the texture holds gamma-encoded rather than linear values.
|
||||
non_linear: bool,
|
||||
/// Which upload this is, unique for the life of the process. See
|
||||
/// [`Self::id`].
|
||||
id: u64,
|
||||
}
|
||||
|
||||
/// The next [`DemosaicedImage::id`].
|
||||
fn next_image_id() -> u64 {
|
||||
static NEXT: std::sync::atomic::AtomicU64 = std::sync::atomic::AtomicU64::new(1);
|
||||
NEXT.fetch_add(1, std::sync::atomic::Ordering::Relaxed)
|
||||
}
|
||||
|
||||
impl DemosaicedImage {
|
||||
@@ -112,6 +147,18 @@ impl DemosaicedImage {
|
||||
(self.width, self.height)
|
||||
}
|
||||
|
||||
/// Which texture this is, as a number that is never reused.
|
||||
///
|
||||
/// For a cache that has to know it is still looking at the same pixels
|
||||
/// (`AdjustPass`'s sample cache) without holding the texture alive to find
|
||||
/// out: keeping a handle would keep half a gigabyte of a closed photograph
|
||||
/// on the device, and comparing addresses would mistake a new upload for an
|
||||
/// old one the moment the allocator reused the slot. A texture here is
|
||||
/// never written after it is built, so the same id is the same pixels.
|
||||
pub(crate) fn id(&self) -> u64 {
|
||||
self.id
|
||||
}
|
||||
|
||||
/// Camera RGB → linear sRGB, row-major. Identity where the body is
|
||||
/// uncalibrated, so the image renders uncalibrated rather than black.
|
||||
pub fn color_matrix(&self) -> [f32; 9] {
|
||||
@@ -246,6 +293,7 @@ impl DemosaicedImage {
|
||||
// the highlights of an image that was already finished.
|
||||
base_curve: BaseCurve::IDENTITY,
|
||||
non_linear: true,
|
||||
id: next_image_id(),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -322,6 +370,7 @@ impl DemosaicedImage {
|
||||
as_shot_wb: [raw.wb_coeffs[0], raw.wb_coeffs[1], raw.wb_coeffs[2]],
|
||||
base_curve: raw.base_curve,
|
||||
non_linear: false,
|
||||
id: next_image_id(),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -414,6 +463,8 @@ pub struct Demosaicer {
|
||||
pipeline: wgpu::ComputePipeline,
|
||||
xtrans_pipeline: wgpu::ComputePipeline,
|
||||
bind_group_layout: wgpu::BindGroupLayout,
|
||||
hot_pixel_pipeline: wgpu::ComputePipeline,
|
||||
hot_pixel_layout: wgpu::BindGroupLayout,
|
||||
}
|
||||
|
||||
impl Demosaicer {
|
||||
@@ -504,11 +555,15 @@ impl Demosaicer {
|
||||
cache: None,
|
||||
});
|
||||
|
||||
let (hot_pixel_pipeline, hot_pixel_layout) = hot_pixel_pipeline(ctx);
|
||||
|
||||
Ok(Self {
|
||||
ctx: ctx.clone(),
|
||||
pipeline,
|
||||
xtrans_pipeline,
|
||||
bind_group_layout,
|
||||
hot_pixel_pipeline,
|
||||
hot_pixel_layout,
|
||||
})
|
||||
}
|
||||
|
||||
@@ -535,8 +590,12 @@ impl Demosaicer {
|
||||
// the buffer outlive the `if` that chose them.
|
||||
let bayer_params;
|
||||
let xtrans_params;
|
||||
// Kept for the hot-pixel repair: finding the X-Trans phase reads the
|
||||
// whole frame on the CPU, and once per photograph is enough.
|
||||
let mut xtrans_tile = None;
|
||||
let (pipeline, params_bytes) = if raw.cfa_pattern.is_xtrans() {
|
||||
xtrans_params = xtrans_params_for(raw, width, height);
|
||||
xtrans_tile = Some(xtrans_params.tile);
|
||||
(&self.xtrans_pipeline, bytemuck::bytes_of(&xtrans_params))
|
||||
} else {
|
||||
let pattern = match raw.cfa_pattern {
|
||||
@@ -575,6 +634,64 @@ impl Demosaicer {
|
||||
usage: wgpu::BufferUsages::STORAGE,
|
||||
});
|
||||
|
||||
// TRACES: FR-RAW-3
|
||||
// The mosaic the demosaic actually reads: the readout with its hot and
|
||||
// dead photosites repaired. A second buffer rather than in place,
|
||||
// because every photosite's verdict reads its neighbours' originals.
|
||||
let repaired = self.ctx.device.create_buffer(&wgpu::BufferDescriptor {
|
||||
label: Some("raw-repaired"),
|
||||
size: raw_buf.size(),
|
||||
usage: wgpu::BufferUsages::STORAGE,
|
||||
mapped_at_creation: false,
|
||||
});
|
||||
let words = packed.len() as u32;
|
||||
let groups = words.div_ceil(HOT_PIXEL_GROUP).max(1);
|
||||
// A 24 MP readout is 190,000 workgroups, past the 65,535 one
|
||||
// dispatch dimension may hold, so the grid folds into rows.
|
||||
let groups_x = groups.min(
|
||||
self.ctx
|
||||
.device
|
||||
.limits()
|
||||
.max_compute_workgroups_per_dimension,
|
||||
);
|
||||
let groups_y = groups.div_ceil(groups_x);
|
||||
let hot_params = hot_pixel_params(
|
||||
raw,
|
||||
(width, height),
|
||||
words,
|
||||
groups_x * HOT_PIXEL_GROUP,
|
||||
xtrans_tile,
|
||||
);
|
||||
let hot_params_buf =
|
||||
self.ctx
|
||||
.device
|
||||
.create_buffer_init(&wgpu::util::BufferInitDescriptor {
|
||||
label: Some("hot-pixel-params"),
|
||||
contents: bytemuck::bytes_of(&hot_params),
|
||||
usage: wgpu::BufferUsages::UNIFORM,
|
||||
});
|
||||
let hot_bind_group = self
|
||||
.ctx
|
||||
.device
|
||||
.create_bind_group(&wgpu::BindGroupDescriptor {
|
||||
label: Some("hot-pixel-bg"),
|
||||
layout: &self.hot_pixel_layout,
|
||||
entries: &[
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 0,
|
||||
resource: raw_buf.as_entire_binding(),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 1,
|
||||
resource: hot_params_buf.as_entire_binding(),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 2,
|
||||
resource: repaired.as_entire_binding(),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
let params_buf = self
|
||||
.ctx
|
||||
.device
|
||||
@@ -613,7 +730,7 @@ impl Demosaicer {
|
||||
entries: &[
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 0,
|
||||
resource: raw_buf.as_entire_binding(),
|
||||
resource: repaired.as_entire_binding(),
|
||||
},
|
||||
wgpu::BindGroupEntry {
|
||||
binding: 1,
|
||||
@@ -632,6 +749,18 @@ impl Demosaicer {
|
||||
.create_command_encoder(&wgpu::CommandEncoderDescriptor {
|
||||
label: Some("demosaic-encoder"),
|
||||
});
|
||||
// Two passes in one submission. wgpu orders a storage write in one
|
||||
// pass before a read of the same buffer in the next, so the demosaic
|
||||
// sees every repair.
|
||||
{
|
||||
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
|
||||
label: Some("hot-pixel-pass"),
|
||||
timestamp_writes: None,
|
||||
});
|
||||
pass.set_pipeline(&self.hot_pixel_pipeline);
|
||||
pass.set_bind_group(0, &hot_bind_group, &[]);
|
||||
pass.dispatch_workgroups(groups_x, groups_y, 1);
|
||||
}
|
||||
{
|
||||
let mut pass = enc.begin_compute_pass(&wgpu::ComputePassDescriptor {
|
||||
label: Some("demosaic-pass"),
|
||||
@@ -660,6 +789,7 @@ impl Demosaicer {
|
||||
// normalises against black and white levels and applies no
|
||||
// transfer function.
|
||||
non_linear: false,
|
||||
id: next_image_id(),
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -936,6 +1066,136 @@ fn detect_xtrans_phase(raw: &RawImage) -> (u32, u32) {
|
||||
|
||||
/// TRACES: FR-RAW-5
|
||||
/// Everything the X-Trans shader needs about one image.
|
||||
/// TRACES: FR-RAW-3
|
||||
/// The hot-pixel repair's pipeline and its three bindings: the readout, the
|
||||
/// uniform block, and the repaired copy it writes.
|
||||
fn hot_pixel_pipeline(ctx: &GpuContext) -> (wgpu::ComputePipeline, wgpu::BindGroupLayout) {
|
||||
let shader = ctx
|
||||
.device
|
||||
.create_shader_module(wgpu::ShaderModuleDescriptor {
|
||||
label: Some("hot-pixels"),
|
||||
source: wgpu::ShaderSource::Wgsl(include_str!("shaders/hot_pixels.wgsl").into()),
|
||||
});
|
||||
let storage = |binding, read_only| wgpu::BindGroupLayoutEntry {
|
||||
binding,
|
||||
visibility: wgpu::ShaderStages::COMPUTE,
|
||||
ty: wgpu::BindingType::Buffer {
|
||||
ty: wgpu::BufferBindingType::Storage { read_only },
|
||||
has_dynamic_offset: false,
|
||||
min_binding_size: None,
|
||||
},
|
||||
count: None,
|
||||
};
|
||||
let layout = ctx
|
||||
.device
|
||||
.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
|
||||
label: Some("hot-pixel-bgl"),
|
||||
entries: &[
|
||||
storage(0, true),
|
||||
wgpu::BindGroupLayoutEntry {
|
||||
binding: 1,
|
||||
visibility: wgpu::ShaderStages::COMPUTE,
|
||||
ty: wgpu::BindingType::Buffer {
|
||||
ty: wgpu::BufferBindingType::Uniform,
|
||||
has_dynamic_offset: false,
|
||||
min_binding_size: None,
|
||||
},
|
||||
count: None,
|
||||
},
|
||||
storage(2, false),
|
||||
],
|
||||
});
|
||||
let pipeline_layout = ctx
|
||||
.device
|
||||
.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
|
||||
label: Some("hot-pixel-layout"),
|
||||
bind_group_layouts: &[Some(&layout)],
|
||||
immediate_size: 0,
|
||||
});
|
||||
let pipeline = ctx
|
||||
.device
|
||||
.create_compute_pipeline(&wgpu::ComputePipelineDescriptor {
|
||||
label: Some("hot-pixel-pipeline"),
|
||||
layout: Some(&pipeline_layout),
|
||||
module: &shader,
|
||||
entry_point: Some("main"),
|
||||
compilation_options: Default::default(),
|
||||
cache: None,
|
||||
});
|
||||
(pipeline, layout)
|
||||
}
|
||||
|
||||
/// The colour of each position of a Bayer cell, row-major, for the pattern
|
||||
/// the decoder reported: 0=R, 1=G, 2=B. `None` for anything that is not a
|
||||
/// 2×2 pattern.
|
||||
fn bayer_cell(pattern: CfaPattern) -> Option<[u32; 4]> {
|
||||
match pattern {
|
||||
CfaPattern::Rggb => Some([0, 1, 1, 2]),
|
||||
CfaPattern::Bggr => Some([2, 1, 1, 0]),
|
||||
CfaPattern::Grbg => Some([1, 0, 2, 1]),
|
||||
CfaPattern::Gbrg => Some([1, 2, 0, 1]),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
/// A Bayer cell as the 6×6 sensor-anchored tile the repair indexes.
|
||||
///
|
||||
/// The decoder's pattern is phased for the *crop* origin, and the tile is
|
||||
/// indexed by sensor coordinate, so each position is shifted by the crop.
|
||||
/// Six is even, so a column's parity modulo 6 is its parity outright and the
|
||||
/// cell repeats cleanly.
|
||||
fn pack_bayer_tile(cell: [u32; 4], crop_x: u32, crop_y: u32) -> [u32; 4] {
|
||||
let mut out = [0u32; 4];
|
||||
for row in 0..6u32 {
|
||||
for col in 0..6u32 {
|
||||
let i = (((row + crop_y) & 1) * 2 + ((col + crop_x) & 1)) as usize;
|
||||
out[(row >> 1) as usize] |= cell[i] << ((row & 1) * 12 + col * 2);
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// The repair's uniforms for one readout.
|
||||
///
|
||||
/// `xtrans_tile` is the tile the X-Trans demosaic was given, when it was one;
|
||||
/// anything else must be a Bayer pattern, which `run` has already checked.
|
||||
fn hot_pixel_params(
|
||||
raw: &RawImage,
|
||||
(width, height): (u32, u32),
|
||||
words: u32,
|
||||
row_invocations: u32,
|
||||
xtrans_tile: Option<[u32; 4]>,
|
||||
) -> HotPixelParams {
|
||||
let (black, inv_range, tile) = match (xtrans_tile, bayer_cell(raw.cfa_pattern)) {
|
||||
(Some(tile), _) => {
|
||||
let (black, inv_range) = xtrans_levels(raw);
|
||||
([black; 4], [inv_range; 4], tile)
|
||||
}
|
||||
(None, Some(cell)) => (
|
||||
black_per_cell(raw),
|
||||
inv_range_per_cell(raw),
|
||||
pack_bayer_tile(cell, raw.crop.x, raw.crop.y),
|
||||
),
|
||||
// Not reached from `run`, which refuses any other pattern before
|
||||
// this. A zero tile judges every photosite against all of its
|
||||
// neighbours, which is right for a sensor with no colour filter.
|
||||
(None, None) => (black_per_cell(raw), inv_range_per_cell(raw), [0; 4]),
|
||||
};
|
||||
HotPixelParams {
|
||||
crop_x: raw.crop.x,
|
||||
crop_y: raw.crop.y,
|
||||
width,
|
||||
height,
|
||||
stride: raw.width,
|
||||
words,
|
||||
row_invocations,
|
||||
samples: raw.data.len() as u32,
|
||||
black,
|
||||
inv_range,
|
||||
tile,
|
||||
}
|
||||
}
|
||||
|
||||
fn xtrans_params_for(raw: &RawImage, width: u32, height: u32) -> XTransParams {
|
||||
let (black, inv_range) = xtrans_levels(raw);
|
||||
let wb = wb_gains(raw);
|
||||
@@ -970,6 +1230,24 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
/// The repair's tile is indexed by sensor coordinate, the decoder's
|
||||
/// pattern by crop coordinate. A crop at an odd origin must shift one
|
||||
/// into the other, or the repair compares red with green.
|
||||
#[test]
|
||||
fn the_bayer_tile_is_anchored_to_the_sensor_not_the_crop() {
|
||||
let cell = bayer_cell(CfaPattern::Rggb).unwrap();
|
||||
let colour = |tile: [u32; 4], x: u32, y: u32| {
|
||||
(tile[((y % 6) >> 1) as usize] >> (((y % 6) & 1) * 12 + (x % 6) * 2)) & 3
|
||||
};
|
||||
for (cx, cy) in [(0, 0), (1, 0), (0, 1), (1, 1), (7, 4)] {
|
||||
let tile = pack_bayer_tile(cell, cx, cy);
|
||||
// Red is the crop's first photosite, wherever the crop starts.
|
||||
assert_eq!(colour(tile, cx, cy), 0, "crop at ({cx}, {cy})");
|
||||
assert_eq!(colour(tile, cx + 1, cy + 1), 2, "crop at ({cx}, {cy})");
|
||||
assert_eq!(colour(tile, cx + 1, cy), 1, "crop at ({cx}, {cy})");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unclamped_half_keeps_shadows_signs_and_highlights() {
|
||||
// A 14-bit LSB, normalised: subnormal in f16, and must not be zero.
|
||||
|
||||
@@ -470,7 +470,7 @@ impl FocusPeakPass {
|
||||
// TEXTURE_BINDING to be sampled by the compositor.
|
||||
// RENDER_ATTACHMENT is not used by anything here and is required
|
||||
// anyway: Slint rejects an imported texture without it. COPY_SRC
|
||||
// is for `read_overlay` and its two callers.
|
||||
// is for `read_overlay` and the tests that call it.
|
||||
usage: wgpu::TextureUsages::STORAGE_BINDING
|
||||
| wgpu::TextureUsages::TEXTURE_BINDING
|
||||
| wgpu::TextureUsages::RENDER_ATTACHMENT
|
||||
@@ -490,18 +490,11 @@ impl FocusPeakPass {
|
||||
/// TRACES: AC-8
|
||||
/// Copy the overlay to the CPU, as RGBA8 rows with no padding.
|
||||
///
|
||||
/// **Two callers, and neither is the desktop display path.** The tests
|
||||
/// below are one: an overlay is a claim about which pixels are sharp, and
|
||||
/// there is no way to check that claim without looking at the pixels. The
|
||||
/// other is the Android develop view, which reads the *frame* back for the
|
||||
/// reasons `technical-debt.md` TD-1 records — wgpu's Android swapchain
|
||||
/// tears a portrait window, so Slint is not drawing with wgpu there and no
|
||||
/// texture can be handed over. An overlay that stayed on the device on a
|
||||
/// platform where the picture underneath it does not would simply never be
|
||||
/// seen.
|
||||
///
|
||||
/// On desktop nothing calls this, and ARCH §6.1 holds on the path that
|
||||
/// matters: the overlay reaches the compositor as a texture.
|
||||
/// **The tests below are the only caller, and never the display path.** An
|
||||
/// overlay is a claim about which pixels are sharp, and there is no way to
|
||||
/// check that claim without looking at the pixels. On screen, on desktop
|
||||
/// and Android alike, the overlay reaches the compositor as a texture and
|
||||
/// ARCH §6.1 holds. (Android read it back here until TD-1 was paid off.)
|
||||
pub fn read_overlay(&self) -> Result<(Vec<u8>, u32, u32), GpuError> {
|
||||
let Some(layer) = self.layers[self.current].as_ref() else {
|
||||
return Err(GpuError::Readback("no overlay has been rendered".into()));
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
//! module doc said for eight releases that it held no pipeline and no masks.
|
||||
//! It holds both now, plus demosaic, detail, segmentation masks, two
|
||||
//! histograms and focus peaking. The zero-copy claim is still the one that
|
||||
//! matters, and TD-1 records the one platform where it does not hold.
|
||||
//! matters, and since TD-1 was paid off it holds on Android too.
|
||||
//!
|
||||
//! Deliberately free of UI dependencies (ARCH §6.5a). The texture is handed
|
||||
//! out as a `wgpu::Texture`; who composites it is not this crate's concern.
|
||||
|
||||
@@ -395,6 +395,7 @@ pub struct MaskPass {
|
||||
combine_layout: wgpu::BindGroupLayout,
|
||||
combine_union: wgpu::RenderPipeline,
|
||||
combine_subtract: wgpu::RenderPipeline,
|
||||
combine_intersect: wgpu::RenderPipeline,
|
||||
/// Where a part is drawn before it is joined.
|
||||
///
|
||||
/// One texture for the whole stack rather than one per layer, because
|
||||
@@ -651,6 +652,16 @@ impl MaskPass {
|
||||
"mask-combine-subtract",
|
||||
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::OneMinusSrc),
|
||||
);
|
||||
// TRACES: FR-DEV-19a
|
||||
// `dst * src`: what the mask had, kept only in proportion to how much
|
||||
// of it this part also covers. The same three vertices and the same
|
||||
// scratch, so a third set operation is a third blend state and
|
||||
// nothing more — which is what `Join::apply` states on the CPU and
|
||||
// `the_joins_match_their_definition` holds this to.
|
||||
let combine_intersect = combine(
|
||||
"mask-combine-intersect",
|
||||
blend_state(wgpu::BlendFactor::Zero, wgpu::BlendFactor::Src),
|
||||
);
|
||||
|
||||
// The same, with the deposit thrown away: coverage is only ever taken
|
||||
// off what earlier strokes on this layer put down. There is no negative
|
||||
@@ -681,6 +692,7 @@ impl MaskPass {
|
||||
combine_layout,
|
||||
combine_union,
|
||||
combine_subtract,
|
||||
combine_intersect,
|
||||
scratch: None,
|
||||
array: None,
|
||||
allocations: 0,
|
||||
@@ -1376,6 +1388,7 @@ impl MaskPass {
|
||||
pass.set_pipeline(match join {
|
||||
Join::Union => &self.combine_union,
|
||||
Join::Subtract => &self.combine_subtract,
|
||||
Join::Intersect => &self.combine_intersect,
|
||||
});
|
||||
pass.set_bind_group(0, &bind_group, &[]);
|
||||
pass.draw(0..3, 0..1);
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
// Hot and dead photosite repair, on the raw mosaic, before demosaic.
|
||||
//
|
||||
// A hot photosite reads far above anything the light put there — a leaky
|
||||
// well, lit by its own dark current on a long or high-ISO exposure. Left in,
|
||||
// the demosaic spreads it into its neighbours' interpolated channels and it
|
||||
// becomes a coloured cross, three pixels wide, that no later stage can take
|
||||
// back out: by then it is five pixels of plausible colour rather than one
|
||||
// photosite of nonsense. So it is repaired here, where it is still one value.
|
||||
//
|
||||
// **What counts as hot.** A photosite far above *every* photosite of its own
|
||||
// colour in its 5x5 window, and also far above every one of its eight
|
||||
// immediate neighbours whatever their colour. The second half is what keeps a
|
||||
// star or a glint: real light arrives through a lens and an anti-aliasing
|
||||
// filter, so even the sharpest point lands on a patch of photosites, and the
|
||||
// ones beside it are lit too. A hot photosite's neighbours are as dark as the
|
||||
// rest of the frame. Dead photosites are the mirror image and are handled the
|
||||
// same way.
|
||||
//
|
||||
// **What it becomes.** The brightest (for a hot photosite) or darkest (for a
|
||||
// dead one) same-colour neighbour — the value nearest to what it read that
|
||||
// the neighbourhood can vouch for. An average would soften the one case this
|
||||
// gets wrong, a real highlight that happened to pass both tests; a clamp to
|
||||
// the neighbourhood's range cannot invent anything.
|
||||
//
|
||||
// Written for either colour filter array: the colour of a photosite comes from
|
||||
// a 6x6 tile anchored to the sensor, which holds the X-Trans pattern as it is
|
||||
// and a Bayer 2x2 cell repeated nine times.
|
||||
|
||||
struct HotPixelParams {
|
||||
// The cropped area, in sensor photosites. Only photosites inside it are
|
||||
// judged, and only photosites inside it are asked as neighbours — the
|
||||
// masked border sits at black and would make everything look hot.
|
||||
crop_x: u32,
|
||||
crop_y: u32,
|
||||
width: u32,
|
||||
height: u32,
|
||||
// Row stride of the readout, in samples, and the number of u32 words.
|
||||
stride: u32,
|
||||
words: u32,
|
||||
// How many invocations one row of the dispatch grid holds, so a frame
|
||||
// wider than a dispatch dimension can be addressed as two.
|
||||
row_invocations: u32,
|
||||
// Samples in the readout. One less than twice `words` when the count is
|
||||
// odd, and the padding half of the last word is never judged.
|
||||
samples: u32,
|
||||
// Per-position black levels and reciprocal ranges, indexed by the
|
||||
// photosite's parity within the *crop*: (y&1)*2 + (x&1) counted from its
|
||||
// origin, as the demosaic counts them.
|
||||
black: vec4<f32>,
|
||||
inv_range: vec4<f32>,
|
||||
// The 6x6 colour tile, two bits per photosite, indexed by sensor
|
||||
// coordinate modulo 6: word k holds row 2k in its low 12 bits and row
|
||||
// 2k+1 in the next 12. The fourth word is padding.
|
||||
tile: vec4<u32>,
|
||||
}
|
||||
|
||||
@group(0) @binding(0) var<storage, read> raw: array<u32>;
|
||||
@group(0) @binding(1) var<uniform> params: HotPixelParams;
|
||||
@group(0) @binding(2) var<storage, read_write> repaired: array<u32>;
|
||||
|
||||
// How far above its brightest neighbour a photosite must read to be hot, as a
|
||||
// ratio and a margin in normalised units. Twice the neighbourhood and two
|
||||
// percent of the range above it: far enough that shot noise in a lit area
|
||||
// never qualifies, near enough that a hot photosite in a night sky — reading
|
||||
// a third of the range over a sky at one percent — always does.
|
||||
const HOT_RATIO: f32 = 2.0;
|
||||
const HOT_MARGIN: f32 = 0.02;
|
||||
// A dead photosite reads under half its darkest neighbour, and only counts
|
||||
// where that neighbour is at least this bright: in the shadows, a photosite
|
||||
// at zero is noise that clipped at the black point, not a defect.
|
||||
const DEAD_RATIO: f32 = 0.5;
|
||||
const DEAD_FLOOR: f32 = 0.05;
|
||||
|
||||
fn value_at(index: u32) -> u32 {
|
||||
let word = raw[index >> 1u];
|
||||
return select(word & 0xFFFFu, word >> 16u, (index & 1u) == 1u);
|
||||
}
|
||||
|
||||
fn colour_at(sx: u32, sy: u32) -> u32 {
|
||||
let row = sy % 6u;
|
||||
let col = sx % 6u;
|
||||
let word = params.tile[row >> 1u];
|
||||
return (word >> ((row & 1u) * 12u + col * 2u)) & 3u;
|
||||
}
|
||||
|
||||
// A raw value against its own black level and range. Compared rather than
|
||||
// stored, so it is left unclamped at the top: a hot photosite above white is
|
||||
// still more above white than its neighbours are.
|
||||
fn level(sx: u32, sy: u32, v: u32) -> f32 {
|
||||
let cell = ((sy - params.crop_y) & 1u) * 2u + ((sx - params.crop_x) & 1u);
|
||||
return max(f32(v) - params.black[cell], 0.0) * params.inv_range[cell];
|
||||
}
|
||||
|
||||
// The value to store for the photosite at `index`.
|
||||
fn repair(index: u32) -> u32 {
|
||||
let v = value_at(index);
|
||||
let sx = index % params.stride;
|
||||
let sy = index / params.stride;
|
||||
if (sx < params.crop_x || sy < params.crop_y
|
||||
|| sx >= params.crop_x + params.width || sy >= params.crop_y + params.height) {
|
||||
return v;
|
||||
}
|
||||
|
||||
let centre = level(sx, sy, v);
|
||||
let colour = colour_at(sx, sy);
|
||||
|
||||
var same_hi = -1.0;
|
||||
var same_lo = 1.0e9;
|
||||
var same_hi_raw = v;
|
||||
var same_lo_raw = v;
|
||||
var same_count = 0u;
|
||||
var adjacent_hi = 0.0;
|
||||
var adjacent_lo = 1.0e9;
|
||||
|
||||
for (var dy = -2; dy <= 2; dy++) {
|
||||
for (var dx = -2; dx <= 2; dx++) {
|
||||
if (dx == 0 && dy == 0) {
|
||||
continue;
|
||||
}
|
||||
let nx = i32(sx) + dx;
|
||||
let ny = i32(sy) + dy;
|
||||
if (nx < i32(params.crop_x) || ny < i32(params.crop_y)
|
||||
|| nx >= i32(params.crop_x + params.width)
|
||||
|| ny >= i32(params.crop_y + params.height)) {
|
||||
continue;
|
||||
}
|
||||
let nsx = u32(nx);
|
||||
let nsy = u32(ny);
|
||||
let nv = value_at(nsy * params.stride + nsx);
|
||||
let n = level(nsx, nsy, nv);
|
||||
|
||||
if (abs(dx) <= 1 && abs(dy) <= 1) {
|
||||
adjacent_hi = max(adjacent_hi, n);
|
||||
adjacent_lo = min(adjacent_lo, n);
|
||||
}
|
||||
if (colour_at(nsx, nsy) == colour) {
|
||||
same_count += 1u;
|
||||
if (n > same_hi) {
|
||||
same_hi = n;
|
||||
same_hi_raw = nv;
|
||||
}
|
||||
if (n < same_lo) {
|
||||
same_lo = n;
|
||||
same_lo_raw = nv;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// A corner of the crop can leave a photosite with a single same-colour
|
||||
// neighbour, and one witness is not a neighbourhood.
|
||||
if (same_count < 2u) {
|
||||
return v;
|
||||
}
|
||||
|
||||
let hot_line_same = same_hi * HOT_RATIO + HOT_MARGIN;
|
||||
let hot_line_adjacent = adjacent_hi * HOT_RATIO + HOT_MARGIN;
|
||||
if (centre > hot_line_same && centre > hot_line_adjacent) {
|
||||
return same_hi_raw;
|
||||
}
|
||||
if (same_lo >= DEAD_FLOOR && centre < same_lo * DEAD_RATIO
|
||||
&& centre < adjacent_lo * DEAD_RATIO) {
|
||||
return same_lo_raw;
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
// One invocation per u32 word: two photosites, packed as the demosaic reads
|
||||
// them. A word may straddle two rows when the stride is odd, which `repair`
|
||||
// does not mind — it addresses by sample index.
|
||||
@compute @workgroup_size(64, 1, 1)
|
||||
fn main(@builtin(global_invocation_id) gid: vec3<u32>) {
|
||||
let word = gid.y * params.row_invocations + gid.x;
|
||||
if (word >= params.words) {
|
||||
return;
|
||||
}
|
||||
let lo = repair(word * 2u);
|
||||
// The padding half of an odd-length readout is copied, not judged: it is
|
||||
// not a photosite, and the demosaic never addresses it.
|
||||
var hi = raw[word] >> 16u;
|
||||
if (word * 2u + 1u < params.samples) {
|
||||
hi = repair(word * 2u + 1u);
|
||||
}
|
||||
repaired[word] = (lo & 0xFFFFu) | (hi << 16u);
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
//! Contrast, on a device, at the two ends of the tonal range.
|
||||
//!
|
||||
//! The descriptor tests check that the fragment says the right words; these
|
||||
//! check what those words do to a pixel. Both failures here passed every
|
||||
//! descriptor test for months, because each is a property of the arithmetic
|
||||
//! at the extremes rather than of the shape of the code.
|
||||
|
||||
use dr_gpu::{AdjustPass, DemosaicedImage, GpuContext};
|
||||
use dr_pipeline::descriptor::ParamId;
|
||||
use dr_pipeline::operation::compose;
|
||||
use dr_pipeline::ops;
|
||||
|
||||
const SIZE: u32 = 8;
|
||||
|
||||
fn ctx() -> Option<GpuContext> {
|
||||
pollster::block_on(GpuContext::new_headless()).ok()
|
||||
}
|
||||
|
||||
/// A flat frame of one sRGB colour, contrast set to `amount`, rendered and
|
||||
/// read back as the colour of one pixel.
|
||||
fn render(ctx: &GpuContext, rgb: [u8; 3], amount: f32) -> [u8; 3] {
|
||||
let data: Vec<u8> = (0..SIZE * SIZE)
|
||||
.flat_map(|_| [rgb[0], rgb[1], rgb[2], 255])
|
||||
.collect();
|
||||
let source = DemosaicedImage::from_rgba8(ctx, &data, SIZE, SIZE).expect("upload");
|
||||
|
||||
let mut chain = ops::chain();
|
||||
let op = chain
|
||||
.iter_mut()
|
||||
.find(|o| o.descriptor().id.0 == "contrast")
|
||||
.expect("contrast is in the chain");
|
||||
op.set_param(ParamId("contrast"), amount);
|
||||
let shader = compose(&chain);
|
||||
|
||||
let mut adjust = AdjustPass::new(ctx);
|
||||
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
|
||||
let pixels = adjust.export_pixels().expect("readback").0;
|
||||
[pixels[0], pixels[1], pixels[2]]
|
||||
}
|
||||
|
||||
/// **The pink-blacks bug.** A near-black pixel whose red and blue sit a count
|
||||
/// above its green — what white-balanced sensor noise in a night shadow looks
|
||||
/// like — must come out grey when contrast is reduced, not magenta.
|
||||
///
|
||||
/// The ratio form lifted it by a gain of well over a hundred, and a hundred
|
||||
/// times a one-count cast is a saturated colour.
|
||||
#[test]
|
||||
fn reducing_contrast_lifts_a_black_to_grey_not_to_magenta() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let [r, g, b] = render(&ctx, [4, 1, 4], -50.0);
|
||||
let spread = r.max(g).max(b) - r.min(g).min(b);
|
||||
assert!(
|
||||
g > 40,
|
||||
"a black at half contrast should be lifted toward grey, got ({r}, {g}, {b})"
|
||||
);
|
||||
assert!(
|
||||
spread <= 6,
|
||||
"the lift must be neutral: ({r}, {g}, {b}) has a cast of {spread}"
|
||||
);
|
||||
}
|
||||
|
||||
/// **The pinned highlights.** A light tone, above twice middle grey, must not
|
||||
/// be pulled down to the top of the curve by the smallest positive contrast.
|
||||
#[test]
|
||||
fn a_little_contrast_leaves_a_highlight_where_it_was() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let before = render(&ctx, [230, 230, 230], 0.0)[1];
|
||||
let after = render(&ctx, [230, 230, 230], 10.0)[1];
|
||||
assert!(
|
||||
after >= before.saturating_sub(2),
|
||||
"contrast +10 took a highlight from {before} to {after}"
|
||||
);
|
||||
}
|
||||
@@ -16,9 +16,12 @@
|
||||
//! the CPU model.
|
||||
|
||||
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
|
||||
use dr_film::bake::{bake, Recipe};
|
||||
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
|
||||
use dr_pipeline::ops::FilmTables;
|
||||
use dr_film::bake::{bake, Recipe, Settings};
|
||||
use dr_gpu::{AdjustPass, Demosaicer, GpuContext, LabelField, MaskPass};
|
||||
use dr_pipeline::mask::{MaskLayer, MaskSource};
|
||||
use dr_pipeline::ops::film_sim;
|
||||
use dr_pipeline::ops::film_sim::FORMAT_COUNT;
|
||||
use dr_pipeline::ops::{FilmTables, PaperTables};
|
||||
use dr_pipeline::EditGraph;
|
||||
|
||||
const SIZE: u32 = 16;
|
||||
@@ -77,15 +80,31 @@ fn tables(baked: &dr_film::Baked) -> FilmTables {
|
||||
/// split a grain that never left the CPU would look exactly like a passing
|
||||
/// test suite.
|
||||
fn tables_with_grain(baked: &dr_film::Baked, particles: [f32; 3]) -> FilmTables {
|
||||
// The paper, when there is one, rides behind the film: its curve as one
|
||||
// more row, its cube stacked after the film's.
|
||||
let mut curves = baked.curves.clone();
|
||||
let mut lut = baked.lut.clone();
|
||||
let paper = baked.paper.as_ref().map(|p| {
|
||||
curves.extend_from_slice(&p.curves);
|
||||
lut.extend_from_slice(&p.lut);
|
||||
PaperTables {
|
||||
balance: p.balance,
|
||||
log_min: p.log_min,
|
||||
log_max: p.log_max,
|
||||
density_max: p.density_max,
|
||||
}
|
||||
});
|
||||
FilmTables {
|
||||
exposure_matrix: baked.exposure_matrix,
|
||||
curves: baked.curves.clone(),
|
||||
curves,
|
||||
push_stations: baked.push_stations.clone(),
|
||||
curve_log_min: baked.curve_log_min,
|
||||
curve_log_max: baked.curve_log_max,
|
||||
lut: baked.lut.clone(),
|
||||
lut,
|
||||
density_max: baked.density_max,
|
||||
lut_size: baked.lut_size,
|
||||
grain_particles: particles,
|
||||
paper,
|
||||
grain_particles: [particles; FORMAT_COUNT],
|
||||
grain_density_max: [baked.density_max; 3],
|
||||
grain_uniformity: 0.97,
|
||||
}
|
||||
@@ -241,3 +260,241 @@ fn grain_reaches_the_shader_and_scales_with_the_pixel() {
|
||||
"grain never reached the shader: the coarsest setting moved the pixel by {coarse_err}"
|
||||
);
|
||||
}
|
||||
|
||||
/// The same render, with the film's sliders set and mask layers laid over it.
|
||||
///
|
||||
/// Every layer is a `Regions` mask over a field splitting the frame down the
|
||||
/// middle: region 0, the left half, at full weight, and the right half
|
||||
/// untouched. Returns the left and right centre pixels, linear.
|
||||
fn rendered_split(
|
||||
ctx: &GpuContext,
|
||||
level: u16,
|
||||
tables: FilmTables,
|
||||
global: Settings,
|
||||
layers: Vec<MaskLayer>,
|
||||
) -> ([f32; 3], [f32; 3]) {
|
||||
let source = Demosaicer::new(ctx)
|
||||
.expect("demosaicer")
|
||||
.run(&flat_raw(level))
|
||||
.expect("demosaic");
|
||||
|
||||
let mut graph = EditGraph::default_chain();
|
||||
graph.set_film(Some(dr_pipeline::graph::Film {
|
||||
stock: "under_test".to_string(),
|
||||
print: None,
|
||||
tables: tables.clone(),
|
||||
}));
|
||||
graph.set_param(film_sim::ID, film_sim::EXPOSURE, global.exposure_ev);
|
||||
graph.set_param(film_sim::ID, film_sim::PUSH, global.push_stops);
|
||||
graph.set_param(
|
||||
film_sim::ID,
|
||||
film_sim::PRINT_EXPOSURE,
|
||||
global.print_exposure_ev,
|
||||
);
|
||||
for layer in layers {
|
||||
graph.masks_mut().push(layer);
|
||||
}
|
||||
let shader = graph.compose();
|
||||
|
||||
let labels: Vec<u32> = (0..SIZE * SIZE)
|
||||
.map(|i| u32::from(i % SIZE >= SIZE / 2))
|
||||
.collect();
|
||||
let field = LabelField::upload(ctx, &labels, SIZE, SIZE, 2).expect("label upload");
|
||||
let mut masks = MaskPass::new(ctx).expect("mask pass");
|
||||
let array = masks
|
||||
.render(graph.masks(), Some(&field), None, None, SIZE, SIZE)
|
||||
.expect("rasterise");
|
||||
|
||||
let mut adjust = AdjustPass::new(ctx);
|
||||
adjust.set_film(Some(&tables));
|
||||
adjust
|
||||
.render_masked(&source, &shader, SIZE, SIZE, Some(array))
|
||||
.expect("render");
|
||||
let (pixels, _, _) = adjust.export_pixels().expect("readback");
|
||||
let at = |x: u32| {
|
||||
let c = (((SIZE / 2) * SIZE + x) * 4) as usize;
|
||||
[0, 1, 2].map(|i| srgb_to_linear(f32::from(pixels[c + i]) / 255.0))
|
||||
};
|
||||
(at(SIZE / 4), at(3 * SIZE / 4))
|
||||
}
|
||||
|
||||
/// A layer over the left half holding these film offsets.
|
||||
fn left_half(id: &str, offsets: &[(dr_pipeline::descriptor::ParamId, f32)]) -> MaskLayer {
|
||||
let mut layer = MaskLayer::new(
|
||||
id,
|
||||
MaskSource::Regions {
|
||||
signature: 1,
|
||||
level: 2,
|
||||
ids: vec![0],
|
||||
},
|
||||
);
|
||||
for (param, v) in offsets {
|
||||
layer.set_param(film_sim::ID.0, *param, *v);
|
||||
}
|
||||
layer
|
||||
}
|
||||
|
||||
fn assert_close(got: [f32; 3], want: [f32; 3], what: &str) {
|
||||
for c in 0..3 {
|
||||
assert!(
|
||||
(got[c] - want[c]).abs() < 0.02,
|
||||
"{what}, channel {c}: GPU gave {got:?}, the model says {want:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_print_renders_on_the_gpu_the_way_it_does_on_the_cpu_at_any_setting() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// The print path — the film's lookup into the paper's log exposure, the
|
||||
// enlarger added between, the paper's curve and its own lookup — is read
|
||||
// from the same two textures as the film, at offsets. Every one of those
|
||||
// offsets is a way to render a plausible print of the wrong thing.
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let film = dr_film::find("kodak_portra_400").expect("stock");
|
||||
let paper = dr_film::default_print(film).expect("paper");
|
||||
let baked = bake(&Recipe::new(film, Some(paper)));
|
||||
|
||||
for settings in [
|
||||
Settings::default(),
|
||||
Settings {
|
||||
print_exposure_ev: -1.3,
|
||||
..Settings::default()
|
||||
},
|
||||
Settings {
|
||||
exposure_ev: 0.4,
|
||||
print_exposure_ev: 0.8,
|
||||
..Settings::default()
|
||||
},
|
||||
] {
|
||||
for level in [6_000u16, 20_000] {
|
||||
let input = f32::from(level) / f32::from(u16::MAX);
|
||||
let (got, _) = rendered_split(&ctx, level, tables(&baked), settings, Vec::new());
|
||||
assert_close(
|
||||
got,
|
||||
baked.apply_at([input; 3], &settings),
|
||||
&format!("{settings:?} at {level}"),
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_push_between_two_measured_processes_renders_as_the_model_does() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// Double-X measures five processes; a push between two is a mix of two
|
||||
// rows of the curve texture, found by searching the stations uniform.
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let film = dr_film::find("kodak_doublex").expect("stock");
|
||||
let baked = bake(&Recipe::new(film, None));
|
||||
assert!(baked.curve_rows > 2, "Double-X has a development series");
|
||||
|
||||
for push in [-0.8f32, 0.4, 1.3, 2.9] {
|
||||
let settings = Settings {
|
||||
push_stops: push,
|
||||
..Settings::default()
|
||||
};
|
||||
let level = 12_000u16;
|
||||
let input = f32::from(level) / f32::from(u16::MAX);
|
||||
let (got, _) = rendered_split(&ctx, level, tables(&baked), settings, Vec::new());
|
||||
assert_close(
|
||||
got,
|
||||
baked.apply_at([input; 3], &settings),
|
||||
&format!("push {push}"),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_layer_develops_its_region_on_its_own_settings() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// Offsets to the photograph's: print exposure +1 on a photograph at +0.5
|
||||
// is +1.5 under the layer, and the rest of the print is untouched. Before
|
||||
// film was blended as settings the layer's sliders moved and nothing
|
||||
// happened, because the layer's copy of the node had no stock.
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let film = dr_film::find("kodak_portra_400").expect("stock");
|
||||
let paper = dr_film::default_print(film).expect("paper");
|
||||
let baked = bake(&Recipe::new(film, Some(paper)));
|
||||
let level = 12_000u16;
|
||||
let input = f32::from(level) / f32::from(u16::MAX);
|
||||
|
||||
let global = Settings {
|
||||
print_exposure_ev: 0.5,
|
||||
..Settings::default()
|
||||
};
|
||||
let layer = left_half(
|
||||
"burn",
|
||||
&[(film_sim::PRINT_EXPOSURE, 1.0), (film_sim::EXPOSURE, -0.5)],
|
||||
);
|
||||
let (left, right) = rendered_split(&ctx, level, tables(&baked), global, vec![layer]);
|
||||
|
||||
let under = Settings {
|
||||
exposure_ev: -0.5,
|
||||
print_exposure_ev: 1.5,
|
||||
..Settings::default()
|
||||
};
|
||||
assert_close(left, baked.apply_at([input; 3], &under), "under the layer");
|
||||
assert_close(right, baked.apply_at([input; 3], &global), "outside it");
|
||||
assert!(
|
||||
left[1] < right[1] - 0.01,
|
||||
"the burn did not darken: {left:?} vs {right:?}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overlapping_layers_take_the_average_of_their_settings() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// Three layers over the same pixels at full weight: the plain mean of what
|
||||
// each asks for, and the global setting has no weight left. Summed, the
|
||||
// offsets would be -2 stops of print exposure and +1 of exposure; the mean
|
||||
// is (-1, -1, 0) / 3 and (0, 0, +1) / 3.
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let film = dr_film::find("kodak_portra_400").expect("stock");
|
||||
let paper = dr_film::default_print(film).expect("paper");
|
||||
let baked = bake(&Recipe::new(film, Some(paper)));
|
||||
let level = 12_000u16;
|
||||
let input = f32::from(level) / f32::from(u16::MAX);
|
||||
|
||||
let layers = vec![
|
||||
left_half("a", &[(film_sim::PRINT_EXPOSURE, -1.0)]),
|
||||
left_half("b", &[(film_sim::PRINT_EXPOSURE, -1.0)]),
|
||||
left_half("c", &[(film_sim::EXPOSURE, 1.0)]),
|
||||
];
|
||||
let (left, right) = rendered_split(&ctx, level, tables(&baked), Settings::default(), layers);
|
||||
|
||||
let mean = Settings {
|
||||
exposure_ev: 1.0 / 3.0,
|
||||
print_exposure_ev: -2.0 / 3.0,
|
||||
..Settings::default()
|
||||
};
|
||||
let summed = Settings {
|
||||
exposure_ev: 1.0,
|
||||
print_exposure_ev: -2.0,
|
||||
..Settings::default()
|
||||
};
|
||||
let (m, s) = (
|
||||
baked.apply_at([input; 3], &mean)[1],
|
||||
baked.apply_at([input; 3], &summed)[1],
|
||||
);
|
||||
// Three times the tolerance the GPU is held to below, or a sum could pass
|
||||
// for a mean.
|
||||
assert!(
|
||||
(m - s).abs() > 0.06,
|
||||
"the mean and the sum render alike ({m} vs {s}), so this proves nothing"
|
||||
);
|
||||
assert_close(left, baked.apply_at([input; 3], &mean), "under all three");
|
||||
assert_close(right, baked.apply([input; 3]), "outside them");
|
||||
}
|
||||
|
||||
@@ -0,0 +1,142 @@
|
||||
//! TRACES: FR-RAW-3
|
||||
//! Hot and dead photosite repair, end to end on a device.
|
||||
//!
|
||||
//! Each test renders a frame twice — once with a defect, once without — and
|
||||
//! compares the finished pixels. That is the only comparison that means
|
||||
//! anything: the repair happens on the mosaic, and what a photographer would
|
||||
//! see of a defect it missed is the coloured cross the demosaic makes of it.
|
||||
|
||||
use dr_decode::{BaseCurve, CfaPattern, CropRect, RawImage};
|
||||
use dr_gpu::{AdjustPass, Demosaicer, GpuContext};
|
||||
use dr_pipeline::EditGraph;
|
||||
|
||||
const SIZE: u32 = 36;
|
||||
const WHITE: u16 = 4095;
|
||||
|
||||
fn ctx() -> Option<GpuContext> {
|
||||
pollster::block_on(GpuContext::new_headless()).ok()
|
||||
}
|
||||
|
||||
/// A flat frame at `level`, with `set` applied to its photosites.
|
||||
fn frame(pattern: CfaPattern, level: u16, set: &[(u32, u32, u16)]) -> RawImage {
|
||||
let mut data = vec![level; (SIZE * SIZE) as usize];
|
||||
for &(x, y, v) in set {
|
||||
data[(y * SIZE + x) as usize] = v;
|
||||
}
|
||||
RawImage {
|
||||
width: SIZE,
|
||||
height: SIZE,
|
||||
data,
|
||||
cfa_pattern: pattern,
|
||||
black_level: [0; 4],
|
||||
white_level: WHITE,
|
||||
wb_coeffs: [1.0, 1.0, 1.0, 1.0],
|
||||
color_matrix: Some([1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]),
|
||||
base_curve: BaseCurve::IDENTITY,
|
||||
samples_per_pixel: 1,
|
||||
profile: None,
|
||||
make: String::new(),
|
||||
model: String::new(),
|
||||
crop: CropRect {
|
||||
x: 0,
|
||||
y: 0,
|
||||
width: SIZE,
|
||||
height: SIZE,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
fn render(ctx: &GpuContext, raw: &RawImage) -> Vec<u8> {
|
||||
let source = Demosaicer::new(ctx)
|
||||
.expect("demosaicer")
|
||||
.run(raw)
|
||||
.expect("demosaic");
|
||||
let shader = EditGraph::default_chain().compose();
|
||||
let mut adjust = AdjustPass::new(ctx);
|
||||
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
|
||||
adjust.export_pixels().expect("readback").0
|
||||
}
|
||||
|
||||
/// The largest channel difference between two renders.
|
||||
fn worst(a: &[u8], b: &[u8]) -> u8 {
|
||||
a.iter().zip(b).map(|(x, y)| x.abs_diff(*y)).max().unwrap()
|
||||
}
|
||||
|
||||
const MIDDLE: u32 = SIZE / 2;
|
||||
|
||||
/// **The feature.** A photosite at white in a dark frame — a hot pixel in a
|
||||
/// night sky — leaves no trace in the rendered picture.
|
||||
#[test]
|
||||
fn a_hot_photosite_in_a_dark_frame_is_invisible() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let clean = render(&ctx, &frame(CfaPattern::Rggb, 40, &[]));
|
||||
for (x, y) in [
|
||||
(MIDDLE, MIDDLE),
|
||||
(MIDDLE + 1, MIDDLE),
|
||||
(MIDDLE + 1, MIDDLE + 1),
|
||||
] {
|
||||
let hot = render(&ctx, &frame(CfaPattern::Rggb, 40, &[(x, y, WHITE)]));
|
||||
let diff = worst(&clean, &hot);
|
||||
assert!(
|
||||
diff <= 1,
|
||||
"a hot photosite at ({x}, {y}) still shows, by {diff}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// The same for one stuck dark in a lit area.
|
||||
#[test]
|
||||
fn a_dead_photosite_in_a_lit_frame_is_invisible() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let clean = render(&ctx, &frame(CfaPattern::Rggb, 1600, &[]));
|
||||
let dead = render(&ctx, &frame(CfaPattern::Rggb, 1600, &[(MIDDLE, MIDDLE, 0)]));
|
||||
let diff = worst(&clean, &dead);
|
||||
assert!(diff <= 1, "a dead photosite still shows, by {diff}");
|
||||
}
|
||||
|
||||
/// **What it must not eat.** A point of real light lands on a patch of
|
||||
/// photosites, not one — so a 3×3 highlight survives, even at its brightest.
|
||||
#[test]
|
||||
fn a_small_real_highlight_survives() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let mut star = Vec::new();
|
||||
for dy in 0..3 {
|
||||
for dx in 0..3 {
|
||||
star.push((MIDDLE - 1 + dx, MIDDLE - 1 + dy, WHITE));
|
||||
}
|
||||
}
|
||||
let clean = render(&ctx, &frame(CfaPattern::Rggb, 40, &[]));
|
||||
let lit = render(&ctx, &frame(CfaPattern::Rggb, 40, &star));
|
||||
let at = ((MIDDLE * SIZE + MIDDLE) * 4 + 1) as usize;
|
||||
assert!(
|
||||
lit[at] > clean[at] + 100,
|
||||
"the highlight was repaired away: {} against a background of {}",
|
||||
lit[at],
|
||||
clean[at]
|
||||
);
|
||||
}
|
||||
|
||||
/// The Fujifilm path goes through the same repair, with its own tile.
|
||||
#[test]
|
||||
fn a_hot_photosite_on_x_trans_is_invisible() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let clean = render(&ctx, &frame(CfaPattern::XTrans, 40, &[]));
|
||||
let hot = render(
|
||||
&ctx,
|
||||
&frame(CfaPattern::XTrans, 40, &[(MIDDLE, MIDDLE, WHITE)]),
|
||||
);
|
||||
let diff = worst(&clean, &hot);
|
||||
assert!(diff <= 1, "a hot X-Trans photosite still shows, by {diff}");
|
||||
}
|
||||
@@ -790,6 +790,119 @@ fn the_order_parts_are_joined_in_is_the_mask() {
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// The truth table `docs/dev/mask-editing.md` §13 asks for: one base, one
|
||||
/// part that half-covers it, joined each of the three ways. The base is the
|
||||
/// left half of the frame and the part a dab in the middle, so the four
|
||||
/// quarters of the table are four pixels.
|
||||
#[test]
|
||||
fn a_part_unioned_subtracted_and_intersected_gives_the_three_fields() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
let field = split_field(&ctx);
|
||||
let joined = |join: Join| {
|
||||
let mut layer = brighten(MaskSource::Regions {
|
||||
signature: 1,
|
||||
level: 2,
|
||||
ids: vec![0],
|
||||
});
|
||||
assert!(layer.push_part(MaskPart::painted("p2", join)));
|
||||
paint(&mut layer, 1, false, &[(0.5, 0.5)]);
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(layer);
|
||||
render(&ctx, &stack, Some(&field))
|
||||
};
|
||||
|
||||
// (base, part): left outside the dab, left inside, right inside, right
|
||||
// outside.
|
||||
let cells = [(4, 16), (14, 16), (18, 16), (27, 16)];
|
||||
let lit = |pixels: &[u8]| cells.map(|(x, y)| luma_at(pixels, x, y) > 200);
|
||||
|
||||
assert_eq!(
|
||||
lit(&joined(Join::Union)),
|
||||
[true, true, true, false],
|
||||
"union: either"
|
||||
);
|
||||
assert_eq!(
|
||||
lit(&joined(Join::Subtract)),
|
||||
[true, false, false, false],
|
||||
"subtract: the base without the dab"
|
||||
);
|
||||
assert_eq!(
|
||||
lit(&joined(Join::Intersect)),
|
||||
[false, true, false, false],
|
||||
"intersect: only where both are"
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// Intersection on soft coverage is the product `Join::apply` defines, on
|
||||
/// either side of the join: a gradient intersected with a region it fills is
|
||||
/// the gradient there and nothing elsewhere, and a region intersected with a
|
||||
/// gradient is the gradient wherever the region is.
|
||||
#[test]
|
||||
fn the_joins_match_their_definition() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no adapter; skipping");
|
||||
return;
|
||||
};
|
||||
|
||||
let field = split_field(&ctx);
|
||||
let ramp = || MaskSource::Linear {
|
||||
centre: (0.5, 0.5),
|
||||
angle: 0.0,
|
||||
width: 1.0,
|
||||
};
|
||||
let right_half = || MaskSource::Regions {
|
||||
signature: 1,
|
||||
level: 2,
|
||||
ids: vec![1],
|
||||
};
|
||||
let draw = |layer: MaskLayer| {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(layer);
|
||||
render(&ctx, &stack, Some(&field))
|
||||
};
|
||||
|
||||
let alone = draw(brighten(ramp()));
|
||||
|
||||
let mut ramp_then_region = brighten(ramp());
|
||||
assert!(ramp_then_region.push_part(MaskPart::new("p2", Join::Intersect, right_half())));
|
||||
let ramp_then_region = draw(ramp_then_region);
|
||||
|
||||
let mut region_then_ramp = brighten(whole_frame());
|
||||
assert!(region_then_ramp.push_part(MaskPart::new("p2", Join::Intersect, ramp())));
|
||||
let region_then_ramp = draw(region_then_ramp);
|
||||
|
||||
for x in 0..SIZE {
|
||||
let y = SIZE / 2;
|
||||
let want = luma_at(&alone, x, y);
|
||||
// dst · 1 = dst on the right; dst · 0 = 0 on the left. Pixels
|
||||
// within two of the seam are left out: the region's own edge
|
||||
// is soft there, so neither side of the table is 0 or 1.
|
||||
let got = luma_at(&ramp_then_region, x, y);
|
||||
if x.abs_diff(SIZE / 2) <= 2 {
|
||||
// The seam.
|
||||
} else if x > SIZE / 2 {
|
||||
assert!(
|
||||
got.abs_diff(want) <= 1,
|
||||
"x={x}: the ramp survives where the region is ({got} vs {want})"
|
||||
);
|
||||
} else {
|
||||
assert_eq!(got, 128, "x={x}: and nothing survives where it is not");
|
||||
}
|
||||
// 1 · src = src everywhere.
|
||||
let got = luma_at(®ion_then_ramp, x, y);
|
||||
assert!(
|
||||
got.abs_diff(want) <= 1,
|
||||
"x={x}: a full base intersected with the ramp is the ramp ({got} vs {want})"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// --- seeing the mask (FR-DEV-19c) ------------------------------------------
|
||||
|
||||
/// A radial that covers the middle of the frame and nothing near the corners.
|
||||
@@ -1022,3 +1135,76 @@ fn two_shown_masks_are_drawn_each_in_its_own_colour() {
|
||||
"between them, alpha shows black: ({r}, {g}, {b})"
|
||||
);
|
||||
}
|
||||
|
||||
/// Render a mid-grey-and-shadows frame through `chain` and `stack`.
|
||||
fn render_chain(
|
||||
ctx: &GpuContext,
|
||||
chain: &[Box<dyn dr_pipeline::operation::Operation>],
|
||||
stack: &MaskStack,
|
||||
field: Option<&LabelField>,
|
||||
) -> Vec<u8> {
|
||||
// A ramp, so both ends of the tonal range are in the comparison.
|
||||
let data: Vec<u8> = (0..SIZE * SIZE)
|
||||
.flat_map(|i| {
|
||||
let v = ((i % SIZE) * 255 / (SIZE - 1)) as u8;
|
||||
[v, v / 2, v, 255]
|
||||
})
|
||||
.collect();
|
||||
let source = DemosaicedImage::from_rgba8(ctx, &data, SIZE, SIZE).expect("upload");
|
||||
let shader = compose_full(
|
||||
chain,
|
||||
&Framing::new(),
|
||||
ColourSpace::Srgb,
|
||||
stack,
|
||||
&SpotSet::new(),
|
||||
&[],
|
||||
);
|
||||
let mut masks = MaskPass::new(ctx).expect("mask pass");
|
||||
let array = masks
|
||||
.render(stack, field, None, None, SIZE, SIZE)
|
||||
.expect("rasterise");
|
||||
let mut adjust = AdjustPass::new(ctx);
|
||||
adjust
|
||||
.render_masked(&source, &shader, SIZE, SIZE, Some(array))
|
||||
.expect("render");
|
||||
adjust.export_pixels().expect("readback").0
|
||||
}
|
||||
|
||||
fn contrast_chain(v: f32) -> Vec<Box<dyn dr_pipeline::operation::Operation>> {
|
||||
let mut chain = ops::chain();
|
||||
chain
|
||||
.iter_mut()
|
||||
.find(|o| o.descriptor().id.0 == "contrast")
|
||||
.expect("contrast")
|
||||
.set_param(ParamId("contrast"), v);
|
||||
chain
|
||||
}
|
||||
|
||||
/// A layer's setting is an offset to the global one, applied once: global
|
||||
/// −30 with a whole-frame layer at −20 is exactly global −50 — not −30 and
|
||||
/// then −20 again on the result, which is what a layer used to do.
|
||||
#[test]
|
||||
fn a_whole_frame_layer_adds_its_setting_to_the_global_one() {
|
||||
let Some(ctx) = ctx() else {
|
||||
eprintln!("no GPU adapter; skipping");
|
||||
return;
|
||||
};
|
||||
let mut layer = MaskLayer::new("m1", whole_frame());
|
||||
layer.set_param("contrast", ParamId("contrast"), -20.0);
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(layer);
|
||||
|
||||
let field = split_field(&ctx);
|
||||
let offset = render_chain(&ctx, &contrast_chain(-30.0), &stack, Some(&field));
|
||||
let direct = render_chain(&ctx, &contrast_chain(-50.0), &MaskStack::new(), None);
|
||||
let worst = offset
|
||||
.iter()
|
||||
.zip(&direct)
|
||||
.map(|(a, b)| a.abs_diff(*b))
|
||||
.max()
|
||||
.unwrap();
|
||||
assert!(
|
||||
worst <= 1,
|
||||
"layer offset differs from the summed setting by {worst}"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -35,41 +35,62 @@ helpers: [luminance, apply_tone_gain]
|
||||
|
||||
define:
|
||||
contrast_curve: |
|
||||
// A symmetric S-curve on a 0..1 perceptual position.
|
||||
// The steepening S, on a 0..1 perceptual position.
|
||||
//
|
||||
// `amount` above zero steepens, below zero flattens. The smoothstep form is
|
||||
// used for the steepening direction because it has zero gradient at both
|
||||
// ends, so the curve cannot invert however hard it is pushed — the failure
|
||||
// that makes naive gain-about-a-pivot unusable past moderate settings.
|
||||
// Blends toward a smoothstep, which has zero gradient at both ends, so the
|
||||
// curve cannot invert however hard it is pushed — the failure that makes
|
||||
// naive gain-about-a-pivot unusable past moderate settings. Only the
|
||||
// positive direction comes here: flattening is not a curve at all (see
|
||||
// the fragment).
|
||||
fn contrast_curve(x: f32, amount: f32) -> f32 {
|
||||
let clamped = clamp(x, 0.0, 1.0);
|
||||
if (amount >= 0.0) {
|
||||
// Blend toward a smoothstep, which is the S.
|
||||
let s = clamped * clamped * (3.0 - 2.0 * clamped);
|
||||
return mix(clamped, s, amount);
|
||||
}
|
||||
// Flattening: pull toward the mid-point. At amount = -1 every tone
|
||||
// collapses to 0.5, which is the meaningful limit of 'no contrast'.
|
||||
return mix(clamped, 0.5, -amount);
|
||||
let s = clamped * clamped * (3.0 - 2.0 * clamped);
|
||||
return mix(clamped, s, amount);
|
||||
}
|
||||
|
||||
wgsl: |
|
||||
let luma = luminance(c);
|
||||
if (luma > 0.0001) {
|
||||
// Work on luminance and rescale the colour by the ratio, rather than
|
||||
// curving each channel independently. Per-channel contrast shifts hue
|
||||
// wherever the channels differ — the classic symptom being skies going
|
||||
// cyan as contrast rises.
|
||||
if (amount < 0.0) {
|
||||
// **Flattening mixes toward middle grey; it does not scale.**
|
||||
//
|
||||
// MIDDLE_GREY is 0.18: the linear value the eye reads as mid-tone. The
|
||||
// curve operates on luma/(2*0.18) so that middle grey lands at the
|
||||
// curve's own 0.5 pivot.
|
||||
let pos = clamp(luma / 0.36, 0.0, 1.0);
|
||||
let curved = contrast_curve(pos, amount);
|
||||
// Not `target`: that is a WGSL reserved keyword, and using it produces a
|
||||
// parse error in generated code rather than anywhere a reader would look.
|
||||
let curved_luma = curved * 0.36;
|
||||
c = apply_tone_gain(c, curved_luma / luma);
|
||||
// Every tone moves the same fraction of the way to 0.18, which at -1
|
||||
// collapses the picture to grey — the meaningful limit of 'no contrast'.
|
||||
// In luminance this is exactly what the ratio form below would compute,
|
||||
// but the ratio form reaches it by multiplying: a pixel at 0.001 has to
|
||||
// be lifted to 0.09, a gain of ninety, and in the deepest shadows the
|
||||
// channels are sensor noise, not a colour. After white balance the red
|
||||
// and blue noise sits above the green (their multipliers are nearly
|
||||
// twice its), so ninety times that noise is magenta — every black in the
|
||||
// frame turned pink. Mixing adds the lift as a neutral, so a black goes
|
||||
// to grey and its noise stays the size it was.
|
||||
//
|
||||
// The grey is (1, 1, 1) scaled, because this runs after white balance
|
||||
// in the camera's space, where that is what neutral is.
|
||||
c = mix(c, vec3<f32>(0.18), -amount);
|
||||
} else {
|
||||
let luma = luminance(c);
|
||||
// Only up to twice middle grey, which is the curve's whole domain. Above
|
||||
// it the curve's value is 1 and its slope 0, so leaving those tones
|
||||
// alone is the continuous continuation — where scaling them to the
|
||||
// curve's top, as this once did through a clamp, pinned every highlight
|
||||
// in the photograph to 0.36 at the smallest touch of the slider.
|
||||
if (luma > 0.0001 && luma < 0.36) {
|
||||
// Work on luminance and rescale the colour by the ratio, rather than
|
||||
// curving each channel independently. Per-channel contrast shifts hue
|
||||
// wherever the channels differ — the classic symptom being skies going
|
||||
// cyan as contrast rises. Safe here where it was not for flattening:
|
||||
// the S only ever pulls a shadow down, so the gain is at most one
|
||||
// below the pivot and noise is never amplified.
|
||||
//
|
||||
// MIDDLE_GREY is 0.18: the linear value the eye reads as mid-tone.
|
||||
// The curve operates on luma/(2*0.18) so that middle grey lands at
|
||||
// the curve's own 0.5 pivot.
|
||||
let pos = luma / 0.36;
|
||||
let curved = contrast_curve(pos, amount);
|
||||
// Not `target`: that is a WGSL reserved keyword, and using it produces a
|
||||
// parse error in generated code rather than anywhere a reader would look.
|
||||
let curved_luma = curved * 0.36;
|
||||
c = apply_tone_gain(c, curved_luma / luma);
|
||||
}
|
||||
}
|
||||
c = max(c, vec3<f32>(0.0));
|
||||
|
||||
@@ -104,6 +125,19 @@ tests:
|
||||
propagate through everything downstream.
|
||||
expect_wgsl: ["luma > 0.0001"]
|
||||
|
||||
- name: flattening_mixes_toward_grey_rather_than_scaling
|
||||
why: |
|
||||
Lifting a shadow by a luminance ratio multiplies its noise by the same
|
||||
ratio — ninety at the bottom of a night photograph — and after white
|
||||
balance that noise is magenta. A mix adds the lift as a neutral.
|
||||
expect_wgsl: ["mix(c, vec3<f32>(0.18), -amount)"]
|
||||
|
||||
- name: highlights_are_not_pinned_to_the_top_of_the_curve
|
||||
why: |
|
||||
The curve covers 0..0.36. A clamp into that range scaled every brighter
|
||||
pixel down to 0.36; tones above it are left as they are.
|
||||
expect_wgsl: ["luma < 0.36"]
|
||||
|
||||
- name: the_curve_cannot_invert
|
||||
why: |
|
||||
A gain-about-a-pivot form produces a non-monotonic curve past moderate
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
drpl 1
|
||||
|
||||
# Black and white film: each preset is one measured stock, developed and — for a
|
||||
# negative — printed on the paper its profile names, with every other
|
||||
# control left where it was. The stock is the look; a grade on top is the
|
||||
# photographer's to add.
|
||||
#
|
||||
# The measurements are spektrafilm's (Andrea Volpato, CC BY-SA 4.0), as
|
||||
# converted in core/dr-film/profiles. A preset names a stock by id, so
|
||||
# these lines are that attribution's reach: nothing of the data is here.
|
||||
|
||||
[preset Ilford Delta 100]
|
||||
film = ilford_delta_100
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Ilford Delta 400]
|
||||
film = ilford_delta_400
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Ilford FP4 Plus]
|
||||
film = ilford_fp4_plus
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Ilford HP5 Plus]
|
||||
film = ilford_hp5_plus
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Ilford Pan F Plus]
|
||||
film = ilford_pan_f_plus
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Kodak Double-X 5222]
|
||||
film = kodak_doublex
|
||||
film_print = kodak_2302
|
||||
|
||||
[preset Kodak Tri-X Reversal 7266]
|
||||
film = kodak_trix
|
||||
@@ -0,0 +1,30 @@
|
||||
drpl 1
|
||||
|
||||
# Cinema film: each preset is one measured stock, developed and — for a
|
||||
# negative — printed on the paper its profile names, with every other
|
||||
# control left where it was. The stock is the look; a grade on top is the
|
||||
# photographer's to add.
|
||||
#
|
||||
# The measurements are spektrafilm's (Andrea Volpato, CC BY-SA 4.0), as
|
||||
# converted in core/dr-film/profiles. A preset names a stock by id, so
|
||||
# these lines are that attribution's reach: nothing of the data is here.
|
||||
|
||||
[preset Kodak Verita 200D]
|
||||
film = kodak_verita_200d
|
||||
film_print = kodak_2383
|
||||
|
||||
[preset Kodak Vision3 200T]
|
||||
film = kodak_vision3_200t
|
||||
film_print = kodak_2383
|
||||
|
||||
[preset Kodak Vision3 250D]
|
||||
film = kodak_vision3_250d
|
||||
film_print = kodak_2383
|
||||
|
||||
[preset Kodak Vision3 500T]
|
||||
film = kodak_vision3_500t
|
||||
film_print = kodak_2383
|
||||
|
||||
[preset Kodak Vision3 50D]
|
||||
film = kodak_vision3_50d
|
||||
film_print = kodak_2383
|
||||
@@ -0,0 +1,66 @@
|
||||
drpl 1
|
||||
|
||||
# Colour film: each preset is one measured stock, developed and — for a
|
||||
# negative — printed on the paper its profile names, with every other
|
||||
# control left where it was. The stock is the look; a grade on top is the
|
||||
# photographer's to add.
|
||||
#
|
||||
# The measurements are spektrafilm's (Andrea Volpato, CC BY-SA 4.0), as
|
||||
# converted in core/dr-film/profiles. A preset names a stock by id, so
|
||||
# these lines are that attribution's reach: nothing of the data is here.
|
||||
|
||||
[preset Fujifilm C200]
|
||||
film = fujifilm_c200
|
||||
film_print = fujifilm_crystal_archive_typeii
|
||||
|
||||
[preset Fujifilm Pro 400H]
|
||||
film = fujifilm_pro_400h
|
||||
film_print = fujifilm_crystal_archive_typeii
|
||||
|
||||
[preset Fujifilm Provia 100F]
|
||||
film = fujifilm_provia_100f
|
||||
|
||||
[preset Fujifilm Velvia 100]
|
||||
film = fujifilm_velvia_100
|
||||
|
||||
[preset Fujifilm X-Tra 400]
|
||||
film = fujifilm_xtra_400
|
||||
film_print = fujifilm_crystal_archive_typeii
|
||||
|
||||
[preset Kodak Ektachrome 100]
|
||||
film = kodak_ektachrome_100
|
||||
|
||||
[preset Kodak Ektar 100]
|
||||
film = kodak_ektar_100
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Gold 200]
|
||||
film = kodak_gold_200
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Kodachrome 64]
|
||||
film = kodak_kodachrome_64
|
||||
|
||||
[preset Kodak Portra 160]
|
||||
film = kodak_portra_160
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Portra 400]
|
||||
film = kodak_portra_400
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Portra 800]
|
||||
film = kodak_portra_800
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Portra 800 pushed one stop]
|
||||
film = kodak_portra_800_push1
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Portra 800 pushed two stops]
|
||||
film = kodak_portra_800_push2
|
||||
film_print = kodak_portra_endura
|
||||
|
||||
[preset Kodak Ultramax 400]
|
||||
film = kodak_ultramax_400
|
||||
film_print = kodak_portra_endura
|
||||
@@ -0,0 +1,41 @@
|
||||
drpl 1
|
||||
|
||||
# Essentials: small, general corrections written against this pipeline.
|
||||
# These were the first-run starter set; they ship here now so that a
|
||||
# new release can improve them without rewriting anybody's own presets.
|
||||
# Deliberately mild — see bundled.rs.
|
||||
|
||||
[preset Crisp detail]
|
||||
capture_sharpen.amount = 35
|
||||
clarity.amount = 10
|
||||
texture.amount = 20
|
||||
|
||||
[preset Lift the shadows]
|
||||
blacks_whites.blacks = 12
|
||||
contrast.contrast = -5
|
||||
highlights_shadows.shadows = 40
|
||||
|
||||
[preset Muted]
|
||||
contrast.contrast = -10
|
||||
highlights_shadows.shadows = 12
|
||||
saturation.saturation = -30
|
||||
vibrance.vibrance = 10
|
||||
|
||||
[preset Punch]
|
||||
blacks_whites.blacks = -8
|
||||
clarity.amount = 12
|
||||
contrast.contrast = 18
|
||||
vibrance.vibrance = 18
|
||||
|
||||
[preset Recover the sky]
|
||||
blacks_whites.whites = -10
|
||||
highlights_shadows.highlights = -55
|
||||
highlights_shadows.shadows = 35
|
||||
|
||||
[preset Soft portrait]
|
||||
clarity.amount = -10
|
||||
contrast.contrast = -8
|
||||
highlights_shadows.highlights = -20
|
||||
highlights_shadows.shadows = 15
|
||||
saturation.saturation = -5
|
||||
vibrance.vibrance = 10
|
||||
@@ -0,0 +1,52 @@
|
||||
drpl 1
|
||||
|
||||
# Skies: a bluer, deeper sky without touching the rest of the picture.
|
||||
#
|
||||
# Written against this pipeline, not derived from anybody's preset. Each
|
||||
# works the colour mixer's azure and blue bands — the hues a clear sky
|
||||
# occupies, 210° and 240° — darkening them and adding chroma, which is what
|
||||
# a polarising filter does to a sky and why it reads as "more blue" rather
|
||||
# than "more saturated". A grey sky has no hue for the bands to find, so on
|
||||
# an overcast frame these do little, by construction: they cannot invent a
|
||||
# sky, and a preset that tinted grey clouds blue would be one nobody trusted.
|
||||
#
|
||||
# Highlights come down with the sky in the stronger ones, because a darker
|
||||
# blue beside a clipped white cloud looks like a mask edge.
|
||||
|
||||
[preset Blue sky]
|
||||
colour_mixer.azure_lum = -20
|
||||
colour_mixer.azure_sat = 25
|
||||
colour_mixer.blue_lum = -15
|
||||
colour_mixer.blue_sat = 20
|
||||
highlights_shadows.highlights = -15
|
||||
|
||||
[preset Deep blue sky]
|
||||
colour_mixer.azure_hue = 10
|
||||
colour_mixer.azure_lum = -30
|
||||
colour_mixer.azure_sat = 35
|
||||
colour_mixer.blue_lum = -25
|
||||
colour_mixer.blue_sat = 30
|
||||
colour_mixer.cyan_sat = 10
|
||||
highlights_shadows.highlights = -30
|
||||
|
||||
[preset Polariser]
|
||||
colour_mixer.azure_hue = 10
|
||||
colour_mixer.azure_lum = -35
|
||||
colour_mixer.azure_sat = 40
|
||||
colour_mixer.blue_lum = -30
|
||||
colour_mixer.blue_sat = 35
|
||||
colour_mixer.cyan_lum = -10
|
||||
colour_mixer.cyan_sat = 15
|
||||
dehaze.amount = 20
|
||||
highlights_shadows.highlights = -35
|
||||
vibrance.vibrance = 10
|
||||
|
||||
[preset Blue sky, golden land]
|
||||
colour_mixer.azure_lum = -20
|
||||
colour_mixer.azure_sat = 25
|
||||
colour_mixer.blue_lum = -15
|
||||
colour_mixer.blue_sat = 20
|
||||
colour_mixer.orange_sat = 12
|
||||
colour_mixer.yellow_hue = -10
|
||||
colour_mixer.yellow_sat = 15
|
||||
highlights_shadows.highlights = -20
|
||||
@@ -0,0 +1,445 @@
|
||||
//! TRACES: FR-DEV-6
|
||||
//! The presets that ship with the application.
|
||||
//!
|
||||
//! # Shipped, not seeded
|
||||
//!
|
||||
//! The first six used to be *copied* into the photographer's own library on
|
||||
//! the first run and were theirs from then on. That was the right answer for
|
||||
//! six, and it cannot grow: a copy is frozen at the version that made it, so a
|
||||
//! better "Portra" in the next release would reach nobody who already had the
|
||||
//! old one, and re-seeding would overwrite a preset someone had tuned. So the
|
||||
//! shipped set is now read from the binary every time, never written to the
|
||||
//! user's file, and changes when the application does.
|
||||
//!
|
||||
//! # Your copy wins, as long as it keeps the name
|
||||
//!
|
||||
//! Saving over a shipped preset's name makes the photographer's version the
|
||||
//! one that name means, here and in every apply. It is still *that* preset —
|
||||
//! listed where the shipped one was, marked as changed — and deleting it
|
||||
//! reveals the shipped one again, which is what "revert" means to the person
|
||||
//! pressing it. Renaming it cuts the link: it becomes one of their own, and
|
||||
//! the shipped preset reappears beside it. The lookup is by name because the
|
||||
//! name is what the photographer sees and chooses by; an id they never see
|
||||
//! would link two presets they believe are different.
|
||||
//!
|
||||
//! # Looks, not whole edits
|
||||
//!
|
||||
//! Every shipped preset reaches only the operations it names
|
||||
//! ([`Reach::Named`]): a look applied to a corrected photograph must keep the
|
||||
//! correction. A photographer's own saved edits keep [`Reach::Whole`], which
|
||||
//! is what saving an edit has always meant.
|
||||
//!
|
||||
//! # Why the data is text files
|
||||
//!
|
||||
//! The same format the user's library is written in, so a shipped preset can
|
||||
//! be read, diffed and copied into one's own library by hand, and each file
|
||||
//! can say in a comment where its looks came from — the attribution a licence
|
||||
//! may require travels with the data it covers.
|
||||
//!
|
||||
//! # Why this lives in the core
|
||||
//!
|
||||
//! It names operations — "Punch" is a statement about contrast and clarity —
|
||||
//! and nothing in `ui/` may (`ui_names_no_operation.rs`, ARCH §4.3a). The
|
||||
//! frontend asks for the listing and applies what it is handed.
|
||||
|
||||
use crate::preset::{Preset, PresetLibrary, Reach};
|
||||
|
||||
/// One group of shipped presets, as the sheet lists it.
|
||||
pub struct Section {
|
||||
/// A stable identifier, for a frontend that remembers which sections a
|
||||
/// photographer folded away. Never shown.
|
||||
pub id: &'static str,
|
||||
/// What the section is called on screen.
|
||||
pub title: &'static str,
|
||||
/// The presets in it, every one reaching only what it names.
|
||||
pub presets: PresetLibrary,
|
||||
}
|
||||
|
||||
/// The files, in the order the sheet lists them.
|
||||
const SECTIONS: &[(&str, &str, &str)] = &[
|
||||
(
|
||||
"essentials",
|
||||
"Essentials",
|
||||
include_str!("../presets/essentials.drpl"),
|
||||
),
|
||||
("skies", "Skies", include_str!("../presets/skies.drpl")),
|
||||
(
|
||||
"colour_film",
|
||||
"Colour film",
|
||||
include_str!("../presets/colour_film.drpl"),
|
||||
),
|
||||
(
|
||||
"cinema_film",
|
||||
"Cinema film",
|
||||
include_str!("../presets/cinema_film.drpl"),
|
||||
),
|
||||
(
|
||||
"bw_film",
|
||||
"Black and white film",
|
||||
include_str!("../presets/bw_film.drpl"),
|
||||
),
|
||||
];
|
||||
|
||||
/// Every shipped section, parsed.
|
||||
///
|
||||
/// Parsed on each call rather than held: it is a few kilobytes read when the
|
||||
/// preset sheet is drawn, and a static would be one more thing to keep
|
||||
/// consistent with the files in a test. A file that fails to parse costs its
|
||||
/// section, with a warning, rather than the sheet — and the tests below make
|
||||
/// sure none does.
|
||||
pub fn sections() -> Vec<Section> {
|
||||
SECTIONS
|
||||
.iter()
|
||||
.filter_map(|(id, title, text)| match PresetLibrary::parse(text) {
|
||||
Ok(library) => Some(Section {
|
||||
id,
|
||||
title,
|
||||
presets: as_looks(library),
|
||||
}),
|
||||
Err(e) => {
|
||||
log::warn!("shipped preset section {id} is unreadable ({e}); skipping");
|
||||
None
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Mark every preset in `library` as a look.
|
||||
///
|
||||
/// Here rather than as a `reach = named` line in every block of every file:
|
||||
/// the rule is about where a preset came from, and a file that forgot the
|
||||
/// line would ship a preset that wiped a photographer's corrections.
|
||||
fn as_looks(library: PresetLibrary) -> PresetLibrary {
|
||||
let mut looks = PresetLibrary::default();
|
||||
for (name, preset) in library.iter() {
|
||||
let _ = looks.insert(name, preset.clone().with_reach(Reach::Named));
|
||||
}
|
||||
looks
|
||||
}
|
||||
|
||||
/// Where a listed preset comes from.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum Origin {
|
||||
/// The photographer's own, with no shipped preset of that name.
|
||||
Yours,
|
||||
/// Shipped, and not overridden.
|
||||
Shipped,
|
||||
/// Shipped, and overridden by the photographer's copy under the same
|
||||
/// name. The copy is what applies; deleting it reverts to the shipped one.
|
||||
Changed,
|
||||
}
|
||||
|
||||
/// One row of the preset sheet.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Listed {
|
||||
pub name: String,
|
||||
pub origin: Origin,
|
||||
}
|
||||
|
||||
/// One group of rows: the photographer's own, then each shipped section.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ListedSection {
|
||||
/// `None` for the photographer's own presets.
|
||||
pub id: Option<&'static str>,
|
||||
pub title: &'static str,
|
||||
pub rows: Vec<Listed>,
|
||||
}
|
||||
|
||||
/// Everything the sheet lists, in order, with the photographer's copies
|
||||
/// standing in for the shipped presets they override.
|
||||
///
|
||||
/// Their own presets come first, because a photographer reaches for their own
|
||||
/// work more than for anybody's defaults, and a copy of a shipped preset is
|
||||
/// listed in the shipped section rather than among their own — it is still
|
||||
/// that preset, changed, and belongs where they would look for it.
|
||||
pub fn listing(yours: &PresetLibrary) -> Vec<ListedSection> {
|
||||
let shipped = sections();
|
||||
let is_shipped = |name: &str| shipped.iter().any(|section| section.presets.contains(name));
|
||||
|
||||
let mut out = vec![ListedSection {
|
||||
id: None,
|
||||
title: "Yours",
|
||||
rows: yours
|
||||
.names()
|
||||
.filter(|name| !is_shipped(name))
|
||||
.map(|name| Listed {
|
||||
name: name.to_string(),
|
||||
origin: Origin::Yours,
|
||||
})
|
||||
.collect(),
|
||||
}];
|
||||
out.extend(shipped.iter().map(|section| {
|
||||
ListedSection {
|
||||
id: Some(section.id),
|
||||
title: section.title,
|
||||
rows: section
|
||||
.presets
|
||||
.names()
|
||||
.map(|name| Listed {
|
||||
name: name.to_string(),
|
||||
origin: if yours.contains(name) {
|
||||
Origin::Changed
|
||||
} else {
|
||||
Origin::Shipped
|
||||
},
|
||||
})
|
||||
.collect(),
|
||||
}
|
||||
}));
|
||||
out
|
||||
}
|
||||
|
||||
/// The preset a name means: the photographer's if they have one, the shipped
|
||||
/// one otherwise.
|
||||
pub fn lookup(yours: &PresetLibrary, name: &str) -> Option<Preset> {
|
||||
yours.get(name).cloned().or_else(|| {
|
||||
sections()
|
||||
.into_iter()
|
||||
.find_map(|section| section.presets.get(name).cloned())
|
||||
})
|
||||
}
|
||||
|
||||
/// Whether `name` is a shipped preset's.
|
||||
pub fn is_shipped(name: &str) -> bool {
|
||||
sections()
|
||||
.iter()
|
||||
.any(|section| section.presets.contains(name))
|
||||
}
|
||||
|
||||
/// Remove the copies a first run used to seed, where they are still exactly
|
||||
/// as seeded. Returns how many went.
|
||||
///
|
||||
/// Those copies would otherwise all list as changed — overriding a shipped
|
||||
/// preset with an identical one — and would freeze the six at their old
|
||||
/// values forever. One that differs in any way was tuned by somebody and is
|
||||
/// kept: it is theirs, and it now overrides the shipped one, which is the
|
||||
/// rule above doing what it is for.
|
||||
///
|
||||
/// Compared on the parameters and the film and not on [`Reach`]: the seeded
|
||||
/// copies were whole edits, the shipped ones are looks, and that difference
|
||||
/// is the thing this migration exists to deliver.
|
||||
pub fn forget_unchanged_copies(yours: &mut PresetLibrary) -> usize {
|
||||
let shipped = sections();
|
||||
let stale: Vec<String> = yours
|
||||
.iter()
|
||||
.filter(|(name, preset)| {
|
||||
shipped.iter().any(|section| {
|
||||
section
|
||||
.presets
|
||||
.get(name)
|
||||
.is_some_and(|s| s.params() == preset.params() && s.film() == preset.film())
|
||||
})
|
||||
})
|
||||
.map(|(name, _)| name.to_string())
|
||||
.collect();
|
||||
for name in &stale {
|
||||
yours.remove(name);
|
||||
}
|
||||
stale.len()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{EditGraph, Scope};
|
||||
|
||||
fn all() -> Vec<(&'static str, String, Preset)> {
|
||||
sections()
|
||||
.into_iter()
|
||||
.flat_map(|s| {
|
||||
let id = s.id;
|
||||
s.presets
|
||||
.iter()
|
||||
.map(|(n, p)| (id, n.to_string(), p.clone()))
|
||||
.collect::<Vec<_>>()
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_file_parses_and_every_line_is_understood() {
|
||||
// A misspelt key would otherwise be kept as a line this build does
|
||||
// not understand — preserved faithfully, and doing nothing.
|
||||
assert_eq!(
|
||||
sections().len(),
|
||||
SECTIONS.len(),
|
||||
"a section failed to parse"
|
||||
);
|
||||
for (id, _, text) in SECTIONS {
|
||||
let library = PresetLibrary::parse(text).unwrap();
|
||||
assert_eq!(library.unread_lines(), 0, "{id} has lines nobody reads");
|
||||
assert!(!library.is_empty(), "{id} is empty");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_shipped_name_is_unique_across_sections() {
|
||||
// A name is what the lookup and the override key on. Two shipped
|
||||
// presets sharing one would make one of them unreachable.
|
||||
let mut names: Vec<String> = all().into_iter().map(|(_, n, _)| n).collect();
|
||||
let before = names.len();
|
||||
names.sort();
|
||||
names.dedup();
|
||||
assert_eq!(names.len(), before);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_shipped_preset_is_a_look() {
|
||||
for (id, name, preset) in all() {
|
||||
assert_eq!(preset.reach(), Reach::Named, "{id}/{name}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_shipped_preset_names_parameters_this_build_actually_has() {
|
||||
// A renamed parameter must break the build rather than ship a preset
|
||||
// that quietly does nothing.
|
||||
let graph = EditGraph::default_chain();
|
||||
let capabilities = graph.capabilities();
|
||||
for (id, name, preset) in all() {
|
||||
for (op, param) in preset.params().keys() {
|
||||
let capability = capabilities
|
||||
.iter()
|
||||
.find(|c| c.id.0 == op)
|
||||
.unwrap_or_else(|| panic!("{id}/{name}: no operation {op:?}"));
|
||||
assert!(
|
||||
capability.params.iter().any(|p| p.id.0 == param),
|
||||
"{id}/{name}: operation {op:?} has no parameter {param:?}"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_shipped_preset_changes_something() {
|
||||
// A preset that applies to nothing teaches the photographer that the
|
||||
// list does not work.
|
||||
for (id, name, preset) in all() {
|
||||
let mut graph = EditGraph::default_chain();
|
||||
let rebake = preset.apply(&mut graph, Scope::adjustments());
|
||||
assert!(
|
||||
rebake.wanted().is_some() || Preset::capture(&graph) != Preset::default(),
|
||||
"{id}/{name} left the graph at its defaults"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_shipped_preset_carries_a_crop() {
|
||||
for (id, name, preset) in all() {
|
||||
assert!(!preset.touches_framing(), "{id}/{name} carries framing");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_values_stay_inside_what_the_controls_accept() {
|
||||
// Clamping happens on apply, so an out-of-range literal would be
|
||||
// silently trimmed and the preset would not be the one written.
|
||||
for (id, name, preset) in all() {
|
||||
let mut graph = EditGraph::default_chain();
|
||||
preset
|
||||
.clone()
|
||||
.with_film(None)
|
||||
.apply(&mut graph, Scope::everything())
|
||||
.expect_no_film();
|
||||
for ((op, param), value) in preset.params() {
|
||||
let (op_id, param_id) = crate::preset::resolve(&graph, op, param).unwrap();
|
||||
assert_eq!(
|
||||
graph.param(op_id, param_id),
|
||||
Some(*value),
|
||||
"{id}/{name}: {op}.{param} = {value} was clamped"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- the listing and the override ------------------------------------
|
||||
|
||||
fn yours_with(names: &[(&str, Preset)]) -> PresetLibrary {
|
||||
let mut lib = PresetLibrary::default();
|
||||
for (n, p) in names {
|
||||
lib.insert(n, p.clone()).unwrap();
|
||||
}
|
||||
lib
|
||||
}
|
||||
|
||||
fn first_shipped() -> (String, Preset) {
|
||||
let (_, name, preset) = all().into_iter().next().unwrap();
|
||||
(name, preset)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn your_copy_under_a_shipped_name_is_what_that_name_applies() {
|
||||
let (name, shipped) = first_shipped();
|
||||
let mine = Preset::default();
|
||||
let yours = yours_with(&[(&name, mine.clone())]);
|
||||
|
||||
assert_eq!(lookup(&yours, &name), Some(mine));
|
||||
assert_eq!(lookup(&PresetLibrary::default(), &name), Some(shipped));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn your_copy_is_listed_in_the_shipped_section_as_changed() {
|
||||
let (name, _) = first_shipped();
|
||||
let yours = yours_with(&[(&name, Preset::default()), ("Mine", Preset::default())]);
|
||||
let listing = listing(&yours);
|
||||
|
||||
assert_eq!(listing[0].id, None);
|
||||
assert_eq!(
|
||||
listing[0].rows,
|
||||
vec![Listed {
|
||||
name: "Mine".into(),
|
||||
origin: Origin::Yours
|
||||
}],
|
||||
"an override must not be listed twice"
|
||||
);
|
||||
let row = listing[1..]
|
||||
.iter()
|
||||
.flat_map(|s| &s.rows)
|
||||
.find(|r| r.name == name)
|
||||
.unwrap();
|
||||
assert_eq!(row.origin, Origin::Changed);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn deleting_your_copy_reverts_to_the_shipped_one() {
|
||||
let (name, shipped) = first_shipped();
|
||||
let mut yours = yours_with(&[(&name, Preset::default())]);
|
||||
yours.remove(&name);
|
||||
assert_eq!(lookup(&yours, &name), Some(shipped));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn renaming_your_copy_cuts_the_link() {
|
||||
let (name, shipped) = first_shipped();
|
||||
let mut yours = yours_with(&[(&name, Preset::default())]);
|
||||
yours.rename(&name, "My version").unwrap();
|
||||
|
||||
assert_eq!(lookup(&yours, &name), Some(shipped));
|
||||
assert_eq!(lookup(&yours, "My version"), Some(Preset::default()));
|
||||
assert_eq!(listing(&yours)[0].rows[0].name, "My version");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_old_seeded_copies_are_forgotten_and_tuned_ones_kept() {
|
||||
// The six as a first run wrote them: the same parameters, as whole
|
||||
// edits, because that is what a seeded copy was.
|
||||
let essentials = sections().into_iter().next().unwrap().presets;
|
||||
let mut yours = PresetLibrary::default();
|
||||
for (name, preset) in essentials.iter() {
|
||||
yours
|
||||
.insert(name, preset.clone().with_reach(Reach::Whole))
|
||||
.unwrap();
|
||||
}
|
||||
let tuned = essentials.names().next().unwrap().to_string();
|
||||
yours.insert(&tuned, Preset::default()).unwrap();
|
||||
yours.insert("Mine", Preset::default()).unwrap();
|
||||
|
||||
let forgotten = forget_unchanged_copies(&mut yours);
|
||||
|
||||
assert_eq!(forgotten, essentials.len() - 1);
|
||||
assert!(yours.contains(&tuned), "a tuned copy was thrown away");
|
||||
assert!(yours.contains("Mine"));
|
||||
assert_eq!(yours.len(), 2);
|
||||
}
|
||||
}
|
||||
@@ -431,6 +431,24 @@ pub struct DetailPass {
|
||||
pub storage: Vec<[f32; 4]>,
|
||||
}
|
||||
|
||||
impl DetailPass {
|
||||
/// Whether this pass hands on exactly what it was given: a full-size pass
|
||||
/// with nothing to bind and a body with no code in it, only comments.
|
||||
///
|
||||
/// The composer drops such a pass where that is exact (see
|
||||
/// [`compose_detail_with`]); the operation still emits it, because whether
|
||||
/// dropping it is exact depends on what is around it in the chain.
|
||||
pub fn is_identity(&self) -> bool {
|
||||
self.output_scale <= 1
|
||||
&& self.storage.is_empty()
|
||||
&& self
|
||||
.wgsl
|
||||
.lines()
|
||||
.map(|l| l.split("//").next().unwrap_or("").trim())
|
||||
.all(str::is_empty)
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3 | FR-DEV-8
|
||||
/// An operation that reads pixels other than the one it is writing.
|
||||
///
|
||||
@@ -646,6 +664,40 @@ pub fn compose_detail_with(
|
||||
};
|
||||
}
|
||||
|
||||
// TRACES: NFR-P5
|
||||
// A pass whose body is empty changes nothing but where the pixels are: it
|
||||
// reads the intermediate and writes the same values to the other one.
|
||||
// Capture sharpening emits exactly that at a scale too coarse to draw its
|
||||
// radius (`nothing_to_sharpen`), and at fit on any modern sensor that is
|
||||
// most of the time — so an edit with sharpening *and* another kernel paid
|
||||
// a full render-sized read and write for it on every frame, 4.4 ms of a
|
||||
// 2560 x 1600 frame on the reference laptop with its clocks held down.
|
||||
//
|
||||
// Dropped here, where the chain is still a list, and only where dropping
|
||||
// it is exact:
|
||||
//
|
||||
// - **Not the last pass.** The last pass performs the output transform on
|
||||
// what it read from an `rgba16float` intermediate. Moving that transform
|
||||
// onto the pass before would apply it to that pass's `f32` result
|
||||
// instead, which is a different rounding of the same picture.
|
||||
// - **Not after a reduced pass.** A full-resolution pass ends the reduced
|
||||
// chain (see `DetailRunner::encode`), so one that follows a scaled pass
|
||||
// is what stops the next operation reading the last one's base. None of
|
||||
// today's operations leave a reduced chain open, but a declared one may.
|
||||
//
|
||||
// Everywhere else the pass before and the pass after exchange the same
|
||||
// `rgba16float` texels either way, `aux` included.
|
||||
let mut kept: Vec<(&str, &[Helper], DetailPass, usize)> = Vec::with_capacity(planned.len());
|
||||
let total = planned.len();
|
||||
for (position, entry) in planned.into_iter().enumerate() {
|
||||
let after_full = kept.last().is_none_or(|(_, _, p, _)| p.output_scale <= 1);
|
||||
let droppable = position + 1 < total && after_full && entry.2.is_identity();
|
||||
if !droppable {
|
||||
kept.push(entry);
|
||||
}
|
||||
}
|
||||
let planned = kept;
|
||||
|
||||
let last = planned.len().saturating_sub(1);
|
||||
let passes = planned
|
||||
.into_iter()
|
||||
@@ -1211,6 +1263,56 @@ mod tests {
|
||||
assert_eq!(fused(&ops).output_mode, OutputMode::LinearWorking);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_pass_that_changes_nothing_is_dropped_where_that_is_exact() {
|
||||
// TRACES: NFR-P5
|
||||
// Capture sharpening at a scale too coarse to draw its radius emits a
|
||||
// pass with an empty body. Between two other passes it costs a
|
||||
// render-sized read and write and changes no texel, so it goes; as the
|
||||
// last pass it performs the output transform on the intermediate, and
|
||||
// moving that onto the pass before would round differently, so it
|
||||
// stays.
|
||||
use crate::ops::{capture_sharpen, CaptureSharpen, NoiseReduction};
|
||||
let sharpen = || -> Box<dyn Operation> {
|
||||
let mut op = CaptureSharpen::new();
|
||||
op.set_param(capture_sharpen::AMOUNT, 60.0);
|
||||
Box::new(op)
|
||||
};
|
||||
let chroma = || -> Box<dyn Operation> { Box::new(NoiseReduction::with_amounts(0.0, 60.0)) };
|
||||
// A 24 MP frame fitted to a panel: a one-source-pixel radius is a
|
||||
// quarter of a render pixel.
|
||||
let scale = RenderScale::new((1500, 1000), (6000, 4000));
|
||||
let unresolved = sharpen().detail().expect("a detail stage").passes(scale);
|
||||
assert!(
|
||||
unresolved.len() == 1 && unresolved[0].is_identity(),
|
||||
"the premise: sharpening at this scale is one pass that does nothing"
|
||||
);
|
||||
let labels = |ops: &[Box<dyn Operation>]| -> Vec<String> {
|
||||
compose_detail(ops, scale, dr_types::ColourSpace::Srgb)
|
||||
.passes
|
||||
.iter()
|
||||
.map(|p| p.label.clone())
|
||||
.collect()
|
||||
};
|
||||
|
||||
// First, ahead of the chroma passes: dropped.
|
||||
let first = labels(&[sharpen(), chroma()]);
|
||||
assert_eq!(
|
||||
first,
|
||||
[
|
||||
"noise_reduction/chroma-horizontal",
|
||||
"noise_reduction/chroma-vertical"
|
||||
]
|
||||
);
|
||||
// Last, after them: kept, and it is the pass that encodes.
|
||||
let last = labels(&[chroma(), sharpen()]);
|
||||
assert_eq!(last.len(), 3);
|
||||
assert_eq!(last[2], "capture_sharpen/unresolved");
|
||||
// Alone: kept, because the fused pass stopped short and something has
|
||||
// to finish the frame.
|
||||
assert_eq!(labels(&[sharpen()]), ["capture_sharpen/unresolved"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_pass_can_hand_a_scalar_to_the_next_one_alongside_the_colour() {
|
||||
// What makes an unsharp mask — sharpening, clarity, texture, dehaze —
|
||||
|
||||
@@ -27,6 +27,33 @@
|
||||
//! chain exactly the space it documents: normalised, centred, `r == 1` at the
|
||||
//! corner. Neither stage needs to know the other exists.
|
||||
//!
|
||||
//! # Perspective sits inside framing (FR-DEV-20)
|
||||
//!
|
||||
//! A keystone correction is composition too — straightening converging
|
||||
//! verticals reframes the photograph — so it is a step *of* framing rather
|
||||
//! than a stage beside it, and inherits framing's `Compose` attribute, its
|
||||
//! place in the sidecar and its exclusion from a default paste. Expanded, the
|
||||
//! chain reads:
|
||||
//!
|
||||
//! ```text
|
||||
//! output pixel → crop → straighten → perspective → orientation → warp (lens) → sample
|
||||
//! ```
|
||||
//!
|
||||
//! After the straightening, because the angle is a nudge applied to the
|
||||
//! corrected picture: the verticals are made parallel and *then* the whole is
|
||||
//! levelled. Before the stored orientation, because "vertical" means vertical
|
||||
//! in the photograph as it is shown — a portrait frame the camera stored on
|
||||
//! its side must converge along its displayed height, not along the sensor's
|
||||
//! rows. And before the lens warp, which still sees the whole frame it
|
||||
//! corrects, for the reason given above.
|
||||
//!
|
||||
//! The correction maps the output frame onto a trapezoid **inside** the
|
||||
//! source rather than pulling the source edges in. So a keystone on its own
|
||||
//! never exposes an empty corner, and the crop the user drew is still valid
|
||||
//! after it; only in combination with a straightening angle does the
|
||||
//! inscribed crop have anything to account for — see
|
||||
//! [`Framing::max_inscribed_crop`].
|
||||
//!
|
||||
//! # Why sampling changes with the angle
|
||||
//!
|
||||
//! At 90° steps and flips, output pixels land exactly on source pixels, so
|
||||
@@ -58,6 +85,13 @@ pub const CROP_X: ParamId = ParamId("crop_x");
|
||||
pub const CROP_Y: ParamId = ParamId("crop_y");
|
||||
pub const CROP_W: ParamId = ParamId("crop_w");
|
||||
pub const CROP_H: ParamId = ParamId("crop_h");
|
||||
/// TRACES: FR-DEV-20
|
||||
/// Vertical keystone. Positive spreads the top of the frame — the correction
|
||||
/// for a building photographed looking up, whose verticals lean together.
|
||||
pub const KEYSTONE_V: ParamId = ParamId("keystone_v");
|
||||
/// TRACES: FR-DEV-20
|
||||
/// Horizontal keystone. Positive spreads the right-hand side of the frame.
|
||||
pub const KEYSTONE_H: ParamId = ParamId("keystone_h");
|
||||
|
||||
/// Widest straightening the control offers, in degrees either way.
|
||||
///
|
||||
@@ -66,12 +100,28 @@ pub const CROP_H: ParamId = ParamId("crop_h");
|
||||
/// the edits actually are.
|
||||
pub const MAX_STRAIGHTEN: f32 = 45.0;
|
||||
|
||||
/// The keystone sliders' travel either way.
|
||||
///
|
||||
/// A plain amount rather than degrees of tilt: the angle a camera was tilted
|
||||
/// by depends on a focal length the correction does not know, and a number
|
||||
/// that claimed to be one would be wrong for every lens but one.
|
||||
pub const MAX_KEYSTONE: f32 = 100.0;
|
||||
|
||||
/// How far a full keystone narrows the far edge of the frame, as a fraction
|
||||
/// of its width: at `MAX_KEYSTONE` the source trapezoid's short side is half
|
||||
/// its long one.
|
||||
///
|
||||
/// Enough for a tall building from its own pavement, and short of the point
|
||||
/// where the stretched edge is so magnified that the correction reads as a
|
||||
/// fault of its own.
|
||||
const KEYSTONE_REACH: f64 = 0.5;
|
||||
|
||||
/// The parameters the framing widget owns — every one of them.
|
||||
///
|
||||
/// In the order the widget expects: the rect first, then the angle it is
|
||||
/// straightened by, then the exact reorientations.
|
||||
static FRAMING_PARAMS: [ParamId; 8] = [
|
||||
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, ROTATION, FLIP_H, FLIP_V,
|
||||
static FRAMING_PARAMS: [ParamId; 10] = [
|
||||
CROP_X, CROP_Y, CROP_W, CROP_H, ANGLE, KEYSTONE_V, KEYSTONE_H, ROTATION, FLIP_H, FLIP_V,
|
||||
];
|
||||
|
||||
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
@@ -117,6 +167,30 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
ParamDescriptor::fraction("crop_y", "param.crop_y", 0.0),
|
||||
ParamDescriptor::fraction("crop_w", "param.crop_w", 1.0),
|
||||
ParamDescriptor::fraction("crop_h", "param.crop_h", 1.0),
|
||||
// TRACES: FR-DEV-20
|
||||
// Perspective. Last so every sidecar written before these existed
|
||||
// reads exactly as it did: a missing parameter is its default, and
|
||||
// the default is no correction.
|
||||
ParamDescriptor::scalar(
|
||||
"keystone_v",
|
||||
"param.keystone_v",
|
||||
-MAX_KEYSTONE,
|
||||
MAX_KEYSTONE,
|
||||
0.0,
|
||||
Unit::None,
|
||||
Scale::Linear,
|
||||
0,
|
||||
),
|
||||
ParamDescriptor::scalar(
|
||||
"keystone_h",
|
||||
"param.keystone_h",
|
||||
-MAX_KEYSTONE,
|
||||
MAX_KEYSTONE,
|
||||
0.0,
|
||||
Unit::None,
|
||||
Scale::Linear,
|
||||
0,
|
||||
),
|
||||
],
|
||||
})
|
||||
});
|
||||
@@ -311,6 +385,86 @@ fn finite(v: f32, fallback: f32) -> f32 {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-20
|
||||
/// A plane projective map, row-major, acting on `(x, y, 1)`.
|
||||
///
|
||||
/// Held in `f64` because it is built by solving for four corners and then
|
||||
/// inverted for [`Framing::output_at`]; the shader gets `f32` copies of the
|
||||
/// forward map only.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
struct Homography([[f64; 3]; 3]);
|
||||
|
||||
impl Homography {
|
||||
/// The map taking the square `[-0.5, 0.5]²` onto the quadrilateral whose
|
||||
/// corners are `q`, listed top-left, top-right, bottom-right, bottom-left.
|
||||
///
|
||||
/// Heckbert's closed form for the unit square, composed with the shift
|
||||
/// from the centred square onto it.
|
||||
fn square_to_quad(q: [(f64, f64); 4]) -> Self {
|
||||
let [(x0, y0), (x1, y1), (x2, y2), (x3, y3)] = q;
|
||||
let (dx1, dx2, dx3) = (x1 - x2, x3 - x2, x0 - x1 + x2 - x3);
|
||||
let (dy1, dy2, dy3) = (y1 - y2, y3 - y2, y0 - y1 + y2 - y3);
|
||||
let den = dx1 * dy2 - dx2 * dy1;
|
||||
let (g, h) = if den.abs() < 1e-12 {
|
||||
(0.0, 0.0)
|
||||
} else {
|
||||
((dx3 * dy2 - dx2 * dy3) / den, (dx1 * dy3 - dx3 * dy1) / den)
|
||||
};
|
||||
// Unit square (u, v) -> quad.
|
||||
let unit = [
|
||||
[x1 - x0 + g * x1, x3 - x0 + h * x3, x0],
|
||||
[y1 - y0 + g * y1, y3 - y0 + h * y3, y0],
|
||||
[g, h, 1.0],
|
||||
];
|
||||
// Centred square -> unit square is `u = x + 0.5`, so fold the shift
|
||||
// into the constant column.
|
||||
let mut m = unit;
|
||||
for row in &mut m {
|
||||
row[2] += 0.5 * (row[0] + row[1]);
|
||||
}
|
||||
Self(m)
|
||||
}
|
||||
|
||||
/// Where `(x, y)` lands, or `None` past the line the map sends to
|
||||
/// infinity — a point with no image, which the caller treats as outside
|
||||
/// the source.
|
||||
fn apply(&self, (x, y): (f64, f64)) -> Option<(f64, f64)> {
|
||||
let m = &self.0;
|
||||
let w = m[2][0] * x + m[2][1] * y + m[2][2];
|
||||
if w <= 1e-9 {
|
||||
return None;
|
||||
}
|
||||
Some((
|
||||
(m[0][0] * x + m[0][1] * y + m[0][2]) / w,
|
||||
(m[1][0] * x + m[1][1] * y + m[1][2]) / w,
|
||||
))
|
||||
}
|
||||
|
||||
/// The inverse map, by the adjugate. Scale is irrelevant to a projective
|
||||
/// map, so the determinant is only divided out to keep `w` positive and
|
||||
/// near one — which is what [`Self::apply`]'s horizon test relies on.
|
||||
fn inverse(&self) -> Self {
|
||||
let m = &self.0;
|
||||
let c = |r0: usize, c0: usize, r1: usize, c1: usize| {
|
||||
m[r0][c0] * m[r1][c1] - m[r0][c1] * m[r1][c0]
|
||||
};
|
||||
let adj = [
|
||||
[c(1, 1, 2, 2), -c(0, 1, 2, 2), c(0, 1, 1, 2)],
|
||||
[-c(1, 0, 2, 2), c(0, 0, 2, 2), -c(0, 0, 1, 2)],
|
||||
[c(1, 0, 2, 1), -c(0, 0, 2, 1), c(0, 0, 1, 1)],
|
||||
];
|
||||
let det = m[0][0] * adj[0][0] + m[0][1] * adj[1][0] + m[0][2] * adj[2][0];
|
||||
let det = if det.abs() < 1e-12 { 1.0 } else { det };
|
||||
let mut inv = adj;
|
||||
for row in &mut inv {
|
||||
for v in row.iter_mut() {
|
||||
*v /= det;
|
||||
}
|
||||
}
|
||||
Self(inv)
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3 | FR-DEV-3d
|
||||
/// Crop, straighten, rotation and flips for one image.
|
||||
///
|
||||
@@ -325,6 +479,10 @@ pub struct Framing {
|
||||
quarter_turns: u8,
|
||||
flip_h: bool,
|
||||
flip_v: bool,
|
||||
/// TRACES: FR-DEV-20
|
||||
/// Vertical and horizontal keystone, each `-MAX_KEYSTONE..=MAX_KEYSTONE`.
|
||||
keystone_v: f32,
|
||||
keystone_h: f32,
|
||||
/// TRACES: FR-DEV-3h
|
||||
/// How the file's pixels were stored, from its EXIF orientation.
|
||||
///
|
||||
@@ -376,6 +534,8 @@ impl Default for Framing {
|
||||
quarter_turns: 0,
|
||||
flip_h: false,
|
||||
flip_v: false,
|
||||
keystone_v: 0.0,
|
||||
keystone_h: 0.0,
|
||||
baseline: dr_types::Orientation::NORMAL,
|
||||
crop: CropRect::default(),
|
||||
view: CropRect::default(),
|
||||
@@ -396,7 +556,7 @@ impl Framing {
|
||||
/// How framing would like to be presented.
|
||||
///
|
||||
/// **This is what stops a frontend having to name this stage.** Rendered
|
||||
/// generically these eight parameters are eight bad controls: four crop
|
||||
/// generically these ten parameters are ten bad controls: four crop
|
||||
/// edges the photographer would have to type coordinates into, a "rotate"
|
||||
/// slider running 0..3, and two switches. Every one of them is a worse
|
||||
/// control than the gesture it stands for — a crop is dragged on the
|
||||
@@ -407,10 +567,10 @@ impl Framing {
|
||||
/// a second frontend would have had to learn the same special case, and
|
||||
/// nothing in the capability output said why. Now the preference is
|
||||
/// declared, the demand says what the widget needs, and a frontend that
|
||||
/// cannot meet it falls back to the eight sliders — tedious, but complete,
|
||||
/// cannot meet it falls back to the ten sliders — tedious, but complete,
|
||||
/// which is the guarantee the whole hint mechanism rests on.
|
||||
///
|
||||
/// The widget owns **all eight** parameters rather than only the rect: a
|
||||
/// The widget owns **all ten** parameters rather than only the rect: a
|
||||
/// frontend that takes this on is taking on the whole framing control
|
||||
/// surface, and leaving rotation and the flips behind would scatter them
|
||||
/// into the generated panel underneath a crop control that already exists.
|
||||
@@ -476,6 +636,52 @@ impl Framing {
|
||||
(self.flip_h, self.flip_v)
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-20
|
||||
/// The vertical and horizontal keystone, as the sliders show them.
|
||||
pub fn keystone(&self) -> (f32, f32) {
|
||||
(self.keystone_v, self.keystone_h)
|
||||
}
|
||||
|
||||
/// Whether a perspective correction is applied at all.
|
||||
pub fn has_keystone(&self) -> bool {
|
||||
self.keystone_v != 0.0 || self.keystone_h != 0.0
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-20
|
||||
/// The perspective map, from the straightened output frame to the upright
|
||||
/// source frame, both measured as the centred square `[-0.5, 0.5]²`.
|
||||
///
|
||||
/// **Measured in fractions of the frame, not in the aspect-scaled space
|
||||
/// the rest of the prologue works in**, so the map is the same for every
|
||||
/// frame shape and the uniforms need no image size. The prologue divides
|
||||
/// `p.x` by the frame's aspect on the way in and multiplies it back on
|
||||
/// the way out.
|
||||
///
|
||||
/// The output frame's corners go to a trapezoid inside the source: a
|
||||
/// positive vertical keystone brings the top corners in, so the top of
|
||||
/// the source is spread across the full width of the output and lines
|
||||
/// that converged upward come out parallel. Nothing is ever mapped from
|
||||
/// outside the source, which is why a keystone alone needs no crop.
|
||||
fn keystone_map(&self) -> Option<Homography> {
|
||||
if !self.has_keystone() {
|
||||
return None;
|
||||
}
|
||||
let amount = |v: f32| f64::from(v / MAX_KEYSTONE).clamp(-1.0, 1.0) * KEYSTONE_REACH;
|
||||
let (tv, th) = (amount(self.keystone_v), amount(self.keystone_h));
|
||||
// How much of each edge survives: the top and bottom rows' widths,
|
||||
// the left and right columns' heights.
|
||||
let top = 1.0 - tv.max(0.0);
|
||||
let bottom = 1.0 + tv.min(0.0);
|
||||
let right = 1.0 - th.max(0.0);
|
||||
let left = 1.0 + th.min(0.0);
|
||||
Some(Homography::square_to_quad([
|
||||
(-0.5 * top, -0.5 * left),
|
||||
(0.5 * top, -0.5 * right),
|
||||
(0.5 * bottom, 0.5 * right),
|
||||
(-0.5 * bottom, 0.5 * left),
|
||||
]))
|
||||
}
|
||||
|
||||
/// Add quarter turns, wrapping. The rotate-left/right buttons.
|
||||
pub fn rotate_quarters(&mut self, turns: i32) {
|
||||
self.quarter_turns = (i32::from(self.quarter_turns) + turns).rem_euclid(4) as u8;
|
||||
@@ -544,6 +750,7 @@ impl Framing {
|
||||
pub fn is_active(&self) -> bool {
|
||||
let (turns, flip_h, flip_v) = self.effective();
|
||||
self.angle != 0.0
|
||||
|| self.has_keystone()
|
||||
// Effective, not the user's: a file stored sideways needs the
|
||||
// prologue emitted even on an untouched image, or it renders
|
||||
// through the identity map and lies on its side.
|
||||
@@ -572,6 +779,7 @@ impl Framing {
|
||||
/// edited, the file was merely read correctly.
|
||||
pub fn edits_image(&self) -> bool {
|
||||
self.angle != 0.0
|
||||
|| self.has_keystone()
|
||||
|| self.quarter_turns != 0
|
||||
|| self.flip_h
|
||||
|| self.flip_v
|
||||
@@ -591,7 +799,9 @@ impl Framing {
|
||||
/// warp being active forces interpolation regardless, which is the
|
||||
/// composer's call to make rather than this stage's.
|
||||
pub fn needs_interpolation(&self) -> bool {
|
||||
self.angle != 0.0
|
||||
// A keystone stretches the frame by a different amount at every row,
|
||||
// so it lands between pixels everywhere but on its centre line.
|
||||
self.angle != 0.0 || self.has_keystone()
|
||||
}
|
||||
|
||||
pub fn set_param(&mut self, id: ParamId, value: f32) {
|
||||
@@ -629,6 +839,8 @@ impl Framing {
|
||||
}
|
||||
.normalised()
|
||||
}
|
||||
KEYSTONE_V => self.keystone_v = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
|
||||
KEYSTONE_H => self.keystone_h = finite(value, 0.0).clamp(-MAX_KEYSTONE, MAX_KEYSTONE),
|
||||
_ => log::warn!("framing: unknown parameter {id}"),
|
||||
}
|
||||
}
|
||||
@@ -643,6 +855,8 @@ impl Framing {
|
||||
CROP_Y => self.crop.y,
|
||||
CROP_W => self.crop.width,
|
||||
CROP_H => self.crop.height,
|
||||
KEYSTONE_V => self.keystone_v,
|
||||
KEYSTONE_H => self.keystone_h,
|
||||
_ => 0.0,
|
||||
}
|
||||
}
|
||||
@@ -707,8 +921,22 @@ impl Framing {
|
||||
///
|
||||
/// The standard largest-inscribed-rectangle result for a rotated
|
||||
/// rectangle of the same aspect ratio.
|
||||
///
|
||||
/// TRACES: FR-DEV-20
|
||||
/// **With a keystone the closed form no longer applies**: the area with a
|
||||
/// source pixel behind it is the source rectangle pulled back through the
|
||||
/// perspective map and then turned, a quadrilateral no textbook result
|
||||
/// describes. That case is searched instead — see
|
||||
/// [`Self::inscribed_by_search`]. A keystone alone never needs a crop, so
|
||||
/// the search returns the whole frame for it, exactly.
|
||||
pub fn max_inscribed_crop(&self, width: u32, height: u32) -> CropRect {
|
||||
if self.angle == 0.0 || width == 0 || height == 0 {
|
||||
if width == 0 || height == 0 {
|
||||
return CropRect::default();
|
||||
}
|
||||
if self.has_keystone() {
|
||||
return self.inscribed_by_search(width, height);
|
||||
}
|
||||
if self.angle == 0.0 {
|
||||
return CropRect::default();
|
||||
}
|
||||
|
||||
@@ -750,6 +978,123 @@ impl Framing {
|
||||
.normalised()
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-20
|
||||
/// The largest centred crop with a source pixel behind every point, found
|
||||
/// by search rather than by formula.
|
||||
///
|
||||
/// The area that has a source pixel behind it is convex — the source
|
||||
/// rectangle pulled back through a projective map whose horizon lies
|
||||
/// outside it, then turned — and a rectangle lies inside a convex region
|
||||
/// exactly when its four corners do. For a given width the tallest
|
||||
/// rectangle that fits is therefore found by bisection, and the area
|
||||
/// `width × tallest(width)` is unimodal in the width (a positive concave
|
||||
/// function times a line), so a golden-section search finds the best
|
||||
/// width. Every rectangle returned has been tested corner by corner, so
|
||||
/// the answer errs inside, never outside.
|
||||
///
|
||||
/// A few hundred corner tests, on a gesture's release, is nothing next to
|
||||
/// the render that follows it.
|
||||
fn inscribed_by_search(&self, width: u32, height: u32) -> CropRect {
|
||||
let (w, h) = if self.swaps_axes() {
|
||||
(f64::from(height), f64::from(width))
|
||||
} else {
|
||||
(f64::from(width), f64::from(height))
|
||||
};
|
||||
let fa = w / h;
|
||||
let rad = f64::from(self.angle).to_radians();
|
||||
let (sn, cs) = (rad.sin(), rad.cos());
|
||||
let map = self.keystone_map();
|
||||
|
||||
// Whether the output point `(fx, fy)`, in fractions of the frame from
|
||||
// its centre, has a source pixel behind it. The prologue's steps, in
|
||||
// its order, stopping short of the turns: those are a permutation of
|
||||
// the frame and cannot move a point across its edge.
|
||||
let defined = |fx: f64, fy: f64| {
|
||||
let p = (fx * fa, fy);
|
||||
let q = (p.0 * cs - p.1 * sn, p.0 * sn + p.1 * cs);
|
||||
let n = (q.0 / fa, q.1);
|
||||
let src = match &map {
|
||||
Some(m) => m.apply(n),
|
||||
None => Some(n),
|
||||
};
|
||||
const EDGE: f64 = 0.5 + 1e-9;
|
||||
src.is_some_and(|(x, y)| x.abs() <= EDGE && y.abs() <= EDGE)
|
||||
};
|
||||
let fits = |hw: f64, hh: f64| {
|
||||
[(-1.0, -1.0), (1.0, -1.0), (1.0, 1.0), (-1.0, 1.0)]
|
||||
.iter()
|
||||
.all(|(sx, sy)| defined(sx * hw, sy * hh))
|
||||
};
|
||||
|
||||
if fits(0.5, 0.5) {
|
||||
return CropRect::default();
|
||||
}
|
||||
|
||||
// Half the tallest height that fits at half-width `hw`.
|
||||
let tallest = |hw: f64| {
|
||||
if fits(hw, 0.5) {
|
||||
return 0.5;
|
||||
}
|
||||
if !fits(hw, 0.0) {
|
||||
return 0.0;
|
||||
}
|
||||
let (mut lo, mut hi) = (0.0, 0.5);
|
||||
for _ in 0..40 {
|
||||
let mid = 0.5 * (lo + hi);
|
||||
if fits(hw, mid) {
|
||||
lo = mid;
|
||||
} else {
|
||||
hi = mid;
|
||||
}
|
||||
}
|
||||
lo
|
||||
};
|
||||
let area = |hw: f64| hw * tallest(hw);
|
||||
|
||||
let ratio = (5.0_f64.sqrt() - 1.0) * 0.5;
|
||||
let (mut a, mut b) = (0.0, 0.5);
|
||||
let mut c = b - ratio * (b - a);
|
||||
let mut d = a + ratio * (b - a);
|
||||
let (mut fc, mut fd) = (area(c), area(d));
|
||||
for _ in 0..48 {
|
||||
if fc < fd {
|
||||
a = c;
|
||||
c = d;
|
||||
fc = fd;
|
||||
d = a + ratio * (b - a);
|
||||
fd = area(d);
|
||||
} else {
|
||||
b = d;
|
||||
d = c;
|
||||
fd = fc;
|
||||
c = b - ratio * (b - a);
|
||||
fc = area(c);
|
||||
}
|
||||
}
|
||||
let hw = 0.5 * (a + b);
|
||||
let hh = tallest(hw);
|
||||
// Tested at `hw` as returned, so a width that the search's last step
|
||||
// nudged past the boundary cannot come back with a height that no
|
||||
// longer fits it.
|
||||
let (hw, hh) = if hh > 0.0 && fits(hw, hh) {
|
||||
(hw, hh)
|
||||
} else {
|
||||
(c.min(d), tallest(c.min(d)))
|
||||
};
|
||||
|
||||
let fw = (2.0 * hw) as f32;
|
||||
let fh = (2.0 * hh) as f32;
|
||||
let fw = fw.clamp(CropRect::MIN_EXTENT, 1.0);
|
||||
let fh = fh.clamp(CropRect::MIN_EXTENT, 1.0);
|
||||
CropRect {
|
||||
x: (1.0 - fw) * 0.5,
|
||||
y: (1.0 - fh) * 0.5,
|
||||
width: fw,
|
||||
height: fh,
|
||||
}
|
||||
.normalised()
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3
|
||||
/// Where an output point comes from in the source, both in normalised
|
||||
/// `0..1` coordinates.
|
||||
@@ -782,6 +1127,15 @@ impl Framing {
|
||||
p = (p.0 * c - p.1 * s, p.0 * s + p.1 * c);
|
||||
}
|
||||
|
||||
if let Some(m) = self.keystone_map() {
|
||||
p = match m.apply((f64::from(p.0 / fx), f64::from(p.1))) {
|
||||
Some((x, y)) => (x as f32 * fx, y as f32),
|
||||
// Beyond the map's horizon: no source point at all, reported
|
||||
// as one far outside the frame rather than as a NaN.
|
||||
None => (1e6, 1e6),
|
||||
};
|
||||
}
|
||||
|
||||
let (turns, flip_h, flip_v) = self.effective();
|
||||
p = match turns {
|
||||
1 => (p.1 * ax, -p.0 / fx),
|
||||
@@ -824,6 +1178,13 @@ impl Framing {
|
||||
_ => p,
|
||||
};
|
||||
|
||||
if let Some(m) = self.keystone_map() {
|
||||
p = match m.inverse().apply((f64::from(p.0 / fx), f64::from(p.1))) {
|
||||
Some((x, y)) => (x as f32 * fx, y as f32),
|
||||
None => (1e6, 1e6),
|
||||
};
|
||||
}
|
||||
|
||||
if self.angle != 0.0 {
|
||||
let rad = -self.angle * PI / 180.0;
|
||||
let (s, c) = (rad.sin(), rad.cos());
|
||||
@@ -865,6 +1226,19 @@ impl Framing {
|
||||
// identical whether or not the user is zoomed in, and costs no extra
|
||||
// uniform slot.
|
||||
let rect = self.visible_rect();
|
||||
// The perspective map by columns, so the prologue can apply it as
|
||||
// three multiply-adds. The identity when there is none: the slots
|
||||
// exist either way and the prologue does not read them.
|
||||
let m = self
|
||||
.keystone_map()
|
||||
.unwrap_or(Homography([
|
||||
[1.0, 0.0, 0.0],
|
||||
[0.0, 1.0, 0.0],
|
||||
[0.0, 0.0, 1.0],
|
||||
]))
|
||||
.0;
|
||||
let col = |c: usize| [m[0][c] as f32, m[1][c] as f32, m[2][c] as f32, 0.0];
|
||||
let [c0, c1, c2] = [col(0), col(1), col(2)];
|
||||
[
|
||||
rect.x,
|
||||
rect.y,
|
||||
@@ -874,6 +1248,18 @@ impl Framing {
|
||||
rad.cos(),
|
||||
0.0,
|
||||
0.0,
|
||||
c0[0],
|
||||
c0[1],
|
||||
c0[2],
|
||||
c0[3],
|
||||
c1[0],
|
||||
c1[1],
|
||||
c1[2],
|
||||
c1[3],
|
||||
c2[0],
|
||||
c2[1],
|
||||
c2[2],
|
||||
c2[3],
|
||||
]
|
||||
}
|
||||
|
||||
@@ -972,6 +1358,28 @@ impl Framing {
|
||||
);
|
||||
}
|
||||
|
||||
if self.has_keystone() {
|
||||
// TRACES: FR-DEV-20
|
||||
// After the straightening and before the turns, so the keystone
|
||||
// acts on the photograph as it is shown. The map is measured in
|
||||
// fractions of the frame (see `Framing::keystone_map`), hence the
|
||||
// aspect divided out and put back. A point past the map's horizon
|
||||
// has no source at all and is sent far outside it, where the
|
||||
// sampler's bounds test renders it void.
|
||||
s.push_str(
|
||||
"
|
||||
// Perspective: the straightened frame onto a trapezoid of the source.
|
||||
let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);
|
||||
let key_h = u.keystone_c0.xyz * key_n.x + u.keystone_c1.xyz * key_n.y + u.keystone_c2.xyz;
|
||||
p = select(
|
||||
vec2<f32>(1.0e6),
|
||||
vec2<f32>(key_h.x / key_h.z * frame_aspect.x, key_h.y / key_h.z),
|
||||
key_h.z > 1.0e-6,
|
||||
);
|
||||
",
|
||||
);
|
||||
}
|
||||
|
||||
// The user's turns and mirrors composed with the file's stored
|
||||
// orientation. One permutation covers both, so honouring the EXIF tag
|
||||
// adds no per-pixel work over an untagged file.
|
||||
@@ -1040,13 +1448,15 @@ impl Framing {
|
||||
| u64::from(flip_v) << 3
|
||||
| u64::from(turns) << 4
|
||||
| u64::from(self.is_active()) << 6
|
||||
| u64::from(self.has_keystone()) << 7
|
||||
}
|
||||
}
|
||||
|
||||
/// Floats the framing block occupies in the generated uniform struct.
|
||||
///
|
||||
/// Two `vec4`s: the crop rect, and the angle's sin/cos with padding.
|
||||
pub const FRAMING_UNIFORM_FIELDS: usize = 8;
|
||||
/// Five `vec4`s: the crop rect, the angle's sin/cos with padding, and the
|
||||
/// perspective map's three columns, each padded.
|
||||
pub const FRAMING_UNIFORM_FIELDS: usize = 20;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
@@ -2025,6 +2435,8 @@ mod tests {
|
||||
(CROP_Y, 0.2),
|
||||
(CROP_W, 0.5),
|
||||
(CROP_H, 0.4),
|
||||
(KEYSTONE_V, 35.0),
|
||||
(KEYSTONE_H, -20.0),
|
||||
] {
|
||||
f.set_param(id, v);
|
||||
assert_eq!(f.param(id), v, "{id} did not round-trip");
|
||||
@@ -2241,6 +2653,8 @@ mod tests {
|
||||
f.rotate_quarters(turns);
|
||||
f.set_param(FLIP_H, 1.0);
|
||||
f.set_param(FLIP_V, 1.0);
|
||||
f.set_param(KEYSTONE_V, 60.0);
|
||||
f.set_param(KEYSTONE_H, -25.0);
|
||||
|
||||
for out in [(0.0, 0.0), (0.5, 0.5), (0.2, 0.9), (0.95, 0.05)] {
|
||||
let src = f.source_at(out, SRC.0, SRC.1);
|
||||
@@ -2316,6 +2730,27 @@ mod tests {
|
||||
"the three-turn permutation moved; `source_at` must move with it"
|
||||
);
|
||||
|
||||
// The perspective step: the map applied in fractions of the frame,
|
||||
// which is what `source_at` divides the aspect out for.
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 40.0);
|
||||
let prologue = f.wgsl_prologue();
|
||||
assert!(
|
||||
prologue.contains("let key_n = vec2<f32>(p.x / frame_aspect.x, p.y);")
|
||||
&& prologue.contains("key_h.x / key_h.z * frame_aspect.x"),
|
||||
"the perspective step moved; `source_at` must move with it"
|
||||
);
|
||||
// After the straightening and before the turns, as `source_at` has it.
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 40.0);
|
||||
f.set_param(ANGLE, 3.0);
|
||||
f.rotate_quarters(1);
|
||||
let prologue = f.wgsl_prologue();
|
||||
let straighten = prologue.find("// Straighten").unwrap();
|
||||
let keystone = prologue.find("// Perspective").unwrap();
|
||||
let turn = prologue.find("90° clockwise").unwrap();
|
||||
assert!(straighten < keystone && keystone < turn, "{prologue}");
|
||||
|
||||
// And the sampler's last step, which lives in `operation.rs` and is
|
||||
// the half of the map this file does not emit.
|
||||
assert!(
|
||||
@@ -2323,4 +2758,241 @@ mod tests {
|
||||
"the sampler's return to texture coordinates moved"
|
||||
);
|
||||
}
|
||||
|
||||
// ---- perspective (FR-DEV-20) -----------------------------------------
|
||||
|
||||
/// A grid over the whole output frame, edges included.
|
||||
fn grid() -> impl Iterator<Item = (f32, f32)> {
|
||||
(0..=10).flat_map(|j| (0..=10).map(move |i| (i as f32 / 10.0, j as f32 / 10.0)))
|
||||
}
|
||||
|
||||
fn inside(p: (f32, f32)) -> bool {
|
||||
(-1e-4..=1.0 + 1e-4).contains(&p.0) && (-1e-4..=1.0 + 1e-4).contains(&p.1)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_keystone_is_an_edit_and_a_resample() {
|
||||
let mut f = Framing::new();
|
||||
let neutral = f.structure_key();
|
||||
f.set_param(KEYSTONE_V, 30.0);
|
||||
assert!(f.is_active());
|
||||
assert!(f.edits_image(), "a keystone must light the modified dot");
|
||||
assert!(f.needs_interpolation());
|
||||
assert_ne!(f.structure_key(), neutral);
|
||||
assert!(f.wgsl_prologue().contains("u.keystone_c0"));
|
||||
// Neither the output size nor the crop moves: the frame is reshaped
|
||||
// inside itself, so what the user cropped stays cropped.
|
||||
assert_eq!(f.output_size(6000, 4000), (6000, 4000));
|
||||
assert!(f.crop().is_full());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_keystone_magnitude_does_not_reach_the_structure_key() {
|
||||
// Dragging the slider is a uniform upload, never a shader build.
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 10.0);
|
||||
let key = f.structure_key();
|
||||
for (v, h) in [(80.0, 0.0), (-45.0, 30.0), (1.0, -100.0)] {
|
||||
f.set_param(KEYSTONE_V, v);
|
||||
f.set_param(KEYSTONE_H, h);
|
||||
assert_eq!(f.structure_key(), key, "{v}/{h} forced a recompile");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_keystone_is_clamped_to_its_travel() {
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 1e9);
|
||||
f.set_param(KEYSTONE_H, f32::NAN);
|
||||
assert_eq!(f.keystone(), (MAX_KEYSTONE, 0.0));
|
||||
assert!(f.uniforms().iter().all(|v| v.is_finite()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_keystone_alone_never_reaches_outside_the_source() {
|
||||
// The design decision the crop relies on: the output frame is mapped
|
||||
// onto a trapezoid *inside* the source, so no corner goes empty and
|
||||
// a crop drawn before the keystone is still a crop of the picture.
|
||||
for (v, h) in [
|
||||
(100.0, 0.0),
|
||||
(-100.0, 0.0),
|
||||
(0.0, 100.0),
|
||||
(0.0, -100.0),
|
||||
(100.0, 100.0),
|
||||
(-100.0, 100.0),
|
||||
(37.0, -64.0),
|
||||
] {
|
||||
for turns in 0..4 {
|
||||
let mut f = Framing::new();
|
||||
f.rotate_quarters(turns);
|
||||
f.set_param(KEYSTONE_V, v);
|
||||
f.set_param(KEYSTONE_H, h);
|
||||
for out in grid() {
|
||||
let src = f.source_at(out, SRC.0, SRC.1);
|
||||
assert!(inside(src), "{v}/{h}, {turns} turn(s): {out:?} -> {src:?}");
|
||||
}
|
||||
assert!(f.max_inscribed_crop(SRC.0, SRC.1).is_full());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_vertical_keystone_makes_upward_converging_lines_parallel() {
|
||||
// What the control is for. Output columns are straight verticals;
|
||||
// with a positive keystone each must come from a straight source line
|
||||
// that leans in toward the centre as it rises — the shape a building
|
||||
// has when photographed looking up.
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 60.0);
|
||||
|
||||
for x in [0.1f32, 0.3, 0.7, 0.9] {
|
||||
let bottom = f.source_at((x, 1.0), SRC.0, SRC.1);
|
||||
let middle = f.source_at((x, 0.5), SRC.0, SRC.1);
|
||||
let top = f.source_at((x, 0.0), SRC.0, SRC.1);
|
||||
|
||||
// Straight: the middle sits on the line through the two ends.
|
||||
let cross = (top.0 - bottom.0) * (middle.1 - bottom.1)
|
||||
- (top.1 - bottom.1) * (middle.0 - bottom.0);
|
||||
assert!(cross.abs() < 1e-4, "column {x} is not a straight line");
|
||||
// Leaning in: the top is nearer the centre than the bottom.
|
||||
assert!(
|
||||
(top.0 - 0.5).abs() < (bottom.0 - 0.5).abs(),
|
||||
"column {x}: top {top:?} is not inside bottom {bottom:?}"
|
||||
);
|
||||
}
|
||||
|
||||
// The bottom row is left where it was; the top row is the one spread.
|
||||
close(
|
||||
f.source_at((0.0, 1.0), SRC.0, SRC.1),
|
||||
(0.0, 1.0),
|
||||
"bottom-left",
|
||||
);
|
||||
close(
|
||||
f.source_at((0.0, 0.0), SRC.0, SRC.1),
|
||||
(0.15, 0.0),
|
||||
"top-left",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_horizontal_keystone_spreads_the_right_hand_side() {
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_H, 100.0);
|
||||
// The right-hand column comes from half the source's height.
|
||||
close(
|
||||
f.source_at((1.0, 0.0), SRC.0, SRC.1),
|
||||
(1.0, 0.25),
|
||||
"top-right",
|
||||
);
|
||||
close(
|
||||
f.source_at((1.0, 1.0), SRC.0, SRC.1),
|
||||
(1.0, 0.75),
|
||||
"bottom-right",
|
||||
);
|
||||
close(
|
||||
f.source_at((0.0, 0.0), SRC.0, SRC.1),
|
||||
(0.0, 0.0),
|
||||
"top-left",
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_keystone_acts_on_the_frame_as_shown() {
|
||||
// A portrait frame the camera stored on its side: "vertical" is the
|
||||
// frame's displayed height, so the same keystone must move the same
|
||||
// *displayed* points whatever the file's stored orientation.
|
||||
let mut upright = Framing::new();
|
||||
upright.set_param(KEYSTONE_V, 50.0);
|
||||
let mut sideways = Framing::new();
|
||||
sideways.set_baseline(dr_types::Orientation::from_exif(6));
|
||||
sideways.set_param(KEYSTONE_V, 50.0);
|
||||
|
||||
// Compare in the displayed frame: map the sideways result back
|
||||
// through the orientation alone.
|
||||
let mut turn_only = Framing::new();
|
||||
turn_only.set_baseline(dr_types::Orientation::from_exif(6));
|
||||
for out in grid() {
|
||||
let a = upright.source_at(out, SRC.1, SRC.0);
|
||||
let b = turn_only.output_at(sideways.source_at(out, SRC.0, SRC.1), SRC.0, SRC.1);
|
||||
close(a, b, "the keystone turned with the file");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_inscribed_crop_accounts_for_the_keystone() {
|
||||
// Straightening a keystoned frame: the empty area is no longer the
|
||||
// rotated rectangle's, and the crop must avoid the area that is.
|
||||
for (angle, v, h) in [
|
||||
(5.0f32, 50.0f32, 0.0f32),
|
||||
(-8.0, -70.0, 20.0),
|
||||
(12.0, 100.0, 100.0),
|
||||
(2.0, 0.0, -40.0),
|
||||
] {
|
||||
for turns in [0, 1] {
|
||||
let mut f = Framing::new();
|
||||
f.rotate_quarters(turns);
|
||||
f.set_param(ANGLE, angle);
|
||||
f.set_param(KEYSTONE_V, v);
|
||||
f.set_param(KEYSTONE_H, h);
|
||||
let c = f.max_inscribed_crop(SRC.0, SRC.1);
|
||||
assert!(
|
||||
c.width > 0.3 && c.height > 0.3 && !c.is_full(),
|
||||
"{angle}°/{v}/{h}: {c:?}"
|
||||
);
|
||||
assert!(
|
||||
((c.x + c.width * 0.5) - 0.5).abs() < 1e-4
|
||||
&& ((c.y + c.height * 0.5) - 0.5).abs() < 1e-4,
|
||||
"{c:?} is not centred"
|
||||
);
|
||||
|
||||
// Every point of it, edges included, has a source pixel.
|
||||
f.set_crop(c);
|
||||
for out in grid() {
|
||||
let src = f.source_at(out, SRC.0, SRC.1);
|
||||
assert!(inside(src), "{angle}°/{v}/{h}: {out:?} -> {src:?}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_inscribed_crop_with_a_keystone_is_not_needlessly_small() {
|
||||
// The search must find the best rectangle, not merely a safe one. A
|
||||
// tenth larger in either direction has to reach outside the source.
|
||||
let mut f = Framing::new();
|
||||
f.set_param(ANGLE, 6.0);
|
||||
f.set_param(KEYSTONE_V, 60.0);
|
||||
let c = f.max_inscribed_crop(SRC.0, SRC.1);
|
||||
for (gw, gh) in [(1.1, 1.0), (1.0, 1.1)] {
|
||||
let mut g = f;
|
||||
let (w, h) = (c.width * gw, c.height * gh);
|
||||
g.set_crop(CropRect {
|
||||
x: 0.5 - w * 0.5,
|
||||
y: 0.5 - h * 0.5,
|
||||
width: w,
|
||||
height: h,
|
||||
});
|
||||
let spills = [(0.0, 0.0), (1.0, 0.0), (1.0, 1.0), (0.0, 1.0)]
|
||||
.into_iter()
|
||||
.any(|out| !inside(g.source_at(out, SRC.0, SRC.1)));
|
||||
// Either the grown rect spills, or it could not grow at all
|
||||
// because the crop was already at the frame's edge on that axis.
|
||||
assert!(
|
||||
spills
|
||||
|| (gw > 1.0 && c.width >= 1.0 - 1e-4)
|
||||
|| (gh > 1.0 && c.height >= 1.0 - 1e-4),
|
||||
"{c:?} grown by {gw}x{gh} still fits"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn reset_clears_the_keystone() {
|
||||
let mut f = Framing::new();
|
||||
f.set_param(KEYSTONE_V, 30.0);
|
||||
f.set_param(KEYSTONE_H, -30.0);
|
||||
f.reset();
|
||||
assert_eq!(f.keystone(), (0.0, 0.0));
|
||||
assert!(!f.is_active());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -623,7 +623,9 @@ impl EditGraph {
|
||||
} = self;
|
||||
|
||||
EditState {
|
||||
params: Preset::capture(self),
|
||||
// Without the film: it has a field of its own below, and one
|
||||
// edit must not have two places to disagree about its stock.
|
||||
params: Preset::capture_params(self),
|
||||
// A refcount bump. See `EditState::masks` for why that matters on
|
||||
// a path called once a frame.
|
||||
masks: Arc::clone(masks),
|
||||
@@ -663,7 +665,11 @@ impl EditGraph {
|
||||
// At full scope. `Scope` is a question about what a paste carries
|
||||
// *between* photographs; this is one photograph's own edit being put
|
||||
// back, so there is nothing to leave behind.
|
||||
params.apply(self, Scope::everything());
|
||||
//
|
||||
// What the parameters say about the film is discarded: the stock is
|
||||
// `film`'s to decide, below, and clearing it is what happens there
|
||||
// either way.
|
||||
let _ = params.apply(self, Scope::everything());
|
||||
|
||||
self.masks = Arc::clone(masks);
|
||||
self.spots = spots.clone();
|
||||
|
||||
@@ -581,7 +581,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 8],
|
||||
density_max: 2.0,
|
||||
lut_size: 2,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [2.0; 3],
|
||||
grain_uniformity: 1.0,
|
||||
},
|
||||
|
||||
@@ -32,6 +32,7 @@
|
||||
//! single multiply and white balance a per-channel scale; on gamma-encoded
|
||||
//! data neither would be physically meaningful (ARCH §5.2).
|
||||
|
||||
pub mod bundled;
|
||||
pub mod coverage;
|
||||
pub mod declared;
|
||||
pub mod descriptor;
|
||||
@@ -44,10 +45,10 @@ pub mod mask;
|
||||
pub mod neutral;
|
||||
pub mod operation;
|
||||
pub mod ops;
|
||||
pub mod orphan;
|
||||
pub mod preset;
|
||||
pub mod sidecar;
|
||||
pub mod spot;
|
||||
pub mod starter;
|
||||
pub mod state;
|
||||
|
||||
pub use coverage::Coverage;
|
||||
@@ -66,9 +67,9 @@ pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
|
||||
pub use operation::{
|
||||
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
|
||||
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
|
||||
RESERVED_UNIFORM_FIELDS,
|
||||
RESERVED_UNIFORM_FIELDS, SAMPLE_CACHE_UNIFORM_OFFSET,
|
||||
};
|
||||
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
|
||||
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Reach, Scope};
|
||||
pub use sidecar::{Sidecar, Version};
|
||||
pub use spot::{Spot, SpotMode, SpotSet};
|
||||
pub use state::{EditState, FilmRebake, FilmRef};
|
||||
@@ -131,7 +132,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 32 * 32 * 32],
|
||||
density_max: 3.0,
|
||||
lut_size: 32,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [3.0; 3],
|
||||
grain_uniformity: 0.97,
|
||||
},
|
||||
|
||||
+518
-92
@@ -73,7 +73,7 @@ use std::fmt::Write as _;
|
||||
use std::sync::Arc;
|
||||
|
||||
use crate::coverage::Coverage;
|
||||
use crate::descriptor::{Attribute, OpDescriptor, ParamId};
|
||||
use crate::descriptor::{Attribute, OpDescriptor, ParamId, ParamKind};
|
||||
use crate::operation::Operation;
|
||||
use crate::ops;
|
||||
|
||||
@@ -926,6 +926,21 @@ pub enum Join {
|
||||
/// erase stroke is a hole in the part it was painted into and reads as
|
||||
/// nothing at all.
|
||||
Subtract,
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// Only where both agree. "Keep the part of this mask that is also that."
|
||||
///
|
||||
/// The join that makes cheap criteria precise: a sky is a category *and*
|
||||
/// a luminance band, skin is a subject *and* a hue. Neither alone is the
|
||||
/// selection, and no feather on either makes it one.
|
||||
///
|
||||
/// The product of the two coverages rather than their minimum, because
|
||||
/// that is what one fixed-function blend gives on the device
|
||||
/// (`dst · src`, `docs/dev/mask-editing.md` §5.2) and it agrees with the
|
||||
/// minimum wherever either side is fully in or fully out. Between two soft
|
||||
/// edges it is the softer of the two readings, which is the right way to
|
||||
/// be wrong: an overlap of two partial selections is less certainly
|
||||
/// selected than either.
|
||||
Intersect,
|
||||
}
|
||||
|
||||
impl Join {
|
||||
@@ -933,6 +948,7 @@ impl Join {
|
||||
match self {
|
||||
Self::Union => "union",
|
||||
Self::Subtract => "subtract",
|
||||
Self::Intersect => "intersect",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -940,12 +956,30 @@ impl Join {
|
||||
Some(match name {
|
||||
"union" => Self::Union,
|
||||
"subtract" => Self::Subtract,
|
||||
"intersect" => Self::Intersect,
|
||||
_ => return None,
|
||||
})
|
||||
}
|
||||
|
||||
/// Every variant, for a UI building a choice control.
|
||||
pub const ALL: [Join; 2] = [Join::Union, Join::Subtract];
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// The coverage this join leaves at one point, given what the mask had
|
||||
/// there (`dst`) and what the part covers (`src`), both in `0..=1`.
|
||||
///
|
||||
/// The definition the device's blend states implement, spelled out once
|
||||
/// on the CPU so a test can hold the GPU to it and a reader can see the
|
||||
/// three set operations side by side without reading `wgpu` enums.
|
||||
pub fn apply(self, dst: f32, src: f32) -> f32 {
|
||||
match self {
|
||||
Self::Union => dst.max(src),
|
||||
Self::Subtract => dst * (1.0 - src),
|
||||
Self::Intersect => dst * src,
|
||||
}
|
||||
}
|
||||
|
||||
/// Every variant, for a UI building a choice control. The order is the
|
||||
/// panel's: a chip cycles through it and a stored index names a place in
|
||||
/// it, so a new join goes on the end.
|
||||
pub const ALL: [Join; 3] = [Join::Union, Join::Subtract, Join::Intersect];
|
||||
}
|
||||
|
||||
/// One selection inside a layer's mask.
|
||||
@@ -1334,17 +1368,19 @@ pub struct MaskLayer {
|
||||
/// This layer's adjustments.
|
||||
///
|
||||
/// A full chain, the same one [`crate::EditGraph`] holds. That is the
|
||||
/// whole reason local adjustments need no per-operation support: the
|
||||
/// composer already knows how to turn a chain into WGSL, and a mask layer
|
||||
/// is a chain that happens to be multiplied by a mask afterwards.
|
||||
/// whole reason local adjustments need no per-operation support: each
|
||||
/// setting here is an offset from its default, added to the global chain's
|
||||
/// setting and run where that operation runs, weighted by the mask (see
|
||||
/// [`offset_onto`]).
|
||||
pub ops: Vec<Box<dyn Operation>>,
|
||||
}
|
||||
|
||||
/// The chain a mask layer holds: every point operation, and neither the
|
||||
/// neighbourhood ones nor the optical corrections.
|
||||
///
|
||||
/// A layer's adjustments are fused into the colour dispatch and multiplied by
|
||||
/// the mask afterwards, which is exactly why a layer needs no per-operation
|
||||
/// A layer's adjustments are fused into the colour dispatch, each beside the
|
||||
/// global operation it offsets and weighted by the mask, which is exactly why
|
||||
/// a layer needs no per-operation
|
||||
/// support — the composer already knows how to turn a chain into WGSL. A
|
||||
/// neighbourhood operation cannot go through that path at all: it runs as its
|
||||
/// own dispatch in [`crate::detail`], after the fused pass and after the masks
|
||||
@@ -1580,9 +1616,20 @@ impl MaskLayer {
|
||||
// A hidden part is not in the build, whichever way it joins — and a
|
||||
// hidden base hands its role to the first part that is shown, which
|
||||
// is why "adds" is asked of the shown parts rather than of index 0.
|
||||
//
|
||||
// Folded rather than asked with `any`, because an intersection can
|
||||
// take away everything the parts before it added: a subject
|
||||
// intersected with an unpainted brush covers nothing. An inverted
|
||||
// part is taken to cover, whatever its source — an inverted empty
|
||||
// brush is the whole frame, and saying "covers nothing" of a layer
|
||||
// that does would hide an adjustment the photographer made.
|
||||
self.shown_parts()
|
||||
.enumerate()
|
||||
.any(|(i, p)| (i == 0 || p.join == Join::Union) && p.covers())
|
||||
.fold(false, |acc, (i, p)| match (i, p.join) {
|
||||
(0, _) | (_, Join::Union) => acc || p.covers(),
|
||||
(_, Join::Subtract) => acc,
|
||||
(_, Join::Intersect) => acc && (p.covers() || p.invert),
|
||||
})
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
@@ -1597,8 +1644,8 @@ impl MaskLayer {
|
||||
/// The operations in this layer's chain that reach the shader.
|
||||
///
|
||||
/// Neighbourhood operations are excluded, and not as an oversight. A
|
||||
/// layer's chain is *fused into the point-operation pass* and multiplied
|
||||
/// by the mask afterwards; the detail stage runs once, over the whole
|
||||
/// layer's chain is *fused into the point-operation pass*, weighted by the
|
||||
/// mask at each operation; the detail stage runs once, over the whole
|
||||
/// frame, after that pass has finished (see [`crate::detail`]). There is
|
||||
/// nowhere in that arrangement for a sharpening confined to one mask to
|
||||
/// happen, so a detail operation in a layer would contribute an empty
|
||||
@@ -1609,7 +1656,7 @@ impl MaskLayer {
|
||||
self.ops
|
||||
.iter()
|
||||
.map(|o| o.as_ref())
|
||||
.filter(|o| o.is_active() && o.detail().is_none())
|
||||
.filter(|o| moves(*o) && o.detail().is_none())
|
||||
}
|
||||
|
||||
/// Whether any part of this mask belongs to a different segmentation.
|
||||
@@ -1930,7 +1977,9 @@ impl MaskStack {
|
||||
Some(self.layers.remove(i))
|
||||
}
|
||||
|
||||
/// Reorder, since later layers composite over earlier ones.
|
||||
/// Reorder. Layers add their changes, so order no longer decides the
|
||||
/// picture — but it is the order the panel lists them in and the order
|
||||
/// their slots are assigned.
|
||||
pub fn move_to(&mut self, id: &str, index: usize) {
|
||||
let Some(from) = self.layers.iter().position(|l| l.id == id) else {
|
||||
return;
|
||||
@@ -1996,25 +2045,104 @@ impl MaskStack {
|
||||
}
|
||||
}
|
||||
|
||||
/// One layer's contribution to the generated shader.
|
||||
/// The layers' contribution to the generated shader.
|
||||
///
|
||||
/// Not a block of its own any more. A layer's adjustments are *offsets to the
|
||||
/// global ones*, applied at each operation's own place in the chain, so what
|
||||
/// this hands back is pieces the composer threads through its loop over the
|
||||
/// global operations: the weights, sampled once before the first operation,
|
||||
/// and one [`LocalOp`] per layer per operation the layer moved.
|
||||
pub(crate) struct LayerShader {
|
||||
pub uniform_fields: String,
|
||||
pub uniform_values: Vec<f32>,
|
||||
pub body: String,
|
||||
/// Each layer's shaped mask, `mask_w{slot}`, sampled once ahead of the
|
||||
/// operations that read it. Empty when no layer changes a pixel.
|
||||
pub weights: String,
|
||||
/// Every layer's version of every operation it moved, in layer order and
|
||||
/// then chain order — see [`LocalOp`].
|
||||
pub ops: Vec<LocalOp>,
|
||||
pub helpers: Vec<crate::operation::Helper>,
|
||||
/// TRACES: FR-DEV-19c
|
||||
/// The block that draws one layer's mask over the finished picture, empty
|
||||
/// when nothing is being revealed.
|
||||
///
|
||||
/// Kept apart from `body` because it belongs at the other end of the
|
||||
/// shader. Everything in `body` runs on scene-referred colour in the
|
||||
/// working space, where a flat tint would then be pushed through the base
|
||||
/// curve and the camera matrix and arrive as some other colour, and a
|
||||
/// Kept apart from the rest because it belongs at the other end of the
|
||||
/// shader. Everything else runs on scene-referred colour in the working
|
||||
/// space, where a flat tint would then be pushed through the base curve
|
||||
/// and the camera matrix and arrive as some other colour, and a
|
||||
/// white-on-black alpha would arrive as neither. This runs after the
|
||||
/// output transform, so what is written is what is seen.
|
||||
pub reveal: String,
|
||||
}
|
||||
|
||||
/// One layer's version of one operation: the global settings with the layer's
|
||||
/// offsets added, as a fragment reading this layer's own uniforms.
|
||||
///
|
||||
/// The composer runs it beside the global fragment on the same input colour,
|
||||
/// and moves the pixel toward its result by the layer's weight — see
|
||||
/// `operation::local_block`.
|
||||
pub(crate) struct LocalOp {
|
||||
pub op: &'static str,
|
||||
pub slot: usize,
|
||||
/// Empty when the offsets cancel the global setting back to neutral. That
|
||||
/// is still an entry, because it still means something: inside the mask
|
||||
/// this operation does nothing at all.
|
||||
///
|
||||
/// Empty, too, for an operation blended as settings: that entry stands
|
||||
/// for this layer's uniforms, `mask{slot}_{op}_*`, which the composer
|
||||
/// averages into the one fragment it runs.
|
||||
pub fragment: String,
|
||||
}
|
||||
|
||||
/// Whether a layer's copy of `op` holds an adjustment.
|
||||
///
|
||||
/// An operation's own answer, except for one blended as settings
|
||||
/// ([`Operation::blends_settings`]): that one is active by what it *holds* —
|
||||
/// a film is active when a stock is loaded — and a layer never holds a stock,
|
||||
/// only offsets to the photograph's. So a layer's film counts as moved when
|
||||
/// its sliders are, which is the question being asked.
|
||||
fn moves(op: &dyn Operation) -> bool {
|
||||
op.is_active()
|
||||
|| (op.blends_settings()
|
||||
&& op
|
||||
.descriptor()
|
||||
.params
|
||||
.iter()
|
||||
.any(|p| op.param(p.id) != p.default))
|
||||
}
|
||||
|
||||
/// A layer's settings for one operation, applied as offsets to the global
|
||||
/// operation's.
|
||||
///
|
||||
/// **This is what a local adjustment means**, and the reason it is not a
|
||||
/// second chain run over the finished picture. A photographer who sets
|
||||
/// contrast −30 on the whole frame and −20 on a face means −50 on the face,
|
||||
/// at the place contrast sits in the chain — not −30, then everything after
|
||||
/// contrast, then −20 applied again to the result. Stacked that way the two
|
||||
/// edits compound in ways neither slider shows, and a flattening applied to an
|
||||
/// already-flattened picture is how a shadow's noise ended up magenta.
|
||||
///
|
||||
/// Per parameter: one the layer left at its default takes the global value; a
|
||||
/// moved scalar adds its distance from default to the global value, clamped to
|
||||
/// the parameter's range; a moved switch or choice replaces it, since there is
|
||||
/// no such thing as half a variant.
|
||||
fn offset_onto(dst: &mut dyn Operation, local: &dyn Operation, global: Option<&dyn Operation>) {
|
||||
let desc = local.descriptor();
|
||||
for p in &desc.params {
|
||||
let here = local.param(p.id);
|
||||
let base = global.map_or(p.default, |g| g.param(p.id));
|
||||
let value = if here == p.default {
|
||||
base
|
||||
} else {
|
||||
match p.kind {
|
||||
ParamKind::Scalar { .. } => p.clamp(base + (here - p.default)),
|
||||
ParamKind::Bool | ParamKind::Enum { .. } => here,
|
||||
}
|
||||
};
|
||||
dst.set_param(p.id, value);
|
||||
}
|
||||
}
|
||||
|
||||
/// Emit the WGSL for every layer that renders, and for the mask being looked
|
||||
/// at.
|
||||
///
|
||||
@@ -2023,11 +2151,18 @@ pub(crate) struct LayerShader {
|
||||
/// an adjustment on it, which is why the two are one sequence and why every
|
||||
/// other half of the pipeline has to be given the same `reveal` for the slots
|
||||
/// to mean the same thing.
|
||||
pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal>) -> LayerShader {
|
||||
///
|
||||
/// `global` is the chain the layers are offsets to.
|
||||
pub(crate) fn compose_layers_revealing(
|
||||
stack: &MaskStack,
|
||||
reveal: Option<&Reveal>,
|
||||
global: &[Box<dyn Operation>],
|
||||
) -> LayerShader {
|
||||
let mut out = LayerShader {
|
||||
uniform_fields: String::new(),
|
||||
uniform_values: Vec::new(),
|
||||
body: String::new(),
|
||||
weights: String::new(),
|
||||
ops: Vec::new(),
|
||||
helpers: Vec::new(),
|
||||
reveal: String::new(),
|
||||
};
|
||||
@@ -2059,13 +2194,14 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
|
||||
);
|
||||
out.uniform_values.extend_from_slice(&layer.uniforms());
|
||||
|
||||
let _ = writeln!(
|
||||
out.body,
|
||||
"\n // ======== mask {slot}: {} ({}) ========",
|
||||
layer.display_name(),
|
||||
layer.base().source.kind()
|
||||
);
|
||||
let _ = writeln!(out.body, " {{");
|
||||
// A layer only being looked at moves no pixel, so it needs a slot for
|
||||
// the reveal and no weight.
|
||||
if layer.active_ops().next().is_none() {
|
||||
continue;
|
||||
}
|
||||
|
||||
// The weight, once per pixel, ahead of every operation that reads it.
|
||||
//
|
||||
// **`uv_src`, not `gid.xy`.** The mask array is rasterised in *source*
|
||||
// space, and `uv_src` is the source position this output pixel came
|
||||
// from — after the crop, the zoom, the pan, the straightening and the
|
||||
@@ -2078,33 +2214,62 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
|
||||
// place. A second copy here would be a second thing to keep in step
|
||||
// with `Framing::wgsl_prologue`, and the failure would be a mask that
|
||||
// is subtly wrong only when straightened.
|
||||
let _ = writeln!(out.body, " var m = sample_mask(uv_src, {slot});");
|
||||
let w = format!("mask_w{slot}");
|
||||
let _ = writeln!(
|
||||
out.body,
|
||||
" m = select(m, 1.0 - m, u.{prefix}_invert > 0.5);"
|
||||
out.weights,
|
||||
"\n // ======== mask {slot}: {} ({}) ========",
|
||||
layer.display_name(),
|
||||
layer.base().source.kind()
|
||||
);
|
||||
let _ = writeln!(out.weights, " var {w} = sample_mask(uv_src, {slot});");
|
||||
let _ = writeln!(
|
||||
out.weights,
|
||||
" {w} = select({w}, 1.0 - {w}, u.{prefix}_invert > 0.5);"
|
||||
);
|
||||
let _ = writeln!(
|
||||
out.body,
|
||||
" m = clamp(m * u.{prefix}_opacity, 0.0, 1.0);"
|
||||
out.weights,
|
||||
" {w} = clamp({w} * u.{prefix}_opacity, 0.0, 1.0);"
|
||||
);
|
||||
// Skipping the work where the mask is empty is most of the point of a
|
||||
// local adjustment: a mask covering a tenth of the frame should cost
|
||||
// about a tenth of the shader. Safe as non-uniform control flow —
|
||||
// nothing inside samples with derivatives or synchronises.
|
||||
let _ = writeln!(out.body, " if (m > 0.0) {{");
|
||||
// `masked` is the outer-scope carrier: op fragments write to a `c`
|
||||
// they expect to own, so the inner block shadows `c` and copies the
|
||||
// result back out. Assigning the outer `c` from inside is not possible
|
||||
// precisely because it is shadowed.
|
||||
let _ = writeln!(out.body, " var masked = c;");
|
||||
let _ = writeln!(out.body, " {{");
|
||||
let _ = writeln!(out.body, " var c = masked;");
|
||||
|
||||
for op in layer.active_ops() {
|
||||
let id = op.descriptor().id.0;
|
||||
// A fresh chain to hold the combined settings: the layer's own ops
|
||||
// are its offsets and must stay that way.
|
||||
let mut combined = layer_chain();
|
||||
for (dst, local) in combined.iter_mut().zip(&layer.ops) {
|
||||
let local = local.as_ref();
|
||||
if !moves(local) || local.detail().is_some() {
|
||||
continue;
|
||||
}
|
||||
let id = local.descriptor().id.0;
|
||||
let g = global
|
||||
.iter()
|
||||
.map(|o| o.as_ref())
|
||||
.find(|o| o.descriptor().id.0 == id);
|
||||
// The photograph's stock, lent to the layer's copy: a layer holds
|
||||
// offsets to a film, never one of its own.
|
||||
if let Some(g) = g {
|
||||
dst.set_film_tables(g.film_tables());
|
||||
}
|
||||
offset_onto(dst.as_mut(), local, g);
|
||||
|
||||
if dst.blends_settings() {
|
||||
// Its uniforms, and no fragment: the composer blends this
|
||||
// layer's settings with the others' and runs the global
|
||||
// fragment once. With no stock loaded there is nothing to
|
||||
// blend, and the global side skips the operation too.
|
||||
if !dst.is_active() {
|
||||
continue;
|
||||
}
|
||||
} else if !dst.is_active() {
|
||||
out.ops.push(LocalOp {
|
||||
op: id,
|
||||
slot,
|
||||
fragment: String::new(),
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
let op_prefix = format!("{prefix}_{}", crate::operation::sanitise(id));
|
||||
|
||||
let op_uniforms = op.uniforms();
|
||||
let op_uniforms = dst.uniforms();
|
||||
if !op_uniforms.is_empty() {
|
||||
let _ = writeln!(out.uniform_fields, " // mask {slot}: {id}");
|
||||
}
|
||||
@@ -2113,34 +2278,29 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
|
||||
out.uniform_values.push(u.value);
|
||||
}
|
||||
|
||||
for h in op.helpers() {
|
||||
for h in dst.helpers() {
|
||||
if !out.helpers.iter().any(|e| e.name == h.name) {
|
||||
out.helpers.push(*h);
|
||||
}
|
||||
}
|
||||
|
||||
let mut fragment = op.wgsl_body();
|
||||
for u in &op_uniforms {
|
||||
fragment = crate::operation::rewrite_uniform(
|
||||
&fragment,
|
||||
u.name,
|
||||
&format!("u.{op_prefix}_{}", u.name),
|
||||
);
|
||||
let mut fragment = String::new();
|
||||
if !dst.blends_settings() {
|
||||
fragment = dst.wgsl_body();
|
||||
for u in &op_uniforms {
|
||||
fragment = crate::operation::rewrite_uniform(
|
||||
&fragment,
|
||||
u.name,
|
||||
&format!("u.{op_prefix}_{}", u.name),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let _ = writeln!(out.body, " // ---- {id} ----");
|
||||
let _ = writeln!(out.body, " {{");
|
||||
for line in fragment.lines() {
|
||||
let _ = writeln!(out.body, " {line}");
|
||||
}
|
||||
let _ = writeln!(out.body, " }}");
|
||||
out.ops.push(LocalOp {
|
||||
op: id,
|
||||
slot,
|
||||
fragment,
|
||||
});
|
||||
}
|
||||
|
||||
let _ = writeln!(out.body, " masked = c;");
|
||||
let _ = writeln!(out.body, " }}");
|
||||
let _ = writeln!(out.body, " c = mix(c, masked, m);");
|
||||
let _ = writeln!(out.body, " }}");
|
||||
let _ = writeln!(out.body, " }}");
|
||||
}
|
||||
|
||||
out
|
||||
@@ -2264,6 +2424,78 @@ mod tests {
|
||||
layer
|
||||
}
|
||||
|
||||
/// A stock the shader can index, with values that are not a real one's.
|
||||
fn film_fixture() -> crate::graph::Film {
|
||||
use crate::ops::film_sim::{CURVE_SAMPLES, FORMAT_COUNT};
|
||||
crate::graph::Film {
|
||||
stock: "fixture".into(),
|
||||
print: None,
|
||||
tables: crate::ops::FilmTables {
|
||||
exposure_matrix: [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]],
|
||||
curves: vec![[0.5; 3]; CURVE_SAMPLES],
|
||||
push_stations: vec![0.0],
|
||||
curve_log_min: -3.0,
|
||||
curve_log_max: 1.0,
|
||||
lut: vec![[0.5; 3]; 8],
|
||||
density_max: 2.0,
|
||||
lut_size: 2,
|
||||
paper: None,
|
||||
grain_particles: [[0.0; 3]; FORMAT_COUNT],
|
||||
grain_density_max: [2.0; 3],
|
||||
grain_uniformity: 1.0,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_layers_film_is_blended_as_settings_and_developed_once() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// The film is a rendering: a layer's version of it run beside the
|
||||
// global one and cross-faded would be the photograph developed twice.
|
||||
// So the layer's uniforms are averaged into the global ones by weight
|
||||
// and the fragment appears once.
|
||||
use crate::ops::film_sim;
|
||||
let mut graph = crate::EditGraph::default_chain();
|
||||
graph.set_film(Some(film_fixture()));
|
||||
let mut layer = MaskLayer::new("m1", regions(&[1]));
|
||||
layer.set_param(film_sim::ID.0, film_sim::PRINT_EXPOSURE, 1.0);
|
||||
assert!(layer.is_active(), "a film offset is an adjustment");
|
||||
graph.masks_mut().push(layer);
|
||||
|
||||
let source = graph.compose().source;
|
||||
assert!(source.contains("let set_w = mask_w0;"), "{source}");
|
||||
assert!(source.contains("let set_g = max(1.0 - set_w, 0.0);"));
|
||||
assert!(
|
||||
source.contains("u.mask0_film_sim_pev"),
|
||||
"the layer's setting is not read"
|
||||
);
|
||||
assert_eq!(
|
||||
source.matches("let density = film_curve_pushed(").count(),
|
||||
1,
|
||||
"the film was developed more than once"
|
||||
);
|
||||
assert!(
|
||||
!source.contains("let local_in"),
|
||||
"the film was blended as a result"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_layers_film_without_a_stock_composes_to_nothing() {
|
||||
// TRACES: FR-DEV-3f
|
||||
// The offsets are kept — a stock chosen later brings them back — but
|
||||
// with no film on the photograph there is nothing for them to offset.
|
||||
use crate::ops::film_sim;
|
||||
let mut graph = crate::EditGraph::default_chain();
|
||||
let mut layer = MaskLayer::new("m1", regions(&[1]));
|
||||
layer.set_param(film_sim::ID.0, film_sim::PUSH, 1.0);
|
||||
graph.masks_mut().push(layer);
|
||||
|
||||
let source = graph.compose().source;
|
||||
assert!(!source.contains("---- film_sim ----"), "{source}");
|
||||
assert!(!source.contains("mask0_film_sim"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_layer_with_no_adjustment_is_not_in_the_shader() {
|
||||
let layer = MaskLayer::new("m1", regions(&[1]));
|
||||
@@ -2272,7 +2504,10 @@ mod tests {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(layer);
|
||||
assert!(stack.is_neutral());
|
||||
assert_eq!(compose_layers_revealing(&stack, None).body, "");
|
||||
assert_eq!(
|
||||
compose_layers_revealing(&stack, None, &ops::chain()).weights,
|
||||
""
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2358,11 +2593,11 @@ mod tests {
|
||||
stack.push(lit_layer("m1", 1.0));
|
||||
stack.push(lit_layer("m2", -1.0));
|
||||
|
||||
let shader = compose_layers_revealing(&stack, None);
|
||||
assert!(shader.body.contains("sample_mask(uv_src, 0)"));
|
||||
assert!(shader.body.contains("sample_mask(uv_src, 1)"));
|
||||
assert!(shader.body.contains("u.mask0_opacity"));
|
||||
assert!(shader.body.contains("u.mask1_opacity"));
|
||||
let shader = compose_layers_revealing(&stack, None, &ops::chain());
|
||||
assert!(shader.weights.contains("sample_mask(uv_src, 0)"));
|
||||
assert!(shader.weights.contains("sample_mask(uv_src, 1)"));
|
||||
assert!(shader.weights.contains("u.mask0_opacity"));
|
||||
assert!(shader.weights.contains("u.mask1_opacity"));
|
||||
}
|
||||
|
||||
/// The slot a layer renders through must follow `active()`, not the raw
|
||||
@@ -2375,12 +2610,12 @@ mod tests {
|
||||
stack.push(off);
|
||||
stack.push(lit_layer("m2", -1.0));
|
||||
|
||||
let shader = compose_layers_revealing(&stack, None);
|
||||
let shader = compose_layers_revealing(&stack, None, &ops::chain());
|
||||
assert!(
|
||||
shader.body.contains("sample_mask(uv_src, 0)"),
|
||||
shader.weights.contains("sample_mask(uv_src, 0)"),
|
||||
"the one active layer must use slot 0, not slot 1"
|
||||
);
|
||||
assert!(!shader.body.contains("sample_mask(uv_src, 1)"));
|
||||
assert!(!shader.weights.contains("sample_mask(uv_src, 1)"));
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19c
|
||||
@@ -2400,7 +2635,7 @@ mod tests {
|
||||
stack.push(MaskLayer::new("m2", MaskSource::brush()));
|
||||
|
||||
let reveal = Reveal::one("m2", RevealStyle::Alpha);
|
||||
let shader = compose_layers_revealing(&stack, Some(&reveal));
|
||||
let shader = compose_layers_revealing(&stack, Some(&reveal), &ops::chain());
|
||||
|
||||
assert_eq!(
|
||||
stack.rendered_count(Some(&reveal)),
|
||||
@@ -2438,7 +2673,7 @@ mod tests {
|
||||
],
|
||||
style: RevealStyle::Tint,
|
||||
};
|
||||
let shader = compose_layers_revealing(&stack, Some(&reveal));
|
||||
let shader = compose_layers_revealing(&stack, Some(&reveal), &ops::chain());
|
||||
|
||||
let sky = shader
|
||||
.reveal
|
||||
@@ -2464,7 +2699,9 @@ mod tests {
|
||||
fn nothing_is_revealed_unless_it_was_asked_for() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(lit_layer("m1", 1.0));
|
||||
assert!(compose_layers_revealing(&stack, None).reveal.is_empty());
|
||||
assert!(compose_layers_revealing(&stack, None, &ops::chain())
|
||||
.reveal
|
||||
.is_empty());
|
||||
}
|
||||
|
||||
/// A reveal aimed at a layer that is not in the stack is not a slot, and
|
||||
@@ -2475,9 +2712,11 @@ mod tests {
|
||||
stack.push(lit_layer("m1", 1.0));
|
||||
let reveal = Reveal::one("gone", RevealStyle::Tint);
|
||||
assert_eq!(stack.rendered_count(Some(&reveal)), 1);
|
||||
assert!(compose_layers_revealing(&stack, Some(&reveal))
|
||||
.reveal
|
||||
.is_empty());
|
||||
assert!(
|
||||
compose_layers_revealing(&stack, Some(&reveal), &ops::chain())
|
||||
.reveal
|
||||
.is_empty()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2486,7 +2725,7 @@ mod tests {
|
||||
stack.push(lit_layer("m1", 1.0));
|
||||
stack.push(lit_layer("m2", -1.0));
|
||||
|
||||
let shader = compose_layers_revealing(&stack, None);
|
||||
let shader = compose_layers_revealing(&stack, None, &ops::chain());
|
||||
assert!(shader.uniform_fields.contains("mask0_exposure_"));
|
||||
assert!(shader.uniform_fields.contains("mask1_exposure_"));
|
||||
assert_eq!(
|
||||
@@ -2500,16 +2739,140 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// The whole shader for `stack` over a global chain with `global`
|
||||
/// applied to it.
|
||||
fn composed_with(stack: &MaskStack, global: impl FnOnce(&mut [Box<dyn Operation>])) -> String {
|
||||
let mut chain = ops::chain();
|
||||
global(&mut chain);
|
||||
crate::operation::compose_full(
|
||||
&chain,
|
||||
&crate::Framing::new(),
|
||||
dr_types::ColourSpace::Srgb,
|
||||
stack,
|
||||
&crate::spot::SpotSet::new(),
|
||||
&[],
|
||||
)
|
||||
.source
|
||||
}
|
||||
|
||||
fn set(chain: &mut [Box<dyn Operation>], op: &str, param: &'static str, v: f32) {
|
||||
chain
|
||||
.iter_mut()
|
||||
.find(|o| o.descriptor().id.0 == op)
|
||||
.expect("op in chain")
|
||||
.set_param(ParamId(param), v);
|
||||
}
|
||||
|
||||
fn contrast_layer(v: f32) -> MaskLayer {
|
||||
let mut layer = MaskLayer::new("m1", regions(&[1]));
|
||||
layer.set_param("contrast", ParamId("contrast"), v);
|
||||
layer
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_inner_block_shadows_c_and_copies_back() {
|
||||
fn the_layer_version_shadows_c_and_blends_by_its_difference() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(lit_layer("m1", 1.0));
|
||||
let body = compose_layers_revealing(&stack, None).body;
|
||||
let src = composed_with(&stack, |_| {});
|
||||
|
||||
assert!(body.contains("var masked = c;"));
|
||||
assert!(body.contains("var c = masked;"));
|
||||
assert!(body.contains("masked = c;"));
|
||||
assert!(body.contains("c = mix(c, masked, m);"));
|
||||
assert!(src.contains("var c = local_in;"));
|
||||
assert!(src.contains("local_sum = local_sum + mask_w0 * (c - local_global);"));
|
||||
assert!(src.contains("c = max(local_sum, vec3<f32>(0.0));"));
|
||||
}
|
||||
|
||||
/// **The bug this shape exists for.** A layer's contrast is added to the
|
||||
/// global contrast, not run a second time on top of it.
|
||||
#[test]
|
||||
fn a_layer_setting_is_an_offset_to_the_global_one() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(contrast_layer(-20.0));
|
||||
let mut chain = ops::chain();
|
||||
set(&mut chain, "contrast", "contrast", -30.0);
|
||||
|
||||
let shader = compose_layers_revealing(&stack, None, &chain);
|
||||
let at = shader
|
||||
.uniform_fields
|
||||
.lines()
|
||||
.filter(|l| l.trim_start().starts_with("mask"))
|
||||
.position(|l| l.contains("mask0_contrast_amount"))
|
||||
.expect("the layer carries its own contrast");
|
||||
assert_eq!(
|
||||
shader.uniform_values[at], -0.5,
|
||||
"global -30 and local -20 is -50 inside the mask"
|
||||
);
|
||||
}
|
||||
|
||||
/// Where the operation runs is where the layer's version of it runs —
|
||||
/// between the global operations either side, not after all of them.
|
||||
#[test]
|
||||
fn a_layer_runs_at_its_operations_place_in_the_chain() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(contrast_layer(-20.0));
|
||||
let src = composed_with(&stack, |c| set(c, "saturation", "saturation", 20.0));
|
||||
|
||||
let contrast = src.find("// ---- contrast ----").expect("contrast block");
|
||||
let blend = src.find("mask_w0 * (c - local_global)").expect("blend");
|
||||
let saturation = src
|
||||
.find("// ---- saturation ----")
|
||||
.expect("saturation block");
|
||||
assert!(contrast < blend && blend < saturation);
|
||||
}
|
||||
|
||||
/// **No setting is applied twice.** The layer's version of an operation
|
||||
/// starts from the colour the operation was handed, not from the global
|
||||
/// result, and reads only its own combined setting — so global −30 and
|
||||
/// local −20 is one contrast of −50 inside the mask, never −30 and then
|
||||
/// −50 again, and the operation does not run a second time after the
|
||||
/// chain as it once did.
|
||||
#[test]
|
||||
fn a_setting_is_applied_once_not_stacked() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(contrast_layer(-20.0));
|
||||
let src = composed_with(&stack, |c| set(c, "contrast", "contrast", -30.0));
|
||||
|
||||
assert_eq!(
|
||||
src.matches("// ---- contrast ----").count(),
|
||||
1,
|
||||
"contrast runs at one place in the chain"
|
||||
);
|
||||
let version = &src[src.find("if (mask_w0 > 0.0)").expect("layer version")..];
|
||||
let version = &version[..version.find("local_sum = local_sum").unwrap()];
|
||||
assert!(
|
||||
version.contains("var c = local_in;"),
|
||||
"starts from the operation's input"
|
||||
);
|
||||
assert!(version.contains("u.mask0_contrast_amount"));
|
||||
assert!(
|
||||
!version.contains("u.contrast_amount"),
|
||||
"the global setting is already inside the combined one"
|
||||
);
|
||||
}
|
||||
|
||||
/// An offset that cancels the global setting is not nothing: inside the
|
||||
/// mask the operation is back at neutral, so the layer's version is empty
|
||||
/// and the blend pulls toward the colour the operation was handed.
|
||||
#[test]
|
||||
fn an_offset_back_to_neutral_undoes_the_global_setting() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(contrast_layer(30.0));
|
||||
let mut chain = ops::chain();
|
||||
set(&mut chain, "contrast", "contrast", -30.0);
|
||||
|
||||
let shader = compose_layers_revealing(&stack, None, &chain);
|
||||
let local: Vec<_> = shader.ops.iter().filter(|l| l.op == "contrast").collect();
|
||||
assert_eq!(local.len(), 1);
|
||||
assert!(local[0].fragment.is_empty());
|
||||
}
|
||||
|
||||
/// A global chain with nothing moved still hands a layer's operation a
|
||||
/// place to run: the global side of the blend is simply empty.
|
||||
#[test]
|
||||
fn an_operation_only_a_layer_moved_still_runs_in_its_place() {
|
||||
let mut stack = MaskStack::new();
|
||||
stack.push(contrast_layer(-20.0));
|
||||
let src = composed_with(&stack, |_| {});
|
||||
assert!(src.contains("// ---- contrast ----"));
|
||||
assert!(!src.contains("(local only)"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -3037,6 +3400,69 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// The three joins, pointwise, on every pair of coverages a part and a
|
||||
/// mask can meet at: fully in, fully out, and each soft edge. Intersection
|
||||
/// is the product, which agrees with the minimum wherever either side is
|
||||
/// decided and is the softer reading where both are not.
|
||||
#[test]
|
||||
fn the_joins_are_max_cut_and_product() {
|
||||
let levels = [0.0f32, 0.25, 0.5, 0.75, 1.0];
|
||||
for &dst in &levels {
|
||||
for &src in &levels {
|
||||
assert_eq!(Join::Union.apply(dst, src), dst.max(src));
|
||||
assert_eq!(Join::Subtract.apply(dst, src), dst * (1.0 - src));
|
||||
let meet = Join::Intersect.apply(dst, src);
|
||||
assert_eq!(meet, dst * src);
|
||||
assert!(meet <= dst.min(src), "never more than either side");
|
||||
if dst == 0.0 || dst == 1.0 || src == 0.0 || src == 1.0 {
|
||||
assert_eq!(meet, dst.min(src), "the minimum where either is decided");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// A stored index names a place in [`Join::ALL`], and a sidecar names a
|
||||
/// join by word: both have to survive a third join arriving, which means
|
||||
/// the first two keep their places and every name reads back as itself.
|
||||
#[test]
|
||||
fn every_join_reads_back_by_name_and_keeps_its_place() {
|
||||
assert_eq!(Join::ALL[0], Join::Union);
|
||||
assert_eq!(Join::ALL[1], Join::Subtract);
|
||||
for join in Join::ALL {
|
||||
assert_eq!(Join::from_name(join.name()), Some(join));
|
||||
}
|
||||
assert_eq!(Join::from_name("intersect"), Some(Join::Intersect));
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// An intersection can empty a mask the parts before it filled: a range
|
||||
/// meeting an unpainted brush selects nothing, and rasterising it would
|
||||
/// spend a slice to draw an empty field. Painting the brush, or inverting
|
||||
/// it, gives the intersection something to keep.
|
||||
#[test]
|
||||
fn an_intersection_with_nothing_covers_nothing() {
|
||||
let mut layer = MaskLayer::new("m1", MaskSource::highlights());
|
||||
layer.set_param("exposure", ParamId("exposure"), 1.0);
|
||||
layer.push_part(MaskPart::painted("p2", Join::Intersect));
|
||||
assert!(
|
||||
!layer.is_active(),
|
||||
"a range intersected with an unpainted brush selects nothing"
|
||||
);
|
||||
|
||||
layer.part_mut(1).expect("p2").invert = true;
|
||||
assert!(layer.is_active(), "inverted, the empty brush is everywhere");
|
||||
layer.part_mut(1).expect("p2").invert = false;
|
||||
|
||||
layer.begin_stroke(1, false, 0.1, 0.5, 1.0);
|
||||
layer.extend_stroke(1, 0.5, 0.5);
|
||||
layer.end_stroke(1);
|
||||
assert!(layer.is_active(), "and painted, it keeps what it covers");
|
||||
|
||||
layer.part_mut(1).expect("p2").hidden = true;
|
||||
assert!(layer.is_active(), "hidden, it is out of the build");
|
||||
}
|
||||
|
||||
/// One stale part is a stale layer: the mask is the fold over all of them,
|
||||
/// so a part that would draw a confidently wrong shape makes the result
|
||||
/// wrong whichever way it joins.
|
||||
|
||||
@@ -307,6 +307,32 @@ pub trait Operation: Send + Sync {
|
||||
/// shape it should take.
|
||||
fn set_film_tables(&mut self, _tables: Option<&crate::ops::film_sim::FilmTables>) {}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// The stock's tables this operation holds, for a mask layer's copy of it.
|
||||
///
|
||||
/// The other half of [`Self::set_film_tables`]. A layer holds offsets,
|
||||
/// never a stock of its own — the photograph is made on one film — so the
|
||||
/// composer hands the global operation's tables to the layer's combined
|
||||
/// copy through this pair.
|
||||
fn film_tables(&self) -> Option<&crate::ops::film_sim::FilmTables> {
|
||||
None
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3 | FR-DEV-3f
|
||||
/// Whether a mask layer's version of this operation is blended as
|
||||
/// *settings* rather than as a result.
|
||||
///
|
||||
/// Default `false`: each layer's version runs on the same input as the
|
||||
/// global one and the results are blended by weight, which is right for an
|
||||
/// adjustment. An operation that answers `true` has every uniform linear
|
||||
/// in what it controls, so the composer can blend the uniforms instead —
|
||||
/// the weighted average of every overlapping layer's settings, the global
|
||||
/// setting taking whatever weight the layers leave — and run the fragment
|
||||
/// once. See `local_settings_block`.
|
||||
fn blends_settings(&self) -> bool {
|
||||
false
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3e | FR-DEV-3f
|
||||
/// Whether this operation *is* the rendering, rather than an adjustment to
|
||||
/// one.
|
||||
@@ -456,6 +482,25 @@ pub struct ComposedShader {
|
||||
pub structure_hash: u64,
|
||||
/// What this shader writes. See [`OutputMode`].
|
||||
pub output_mode: OutputMode,
|
||||
/// What decides which source texel each output pixel reads, when that
|
||||
/// texel is read whole — `None` when it is interpolated.
|
||||
///
|
||||
/// A fit view reads one texel in every three or four of a 60 MP source,
|
||||
/// on a stride, and that gather is most of what the fused pass costs
|
||||
/// there: the texel it wants shares a cache line with neighbours nobody
|
||||
/// reads. But the gather depends on the framing and nothing else, so it
|
||||
/// is the same on every frame of a slider drag. The shader can therefore
|
||||
/// write what it gathered to a viewport-sized texture once and read it
|
||||
/// back contiguously thereafter; the flags in the uniform block at
|
||||
/// [`SAMPLE_CACHE_UNIFORM_OFFSET`] say which, and `dr-gpu` decides.
|
||||
///
|
||||
/// This key is the half of that decision only the composer can make: a
|
||||
/// hash of the generated prologue and the framing and warp uniforms, which
|
||||
/// together are everything that maps an output pixel to a source texel.
|
||||
/// The caller mixes in the source image and the render size. `None` for
|
||||
/// the interpolating paths, whose sample is a blend of four texels and
|
||||
/// not representable exactly in the source's own format.
|
||||
pub sample_key: Option<u64>,
|
||||
}
|
||||
|
||||
/// Fields the generated uniform struct always carries, before op uniforms.
|
||||
@@ -466,7 +511,19 @@ pub struct ComposedShader {
|
||||
/// Twelve of the twenty-eight are the camera profile's base curve
|
||||
/// ([`BASE_CURVE_UNIFORM_FIELDS`]); the rest are the matrix, the as-shot
|
||||
/// balance and framing's own block.
|
||||
const BASE_UNIFORM_FIELDS: usize = 16 + BASE_CURVE_UNIFORM_FIELDS;
|
||||
const BASE_UNIFORM_FIELDS: usize = 16 + SAMPLE_CACHE_UNIFORM_FIELDS + BASE_CURVE_UNIFORM_FIELDS;
|
||||
|
||||
/// Slots the sample cache's two flags occupy: read, write, and two spare to
|
||||
/// keep the block a whole `vec4`. See [`ComposedShader::sample_key`].
|
||||
const SAMPLE_CACHE_UNIFORM_FIELDS: usize = 4;
|
||||
|
||||
/// Where the sample cache's flags sit in the generated uniform block: `x` says
|
||||
/// read the source colour from the cache, `y` says write it there.
|
||||
///
|
||||
/// Exported for the reason [`BASE_CURVE_UNIFORM_OFFSET`] is — `dr-gpu` writes
|
||||
/// these by index — and zero in every block the composer hands out, so a
|
||||
/// caller that never heard of the cache gets the direct read it always had.
|
||||
pub const SAMPLE_CACHE_UNIFORM_OFFSET: usize = 16;
|
||||
|
||||
/// TRACES: FR-DEV-3e
|
||||
/// Slots the base curve occupies: five `(x, y)` points and an active flag.
|
||||
@@ -484,7 +541,8 @@ const BASE_CURVE_UNIFORM_FIELDS: usize = 12;
|
||||
/// Exported for the same reason [`RESERVED_UNIFORM_FIELDS`] is: `dr-gpu`
|
||||
/// writes these by index, and an offset computed independently at both ends is
|
||||
/// an offset that will eventually disagree with itself.
|
||||
pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
|
||||
pub const BASE_CURVE_UNIFORM_OFFSET: usize =
|
||||
SAMPLE_CACHE_UNIFORM_OFFSET + SAMPLE_CACHE_UNIFORM_FIELDS;
|
||||
|
||||
/// How many control points a base curve carries.
|
||||
///
|
||||
@@ -565,10 +623,11 @@ pub fn compose_with_framing(
|
||||
/// TRACES: FR-DEV-3
|
||||
/// Compose the global chain, the framing, and the local adjustments.
|
||||
///
|
||||
/// Mask layers are emitted **after** every global operation and before the
|
||||
/// conversion out of camera space, so a local exposure acts on the tones the
|
||||
/// global chain settled on — which is what a photographer means by "and then
|
||||
/// lift the shadows on her face".
|
||||
/// A mask layer's settings are **offsets to the global ones**, applied at each
|
||||
/// operation's own place in the chain: global contrast −30 and a face at −20
|
||||
/// is contrast −50 on the face, run where contrast runs. They once ran as a
|
||||
/// second chain after every global operation, which compounded the two edits
|
||||
/// in ways neither slider showed — see `mask::offset_onto`.
|
||||
///
|
||||
/// The fused-dispatch property survives: three global adjustments and two
|
||||
/// masked ones are still one shader, one read and one write. The masks
|
||||
@@ -756,6 +815,10 @@ fn compose_inner(
|
||||
\x20 // gamma-encoded JPEG, 0.0 for demosaiced sensor data), which the\n\
|
||||
\x20 // prologue reads to decide whether to linearise.\n\
|
||||
\x20 as_shot_wb: vec4<f32>,\n\
|
||||
\x20 // The sample cache (see `ComposedShader::sample_key`): `.x` reads\n\
|
||||
\x20 // the source colour from `sampled`, `.y` writes it to\n\
|
||||
\x20 // `sample_out`. Zero for both is the direct read.\n\
|
||||
\x20 sample_cache: vec4<f32>,\n\
|
||||
\x20 // The camera profile's base curve (FR-DEV-3e): five points on a\n\
|
||||
\x20 // monotone spline, packed as x0..x3, y0..y3, then (x4, y4, on).\n\
|
||||
\x20 // `.z` of the last is the flag, not padding — it is 0 for a\n\
|
||||
@@ -801,7 +864,11 @@ fn compose_inner(
|
||||
\x20 // angle as sin/cos — a trig call per pixel would recompute a\n\
|
||||
\x20 // value that is constant across the dispatch.\n\
|
||||
\x20 crop_rect: vec4<f32>,\n\
|
||||
\x20 framing_angle: vec4<f32>,\n",
|
||||
\x20 framing_angle: vec4<f32>,\n\
|
||||
\x20 // The perspective map (FR-DEV-20) by columns, `.w` unused.\n\
|
||||
\x20 keystone_c0: vec4<f32>,\n\
|
||||
\x20 keystone_c1: vec4<f32>,\n\
|
||||
\x20 keystone_c2: vec4<f32>,\n",
|
||||
);
|
||||
uniform_values.extend_from_slice(&framing.uniforms());
|
||||
|
||||
@@ -818,48 +885,86 @@ fn compose_inner(
|
||||
}
|
||||
}
|
||||
|
||||
for op in &active {
|
||||
// TRACES: FR-DEV-3
|
||||
// The local adjustments. Composed first because they are threaded through
|
||||
// the loop below rather than appended after it: a layer's settings are
|
||||
// offsets to the global ones, applied at each operation's own place in
|
||||
// the chain (see `mask::offset_onto` for why). The weights are sampled
|
||||
// here, once per pixel, ahead of every operation that reads them.
|
||||
let layers = crate::mask::compose_layers_revealing(masks, reveal, ops);
|
||||
body.push_str(&layers.weights);
|
||||
|
||||
// Every point operation that is active globally *or* in some layer. One
|
||||
// that only a layer moved still runs here, at its place in the chain,
|
||||
// with nothing on the global side of its blend.
|
||||
for op in ops
|
||||
.iter()
|
||||
.map(|o| o.as_ref())
|
||||
.filter(|o| o.detail().is_none())
|
||||
{
|
||||
let id = op.descriptor().id.0;
|
||||
let local: Vec<&crate::mask::LocalOp> = layers.ops.iter().filter(|l| l.op == id).collect();
|
||||
if !op.is_active() && local.is_empty() {
|
||||
continue;
|
||||
}
|
||||
let prefix = sanitise(id);
|
||||
|
||||
// Each op's uniforms are prefixed, so two operations may both declare
|
||||
// a field called `amount` without colliding.
|
||||
let op_uniforms = op.uniforms();
|
||||
if !op_uniforms.is_empty() {
|
||||
let _ = writeln!(uniform_fields, " // {id}");
|
||||
}
|
||||
for u in &op_uniforms {
|
||||
let _ = writeln!(uniform_fields, " {prefix}_{}: f32,", u.name);
|
||||
uniform_values.push(u.value);
|
||||
}
|
||||
|
||||
for h in op.helpers() {
|
||||
if !helpers.iter().any(|existing| existing.name == h.name) {
|
||||
helpers.push(*h);
|
||||
let mut fragment = String::new();
|
||||
let mut op_uniforms = Vec::new();
|
||||
if op.is_active() {
|
||||
// Each op's uniforms are prefixed, so two operations may both
|
||||
// declare a field called `amount` without colliding.
|
||||
op_uniforms = op.uniforms();
|
||||
if !op_uniforms.is_empty() {
|
||||
let _ = writeln!(uniform_fields, " // {id}");
|
||||
}
|
||||
for u in &op_uniforms {
|
||||
let _ = writeln!(uniform_fields, " {prefix}_{}: f32,", u.name);
|
||||
uniform_values.push(u.value);
|
||||
}
|
||||
}
|
||||
|
||||
// Rewrite bare uniform names to their prefixed struct fields, so a
|
||||
// fragment is written without knowing about any other operation.
|
||||
let mut fragment = op.wgsl_body();
|
||||
for u in &op_uniforms {
|
||||
fragment = rewrite_uniform(&fragment, u.name, &format!("u.{prefix}_{}", u.name));
|
||||
for h in op.helpers() {
|
||||
if !helpers.iter().any(|existing| existing.name == h.name) {
|
||||
helpers.push(*h);
|
||||
}
|
||||
}
|
||||
fragment = op.wgsl_body();
|
||||
}
|
||||
|
||||
let _ = writeln!(body, "\n // ---- {id} ----");
|
||||
let _ = writeln!(body, " {{");
|
||||
for line in fragment.lines() {
|
||||
let _ = writeln!(body, " {line}");
|
||||
if op.blends_settings() && !local.is_empty() {
|
||||
// TRACES: FR-DEV-3f
|
||||
body.push_str(&local_settings_block(
|
||||
&fragment,
|
||||
&prefix,
|
||||
&op_uniforms,
|
||||
&local,
|
||||
));
|
||||
continue;
|
||||
}
|
||||
let _ = writeln!(body, " }}");
|
||||
// Rewrite bare uniform names to their prefixed struct fields, so a
|
||||
// fragment is written without knowing about any other operation.
|
||||
for u in &op_uniforms {
|
||||
fragment = rewrite_uniform(&fragment, u.name, &format!("u.{prefix}_{}", u.name));
|
||||
}
|
||||
body.push_str(&local_block(&fragment, &local));
|
||||
}
|
||||
|
||||
// A layer's operation the global chain does not hold at all. Not a case
|
||||
// any editor produces — both chains come from `ops::chain` — but a layer
|
||||
// must not lose an edit because a caller composed a shorter chain.
|
||||
let mut orphans: Vec<&'static str> = Vec::new();
|
||||
for l in &layers.ops {
|
||||
if !orphans.contains(&l.op) && !ops.iter().any(|o| o.descriptor().id.0 == l.op) {
|
||||
orphans.push(l.op);
|
||||
}
|
||||
}
|
||||
for id in orphans {
|
||||
let local: Vec<&crate::mask::LocalOp> = layers.ops.iter().filter(|l| l.op == id).collect();
|
||||
let _ = writeln!(body, "\n // ---- {id} (local only) ----");
|
||||
body.push_str(&local_block("", &local));
|
||||
}
|
||||
|
||||
// The local adjustments, after every global one: a masked exposure should
|
||||
// act on the tones the global chain arrived at, not on the ones it started
|
||||
// from. Their uniforms follow the global ops' in the block for the same
|
||||
// reason those follow framing's — slot order is emission order, and
|
||||
// nothing addresses a slot by number.
|
||||
let layers = crate::mask::compose_layers_revealing(masks, reveal);
|
||||
// TRACES: FR-DEV-19c
|
||||
// Held apart from the body, because it belongs after the output transform
|
||||
// rather than among the operations — see `mask::LayerShader::reveal`.
|
||||
@@ -868,7 +973,6 @@ fn compose_inner(
|
||||
let reveal_block = layers.reveal.clone();
|
||||
uniform_fields.push_str(&layers.uniform_fields);
|
||||
uniform_values.extend_from_slice(&layers.uniform_values);
|
||||
body.push_str(&layers.body);
|
||||
for h in &layers.helpers {
|
||||
if !helpers.iter().any(|existing| existing.name == h.name) {
|
||||
helpers.push(*h);
|
||||
@@ -932,6 +1036,19 @@ fn compose_inner(
|
||||
);
|
||||
let sampler_helper = if interpolate { BILINEAR_HELPER } else { "" };
|
||||
|
||||
// Everything that decides which texel an output pixel reads: the code that
|
||||
// computes `coord`, and the uniforms that code reads. Only on the path that
|
||||
// reads a texel whole — see `ComposedShader::sample_key`.
|
||||
let sample_key = (!interpolate && !warp.splits_channels).then(|| {
|
||||
framing
|
||||
.uniforms()
|
||||
.iter()
|
||||
.chain(&warp.uniforms)
|
||||
.fold(hash_source(&prologue), |h, v| {
|
||||
mix(h, u64::from(v.to_bits()))
|
||||
})
|
||||
});
|
||||
|
||||
// The tail, and it is the whole of the difference between the two output
|
||||
// modes. Everything above — the prologue, the fragments, the mask layers,
|
||||
// the camera matrix — is emitted identically either way, so an operation
|
||||
@@ -1083,6 +1200,12 @@ struct Params {{
|
||||
// stock is loaded, which costs eight bytes and no branch.
|
||||
@group(0) @binding(4) var film_curves: texture_2d<f32>;
|
||||
@group(0) @binding(5) var film_lut_texture: texture_3d<f32>;
|
||||
// The sample cache: the source texel each output pixel read on an earlier
|
||||
// frame with this framing, and where this frame writes it when asked. See
|
||||
// `ComposedShader::sample_key`. Declared unconditionally, like the masks, and
|
||||
// bound to 1x1 placeholders whenever the flags say not to touch them.
|
||||
@group(0) @binding(6) var sampled: texture_2d<f32>;
|
||||
@group(0) @binding(7) var sample_out: texture_storage_2d<rgba16float, write>;
|
||||
|
||||
{sampler_helper}{helper_src}{encode_output}
|
||||
// Display-encoded sRGB back to linear, for sources that arrive that way.
|
||||
@@ -1192,9 +1315,129 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
|
||||
uniforms: uniform_values,
|
||||
structure_hash,
|
||||
output_mode,
|
||||
sample_key,
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// One operation's block when its layers are blended as *settings*: every
|
||||
/// uniform the weighted average of the global value and each overlapping
|
||||
/// layer's, and then the fragment once.
|
||||
///
|
||||
/// The weights: each layer's mask weight `w_i`, and the global setting
|
||||
/// whatever the layers leave, `w_g = max(0, 1 − Σ w_i)`. So
|
||||
///
|
||||
/// ```text
|
||||
/// p = (w_g·p_g + Σ w_i·p_i) / (w_g + Σ w_i)
|
||||
/// ```
|
||||
///
|
||||
/// — one layer at weight `w` is `(1 − w)·p_g + w·p_1`, exactly the blend a
|
||||
/// layer has always had, and three layers overlapping at full weight are the
|
||||
/// plain mean of their three settings rather than one piled on another. A
|
||||
/// layer's `p_i` is its combined setting (the global one plus its offset), so
|
||||
/// the average is of what each layer *asks for*.
|
||||
///
|
||||
/// Only layers that move this operation take part. One that left it alone is
|
||||
/// not voting for the global setting; it is not voting.
|
||||
///
|
||||
/// `fragment` is the operation's body with bare uniform names, as
|
||||
/// `wgsl_body` wrote it: they are rewritten here to the blended values.
|
||||
fn local_settings_block(
|
||||
fragment: &str,
|
||||
prefix: &str,
|
||||
uniforms: &[Uniform],
|
||||
local: &[&crate::mask::LocalOp],
|
||||
) -> String {
|
||||
let mut out = String::new();
|
||||
let _ = writeln!(out, " {{");
|
||||
let weights: Vec<String> = local.iter().map(|l| format!("mask_w{}", l.slot)).collect();
|
||||
let _ = writeln!(out, " let set_w = {};", weights.join(" + "));
|
||||
let _ = writeln!(out, " let set_g = max(1.0 - set_w, 0.0);");
|
||||
// Never zero: the global weight is one wherever no layer reaches.
|
||||
let _ = writeln!(out, " let set_n = max(set_g + set_w, 1e-6);");
|
||||
let mut body = fragment.to_string();
|
||||
for u in uniforms {
|
||||
let mut sum = format!("set_g * u.{prefix}_{}", u.name);
|
||||
for l in local {
|
||||
let _ = write!(
|
||||
sum,
|
||||
" + mask_w{slot} * u.mask{slot}_{prefix}_{name}",
|
||||
slot = l.slot,
|
||||
name = u.name
|
||||
);
|
||||
}
|
||||
let _ = writeln!(out, " let set_{} = ({sum}) / set_n;", u.name);
|
||||
body = rewrite_uniform(&body, u.name, &format!("set_{}", u.name));
|
||||
}
|
||||
let _ = writeln!(out, " {{");
|
||||
for line in body.lines() {
|
||||
let _ = writeln!(out, " {line}");
|
||||
}
|
||||
let _ = writeln!(out, " }}");
|
||||
let _ = writeln!(out, " }}");
|
||||
out
|
||||
}
|
||||
|
||||
/// One operation's block: its global fragment, and each layer's version of it
|
||||
/// blended in by that layer's weight.
|
||||
///
|
||||
/// Every version reads the same input — the colour as it arrived at this
|
||||
/// operation — and the pixel moves from the global result by each layer's
|
||||
/// difference from it: `c_g + Σ w_i (c_i − c_g)`. At full weight that is the
|
||||
/// layer's combined setting exactly, at zero it is the global result exactly,
|
||||
/// and two overlapping layers add their changes rather than one repainting
|
||||
/// the other.
|
||||
///
|
||||
/// With no layer touching the operation this is the block the composer always
|
||||
/// emitted, byte for byte: a photograph with no masks compiles to the shader
|
||||
/// it did before layers were offsets.
|
||||
fn local_block(global: &str, local: &[&crate::mask::LocalOp]) -> String {
|
||||
let mut out = String::new();
|
||||
let _ = writeln!(out, " {{");
|
||||
if local.is_empty() {
|
||||
for line in global.lines() {
|
||||
let _ = writeln!(out, " {line}");
|
||||
}
|
||||
let _ = writeln!(out, " }}");
|
||||
return out;
|
||||
}
|
||||
let _ = writeln!(out, " let local_in = c;");
|
||||
let _ = writeln!(out, " {{");
|
||||
for line in global.lines() {
|
||||
let _ = writeln!(out, " {line}");
|
||||
}
|
||||
let _ = writeln!(out, " }}");
|
||||
let _ = writeln!(out, " let local_global = c;");
|
||||
let _ = writeln!(out, " var local_sum = c;");
|
||||
for l in local {
|
||||
let w = format!("mask_w{}", l.slot);
|
||||
// Skipping where the mask is empty is most of the point of a local
|
||||
// adjustment: a mask covering a tenth of the frame should cost about a
|
||||
// tenth of the extra work. Safe as non-uniform control flow — nothing
|
||||
// inside samples with derivatives or synchronises.
|
||||
let _ = writeln!(out, " if ({w} > 0.0) {{");
|
||||
// Fragments write to a `c` they expect to own, so the layer's version
|
||||
// gets one of its own, shadowing the outer one and starting from what
|
||||
// this operation was handed.
|
||||
let _ = writeln!(out, " var c = local_in;");
|
||||
let _ = writeln!(out, " {{");
|
||||
for line in l.fragment.lines() {
|
||||
let _ = writeln!(out, " {line}");
|
||||
}
|
||||
let _ = writeln!(out, " }}");
|
||||
let _ = writeln!(
|
||||
out,
|
||||
" local_sum = local_sum + {w} * (c - local_global);"
|
||||
);
|
||||
let _ = writeln!(out, " }}");
|
||||
}
|
||||
// Two layers pulling the same way can overshoot below zero, and a
|
||||
// negative component poisons every operation after this one.
|
||||
let _ = writeln!(out, " c = max(local_sum, vec3<f32>(0.0));");
|
||||
let _ = writeln!(out, " }}");
|
||||
out
|
||||
}
|
||||
|
||||
/// The WGSL converting linear sRGB into the output space's primaries.
|
||||
///
|
||||
/// A constant matrix rather than a uniform: the space is chosen when the
|
||||
@@ -1411,7 +1654,19 @@ pub(crate) fn sample_source(interpolate: bool, splits_channels: bool) -> &'stati
|
||||
// amount that changes with the aspect ratio. It reads as a correction that
|
||||
// is simply too weak, which is indistinguishable from a bad profile.
|
||||
let radius = length(p) / (0.5 * length(aspect));
|
||||
var c = textureLoad(source, coord, 0).rgb;
|
||||
// The texel itself, from the source or from the cache of it an earlier
|
||||
// frame wrote (see `ComposedShader::sample_key`). Both branches yield the
|
||||
// same bits: the source is `rgba16float` and so is the cache. The flags
|
||||
// are uniforms, so the whole dispatch takes one branch.
|
||||
var c: vec3<f32>;
|
||||
if (u.sample_cache.x > 0.5) {
|
||||
c = textureLoad(sampled, vec2<i32>(gid.xy), 0).rgb;
|
||||
} else {
|
||||
c = textureLoad(source, coord, 0).rgb;
|
||||
if (u.sample_cache.y > 0.5) {
|
||||
textureStore(sample_out, vec2<i32>(gid.xy), vec4<f32>(c, 1.0));
|
||||
}
|
||||
}
|
||||
"
|
||||
}
|
||||
}
|
||||
@@ -1998,7 +2253,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 32 * 32 * 32],
|
||||
density_max: 3.0,
|
||||
lut_size: 32,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [3.0; 3],
|
||||
grain_uniformity: 0.97,
|
||||
}));
|
||||
|
||||
+118
-144
@@ -116,12 +116,34 @@
|
||||
//! mottling across smooth gradients. This decomposition samples nothing — it
|
||||
//! evaluates the exact minimum over every pixel of the window, in two steps.
|
||||
//!
|
||||
//! # Why it is now two passes and not five
|
||||
//!
|
||||
//! The decomposition was run as four erosion passes — run and span along x,
|
||||
//! then along y — and a fifth for the recovery. On the reference laptop the
|
||||
//! taps turned out not to be what a pass costs: with the memory clock held at
|
||||
//! 810 MHz by the power cap, a pass that only reads the render-sized
|
||||
//! `rgba16float` intermediate and writes the other one costs about 4 ms at
|
||||
//! 2560 x 1600, and dehaze's five came to 22 ms, of which the taps were about
|
||||
//! 2. So each axis is now one pass that takes the minimum over the whole
|
||||
//! window directly — 36 texture reads per pixel at that size instead of 12,
|
||||
//! nearly all of them served by the cache — and the recovery rides in the y
|
||||
//! pass, which already has the veil and this pixel's colour in hand. Two
|
||||
//! passes: 9.2 ms.
|
||||
//!
|
||||
//! It is the same picture, bit for bit. A minimum is exact in any order, so
|
||||
//! the minimum over the window's pixels is one value however it is grouped,
|
||||
//! and the window is the one [`Split`] always covered, surplus pixel
|
||||
//! included. The veil reaching the recovery was always exactly representable
|
||||
//! in the `rgba16float` lane it crossed — a minimum of channel values that
|
||||
//! were themselves read from `rgba16float` — so no rounding was lost by not
|
||||
//! storing it between passes.
|
||||
//!
|
||||
//! It is also why this operation does not use the reduced chain that TD-4 gave
|
||||
//! clarity. The runner holds one reduced buffer, so every scaled pass in a
|
||||
//! chain must declare the same `output_scale`; clarity's steps down with the
|
||||
//! viewport, so a second operation choosing its own would disagree with it at
|
||||
//! some window sizes and not others. Four cheap full-resolution passes cost
|
||||
//! less than that coupling, and the decomposition is what makes them cheap.
|
||||
//! some window sizes and not others. Two full-resolution passes cost less than
|
||||
//! that coupling.
|
||||
//!
|
||||
//! # The artefact this does not fix
|
||||
//!
|
||||
@@ -275,22 +297,23 @@ impl Dehaze {
|
||||
///
|
||||
/// See the module documentation: eroding by a contiguous run and then by a set
|
||||
/// of points spaced one run apart erodes by the sum of the two, which is the
|
||||
/// whole window. This is the arithmetic of that split, in one place, because
|
||||
/// both axes need it and a second copy is a second chance to get the centring
|
||||
/// wrong.
|
||||
/// whole window. The passes no longer run the two stages apart (see "Why it is
|
||||
/// now two passes"), but the window they read is still the one this split
|
||||
/// covers — [`Self::first`] and [`Self::width`] — surplus pixel included,
|
||||
/// because that is the window every edit made so far was tuned against.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub struct Split {
|
||||
/// Length of the contiguous run the first pass takes the minimum over.
|
||||
pub run: u32,
|
||||
/// How many runs the second pass chains together, spaced `run` apart.
|
||||
pub span: u32,
|
||||
/// What the second pass subtracts from its offsets to centre the window.
|
||||
/// How far before the pixel being written the window starts.
|
||||
///
|
||||
/// The composite covers `run * span` pixels, which is at least the window
|
||||
/// asked for and can be one or two more; the surplus falls on the far side
|
||||
/// rather than being trimmed, because trimming it would need a third pass
|
||||
/// and a patch a pixel wider on one side is not a visible difference in a
|
||||
/// field this smooth.
|
||||
/// rather than being trimmed, because trimming it would have needed a
|
||||
/// third pass and a patch a pixel wider on one side is not a visible
|
||||
/// difference in a field this smooth.
|
||||
pub shift: i32,
|
||||
}
|
||||
|
||||
@@ -311,14 +334,25 @@ impl Split {
|
||||
}
|
||||
}
|
||||
|
||||
/// The furthest the second pass reads, in pixels.
|
||||
/// The window's first offset from the pixel being written: `-shift`.
|
||||
pub fn first(&self) -> i32 {
|
||||
-self.shift
|
||||
}
|
||||
|
||||
/// How many pixels the window covers, `run * span` — the patch asked for
|
||||
/// and the one or two surplus pixels on the far side the split leaves.
|
||||
pub fn width(&self) -> u32 {
|
||||
self.run * self.span
|
||||
}
|
||||
|
||||
/// The furthest the window reads from the pixel being written, in pixels.
|
||||
///
|
||||
/// Stated rather than assumed symmetric: the composite window is centred
|
||||
/// to within a pixel and not exactly, so the two directions can differ by
|
||||
/// one. An understated radius is a seam at every tile boundary (ARCH
|
||||
/// §5.3), which is the kind of artefact that looks like a driver bug.
|
||||
pub fn reach(&self) -> u32 {
|
||||
let far = (self.span.saturating_sub(1) * self.run) as i32 - self.shift;
|
||||
/// Stated rather than assumed symmetric: the window is centred to within a
|
||||
/// pixel and not exactly, so the two directions can differ by one. An
|
||||
/// understated radius is a seam at every tile boundary (ARCH §5.3), which
|
||||
/// is the kind of artefact that looks like a driver bug.
|
||||
pub fn extent(&self) -> u32 {
|
||||
let far = self.width() as i32 - 1 - self.shift;
|
||||
self.shift.max(far).max(0) as u32
|
||||
}
|
||||
}
|
||||
@@ -367,83 +401,50 @@ impl DetailStage for Dehaze {
|
||||
fn passes(&self, scale: RenderScale) -> Vec<DetailPass> {
|
||||
let split = Split::of(self.patch(scale));
|
||||
|
||||
// The run stage's uniforms are the same on both axes, and so are the
|
||||
// span stage's. Only the offset expression differs, which is what
|
||||
// `erode_run` and `erode_span` take as an argument — each filter
|
||||
// Both axes erode over the same window; only the offset expression
|
||||
// differs, which is what `erode` takes as an argument — one filter
|
||||
// written once, so the two axes cannot drift into being different
|
||||
// filters.
|
||||
let run = vec![Uniform {
|
||||
name: "run",
|
||||
value: split.run as f32,
|
||||
}];
|
||||
let span = vec![
|
||||
let window = vec![
|
||||
Uniform {
|
||||
name: "span",
|
||||
value: split.span as f32,
|
||||
name: "first",
|
||||
value: split.first() as f32,
|
||||
},
|
||||
Uniform {
|
||||
name: "stride",
|
||||
value: split.run as f32,
|
||||
},
|
||||
Uniform {
|
||||
name: "shift",
|
||||
value: split.shift as f32,
|
||||
name: "width",
|
||||
value: split.width() as f32,
|
||||
},
|
||||
];
|
||||
let mut recovery = window.clone();
|
||||
recovery.extend([
|
||||
Uniform {
|
||||
name: "omega",
|
||||
value: self.omega(),
|
||||
},
|
||||
Uniform {
|
||||
name: "min_transmission",
|
||||
value: MIN_TRANSMISSION,
|
||||
},
|
||||
]);
|
||||
|
||||
vec![
|
||||
DetailPass {
|
||||
output_scale: 1,
|
||||
label: "veil-run-x",
|
||||
// The run starts at this pixel and walks forward, so it reads
|
||||
// `run - 1` beyond itself and nothing behind.
|
||||
radius: split.run.saturating_sub(1),
|
||||
label: "veil-x",
|
||||
radius: split.extent(),
|
||||
storage: Vec::new(),
|
||||
uniforms: run.clone(),
|
||||
wgsl: erode_run(Axis::X),
|
||||
uniforms: window,
|
||||
wgsl: erode(Axis::X),
|
||||
},
|
||||
DetailPass {
|
||||
output_scale: 1,
|
||||
label: "veil-span-x",
|
||||
radius: split.reach(),
|
||||
label: "veil-y-clear",
|
||||
radius: split.extent(),
|
||||
storage: Vec::new(),
|
||||
uniforms: span.clone(),
|
||||
wgsl: erode_span(Axis::X),
|
||||
},
|
||||
DetailPass {
|
||||
output_scale: 1,
|
||||
label: "veil-run-y",
|
||||
radius: split.run.saturating_sub(1),
|
||||
storage: Vec::new(),
|
||||
uniforms: run,
|
||||
wgsl: erode_run(Axis::Y),
|
||||
},
|
||||
DetailPass {
|
||||
output_scale: 1,
|
||||
label: "veil-span-y",
|
||||
radius: split.reach(),
|
||||
storage: Vec::new(),
|
||||
uniforms: span,
|
||||
wgsl: erode_span(Axis::Y),
|
||||
},
|
||||
DetailPass {
|
||||
output_scale: 1,
|
||||
label: "clear",
|
||||
// Reads only the pixel it writes: the veil arrived in the
|
||||
// scratch lane four passes ago.
|
||||
radius: 0,
|
||||
storage: Vec::new(),
|
||||
uniforms: vec![
|
||||
Uniform {
|
||||
name: "omega",
|
||||
value: self.omega(),
|
||||
},
|
||||
Uniform {
|
||||
name: "min_transmission",
|
||||
value: MIN_TRANSMISSION,
|
||||
},
|
||||
],
|
||||
wgsl: CLEAR.to_string(),
|
||||
uniforms: recovery,
|
||||
// The erosion in a block of its own, so its locals do not
|
||||
// collide with the recovery's.
|
||||
wgsl: format!("{{\n{}\n}}\n\n{CLEAR}", erode(Axis::Y)),
|
||||
},
|
||||
]
|
||||
}
|
||||
@@ -466,33 +467,33 @@ impl Axis {
|
||||
}
|
||||
}
|
||||
|
||||
/// The first stage: the minimum over a contiguous run.
|
||||
/// The minimum over the window along one axis.
|
||||
///
|
||||
/// Along x it reads the colour and reduces it to the dark channel; along y the
|
||||
/// dark channel is already in the scratch lane, so it reads that instead.
|
||||
/// Doing the channel minimum again on the second axis would be reducing a
|
||||
/// scalar and would quietly discard the x erosion.
|
||||
/// dark channel's x minimum is already in the scratch lane, so it reads that
|
||||
/// instead. Doing the channel minimum again on the second axis would be
|
||||
/// reducing a scalar and would quietly discard the x erosion.
|
||||
///
|
||||
/// Neither stage touches `c`. The recovery needs the original colour *and* the
|
||||
/// veil in the same place at the same time, and the ping-pong hands each pass
|
||||
/// only what the pass before it wrote — so the veil travels in `aux` and the
|
||||
/// colour rides through untouched. See [`DetailPass::wgsl`].
|
||||
fn erode_run(axis: Axis) -> String {
|
||||
/// The x pass does not touch `c`. The recovery needs the original colour *and*
|
||||
/// the veil in the same place at the same time, and the ping-pong hands each
|
||||
/// pass only what the pass before it wrote — so the veil travels in `aux` and
|
||||
/// the colour rides through untouched. See [`DetailPass::wgsl`].
|
||||
fn erode(axis: Axis) -> String {
|
||||
let source = match axis {
|
||||
Axis::X => "dark_channel(tap(coord, OFFSET))",
|
||||
Axis::Y => "tap_aux(coord, OFFSET)",
|
||||
};
|
||||
let first = source.replace("OFFSET", &axis.offset("0"));
|
||||
let rest = source.replace("OFFSET", &axis.offset("i"));
|
||||
let head = source.replace("OFFSET", &axis.offset("o"));
|
||||
let rest = source.replace("OFFSET", &axis.offset("o + i"));
|
||||
|
||||
format!(
|
||||
"\
|
||||
// Half of the erosion's first stage: the minimum over `run` contiguous pixels,
|
||||
// walking forward from this one. The second stage chains these together, and
|
||||
// the two structuring elements add up to the whole patch — which is why this
|
||||
// one is not centred and does not need to be.
|
||||
let n = i32(run);
|
||||
var veil = {first};
|
||||
// The minimum over every pixel of the window along this axis, from `first`
|
||||
// for `width` pixels. A minimum is exact in any order, so this is the same
|
||||
// value, bit for bit, as chaining a run and a span over the same pixels.
|
||||
let o = i32(first);
|
||||
let n = i32(width);
|
||||
var veil = {head};
|
||||
for (var i = 1; i < n; i = i + 1) {{
|
||||
veil = min(veil, {rest});
|
||||
}}
|
||||
@@ -500,38 +501,10 @@ aux = veil;"
|
||||
)
|
||||
}
|
||||
|
||||
/// The second stage: the minimum over `span` points spaced `stride` apart.
|
||||
///
|
||||
/// Each of those points already holds the minimum over the run that starts
|
||||
/// there, so this reads the whole window while touching `span` pixels of it.
|
||||
/// `shift` is what centres the composite on the pixel being written; without
|
||||
/// it the veil would be measured from a patch lying entirely to one side, and
|
||||
/// the correction would appear to lag the picture by half a patch.
|
||||
fn erode_span(axis: Axis) -> String {
|
||||
let offset = axis.offset("j * s - o");
|
||||
let first = axis.offset("-o");
|
||||
|
||||
format!(
|
||||
"\
|
||||
// The erosion's second stage. `span` taps, spaced a whole run apart, each
|
||||
// standing for the run that begins at it — so the minimum over the patch costs
|
||||
// `run + span` taps rather than the `run * span` pixels it covers, and it is
|
||||
// the exact minimum over all of them rather than a sample of them.
|
||||
let n = i32(span);
|
||||
let s = i32(stride);
|
||||
let o = i32(shift);
|
||||
var veil = tap_aux(coord, {first});
|
||||
for (var j = 1; j < n; j = j + 1) {{
|
||||
veil = min(veil, tap_aux(coord, {offset}));
|
||||
}}
|
||||
aux = veil;"
|
||||
)
|
||||
}
|
||||
|
||||
/// The recovery: invert the scattering model with the transmission the erosion
|
||||
/// implies.
|
||||
const CLEAR: &str = "\
|
||||
// The veil the four erosion passes measured: the smallest channel anywhere in
|
||||
// The veil the two erosions measured: the smallest channel anywhere in
|
||||
// the patch around this pixel, which the dark-channel prior reads as the
|
||||
// airlight that has been composited over the scene here.
|
||||
//
|
||||
@@ -581,7 +554,7 @@ mod tests {
|
||||
#[test]
|
||||
fn dehaze_starts_neutral_and_costs_nothing() {
|
||||
// The rule the whole pipeline rests on. An unedited photograph must not
|
||||
// pay for a slider nobody has touched — and this one is five dispatches
|
||||
// pay for a slider nobody has touched — and this one is two dispatches
|
||||
// when it is on, so "nothing" here is a worthwhile amount of nothing.
|
||||
assert!(!Dehaze::new().is_active());
|
||||
assert!(composed(0.0, RenderScale::full((2000, 1500))).is_empty());
|
||||
@@ -623,7 +596,7 @@ mod tests {
|
||||
// develop view about it would read as a bug.
|
||||
let tiny = RenderScale::full((48, 32));
|
||||
assert_eq!(Dehaze::with_amount(50.0).patch(tiny), 1);
|
||||
assert_eq!(composed(50.0, tiny).len(), 5);
|
||||
assert_eq!(composed(50.0, tiny).len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -648,7 +621,8 @@ mod tests {
|
||||
// Centred to within the pixel the odd surplus leaves over: the window
|
||||
// covers [-31, 32] around the pixel being written.
|
||||
assert_eq!(split.shift, 31);
|
||||
assert_eq!(split.reach(), 31);
|
||||
assert_eq!((split.first(), split.width()), (-31, 64));
|
||||
assert_eq!(split.extent(), 32);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -662,33 +636,32 @@ mod tests {
|
||||
assert_eq!(split.span, 2);
|
||||
assert!(split.run * split.span >= 3, "the window is not covered");
|
||||
assert_eq!(split.shift, 1);
|
||||
assert_eq!(split.reach(), 1);
|
||||
// [-1, 2]: the surplus pixel is on the far side.
|
||||
assert_eq!((split.first(), split.width()), (-1, 4));
|
||||
assert_eq!(split.extent(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_chain_is_four_erosions_and_a_recovery() {
|
||||
fn the_chain_is_an_erosion_per_axis_the_second_carrying_the_recovery() {
|
||||
// The shape of the operation, asserted where it is cheap to assert.
|
||||
// The erosions leave the colour alone and hand the veil forward in the
|
||||
// scratch lane; only the last pass touches `c`, which is what makes an
|
||||
// The x erosion leaves the colour alone and hands the veil forward in
|
||||
// the scratch lane; only the y pass touches `c`, which is what makes an
|
||||
// unsharp-mask-shaped operation expressible in a chain that hands each
|
||||
// pass exactly one texture.
|
||||
// pass exactly one texture. Two passes and not five: each extra pass
|
||||
// is a render-sized read and write, which is what a pass costs (see
|
||||
// "Why it is now two passes").
|
||||
let composed = composed(60.0, RenderScale::full((2000, 1500)));
|
||||
let labels: Vec<&str> = composed.passes.iter().map(|p| p.label.as_str()).collect();
|
||||
assert_eq!(
|
||||
labels,
|
||||
[
|
||||
"dehaze/veil-run-x",
|
||||
"dehaze/veil-span-x",
|
||||
"dehaze/veil-run-y",
|
||||
"dehaze/veil-span-y",
|
||||
"dehaze/clear",
|
||||
]
|
||||
);
|
||||
assert_eq!(labels, ["dehaze/veil-x", "dehaze/veil-y-clear"]);
|
||||
|
||||
// Both declare the whole window they read.
|
||||
let split = Split::of(Dehaze::with_amount(60.0).patch(RenderScale::full((2000, 1500))));
|
||||
assert!(composed.passes.iter().all(|p| p.radius == split.extent()));
|
||||
|
||||
// Only the last writes the display texture, so the output transform
|
||||
// happens exactly once (FR-DEV-2).
|
||||
assert!(composed.passes[..4].iter().all(|p| !p.writes_output));
|
||||
assert!(composed.passes[4].writes_output);
|
||||
assert!(!composed.passes[0].writes_output);
|
||||
assert!(composed.passes[1].writes_output);
|
||||
|
||||
// Nothing here uses the reduced chain — see the module documentation
|
||||
// for why a second operation cannot pick its own `output_scale` while
|
||||
@@ -730,7 +703,8 @@ mod tests {
|
||||
// nothing to say so.
|
||||
let composed = composed(60.0, RenderScale::full((2000, 1500)));
|
||||
assert!(composed.passes[0].source.contains("dark_channel(tap(coord"));
|
||||
assert!(!composed.passes[2].source.contains("dark_channel(tap(coord"));
|
||||
assert!(!composed.passes[1].source.contains("dark_channel(tap(coord"));
|
||||
assert!(composed.passes[1].source.contains("tap_aux(coord"));
|
||||
// The helper is still emitted for every pass of the operation, and it
|
||||
// must define the function it is named for or the shader fails to
|
||||
// compile a long way from here.
|
||||
|
||||
@@ -22,9 +22,11 @@
|
||||
//!
|
||||
//! For the same reason [`crate::ops::vignetting`]'s coefficients are not: they
|
||||
//! are measurements of a physical thing, not something a slider moves. The
|
||||
//! sliders here are exposure and print exposure, which are what a photographer
|
||||
//! and a printer actually control. `dr-film` turns a stock plus those two
|
||||
//! numbers into [`FilmTables`]; this node knows only the layout.
|
||||
//! sliders here are exposure, push, print exposure and format, which are what
|
||||
//! a photographer and a printer actually control. `dr-film` turns a stock into
|
||||
//! [`FilmTables`] that hold none of them; the shader applies all four per
|
||||
//! pixel, which is what lets a mask layer hold its own (see
|
||||
//! [`Operation::blends_settings`]). This node knows only the layout.
|
||||
//!
|
||||
//! Declared as a plain struct here rather than imported, so that dr-pipeline
|
||||
//! keeps its no-dependency property (ARCH §6.5a) exactly as `vignetting` does
|
||||
@@ -63,6 +65,15 @@ static FORMATS: [LocalizedKey; 6] = [
|
||||
/// take; [`FilmTables::is_well_formed`] is what stops the two drifting.
|
||||
pub const CURVE_SAMPLES: usize = 256;
|
||||
|
||||
/// The most development times a stock may measure — a curve row and a push
|
||||
/// station each. Must agree with `dr_film::bake::MAX_CURVE_ROWS`, for the
|
||||
/// reason [`CURVE_SAMPLES`] must; the uniform block holds this many stations.
|
||||
pub const MAX_CURVE_ROWS: usize = 8;
|
||||
|
||||
/// How many frames [`FORMATS`] offers, and so how many grain counts a stock
|
||||
/// carries.
|
||||
pub const FORMAT_COUNT: usize = 6;
|
||||
|
||||
/// The uniform field names the fragment reads the exposure matrix from.
|
||||
///
|
||||
/// A table rather than a formatted string, because a `Uniform`'s name is
|
||||
@@ -74,6 +85,24 @@ static MATRIX_FIELDS: [[&str; 3]; 3] = [
|
||||
["m20", "m21", "m22"],
|
||||
];
|
||||
|
||||
/// Grains per pixel, per format and layer: `gn{format}{layer}`.
|
||||
static GRAIN_FIELDS: [[&str; 3]; FORMAT_COUNT] = [
|
||||
["gn00", "gn01", "gn02"],
|
||||
["gn10", "gn11", "gn12"],
|
||||
["gn20", "gn21", "gn22"],
|
||||
["gn30", "gn31", "gn32"],
|
||||
["gn40", "gn41", "gn42"],
|
||||
["gn50", "gn51", "gn52"],
|
||||
];
|
||||
|
||||
/// The push each curve row was developed to, padded with the last.
|
||||
static PUSH_FIELDS: [&str; MAX_CURVE_ROWS] =
|
||||
["ps0", "ps1", "ps2", "ps3", "ps4", "ps5", "ps6", "ps7"];
|
||||
|
||||
/// Which format this is, one-hot. See [`FilmSim::uniforms`] for why a choice
|
||||
/// reaches the shader as six weights rather than an index.
|
||||
static FORMAT_FIELDS: [&str; FORMAT_COUNT] = ["fmt0", "fmt1", "fmt2", "fmt3", "fmt4", "fmt5"];
|
||||
|
||||
static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
Arc::new(OpDescriptor {
|
||||
// Tone and colour both, and not `Effect`: a stock is not something applied
|
||||
@@ -115,48 +144,86 @@ static DESCRIPTOR: LazyLock<Arc<OpDescriptor>> = LazyLock::new(|| {
|
||||
/// Layout is the contract between the two crates, so it is written down here
|
||||
/// and checked rather than assumed:
|
||||
///
|
||||
/// - `exposure_matrix[l][c]` — layer `l`'s response to linear sRGB channel `c`.
|
||||
/// - `curves` — `CURVE_SAMPLES` density triples, uniform over
|
||||
/// `[curve_log_min, curve_log_max]`.
|
||||
/// - `lut` — `lut_size³` linear sRGB triples, uniform over `[0, density_max]`
|
||||
/// on each axis, with the **red axis varying fastest**: index
|
||||
/// `(b * size + g) * size + r`. That is the order a 3D texture upload
|
||||
/// expects, so the consumer hands the slice straight to the driver. Filling
|
||||
/// it the other way round transposes red and blue in the finished picture —
|
||||
/// which is a plausible photograph of the wrong colour, and which the unit
|
||||
/// tests on both sides of this seam happily pass, because each side is
|
||||
/// internally consistent. `dr-film` pins it; `dr-gpu`'s `film_sim` test
|
||||
/// catches it end to end.
|
||||
/// - `exposure_matrix[l][c]` — layer `l`'s response to linear sRGB channel
|
||||
/// `c`, at unit gain: camera exposure is a per-pixel setting.
|
||||
/// - `curves` — one row of `CURVE_SAMPLES` density triples per
|
||||
/// `push_stations` entry, uniform over `[curve_log_min, curve_log_max]`,
|
||||
/// and then, when printed, one more row: the paper's, uniform over
|
||||
/// `[paper.log_min, paper.log_max]`.
|
||||
/// - `lut` — `lut_size³` triples uniform over `[0, density_max]` on each
|
||||
/// axis, with the **red axis varying fastest**: index
|
||||
/// `(b * size + g) * size + r`. Linear sRGB when the film is viewed
|
||||
/// directly; the paper's log₁₀ exposure through the negative when it is
|
||||
/// printed, followed by a second cube, paper density over
|
||||
/// `[0, paper.density_max]` to linear sRGB. That is the order a 3D texture
|
||||
/// upload expects with the cubes stacked in depth, so the consumer hands the
|
||||
/// slice straight to the driver. Filling it the other way round transposes
|
||||
/// red and blue in the finished picture — which is a plausible photograph
|
||||
/// of the wrong colour, and which the unit tests on both sides of this seam
|
||||
/// happily pass, because each side is internally consistent. `dr-film` pins
|
||||
/// it; `dr-gpu`'s `film_sim` test catches it end to end.
|
||||
///
|
||||
/// Everything the sliders move — exposure, push, print exposure, format — is
|
||||
/// absent. They are per-pixel settings the shader applies against these
|
||||
/// tables, which is what lets a mask layer hold its own.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct FilmTables {
|
||||
pub exposure_matrix: [[f32; 3]; 3],
|
||||
pub curves: Vec<[f32; 3]>,
|
||||
/// The push each film row was developed to, ascending: one entry for a
|
||||
/// stock measured at a single process.
|
||||
pub push_stations: Vec<f32>,
|
||||
pub curve_log_min: f32,
|
||||
pub curve_log_max: f32,
|
||||
pub lut: Vec<[f32; 3]>,
|
||||
pub density_max: f32,
|
||||
pub lut_size: usize,
|
||||
/// The print, for a negative printed on paper.
|
||||
pub paper: Option<PaperTables>,
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// Grains in one pixel's patch of film, per layer, with the density
|
||||
/// ceiling and uniformity the variance is taken against. Zero particles
|
||||
/// means no grain, which is how the control is turned off.
|
||||
pub grain_particles: [f32; 3],
|
||||
/// Grains in one pixel's patch of film, per format and then per layer,
|
||||
/// with the density ceiling and uniformity the variance is taken against.
|
||||
/// Zero particles means no grain, which is how the control is turned off.
|
||||
pub grain_particles: [[f32; 3]; FORMAT_COUNT],
|
||||
pub grain_density_max: [f32; 3],
|
||||
pub grain_uniformity: f32,
|
||||
}
|
||||
|
||||
/// The print half of [`FilmTables`]: where the paper's row and cube are read.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub struct PaperTables {
|
||||
/// The enlarger's filtration, per layer, in log₁₀ exposure.
|
||||
pub balance: [f32; 3],
|
||||
pub log_min: f32,
|
||||
pub log_max: f32,
|
||||
pub density_max: f32,
|
||||
}
|
||||
|
||||
impl FilmTables {
|
||||
/// Film rows, not counting the paper's.
|
||||
pub fn curve_rows(&self) -> usize {
|
||||
self.push_stations.len()
|
||||
}
|
||||
|
||||
/// Whether these tables are the shape the shader will index them at.
|
||||
///
|
||||
/// Checked on the way in, because the failure otherwise is a shader
|
||||
/// sampling past the end of a texture: undefined, silent, and different on
|
||||
/// every driver.
|
||||
pub fn is_well_formed(&self) -> bool {
|
||||
self.curves.len() == CURVE_SAMPLES
|
||||
let rows = self.curve_rows();
|
||||
let printed = usize::from(self.paper.is_some());
|
||||
let paper_ok = self
|
||||
.paper
|
||||
.is_none_or(|p| p.density_max > 0.0 && p.log_max > p.log_min);
|
||||
(1..=MAX_CURVE_ROWS).contains(&rows)
|
||||
&& self.push_stations.windows(2).all(|w| w[0] < w[1])
|
||||
&& self.curves.len() == CURVE_SAMPLES * (rows + printed)
|
||||
&& self.lut_size >= 2
|
||||
&& self.lut.len() == self.lut_size.pow(3)
|
||||
&& self.lut.len() == self.lut_size.pow(3) * (1 + printed)
|
||||
&& self.density_max > 0.0
|
||||
&& self.curve_log_max > self.curve_log_min
|
||||
&& paper_ok
|
||||
}
|
||||
}
|
||||
|
||||
@@ -246,60 +313,80 @@ impl Operation for FilmSim {
|
||||
self.set_tables(tables.cloned());
|
||||
}
|
||||
|
||||
fn film_tables(&self) -> Option<&FilmTables> {
|
||||
self.tables.as_ref()
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// A layer's film is its settings, not its own picture blended over the
|
||||
/// global one.
|
||||
///
|
||||
/// Blending outputs would be a photograph developed twice and cross-faded;
|
||||
/// a region on a pushed film is not that. Every uniform below is linear
|
||||
/// in what it controls, so the composer can take each layer's weighted
|
||||
/// average of them and develop the pixel once.
|
||||
fn blends_settings(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// Every value here is linear in what the shader does with it, which is
|
||||
/// what [`Self::blends_settings`] rests on. The format is the one that
|
||||
/// needs arranging: an index averaged between layers is a format nobody
|
||||
/// chose, so it goes out one-hot and the shader mixes the six grain
|
||||
/// counts by it — two layers on 35 mm and 6x7 meet at the average grain.
|
||||
fn uniforms(&self) -> Vec<Uniform> {
|
||||
let Some(t) = &self.tables else {
|
||||
return Vec::new();
|
||||
};
|
||||
let m = t.exposure_matrix;
|
||||
// Exposure rides in the matrix on the CPU when the stock is baked, so
|
||||
// what is left here is the *shader's* copy of the same nine numbers.
|
||||
// Spelled out one at a time because a uniform is a named `f32` in this
|
||||
// pipeline and a matrix would be a second kind of thing for one caller.
|
||||
let mut out = Vec::with_capacity(MATRIX_FIELDS.len() + 5);
|
||||
for (l, row) in m.iter().enumerate() {
|
||||
let mut out = Vec::with_capacity(64);
|
||||
let mut push = |name: &'static str, value: f32| out.push(Uniform { name, value });
|
||||
for (l, row) in t.exposure_matrix.iter().enumerate() {
|
||||
for (c, v) in row.iter().enumerate() {
|
||||
out.push(Uniform {
|
||||
name: MATRIX_FIELDS[l][c],
|
||||
value: *v,
|
||||
});
|
||||
push(MATRIX_FIELDS[l][c], *v);
|
||||
}
|
||||
}
|
||||
for (l, name) in ["gn0", "gn1", "gn2"].into_iter().enumerate() {
|
||||
out.push(Uniform {
|
||||
name,
|
||||
value: t.grain_particles[l],
|
||||
});
|
||||
for (f, per_layer) in t.grain_particles.iter().enumerate() {
|
||||
for (l, v) in per_layer.iter().enumerate() {
|
||||
push(GRAIN_FIELDS[f][l], *v);
|
||||
}
|
||||
}
|
||||
for (l, name) in ["gd0", "gd1", "gd2"].into_iter().enumerate() {
|
||||
out.push(Uniform {
|
||||
name,
|
||||
value: t.grain_density_max[l],
|
||||
});
|
||||
push(name, t.grain_density_max[l]);
|
||||
}
|
||||
out.push(Uniform {
|
||||
name: "grain_u",
|
||||
value: t.grain_uniformity,
|
||||
});
|
||||
out.push(Uniform {
|
||||
name: "log_min",
|
||||
value: t.curve_log_min,
|
||||
});
|
||||
out.push(Uniform {
|
||||
name: "log_max",
|
||||
value: t.curve_log_max,
|
||||
});
|
||||
out.push(Uniform {
|
||||
name: "density_max",
|
||||
value: t.density_max,
|
||||
});
|
||||
out.push(Uniform {
|
||||
name: "lut_size",
|
||||
value: t.lut_size as f32,
|
||||
});
|
||||
out.push(Uniform {
|
||||
name: "print_exposure",
|
||||
value: self.print_exposure,
|
||||
push("grain_u", t.grain_uniformity);
|
||||
push("log_min", t.curve_log_min);
|
||||
push("log_max", t.curve_log_max);
|
||||
push("density_max", t.density_max);
|
||||
push("lut_size", t.lut_size as f32);
|
||||
|
||||
let last = *t.push_stations.last().unwrap_or(&0.0);
|
||||
for (i, name) in PUSH_FIELDS.into_iter().enumerate() {
|
||||
push(name, t.push_stations.get(i).copied().unwrap_or(last));
|
||||
}
|
||||
push("rows", t.curve_rows() as f32);
|
||||
|
||||
let paper = t.paper.unwrap_or(PaperTables {
|
||||
balance: [0.0; 3],
|
||||
log_min: 0.0,
|
||||
log_max: 1.0,
|
||||
density_max: 1.0,
|
||||
});
|
||||
push("printed", if t.paper.is_some() { 1.0 } else { 0.0 });
|
||||
for (l, name) in ["pb0", "pb1", "pb2"].into_iter().enumerate() {
|
||||
push(name, paper.balance[l]);
|
||||
}
|
||||
push("plog_min", paper.log_min);
|
||||
push("plog_max", paper.log_max);
|
||||
push("pdmax", paper.density_max);
|
||||
|
||||
// The sliders.
|
||||
push("ev", self.exposure);
|
||||
push("push", self.push);
|
||||
push("pev", self.print_exposure);
|
||||
let chosen = (self.format.max(0.0).round() as usize).min(FORMAT_COUNT - 1);
|
||||
for (f, name) in FORMAT_FIELDS.into_iter().enumerate() {
|
||||
push(name, if f == chosen { 1.0 } else { 0.0 });
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
@@ -321,8 +408,9 @@ let scene = vec3<f32>(
|
||||
// What each emulsion layer was exposed to. A matrix, exactly: the scene
|
||||
// spectrum reconstructed from an sRGB triple is linear in that triple, so the
|
||||
// integral over wavelength collapsed into these nine numbers when the stock
|
||||
// was baked.
|
||||
let exposure = vec3<f32>(
|
||||
// was baked. The camera's exposure is a gain on it, applied here rather than
|
||||
// baked in so that a layer can hold its own.
|
||||
let exposure = exp2(ev) * vec3<f32>(
|
||||
dot(vec3<f32>(m00, m01, m02), scene),
|
||||
dot(vec3<f32>(m10, m11, m12), scene),
|
||||
dot(vec3<f32>(m20, m21, m22), scene),
|
||||
@@ -331,12 +419,16 @@ let exposure = vec3<f32>(
|
||||
// the curve, and the toe is where it belongs.
|
||||
let log_exposure = log10(max(exposure, vec3<f32>(0.0)) + 1e-10);
|
||||
|
||||
// The characteristic curve: what density each layer develops to. Clamped, not
|
||||
// extrapolated — past the shoulder a real emulsion stops responding, and
|
||||
// extrapolating would turn a blown highlight into a colour cast that grows the
|
||||
// more it is overexposed.
|
||||
let density = film_curve(clamp((log_exposure - log_min) / (log_max - log_min),
|
||||
vec3<f32>(0.0), vec3<f32>(1.0)));
|
||||
// The characteristic curve: what density each layer develops to, at this
|
||||
// pixel's push. Clamped, not extrapolated — past the shoulder a real emulsion
|
||||
// stops responding, and extrapolating would turn a blown highlight into a
|
||||
// colour cast that grows the more it is overexposed.
|
||||
let density = film_curve_pushed(
|
||||
clamp((log_exposure - log_min) / (log_max - log_min), vec3<f32>(0.0), vec3<f32>(1.0)),
|
||||
push,
|
||||
array<f32, 8>(ps0, ps1, ps2, ps3, ps4, ps5, ps6, ps7),
|
||||
u32(rows),
|
||||
);
|
||||
|
||||
// TRACES: FR-DEV-3f
|
||||
// Grain, on the density and before the dye.
|
||||
@@ -346,16 +438,40 @@ let density = film_curve(clamp((log_exposure - log_min) / (log_max - log_min),
|
||||
// through whatever density resulted. Adding noise to the finished colour --
|
||||
// which is what an effect does -- tints the highlights wrong, because that
|
||||
// noise never passes through the dye at all.
|
||||
let grained = film_grain(density, source_px,
|
||||
vec3<f32>(gn0, gn1, gn2),
|
||||
//
|
||||
// The format's grain count, mixed by the one-hot weights: exactly one format's
|
||||
// on the whole photograph, and the weighted average under overlapping layers.
|
||||
let particles = fmt0 * vec3<f32>(gn00, gn01, gn02)
|
||||
+ fmt1 * vec3<f32>(gn10, gn11, gn12)
|
||||
+ fmt2 * vec3<f32>(gn20, gn21, gn22)
|
||||
+ fmt3 * vec3<f32>(gn30, gn31, gn32)
|
||||
+ fmt4 * vec3<f32>(gn40, gn41, gn42)
|
||||
+ fmt5 * vec3<f32>(gn50, gn51, gn52);
|
||||
let grained = film_grain(density, source_px, particles,
|
||||
vec3<f32>(gd0, gd1, gd2),
|
||||
grain_u);
|
||||
|
||||
// Dye absorption, the print through the negative, the paper, the viewing
|
||||
// illuminant and the chromatic adaptation — all of which take exactly three
|
||||
// numbers in, which is why they fit in one lookup.
|
||||
c = film_lut(clamp(grained / density_max, vec3<f32>(0.0), vec3<f32>(1.0)), lut_size);"
|
||||
.into()
|
||||
// Dye absorption through to what comes next — all of it takes exactly three
|
||||
// numbers in, which is why it fits in one lookup. Viewed directly, that is
|
||||
// the picture; printed, it is the light the paper receives through the
|
||||
// negative, in log exposure.
|
||||
let through = film_lut(clamp(grained / density_max, vec3<f32>(0.0), vec3<f32>(1.0)),
|
||||
lut_size, 0);
|
||||
if (printed > 0.5) {
|
||||
// The enlarger: its filtration, and then its exposure, the same stops on
|
||||
// every layer — which is why print exposure is an addition here and not
|
||||
// a table, and so exact at any setting.
|
||||
let paper_log = through + vec3<f32>(pb0, pb1, pb2) + pev * 0.30103;
|
||||
let paper_density = film_curve(
|
||||
clamp((paper_log - plog_min) / (plog_max - plog_min), vec3<f32>(0.0), vec3<f32>(1.0)),
|
||||
u32(rows),
|
||||
);
|
||||
c = film_lut(clamp(paper_density / pdmax, vec3<f32>(0.0), vec3<f32>(1.0)),
|
||||
lut_size, i32(lut_size));
|
||||
} else {
|
||||
c = through;
|
||||
}"
|
||||
.into()
|
||||
}
|
||||
|
||||
fn helpers(&self) -> &'static [crate::operation::Helper] {
|
||||
@@ -363,7 +479,7 @@ c = film_lut(clamp(grained / density_max, vec3<f32>(0.0), vec3<f32>(1.0)), lut_s
|
||||
}
|
||||
}
|
||||
|
||||
static HELPERS: [crate::operation::Helper; 5] = [
|
||||
static HELPERS: [crate::operation::Helper; 6] = [
|
||||
crate::operation::Helper {
|
||||
name: "film_hash",
|
||||
source: "\
|
||||
@@ -453,9 +569,9 @@ fn log10(v: vec3<f32>) -> vec3<f32> {
|
||||
crate::operation::Helper {
|
||||
name: "film_curve",
|
||||
source: "\
|
||||
// Three characteristic curves, sampled from a 256-wide texture and
|
||||
// interpolated by hand. `t` is already normalised to the curve's domain.
|
||||
fn film_curve(t: vec3<f32>) -> vec3<f32> {
|
||||
// Three characteristic curves, one row of a 256-wide texture, interpolated by
|
||||
// hand. `t` is already normalised to the curve's domain.
|
||||
fn film_curve(t: vec3<f32>, row: u32) -> vec3<f32> {
|
||||
let samples = u32(textureDimensions(film_curves).x);
|
||||
let last = f32(samples - 1u);
|
||||
var out = vec3<f32>(0.0);
|
||||
@@ -463,20 +579,45 @@ fn film_curve(t: vec3<f32>) -> vec3<f32> {
|
||||
let x = t[ch] * last;
|
||||
let i = min(u32(floor(x)), samples - 2u);
|
||||
let f = x - f32(i);
|
||||
let a = textureLoad(film_curves, vec2<i32>(i32(i), 0), 0);
|
||||
let b = textureLoad(film_curves, vec2<i32>(i32(i) + 1, 0), 0);
|
||||
let a = textureLoad(film_curves, vec2<i32>(i32(i), i32(row)), 0);
|
||||
let b = textureLoad(film_curves, vec2<i32>(i32(i) + 1, i32(row)), 0);
|
||||
out[ch] = mix(a[ch], b[ch], f);
|
||||
}
|
||||
return out;
|
||||
}",
|
||||
},
|
||||
crate::operation::Helper {
|
||||
name: "film_curve_pushed",
|
||||
source: "\
|
||||
// The curves at a push between two measured processes. Development is
|
||||
// interpolated in log time and push *is* log time, so a straight line between
|
||||
// the neighbouring rows is the stock's own interpolation, not an estimate of
|
||||
// it. Clamped to the first and last process, as the stock is.
|
||||
fn film_curve_pushed(t: vec3<f32>, push: f32, stations: array<f32, 8>, rows: u32) -> vec3<f32> {
|
||||
if (rows < 2u) {
|
||||
return film_curve(t, 0u);
|
||||
}
|
||||
var at = stations;
|
||||
var hi = rows - 1u;
|
||||
for (var i = 1u; i < rows; i = i + 1u) {
|
||||
if (at[i] >= push) {
|
||||
hi = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
let lo = hi - 1u;
|
||||
let f = clamp((push - at[lo]) / max(at[hi] - at[lo], 1e-6), 0.0, 1.0);
|
||||
return mix(film_curve(t, lo), film_curve(t, hi), f);
|
||||
}",
|
||||
},
|
||||
crate::operation::Helper {
|
||||
name: "film_lut",
|
||||
source: "\
|
||||
// Trilinear interpolation of the density lookup, by hand for the same reason
|
||||
// the curve above is: there is no sampler bound, and the eight loads are
|
||||
// cache-neighbours.
|
||||
fn film_lut(t: vec3<f32>, size: f32) -> vec3<f32> {
|
||||
// Trilinear interpolation of one cube of the lookup, by hand for the same
|
||||
// reason the curve above is: there is no sampler bound, and the eight loads
|
||||
// are cache-neighbours. `z0` is where the cube starts in depth: the film's at
|
||||
// zero, the paper's stacked after it.
|
||||
fn film_lut(t: vec3<f32>, size: f32, z0: i32) -> vec3<f32> {
|
||||
let n = i32(size);
|
||||
let x = t * (size - 1.0);
|
||||
let base = min(vec3<i32>(floor(x)), vec3<i32>(n - 2));
|
||||
@@ -489,7 +630,7 @@ fn film_lut(t: vec3<f32>, size: f32) -> vec3<f32> {
|
||||
let wy = select(1.0 - f.y, f.y, dy == 1);
|
||||
for (var dz = 0; dz < 2; dz = dz + 1) {
|
||||
let wz = select(1.0 - f.z, f.z, dz == 1);
|
||||
let p = base + vec3<i32>(dx, dy, dz);
|
||||
let p = base + vec3<i32>(dx, dy, dz + z0);
|
||||
out = out + wx * wy * wz
|
||||
* textureLoad(film_lut_texture, p, 0).rgb;
|
||||
}
|
||||
@@ -513,7 +654,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 32 * 32 * 32],
|
||||
density_max: 3.0,
|
||||
lut_size: 32,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [3.0; 3],
|
||||
grain_uniformity: 0.97,
|
||||
}
|
||||
|
||||
@@ -82,7 +82,7 @@ pub use colour_mixer::ColourMixer;
|
||||
pub use curve::ToneCurve;
|
||||
pub use dehaze::Dehaze;
|
||||
pub use distortion::Distortion;
|
||||
pub use film_sim::{FilmSim, FilmTables};
|
||||
pub use film_sim::{FilmSim, FilmTables, PaperTables};
|
||||
// Clarity and texture are one implementation at two scales; see the module's
|
||||
// documentation for why that is two nodes and not one.
|
||||
pub use local_contrast::{Clarity, Texture};
|
||||
|
||||
@@ -0,0 +1,645 @@
|
||||
//! TRACES: FR-DEV-17
|
||||
//! Whether a crop leaves a mask layer's work outside the frame.
|
||||
//!
|
||||
//! # Why this is a question worth asking
|
||||
//!
|
||||
//! Mask geometry is stored in normalised *source* coordinates (see
|
||||
//! [`MaskSource::Linear`] and [`Stroke::points`]), so a tighter crop never
|
||||
//! destroys a layer. It makes it invisible — which is worse, because nothing
|
||||
//! announces it. The layer is still in the panel, still in the sidecar, still
|
||||
//! costing a rasterisation, and its adjustment lands on pixels nobody will
|
||||
//! ever see. The crop that did that is exactly the kind of edit made early
|
||||
//! and quickly, and the loss is found, if at all, much later.
|
||||
//!
|
||||
//! This module answers one question on the CPU, cheaply enough to ask once
|
||||
//! per committed crop: *which layers did this change of crop take out of the
|
||||
//! picture?* The interface turns the answer into a notice. It never refuses
|
||||
//! the crop — the photographer may well mean it.
|
||||
//!
|
||||
//! # How it is measured
|
||||
//!
|
||||
//! Each layer's mask is sampled on a [`GRID`]×[`GRID`] lattice over the
|
||||
//! source, with the same geometry the mask shader uses (`mask.wgsl`), folded
|
||||
//! part by part with the layer's joins and inversions. The sample points are
|
||||
//! then mapped through the framing — crop, straighten, turns and flips — into
|
||||
//! the frame, and the layer's *share inside* is the coverage that lands in
|
||||
//! the crop over the coverage there is. A layer is hidden by a crop when that
|
||||
//! share falls below [`HIDDEN_SHARE`] and was not already below it.
|
||||
//!
|
||||
//! The lattice is coarse on purpose. The question is "is this layer mostly
|
||||
//! gone", not "which pixels are": the edge treatment — feather, morphology —
|
||||
//! moves a boundary by a few hundredths of the frame and cannot turn a layer
|
||||
//! that is mostly inside into one that is mostly outside, so it is left out.
|
||||
//!
|
||||
//! # What cannot be orphaned
|
||||
//!
|
||||
//! A range ([`MaskSource::Luminance`], [`MaskSource::Colour`]) selects by a
|
||||
//! property of the picture, so wherever the crop falls it selects whatever
|
||||
//! part of the picture remains — there is nothing for a crop to strand. A
|
||||
//! region selection needs the segmentation's label map, which this crate does
|
||||
//! not hold, and a model's selection with no raster to hand has nothing to
|
||||
//! measure. A layer that *adds* any of these is therefore never reported: a
|
||||
//! false alarm on the common path would teach the photographer to dismiss the
|
||||
//! notice unread, which is the failure it exists to prevent. Subtracting one
|
||||
//! is ignored, which can only make the layer look larger — the safe side.
|
||||
|
||||
use std::borrow::Cow;
|
||||
|
||||
use crate::framing::{CropRect, Framing};
|
||||
use crate::mask::{Join, MaskLayer, MaskPart, MaskSource, MaskStack, Stroke};
|
||||
|
||||
/// Samples per axis over the source.
|
||||
///
|
||||
/// 4096 points: enough that a brush dab a twentieth of the frame across is
|
||||
/// several samples wide, and few enough that the whole stack is measured in
|
||||
/// well under a frame when a crop is let go.
|
||||
pub const GRID: usize = 64;
|
||||
|
||||
/// The share of a layer's coverage below which it counts as cropped away.
|
||||
///
|
||||
/// "Mostly outside" rather than "entirely": a gradient reduced to a sliver
|
||||
/// along one edge, or a subject of which one elbow survives, has lost the
|
||||
/// work as surely as one that is wholly gone. Low enough that an ordinary
|
||||
/// recomposition which trims part of a subject is not reported.
|
||||
pub const HIDDEN_SHARE: f32 = 0.1;
|
||||
|
||||
/// A model's selection, as bytes over the source at some proxy size.
|
||||
///
|
||||
/// Supplied by the caller for a part whose own [`MaskPart::coverage`] is not
|
||||
/// set — the live session holds its model output outside the graph and folds
|
||||
/// it into the parts only when saving.
|
||||
pub struct Raster<'a> {
|
||||
pub values: Cow<'a, [u8]>,
|
||||
pub width: usize,
|
||||
pub height: usize,
|
||||
}
|
||||
|
||||
/// One layer's coverage, sampled over the source.
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct Footprint {
|
||||
/// Row-major over the lattice, `0.0..=1.0`, one per cell centre.
|
||||
weights: Vec<f32>,
|
||||
}
|
||||
|
||||
impl Footprint {
|
||||
/// The layer's coverage on the lattice, or `None` when it has none to
|
||||
/// strand — see the module header for which layers those are.
|
||||
///
|
||||
/// `model` is asked for the raster behind a subject or category part that
|
||||
/// carries none of its own.
|
||||
pub fn of<'r>(
|
||||
layer: &MaskLayer,
|
||||
source: (u32, u32),
|
||||
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
|
||||
) -> Option<Self> {
|
||||
let mut acc: Option<Vec<f32>> = None;
|
||||
for (i, part) in layer.shown_parts().enumerate() {
|
||||
let adds = i == 0 || part.join == Join::Union;
|
||||
let Some(values) = part_values(part, source, model) else {
|
||||
if adds {
|
||||
// It follows the picture, or cannot be measured: either
|
||||
// way it is not a shape a crop can leave behind.
|
||||
return None;
|
||||
}
|
||||
continue;
|
||||
};
|
||||
acc = Some(match acc {
|
||||
None => values,
|
||||
Some(mut a) => {
|
||||
for (d, s) in a.iter_mut().zip(values) {
|
||||
*d = part.join.apply(*d, s);
|
||||
}
|
||||
a
|
||||
}
|
||||
});
|
||||
}
|
||||
let mut weights = acc?;
|
||||
if layer.invert {
|
||||
weights.iter_mut().for_each(|w| *w = 1.0 - *w);
|
||||
}
|
||||
Some(Self { weights })
|
||||
}
|
||||
|
||||
/// Of this layer's coverage, the share `inside` marks as in frame.
|
||||
/// `None` when there is no coverage at all.
|
||||
fn share(&self, inside: &[bool]) -> Option<f32> {
|
||||
let (mut total, mut kept) = (0.0f32, 0.0f32);
|
||||
for (w, &i) in self.weights.iter().zip(inside) {
|
||||
total += w;
|
||||
if i {
|
||||
kept += w;
|
||||
}
|
||||
}
|
||||
(total > 1e-3).then(|| kept / total)
|
||||
}
|
||||
|
||||
/// Of this layer's coverage, the share `framing`'s crop keeps.
|
||||
pub fn share_inside(&self, framing: &Framing, source: (u32, u32)) -> Option<f32> {
|
||||
self.share(&in_frame(framing, source))
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-17
|
||||
/// The layers that changing the framing from `before` to `after` took out of
|
||||
/// the picture, in stack order.
|
||||
///
|
||||
/// Only those it *newly* hid: a layer already cropped away by `before` is not
|
||||
/// reported again, or every later adjustment of the crop would repeat a notice
|
||||
/// the photographer has already answered.
|
||||
///
|
||||
/// The view (zoom and pan) of either framing is ignored. It is a way of
|
||||
/// looking, not the frame.
|
||||
pub fn hidden_by_crop<'m, 'r>(
|
||||
masks: &'m MaskStack,
|
||||
before: &Framing,
|
||||
after: &Framing,
|
||||
source: (u32, u32),
|
||||
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
|
||||
) -> Vec<&'m MaskLayer> {
|
||||
if masks.is_empty() || framed_alike(before, after) {
|
||||
return Vec::new();
|
||||
}
|
||||
let was = in_frame(before, source);
|
||||
let now = in_frame(after, source);
|
||||
masks
|
||||
.layers()
|
||||
.iter()
|
||||
.filter(|layer| {
|
||||
let Some(print) = Footprint::of(layer, source, model) else {
|
||||
return false;
|
||||
};
|
||||
let (Some(was), Some(now)) = (print.share(&was), print.share(&now)) else {
|
||||
return false;
|
||||
};
|
||||
was >= HIDDEN_SHARE && now < HIDDEN_SHARE
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Whether the two frame the same part of the source, zoom aside.
|
||||
fn framed_alike(a: &Framing, b: &Framing) -> bool {
|
||||
let mut a = *a;
|
||||
let mut b = *b;
|
||||
a.set_view(CropRect::default());
|
||||
b.set_view(CropRect::default());
|
||||
a == b
|
||||
}
|
||||
|
||||
/// Which lattice cells the framing's crop keeps.
|
||||
fn in_frame(framing: &Framing, source: (u32, u32)) -> Vec<bool> {
|
||||
let mut f = *framing;
|
||||
f.set_view(CropRect::default());
|
||||
lattice()
|
||||
.map(|uv| {
|
||||
let (x, y) = f.output_at(uv, source.0, source.1);
|
||||
(0.0..=1.0).contains(&x) && (0.0..=1.0).contains(&y)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Cell centres, row-major, in normalised source coordinates.
|
||||
fn lattice() -> impl Iterator<Item = (f32, f32)> {
|
||||
let step = 1.0 / GRID as f32;
|
||||
(0..GRID).flat_map(move |y| {
|
||||
(0..GRID).map(move |x| ((x as f32 + 0.5) * step, (y as f32 + 0.5) * step))
|
||||
})
|
||||
}
|
||||
|
||||
/// One part's coverage on the lattice, its own inversion applied, or `None`
|
||||
/// where it cannot be measured as a shape.
|
||||
fn part_values<'r>(
|
||||
part: &MaskPart,
|
||||
source: (u32, u32),
|
||||
model: &dyn Fn(&MaskPart) -> Option<Raster<'r>>,
|
||||
) -> Option<Vec<f32>> {
|
||||
let aspect = source.0.max(1) as f32 / source.1.max(1) as f32;
|
||||
let mut values: Vec<f32> = match &part.source {
|
||||
MaskSource::Linear {
|
||||
centre,
|
||||
angle,
|
||||
width,
|
||||
} => {
|
||||
let axis = (angle.cos(), angle.sin());
|
||||
lattice()
|
||||
.map(|(u, v)| {
|
||||
let d = (u - centre.0) * aspect * axis.0 + (v - centre.1) * axis.1;
|
||||
if *width <= 0.0 {
|
||||
if d >= 0.0 {
|
||||
1.0
|
||||
} else {
|
||||
0.0
|
||||
}
|
||||
} else {
|
||||
smoothstep(-width * 0.5, width * 0.5, d)
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
MaskSource::Radial {
|
||||
centre,
|
||||
radii,
|
||||
angle,
|
||||
feather,
|
||||
} => {
|
||||
let (sa, ca) = (-angle).sin_cos();
|
||||
let radii = (radii.0.max(1e-6), radii.1.max(1e-6));
|
||||
let edge = feather.clamp(0.0, 1.0);
|
||||
lattice()
|
||||
.map(|(u, v)| {
|
||||
let d = ((u - centre.0) * aspect, v - centre.1);
|
||||
let local = (d.0 * ca - d.1 * sa, d.0 * sa + d.1 * ca);
|
||||
let r = (local.0 / radii.0).hypot(local.1 / radii.1);
|
||||
if edge <= 0.0 {
|
||||
if r <= 1.0 {
|
||||
1.0
|
||||
} else {
|
||||
0.0
|
||||
}
|
||||
} else {
|
||||
1.0 - smoothstep(1.0 - edge, 1.0, r)
|
||||
}
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
MaskSource::Brush { strokes } => brush_values(strokes, source),
|
||||
MaskSource::Subject { .. } | MaskSource::Category { .. } => {
|
||||
if let Some(coverage) = &part.coverage {
|
||||
// The lattice is a regular grid over the source, which is
|
||||
// exactly the resample `decode_at` does.
|
||||
coverage
|
||||
.decode_at(GRID, GRID)
|
||||
.into_iter()
|
||||
.map(|b| f32::from(b) / 255.0)
|
||||
.collect()
|
||||
} else {
|
||||
let raster = model(part)?;
|
||||
if raster.width == 0
|
||||
|| raster.height == 0
|
||||
|| raster.values.len() < raster.width * raster.height
|
||||
{
|
||||
return None;
|
||||
}
|
||||
lattice()
|
||||
.map(|(u, v)| {
|
||||
let x = ((u * raster.width as f32) as usize).min(raster.width - 1);
|
||||
let y = ((v * raster.height as f32) as usize).min(raster.height - 1);
|
||||
f32::from(raster.values[y * raster.width + x]) / 255.0
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
}
|
||||
MaskSource::Regions { .. } | MaskSource::Luminance { .. } | MaskSource::Colour { .. } => {
|
||||
return None
|
||||
}
|
||||
};
|
||||
if part.invert {
|
||||
values.iter_mut().for_each(|w| *w = 1.0 - *w);
|
||||
}
|
||||
Some(values)
|
||||
}
|
||||
|
||||
/// Strokes composited in order, the way `fs_brush` and its blend states do.
|
||||
fn brush_values(strokes: &[Stroke], source: (u32, u32)) -> Vec<f32> {
|
||||
// Into units of the shorter edge, as `to_square` does, so a dab is round.
|
||||
let short = source.0.min(source.1).max(1) as f32;
|
||||
let scale = (
|
||||
source.0.max(1) as f32 / short,
|
||||
source.1.max(1) as f32 / short,
|
||||
);
|
||||
let square = |p: (f32, f32)| (p.0 * scale.0, p.1 * scale.1);
|
||||
|
||||
let cells: Vec<(f32, f32)> = lattice().map(square).collect();
|
||||
let mut out = vec![0.0f32; cells.len()];
|
||||
for stroke in strokes.iter().filter(|s| !s.is_empty()) {
|
||||
let points: Vec<(f32, f32)> = stroke.points.iter().copied().map(square).collect();
|
||||
let r = stroke.radius;
|
||||
let inner = r * stroke.hardness.clamp(0.0, 1.0);
|
||||
// Only cells inside the stroke's bounding box can be reached.
|
||||
let (lo, hi) = points.iter().fold(
|
||||
((f32::MAX, f32::MAX), (f32::MIN, f32::MIN)),
|
||||
|(lo, hi), p| {
|
||||
(
|
||||
(lo.0.min(p.0), lo.1.min(p.1)),
|
||||
(hi.0.max(p.0), hi.1.max(p.1)),
|
||||
)
|
||||
},
|
||||
);
|
||||
for (cell, dst) in cells.iter().zip(out.iter_mut()) {
|
||||
if cell.0 < lo.0 - r || cell.0 > hi.0 + r || cell.1 < lo.1 - r || cell.1 > hi.1 + r {
|
||||
continue;
|
||||
}
|
||||
let d = if points.len() == 1 {
|
||||
dist(*cell, points[0])
|
||||
} else {
|
||||
points
|
||||
.windows(2)
|
||||
.map(|s| segment_distance(*cell, s[0], s[1]))
|
||||
.fold(f32::MAX, f32::min)
|
||||
};
|
||||
let c = ((1.0 - smoothstep(inner, r, d)) * stroke.flow).clamp(0.0, 1.0);
|
||||
*dst = if stroke.erase {
|
||||
*dst * (1.0 - c)
|
||||
} else {
|
||||
*dst + c - *dst * c
|
||||
};
|
||||
}
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn dist(a: (f32, f32), b: (f32, f32)) -> f32 {
|
||||
(a.0 - b.0).hypot(a.1 - b.1)
|
||||
}
|
||||
|
||||
fn segment_distance(q: (f32, f32), a: (f32, f32), b: (f32, f32)) -> f32 {
|
||||
let ab = (b.0 - a.0, b.1 - a.1);
|
||||
let len2 = ab.0 * ab.0 + ab.1 * ab.1;
|
||||
if len2 <= 1e-12 {
|
||||
return dist(q, a);
|
||||
}
|
||||
let t = (((q.0 - a.0) * ab.0 + (q.1 - a.1) * ab.1) / len2).clamp(0.0, 1.0);
|
||||
dist(q, (a.0 + ab.0 * t, a.1 + ab.1 * t))
|
||||
}
|
||||
|
||||
/// WGSL's `smoothstep`, including its behaviour when the edges meet.
|
||||
fn smoothstep(e0: f32, e1: f32, x: f32) -> f32 {
|
||||
if e1 <= e0 {
|
||||
return if x < e0 { 0.0 } else { 1.0 };
|
||||
}
|
||||
let t = ((x - e0) / (e1 - e0)).clamp(0.0, 1.0);
|
||||
t * t * (3.0 - 2.0 * t)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use std::sync::Arc;
|
||||
|
||||
use super::*;
|
||||
use crate::coverage::Coverage;
|
||||
|
||||
const SOURCE: (u32, u32) = (3000, 2000);
|
||||
|
||||
fn no_model(_: &MaskPart) -> Option<Raster<'static>> {
|
||||
None
|
||||
}
|
||||
|
||||
fn cropped(x: f32, y: f32, w: f32, h: f32) -> Framing {
|
||||
let mut f = Framing::new();
|
||||
f.set_crop(CropRect {
|
||||
x,
|
||||
y,
|
||||
width: w,
|
||||
height: h,
|
||||
});
|
||||
f
|
||||
}
|
||||
|
||||
/// A small circle near the source's top-left corner.
|
||||
fn top_left_circle() -> MaskLayer {
|
||||
MaskLayer::new(
|
||||
"m1",
|
||||
MaskSource::Radial {
|
||||
centre: (0.15, 0.15),
|
||||
radii: (0.08, 0.08),
|
||||
angle: 0.0,
|
||||
feather: 0.2,
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
fn stack(layers: Vec<MaskLayer>) -> MaskStack {
|
||||
let mut s = MaskStack::new();
|
||||
for l in layers {
|
||||
assert!(s.push(l));
|
||||
}
|
||||
s
|
||||
}
|
||||
|
||||
fn hidden(masks: &MaskStack, before: &Framing, after: &Framing) -> Vec<String> {
|
||||
hidden_by_crop(masks, before, after, SOURCE, &no_model)
|
||||
.into_iter()
|
||||
.map(|l| l.id.clone())
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_crop_away_from_a_shape_hides_it() {
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let after = cropped(0.5, 0.5, 0.5, 0.5);
|
||||
assert_eq!(hidden(&masks, &Framing::new(), &after), vec!["m1"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_crop_that_keeps_the_shape_is_silent() {
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let after = cropped(0.0, 0.0, 0.6, 0.6);
|
||||
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_crop_that_trims_part_of_a_shape_is_silent() {
|
||||
// Half the circle survives, which is a recomposition, not a loss.
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let after = cropped(0.15, 0.0, 0.85, 1.0);
|
||||
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_layer_already_cropped_away_is_not_reported_again() {
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let before = cropped(0.5, 0.5, 0.5, 0.5);
|
||||
let after = cropped(0.6, 0.6, 0.4, 0.4);
|
||||
assert!(hidden(&masks, &before, &after).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn uncropping_brings_a_layer_back_and_says_nothing() {
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let before = cropped(0.5, 0.5, 0.5, 0.5);
|
||||
assert!(hidden(&masks, &before, &Framing::new()).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unchanged_frame_is_silent_whatever_the_zoom() {
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let before = cropped(0.5, 0.5, 0.5, 0.5);
|
||||
let mut after = before;
|
||||
after.set_view(CropRect {
|
||||
x: 0.5,
|
||||
y: 0.5,
|
||||
width: 0.5,
|
||||
height: 0.5,
|
||||
});
|
||||
assert!(hidden(&masks, &Framing::new(), &after).len() == 1);
|
||||
assert!(hidden(&masks, &before, &after).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_range_follows_the_picture_and_is_never_stranded() {
|
||||
let masks = stack(vec![
|
||||
MaskLayer::new("lum", MaskSource::highlights()),
|
||||
MaskLayer::new("skin", MaskSource::skin_tones()),
|
||||
]);
|
||||
let after = cropped(0.9, 0.9, 0.1, 0.1);
|
||||
assert!(hidden(&masks, &Framing::new(), &after).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_linear_gradient_over_the_bottom_is_hidden_by_keeping_the_top() {
|
||||
let masks = stack(vec![MaskLayer::new(
|
||||
"grad",
|
||||
MaskSource::Linear {
|
||||
centre: (0.5, 0.8),
|
||||
angle: std::f32::consts::FRAC_PI_2,
|
||||
width: 0.05,
|
||||
},
|
||||
)]);
|
||||
assert_eq!(
|
||||
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 1.0, 0.5)),
|
||||
vec!["grad"]
|
||||
);
|
||||
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.5, 1.0, 0.5)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_painted_stroke_is_measured_where_it_was_painted() {
|
||||
let mut layer = MaskLayer::new("paint", MaskSource::brush());
|
||||
layer.begin_stroke(0, false, 0.05, 0.8, 1.0);
|
||||
for i in 0..10 {
|
||||
layer.extend_stroke(0, 0.8 + i as f32 * 0.01, 0.8);
|
||||
}
|
||||
layer.end_stroke(0);
|
||||
let masks = stack(vec![layer]);
|
||||
assert_eq!(
|
||||
hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.5, 0.5)),
|
||||
vec!["paint"]
|
||||
);
|
||||
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unpainted_brush_has_nothing_to_lose() {
|
||||
let masks = stack(vec![MaskLayer::new("empty", MaskSource::brush())]);
|
||||
assert!(hidden(&masks, &Framing::new(), &cropped(0.0, 0.0, 0.3, 0.3)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_inverted_layer_is_measured_as_what_it_selects() {
|
||||
// Everything *but* a corner circle: a crop into the other corner
|
||||
// keeps most of it.
|
||||
let mut layer = top_left_circle();
|
||||
layer.invert = true;
|
||||
let masks = stack(vec![layer]);
|
||||
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_subject_is_measured_from_its_stored_coverage_or_the_model() {
|
||||
// A subject occupying the right-hand quarter of a 40x20 proxy.
|
||||
let (w, h) = (40usize, 20usize);
|
||||
let bytes: Vec<u8> = (0..w * h)
|
||||
.map(|i| if i % w >= 30 { 255 } else { 0 })
|
||||
.collect();
|
||||
let subject = MaskSource::Subject {
|
||||
signature: 1,
|
||||
index: 0,
|
||||
class: "dog".into(),
|
||||
score: 0.9,
|
||||
};
|
||||
let left = cropped(0.0, 0.0, 0.5, 1.0);
|
||||
|
||||
let mut stored = MaskLayer::new("stored", subject.clone());
|
||||
stored.base_mut().coverage = Coverage::encode(&bytes, w, h, 2).map(Arc::new);
|
||||
let live = MaskLayer::new("live", subject);
|
||||
let masks = stack(vec![stored, live]);
|
||||
|
||||
// No model to hand: only the layer carrying its raster is measured.
|
||||
assert_eq!(hidden(&masks, &Framing::new(), &left), vec!["stored"]);
|
||||
|
||||
let model = |_: &MaskPart| {
|
||||
Some(Raster {
|
||||
values: Cow::Borrowed(bytes.as_slice()),
|
||||
width: w,
|
||||
height: h,
|
||||
})
|
||||
};
|
||||
let ids: Vec<_> = hidden_by_crop(&masks, &Framing::new(), &left, SOURCE, &model)
|
||||
.into_iter()
|
||||
.map(|l| l.id.as_str())
|
||||
.collect();
|
||||
assert_eq!(ids, vec!["stored", "live"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_crop_is_read_in_the_turned_frame() {
|
||||
// Turned a quarter clockwise, the source's top-left lands at the
|
||||
// frame's top-right, so keeping the right half of the frame keeps it.
|
||||
let masks = stack(vec![top_left_circle()]);
|
||||
let mut before = Framing::new();
|
||||
before.rotate_quarters(1);
|
||||
let mut right = before;
|
||||
right.set_crop(CropRect {
|
||||
x: 0.5,
|
||||
y: 0.0,
|
||||
width: 0.5,
|
||||
height: 1.0,
|
||||
});
|
||||
let mut left = before;
|
||||
left.set_crop(CropRect {
|
||||
x: 0.0,
|
||||
y: 0.0,
|
||||
width: 0.5,
|
||||
height: 1.0,
|
||||
});
|
||||
assert!(hidden(&masks, &before, &right).is_empty());
|
||||
assert_eq!(hidden(&masks, &before, &left), vec!["m1"]);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_subtracted_range_does_not_hide_the_shape_it_cuts() {
|
||||
let mut layer = top_left_circle();
|
||||
assert!(layer.push_part(MaskPart::new(
|
||||
"p2",
|
||||
Join::Subtract,
|
||||
MaskSource::highlights()
|
||||
)));
|
||||
let masks = stack(vec![layer.clone()]);
|
||||
assert_eq!(
|
||||
hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)),
|
||||
vec!["m1"]
|
||||
);
|
||||
// Added instead, the range reaches everywhere and nothing is lost.
|
||||
let mut layer = top_left_circle();
|
||||
assert!(layer.push_part(MaskPart::new("p2", Join::Union, MaskSource::highlights())));
|
||||
let masks = stack(vec![layer]);
|
||||
assert!(hidden(&masks, &Framing::new(), &cropped(0.5, 0.5, 0.5, 0.5)).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_intersection_keeps_only_what_both_parts_cover() {
|
||||
let circle = |id: &str, join, centre, r| {
|
||||
MaskPart::new(
|
||||
id,
|
||||
join,
|
||||
MaskSource::Radial {
|
||||
centre,
|
||||
radii: (r, r),
|
||||
angle: 0.0,
|
||||
feather: 0.2,
|
||||
},
|
||||
)
|
||||
};
|
||||
// Two circles in opposite corners: keeping the bottom-right quarter
|
||||
// keeps one of them, and nothing is lost.
|
||||
let mut layer = top_left_circle();
|
||||
assert!(layer.push_part(circle("p2", Join::Union, (0.85, 0.85), 0.08)));
|
||||
let bottom_right = cropped(0.5, 0.5, 0.5, 0.5);
|
||||
let masks = stack(vec![layer.clone()]);
|
||||
assert!(hidden(&masks, &Framing::new(), &bottom_right).is_empty());
|
||||
// Intersected with a disc around the top-left, only that corner's
|
||||
// circle survives, and the same crop takes it out of the frame.
|
||||
assert!(layer.push_part(circle("p3", Join::Intersect, (0.15, 0.15), 0.3)));
|
||||
let masks = stack(vec![layer]);
|
||||
assert_eq!(hidden(&masks, &Framing::new(), &bottom_right), vec!["m1"]);
|
||||
}
|
||||
}
|
||||
+449
-23
@@ -44,6 +44,7 @@ use std::fmt::Write as _;
|
||||
|
||||
use crate::descriptor::{Attribute, OpId, ParamId};
|
||||
use crate::graph::EditGraph;
|
||||
use crate::state::{FilmRebake, FilmRef};
|
||||
|
||||
/// Which parts of an edit a copy carries.
|
||||
///
|
||||
@@ -173,6 +174,15 @@ impl Scope {
|
||||
self.bits == 0
|
||||
}
|
||||
|
||||
/// Whether this scope carries the film — the stock as well as its sliders.
|
||||
///
|
||||
/// Asked of the film node's own classification rather than decided here,
|
||||
/// so the stock travels under exactly the scopes its exposure slider
|
||||
/// does. Splitting the two would put one stock's exposure on another.
|
||||
pub fn carries_film(self) -> bool {
|
||||
self.covers(crate::ops::film_sim::ID.0)
|
||||
}
|
||||
|
||||
fn bit(attribute: Attribute) -> u8 {
|
||||
1 << Attribute::ALL
|
||||
.iter()
|
||||
@@ -216,6 +226,17 @@ fn attributes_of(op: &str) -> Option<&'static [Attribute]> {
|
||||
TABLE.get(op).map(Vec::as_slice)
|
||||
}
|
||||
|
||||
/// How much of the target an applied preset replaces. See the module note.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
|
||||
pub enum Reach {
|
||||
/// Everything in scope: absence means default. A copy, a saved edit.
|
||||
#[default]
|
||||
Whole,
|
||||
/// Only the operations the preset names, and the film if it names one.
|
||||
/// Everything else in the target is left as it was. A look.
|
||||
Named,
|
||||
}
|
||||
|
||||
/// A set of non-default parameter values, ready to apply elsewhere.
|
||||
///
|
||||
/// Ordered, so two captures of the same edit compare equal and a caller can
|
||||
@@ -223,6 +244,10 @@ fn attributes_of(op: &str) -> Option<&'static [Attribute]> {
|
||||
#[derive(Debug, Clone, PartialEq, Default)]
|
||||
pub struct Preset {
|
||||
params: BTreeMap<(String, String), f32>,
|
||||
reach: Reach,
|
||||
/// TRACES: FR-DEV-3f
|
||||
/// The stock this edit develops on, by id. See the module note.
|
||||
film: Option<FilmRef>,
|
||||
}
|
||||
|
||||
impl Preset {
|
||||
@@ -237,6 +262,21 @@ impl Preset {
|
||||
/// have to be re-copied to change one's mind about framing, and a preset
|
||||
/// that had already discarded the crop could never grow it back.
|
||||
pub fn capture(graph: &EditGraph) -> Self {
|
||||
Self {
|
||||
film: graph.film().map(|f| FilmRef {
|
||||
stock: f.stock.clone(),
|
||||
print: f.print.clone(),
|
||||
}),
|
||||
..Self::capture_params(graph)
|
||||
}
|
||||
}
|
||||
|
||||
/// The parameters alone, without the film.
|
||||
///
|
||||
/// For [`EditGraph::state`], which carries the film in a field of its own
|
||||
/// ([`crate::EditState::film`]). Capturing it twice would give one edit
|
||||
/// two places to disagree about its stock.
|
||||
pub(crate) fn capture_params(graph: &EditGraph) -> Self {
|
||||
let mut params = BTreeMap::new();
|
||||
for cap in graph.capabilities() {
|
||||
for p in &cap.params {
|
||||
@@ -245,12 +285,59 @@ impl Preset {
|
||||
}
|
||||
}
|
||||
}
|
||||
Self { params }
|
||||
Self::from_params(params)
|
||||
}
|
||||
|
||||
/// Build from an already-captured parameter map — a sidecar's, typically.
|
||||
pub fn from_params(params: BTreeMap<(String, String), f32>) -> Self {
|
||||
Self { params }
|
||||
Self {
|
||||
params,
|
||||
reach: Reach::Whole,
|
||||
film: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// The same preset, reaching as far as `reach` says.
|
||||
pub fn with_reach(self, reach: Reach) -> Self {
|
||||
Self { reach, ..self }
|
||||
}
|
||||
|
||||
/// How much of the target this preset replaces.
|
||||
pub fn reach(&self) -> Reach {
|
||||
self.reach
|
||||
}
|
||||
|
||||
/// Whether applying this preset replaces `op` — before any scope is asked.
|
||||
fn reaches(&self, op: &str) -> bool {
|
||||
match self.reach {
|
||||
Reach::Whole => true,
|
||||
Reach::Named => {
|
||||
self.params.keys().any(|(o, _)| o == op)
|
||||
|| (self.film.is_some() && op == crate::ops::film_sim::ID.0)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The same preset, developed on `film`.
|
||||
pub fn with_film(self, film: Option<FilmRef>) -> Self {
|
||||
Self { film, ..self }
|
||||
}
|
||||
|
||||
/// The stock this preset develops on, if it names one.
|
||||
pub fn film(&self) -> Option<&FilmRef> {
|
||||
self.film.as_ref()
|
||||
}
|
||||
|
||||
/// What applying at `scope` does to the target's film.
|
||||
///
|
||||
/// `None` when the scope leaves the film alone; `Some(None)` when it
|
||||
/// develops the target without one. The same two levels the sidecar
|
||||
/// writer takes, which is who asks: the batch path amends files rather
|
||||
/// than graphs, and has to know whether the film line is being written at
|
||||
/// all before it knows what to write.
|
||||
pub fn film_for(&self, scope: Scope) -> Option<Option<&FilmRef>> {
|
||||
(scope.carries_film() && self.reaches(crate::ops::film_sim::ID.0))
|
||||
.then_some(self.film.as_ref())
|
||||
}
|
||||
|
||||
/// The parameters, for a caller that stores them.
|
||||
@@ -270,7 +357,7 @@ impl Preset {
|
||||
/// answers is whether there is a clipboard to offer, which is why the UI
|
||||
/// asks it before enabling a paste.
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.params.is_empty()
|
||||
self.params.is_empty() && self.film.is_none()
|
||||
}
|
||||
|
||||
/// How many parameters were captured.
|
||||
@@ -296,10 +383,14 @@ impl Preset {
|
||||
/// rather than parameters because thirty-six mixer sliders is a number
|
||||
/// about the mixer's shape, not about how much was copied.
|
||||
pub fn op_count(&self, scope: Scope) -> usize {
|
||||
// A stock with every film slider at default is still the film node
|
||||
// at work, and a preset holding nothing else is not "Neutral".
|
||||
let film = self.film.is_some().then_some(crate::ops::film_sim::ID.0);
|
||||
let mut ops: Vec<&str> = self
|
||||
.params
|
||||
.keys()
|
||||
.map(|(op, _)| op.as_str())
|
||||
.chain(film)
|
||||
.filter(|op| scope.covers(op))
|
||||
.collect();
|
||||
ops.sort_unstable();
|
||||
@@ -319,7 +410,13 @@ impl Preset {
|
||||
/// to a fitted view mid-comparison, which reads as the paste having
|
||||
/// navigated somewhere. This mirrors `DevelopSession::reset_framing`, and
|
||||
/// lives here so every caller inherits it rather than each remembering.
|
||||
pub fn apply(&self, graph: &mut EditGraph, scope: Scope) {
|
||||
///
|
||||
/// The **film** is replaced when the scope carries it, and handed back as
|
||||
/// a [`FilmRebake`] because this crate cannot bake a stock — the same debt
|
||||
/// [`EditGraph::set_state`] returns, for the same reason. It is cleared
|
||||
/// even when the preset names the stock already on the graph: the tables
|
||||
/// were baked from the film sliders this call has just replaced.
|
||||
pub fn apply(&self, graph: &mut EditGraph, scope: Scope) -> FilmRebake {
|
||||
let view = graph.framing().view();
|
||||
|
||||
// Clear the scope first, so absence means default (see the module
|
||||
@@ -328,7 +425,7 @@ impl Preset {
|
||||
let clears: Vec<(OpId, ParamId, f32)> = graph
|
||||
.capabilities()
|
||||
.iter()
|
||||
.filter(|cap| scope.covers(cap.id.0))
|
||||
.filter(|cap| scope.covers(cap.id.0) && self.reaches(cap.id.0))
|
||||
.flat_map(|cap| cap.params.iter().map(|p| (cap.id, p.id, p.default)))
|
||||
.collect();
|
||||
for (op, param, default) in clears {
|
||||
@@ -347,6 +444,17 @@ impl Preset {
|
||||
}
|
||||
|
||||
graph.framing_mut().set_view(view);
|
||||
|
||||
match self.film_for(scope) {
|
||||
None => FilmRebake::NotNeeded,
|
||||
Some(film) => {
|
||||
graph.set_film(None);
|
||||
match film {
|
||||
None => FilmRebake::NotNeeded,
|
||||
Some(film) => FilmRebake::Wanted(film.clone()),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Apply to a parameter map — the sidecar of an image that is not open.
|
||||
@@ -361,8 +469,11 @@ impl Preset {
|
||||
/// Same replacement rule as [`Self::apply`]: the target's in-scope keys go,
|
||||
/// the preset's arrive, and out-of-scope keys — the target's own crop, on
|
||||
/// the default scope — are left exactly as they were.
|
||||
///
|
||||
/// A parameter map has no film in it, so the stock is not written here:
|
||||
/// the caller asks [`Self::film_for`] and writes it beside the map.
|
||||
pub fn amend(&self, target: &mut BTreeMap<(String, String), f32>, scope: Scope) {
|
||||
target.retain(|(op, _), _| !scope.covers(op));
|
||||
target.retain(|(op, _), _| !(scope.covers(op) && self.reaches(op)));
|
||||
for ((op, param), value) in &self.params {
|
||||
if scope.covers(op) {
|
||||
target.insert((op.clone(), param.clone()), *value);
|
||||
@@ -556,6 +667,14 @@ impl PresetLibrary {
|
||||
self.presets.is_empty()
|
||||
}
|
||||
|
||||
/// How many lines were kept without being understood.
|
||||
///
|
||||
/// For a file this build wrote itself, or shipped, that should be none: a
|
||||
/// misspelt key is preserved faithfully and does nothing.
|
||||
pub fn unread_lines(&self) -> usize {
|
||||
self.unknown.values().map(Vec::len).sum()
|
||||
}
|
||||
|
||||
/// Serialise to the on-disk form.
|
||||
///
|
||||
/// Deterministic, like the sidecar's: the same library always produces the
|
||||
@@ -567,6 +686,22 @@ impl PresetLibrary {
|
||||
for ((op, param), value) in preset.params() {
|
||||
let _ = writeln!(out, "{op}.{param} = {}", format_value(*value));
|
||||
}
|
||||
// TRACES: FR-DEV-3f
|
||||
// Spelled as the sidecar spells them, so a block can still be
|
||||
// pasted from one file into the other. A build that predates
|
||||
// these lines reads them as lines it does not understand and
|
||||
// writes them back untouched, which is the promise below.
|
||||
// Only when it differs from the default, so every library written
|
||||
// before looks existed still writes the same bytes.
|
||||
if preset.reach() == Reach::Named {
|
||||
let _ = writeln!(out, "reach = named");
|
||||
}
|
||||
if let Some(film) = preset.film() {
|
||||
let _ = writeln!(out, "film = {}", film.stock);
|
||||
if let Some(print) = &film.print {
|
||||
let _ = writeln!(out, "film_print = {print}");
|
||||
}
|
||||
}
|
||||
for line in self.unknown.get(name).into_iter().flatten() {
|
||||
let _ = writeln!(out, "{line}");
|
||||
}
|
||||
@@ -597,6 +732,8 @@ impl PresetLibrary {
|
||||
let mut library = Self::default();
|
||||
let mut current: Option<String> = None;
|
||||
let mut params: BTreeMap<(String, String), f32> = BTreeMap::new();
|
||||
let mut film: Option<FilmRef> = None;
|
||||
let mut reach = Reach::Whole;
|
||||
|
||||
for line in lines {
|
||||
let line = line.trim();
|
||||
@@ -609,9 +746,16 @@ impl PresetLibrary {
|
||||
.and_then(|l| l.strip_suffix(']'))
|
||||
{
|
||||
if let Some(name) = current.take() {
|
||||
library.presets.insert(name, Preset::from_params(params));
|
||||
params = BTreeMap::new();
|
||||
library.presets.insert(
|
||||
name,
|
||||
Preset::from_params(std::mem::take(&mut params))
|
||||
.with_film(film.take())
|
||||
.with_reach(reach),
|
||||
);
|
||||
}
|
||||
params.clear();
|
||||
film = None;
|
||||
reach = Reach::Whole;
|
||||
// A name the writer should never have produced is dropped
|
||||
// rather than taken: accepting it would mean writing a file
|
||||
// back out that no longer parses as this one.
|
||||
@@ -631,6 +775,23 @@ impl PresetLibrary {
|
||||
};
|
||||
|
||||
match line.split_once('=') {
|
||||
// TRACES: FR-DEV-3f
|
||||
// Not checked against the installed stocks: this crate does
|
||||
// not link them, and a preset naming a stock this device
|
||||
// lacks must survive being stored here. Whoever bakes it
|
||||
// reports the miss — the sidecar's rule, for its reason.
|
||||
// `get_or_insert_with` because a hand-edited block may name
|
||||
// the paper first.
|
||||
Some((key, value)) if key.trim() == "reach" && value.trim() == "named" => {
|
||||
reach = Reach::Named;
|
||||
}
|
||||
Some((key, value)) if key.trim() == "film" && !value.trim().is_empty() => {
|
||||
film.get_or_insert_with(FilmRef::default).stock = value.trim().to_string();
|
||||
}
|
||||
Some((key, value)) if key.trim() == "film_print" && !value.trim().is_empty() => {
|
||||
film.get_or_insert_with(FilmRef::default).print =
|
||||
Some(value.trim().to_string());
|
||||
}
|
||||
Some((key, value)) => {
|
||||
let key = key.trim();
|
||||
let value = value.trim();
|
||||
@@ -654,7 +815,21 @@ impl PresetLibrary {
|
||||
}
|
||||
|
||||
if let Some(name) = current {
|
||||
library.presets.insert(name, Preset::from_params(params));
|
||||
library.presets.insert(
|
||||
name,
|
||||
Preset::from_params(params)
|
||||
.with_film(film)
|
||||
.with_reach(reach),
|
||||
);
|
||||
}
|
||||
|
||||
// A paper with no film named beside it is a print of nothing. Dropped
|
||||
// rather than kept, since baking it would have no stock to start from.
|
||||
for preset in library.presets.values_mut() {
|
||||
if preset.film.as_ref().is_some_and(|f| f.stock.is_empty()) {
|
||||
log::warn!("preset library: a paper without a film; ignoring it");
|
||||
preset.film = None;
|
||||
}
|
||||
}
|
||||
|
||||
// A block whose every line was unreadable still produced a preset, and
|
||||
@@ -725,6 +900,7 @@ mod tests {
|
||||
height: 0.5,
|
||||
});
|
||||
g.set_param(framing::ID, framing::ANGLE, -2.0);
|
||||
g.set_param(framing::ID, framing::KEYSTONE_V, 40.0);
|
||||
g
|
||||
}
|
||||
|
||||
@@ -802,7 +978,9 @@ mod tests {
|
||||
};
|
||||
target.set_crop(target_crop);
|
||||
|
||||
preset.apply(&mut target, Scope::adjustments());
|
||||
preset
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
|
||||
assert_eq!(
|
||||
target.param(exposure::ID, exposure::EXPOSURE),
|
||||
@@ -820,15 +998,26 @@ mod tests {
|
||||
Some(0.0),
|
||||
"the source's straightening must not travel on this scope"
|
||||
);
|
||||
// TRACES: FR-DEV-20
|
||||
// Perspective is composition on the same terms as the crop: it is
|
||||
// withheld by the default scope, not pasted over the target's frame.
|
||||
assert_eq!(
|
||||
target.param(framing::ID, framing::KEYSTONE_V),
|
||||
Some(0.0),
|
||||
"the source's keystone must not travel on this scope"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pasting_everything_carries_the_composition_too() {
|
||||
let preset = Preset::capture(&edited());
|
||||
let mut target = EditGraph::default_chain();
|
||||
preset.apply(&mut target, Scope::everything());
|
||||
preset
|
||||
.apply(&mut target, Scope::everything())
|
||||
.expect_no_film();
|
||||
|
||||
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(-2.0));
|
||||
assert_eq!(target.param(framing::ID, framing::KEYSTONE_V), Some(40.0));
|
||||
assert!(
|
||||
(target.crop().width - 0.5).abs() < 1e-5,
|
||||
"{:?}",
|
||||
@@ -847,7 +1036,9 @@ mod tests {
|
||||
target.set_param(exposure::ID, exposure::EXPOSURE, 2.0);
|
||||
target.set_param(saturation::ID, saturation::SATURATION, -50.0);
|
||||
|
||||
neutral.apply(&mut target, Scope::adjustments());
|
||||
neutral
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
|
||||
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(0.0));
|
||||
assert_eq!(
|
||||
@@ -866,7 +1057,9 @@ mod tests {
|
||||
let mut target = EditGraph::default_chain();
|
||||
target.set_param(framing::ID, framing::ANGLE, 3.5);
|
||||
|
||||
neutral.apply(&mut target, Scope::adjustments());
|
||||
neutral
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(3.5));
|
||||
}
|
||||
|
||||
@@ -884,7 +1077,9 @@ mod tests {
|
||||
height: 0.25,
|
||||
});
|
||||
|
||||
preset.apply(&mut target, Scope::everything());
|
||||
preset
|
||||
.apply(&mut target, Scope::everything())
|
||||
.expect_no_film();
|
||||
assert!(
|
||||
target.framing().is_zoomed(),
|
||||
"the paste threw away the viewport: {:?}",
|
||||
@@ -901,7 +1096,9 @@ mod tests {
|
||||
|
||||
let mut sideways = EditGraph::default_chain();
|
||||
sideways.set_orientation(dr_types::Orientation::from_exif(6));
|
||||
preset.apply(&mut sideways, Scope::everything());
|
||||
preset
|
||||
.apply(&mut sideways, Scope::everything())
|
||||
.expect_no_film();
|
||||
|
||||
assert_eq!(
|
||||
sideways.framing().baseline(),
|
||||
@@ -918,7 +1115,9 @@ mod tests {
|
||||
params.insert(("exposure".to_string(), "exposure".to_string()), 1.25);
|
||||
|
||||
let mut target = EditGraph::default_chain();
|
||||
Preset::from_params(params).apply(&mut target, Scope::adjustments());
|
||||
Preset::from_params(params)
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
|
||||
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(1.25));
|
||||
}
|
||||
@@ -929,7 +1128,9 @@ mod tests {
|
||||
params.insert(("exposure".to_string(), "exposure".to_string()), 99.0);
|
||||
|
||||
let mut target = EditGraph::default_chain();
|
||||
Preset::from_params(params).apply(&mut target, Scope::adjustments());
|
||||
Preset::from_params(params)
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(5.0));
|
||||
}
|
||||
|
||||
@@ -991,11 +1192,13 @@ mod tests {
|
||||
|
||||
let mut target_map = Preset::capture(&target_graph).into_params();
|
||||
|
||||
preset.apply(&mut target_graph, scope);
|
||||
preset.apply(&mut target_graph, scope).expect_no_film();
|
||||
preset.amend(&mut target_map, scope);
|
||||
|
||||
let mut rebuilt = EditGraph::default_chain();
|
||||
Preset::from_params(target_map).apply(&mut rebuilt, Scope::everything());
|
||||
Preset::from_params(target_map)
|
||||
.apply(&mut rebuilt, Scope::everything())
|
||||
.expect_no_film();
|
||||
|
||||
for cap in target_graph.capabilities() {
|
||||
for p in &cap.params {
|
||||
@@ -1020,7 +1223,9 @@ mod tests {
|
||||
let preset = Preset::capture(&source);
|
||||
|
||||
let mut target = EditGraph::default_chain();
|
||||
preset.apply(&mut target, Scope::everything());
|
||||
preset
|
||||
.apply(&mut target, Scope::everything())
|
||||
.expect_no_film();
|
||||
|
||||
for cap in source.capabilities() {
|
||||
for p in &cap.params {
|
||||
@@ -1058,6 +1263,190 @@ mod tests {
|
||||
assert_eq!(excluded, vec![framing::ID.0]);
|
||||
}
|
||||
|
||||
// --- the film ------------------------------------------------------------
|
||||
|
||||
/// A graph developing on `stock`. The tables are invented; what is under
|
||||
/// test is whether the *choice* travels.
|
||||
fn on_film(stock: &str) -> EditGraph {
|
||||
let mut g = edited();
|
||||
g.set_film(Some(crate::graph::Film {
|
||||
stock: stock.to_string(),
|
||||
print: Some("kodak_portra_endura".to_string()),
|
||||
tables: crate::ops::FilmTables {
|
||||
exposure_matrix: [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]],
|
||||
curves: vec![[0.5, 0.5, 0.5]; crate::ops::film_sim::CURVE_SAMPLES],
|
||||
curve_log_min: -3.0,
|
||||
curve_log_max: 1.0,
|
||||
lut: vec![[0.5, 0.5, 0.5]; 8],
|
||||
density_max: 2.0,
|
||||
lut_size: 2,
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [2.0; 3],
|
||||
grain_uniformity: 1.0,
|
||||
},
|
||||
}));
|
||||
g
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
#[test]
|
||||
fn a_copy_carries_the_stock_it_was_developed_on() {
|
||||
let preset = Preset::capture(&on_film("kodak_portra_400"));
|
||||
let film = preset.film().expect("the stock was captured");
|
||||
assert_eq!(film.stock, "kodak_portra_400");
|
||||
assert_eq!(film.print.as_deref(), Some("kodak_portra_endura"));
|
||||
assert!(Preset::capture(&edited()).film().is_none());
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
/// The graph's own state keeps the film in one place only.
|
||||
#[test]
|
||||
fn an_edit_state_does_not_carry_the_film_twice() {
|
||||
let state = on_film("kodak_portra_400").state();
|
||||
assert!(state.film.is_some());
|
||||
assert_eq!(state.params.film(), None);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
/// Applying a preset that names a stock asks for it to be baked, and
|
||||
/// clears the tables it found — they came from the sliders it replaced.
|
||||
#[test]
|
||||
fn applying_a_film_preset_asks_for_its_stock() {
|
||||
let preset = Preset::capture(&on_film("kodak_portra_400"));
|
||||
let mut target = on_film("ilford_hp5");
|
||||
|
||||
let rebake = preset.apply(&mut target, Scope::adjustments());
|
||||
assert_eq!(
|
||||
rebake.wanted().map(|f| f.stock.as_str()),
|
||||
Some("kodak_portra_400")
|
||||
);
|
||||
assert!(target.film().is_none(), "the old tables were left standing");
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
/// Replacement, as for every parameter: a preset with no stock, at a
|
||||
/// scope that carries the film, develops the target without one.
|
||||
#[test]
|
||||
fn a_preset_without_a_film_clears_the_targets() {
|
||||
let mut target = on_film("ilford_hp5");
|
||||
Preset::capture(&edited())
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert!(target.film().is_none());
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
/// A scope that leaves the film node behind leaves the stock behind too,
|
||||
/// so an exposure never lands on a stock it was not set for.
|
||||
#[test]
|
||||
fn a_scope_without_the_film_leaves_the_stock_alone() {
|
||||
let preset = Preset::capture(&on_film("kodak_portra_400"));
|
||||
let mut target = on_film("ilford_hp5");
|
||||
|
||||
let tone = Scope::of([Attribute::Tone]);
|
||||
assert!(!tone.carries_film());
|
||||
preset.apply(&mut target, tone).expect_no_film();
|
||||
assert_eq!(target.film().map(|f| f.stock.as_str()), Some("ilford_hp5"));
|
||||
assert_eq!(preset.film_for(tone), None);
|
||||
assert!(preset.film_for(Scope::adjustments()).is_some());
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
#[test]
|
||||
fn a_stock_alone_is_not_a_neutral_preset() {
|
||||
let preset = Preset::default().with_film(Some(FilmRef {
|
||||
stock: "kodak_portra_400".into(),
|
||||
print: None,
|
||||
}));
|
||||
assert!(!preset.is_empty());
|
||||
assert_eq!(preset.op_count(Scope::adjustments()), 1);
|
||||
assert_eq!(preset.op_count(Scope::of([Attribute::Tone])), 0);
|
||||
}
|
||||
|
||||
// --- reach ---------------------------------------------------------------
|
||||
|
||||
/// A look: a stock and a contrast, and nothing else.
|
||||
fn look() -> Preset {
|
||||
let mut params = BTreeMap::new();
|
||||
params.insert(("contrast".to_string(), "contrast".to_string()), 20.0);
|
||||
Preset::from_params(params)
|
||||
.with_film(Some(FilmRef {
|
||||
stock: "kodak_portra_400".into(),
|
||||
print: None,
|
||||
}))
|
||||
.with_reach(Reach::Named)
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6
|
||||
/// The reason `Reach` exists: a look applied over a corrected photograph
|
||||
/// keeps the correction.
|
||||
#[test]
|
||||
fn a_look_leaves_what_it_does_not_name_alone() {
|
||||
let mut target = EditGraph::default_chain();
|
||||
target.set_param(exposure::ID, exposure::EXPOSURE, 1.25);
|
||||
target.set_param(saturation::ID, saturation::SATURATION, -40.0);
|
||||
|
||||
let rebake = look().apply(&mut target, Scope::adjustments());
|
||||
|
||||
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(1.25));
|
||||
assert_eq!(
|
||||
target.param(saturation::ID, saturation::SATURATION),
|
||||
Some(-40.0)
|
||||
);
|
||||
assert_eq!(
|
||||
rebake.wanted().map(|f| f.stock.as_str()),
|
||||
Some("kodak_portra_400")
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6
|
||||
/// A look without a film leaves the target's film alone, where a whole
|
||||
/// edit without one would clear it.
|
||||
#[test]
|
||||
fn a_look_without_a_film_keeps_the_targets() {
|
||||
let mut target = on_film("ilford_hp5");
|
||||
let only_contrast = look().with_film(None);
|
||||
assert_eq!(only_contrast.film_for(Scope::adjustments()), None);
|
||||
only_contrast
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(target.film().map(|f| f.stock.as_str()), Some("ilford_hp5"));
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6
|
||||
/// The batch path agrees: a look amends only the operations it names.
|
||||
#[test]
|
||||
fn a_look_amends_only_what_it_names() {
|
||||
let mut target = BTreeMap::new();
|
||||
target.insert(("exposure".to_string(), "exposure".to_string()), 1.25);
|
||||
target.insert(("contrast".to_string(), "contrast".to_string()), -50.0);
|
||||
look().amend(&mut target, Scope::adjustments());
|
||||
assert_eq!(
|
||||
target.get(&("exposure".to_string(), "exposure".to_string())),
|
||||
Some(&1.25)
|
||||
);
|
||||
assert_eq!(
|
||||
target.get(&("contrast".to_string(), "contrast".to_string())),
|
||||
Some(&20.0)
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6
|
||||
#[test]
|
||||
fn a_look_survives_the_library_round_trip() {
|
||||
let mut lib = named();
|
||||
lib.insert("Look", look()).unwrap();
|
||||
let text = lib.to_text();
|
||||
assert!(text.contains("reach = named\n"), "{text}");
|
||||
let back = PresetLibrary::parse(&text).unwrap();
|
||||
assert_eq!(back, lib);
|
||||
// And the default is not written, so an existing library's bytes are
|
||||
// unchanged by this format ever having grown the line.
|
||||
assert_eq!(named().to_text().matches("reach").count(), 0);
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// Named presets
|
||||
// -----------------------------------------------------------------------
|
||||
@@ -1137,6 +1526,37 @@ mod tests {
|
||||
assert!(out.contains("not_an_op.not_a_param = 0.25"), "{out}");
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
#[test]
|
||||
fn a_film_survives_the_library_round_trip() {
|
||||
let mut lib = named();
|
||||
lib.insert("Portra", Preset::capture(&on_film("kodak_portra_400")))
|
||||
.unwrap();
|
||||
let text = lib.to_text();
|
||||
assert!(text.contains("film = kodak_portra_400\n"), "{text}");
|
||||
assert!(
|
||||
text.contains("film_print = kodak_portra_endura\n"),
|
||||
"{text}"
|
||||
);
|
||||
assert_eq!(PresetLibrary::parse(&text).unwrap(), lib);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6 | FR-DEV-3f
|
||||
/// The paper may come first in a hand-edited block, and a paper with no
|
||||
/// film is dropped rather than baked from nothing.
|
||||
#[test]
|
||||
fn film_lines_read_in_either_order_and_a_lone_paper_is_dropped() {
|
||||
let text = format!(
|
||||
"drpl {LIBRARY_FORMAT_VERSION}\n\n[preset A]\nfilm_print = p\nfilm = s\n\n\
|
||||
[preset B]\nfilm_print = p\nexposure.exposure = 1\n"
|
||||
);
|
||||
let lib = PresetLibrary::parse(&text).unwrap();
|
||||
let a = lib.get("A").unwrap().film().unwrap();
|
||||
assert_eq!((a.stock.as_str(), a.print.as_deref()), ("s", Some("p")));
|
||||
assert_eq!(lib.get("B").unwrap().film(), None);
|
||||
assert_eq!(lib.get("B").unwrap().len(), 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_file_from_a_newer_build_is_refused_rather_than_guessed_at() {
|
||||
let text = format!("drpl {}\n", LIBRARY_FORMAT_VERSION + 1);
|
||||
@@ -1224,7 +1644,9 @@ mod tests {
|
||||
let preset = stored.get("Warm portrait").unwrap();
|
||||
|
||||
let mut target = EditGraph::default_chain();
|
||||
preset.apply(&mut target, Scope::adjustments());
|
||||
preset
|
||||
.apply(&mut target, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(target.param(exposure::ID, exposure::EXPOSURE), Some(0.75));
|
||||
// The target keeps its own framing on the default scope.
|
||||
assert_eq!(target.param(framing::ID, framing::ANGLE), Some(0.0));
|
||||
@@ -1258,7 +1680,9 @@ mod tests {
|
||||
fn a_hand_picked_scope_carries_only_the_kinds_it_names() {
|
||||
let preset = Preset::capture(&edited());
|
||||
let mut target = EditGraph::default_chain();
|
||||
preset.apply(&mut target, Scope::of([Attribute::Tone]));
|
||||
preset
|
||||
.apply(&mut target, Scope::of([Attribute::Tone]))
|
||||
.expect_no_film();
|
||||
|
||||
// Tone was picked, so the exposure travelled.
|
||||
assert_eq!(
|
||||
@@ -1323,7 +1747,9 @@ mod tests {
|
||||
|
||||
let mut target = EditGraph::default_chain();
|
||||
target.set_param(exposure::ID, exposure::EXPOSURE, -1.25);
|
||||
Preset::capture(&edited()).apply(&mut target, empty);
|
||||
Preset::capture(&edited())
|
||||
.apply(&mut target, empty)
|
||||
.expect_no_film();
|
||||
assert_eq!(
|
||||
target.param(exposure::ID, exposure::EXPOSURE),
|
||||
Some(-1.25),
|
||||
|
||||
@@ -93,6 +93,9 @@ pub const MAX_RATING: u8 = 5;
|
||||
/// Highest flag code: 0 unflagged, 1 pick, 2 reject.
|
||||
pub const MAX_FLAG: u8 = 2;
|
||||
|
||||
/// Highest colour-label code: purple. Mirrors `dr_catalog::rating::label_code`.
|
||||
pub const MAX_LABEL: u8 = 5;
|
||||
|
||||
/// TRACES: FR-CAT-8 | FR-NC-8
|
||||
/// One image's sidecar: a keyed set of versions.
|
||||
///
|
||||
@@ -187,6 +190,17 @@ pub struct Version {
|
||||
/// so a value moving between the two stores needs no translation table
|
||||
/// that could drift.
|
||||
pub flag: u8,
|
||||
/// TRACES: FR-CAT-5
|
||||
/// The colour label, `0` for none and `1..=5` red, yellow, green, blue,
|
||||
/// purple — the catalog's `versions.label` codes, for the reason
|
||||
/// [`Self::flag`] shares its encoding.
|
||||
///
|
||||
/// Here for the reason the rating is: a label that lived only in the
|
||||
/// catalog would go with the catalog, and would never reach the
|
||||
/// photographer's other devices, which learn judgements from this file.
|
||||
/// A build that predates the key keeps it as an unknown line and writes
|
||||
/// it back, so an older device passes it on rather than erasing it.
|
||||
pub label: u8,
|
||||
/// The edit itself: `(op, param) -> value`, non-default values only.
|
||||
pub params: BTreeMap<(String, String), f32>,
|
||||
/// TRACES: FR-DEV-3 | FR-NC-9
|
||||
@@ -264,6 +278,7 @@ impl Version {
|
||||
// it in the "not yet looked at" state a cull resumes from.
|
||||
rating: 0,
|
||||
flag: 0,
|
||||
label: 0,
|
||||
params,
|
||||
masks,
|
||||
film,
|
||||
@@ -572,6 +587,10 @@ impl Version {
|
||||
// that never had it.
|
||||
self.rating = merge_judgement(self.rating, remote.rating, remote_wins);
|
||||
self.flag = merge_judgement(self.flag, remote.flag, remote_wins);
|
||||
// TRACES: FR-CAT-5
|
||||
// A label under the same rule: 0 is "none given", so a device that
|
||||
// never labelled a frame cannot clear another's label.
|
||||
self.label = merge_judgement(self.label, remote.label, remote_wins);
|
||||
|
||||
// TRACES: FR-DEV-3f
|
||||
// The film resolves wholesale to the higher revision, like a mask
|
||||
@@ -873,6 +892,9 @@ impl Sidecar {
|
||||
if v.flag > 0 {
|
||||
let _ = writeln!(out, "flag = {}", v.flag);
|
||||
}
|
||||
if v.label > 0 {
|
||||
let _ = writeln!(out, "label = {}", v.label);
|
||||
}
|
||||
// TRACES: FR-DEV-3f
|
||||
// Before the parameters, because it decides what they mean: the
|
||||
// film's exposure slider is a slider on *that stock's* curve.
|
||||
@@ -1068,6 +1090,17 @@ impl Sidecar {
|
||||
Some(value.to_string());
|
||||
}
|
||||
"flag" => version.flag = value.parse::<u8>().unwrap_or(0).min(MAX_FLAG),
|
||||
// TRACES: FR-CAT-5
|
||||
// A code this build does not know reads as none rather than
|
||||
// being clamped onto purple: a wrong colour is a claim, and
|
||||
// no colour is only a gap.
|
||||
"label" => {
|
||||
version.label = value
|
||||
.parse::<u8>()
|
||||
.ok()
|
||||
.filter(|l| *l <= MAX_LABEL)
|
||||
.unwrap_or(0)
|
||||
}
|
||||
// TRACES: FR-DEV-8
|
||||
// Ahead of the `op.param` arm below, which would otherwise try
|
||||
// to read eight numbers as one float and drop the repair with a
|
||||
@@ -1990,7 +2023,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 8],
|
||||
density_max: 2.0,
|
||||
lut_size: 2,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [2.0; 3],
|
||||
grain_uniformity: 1.0,
|
||||
},
|
||||
@@ -2141,6 +2176,8 @@ mod tests {
|
||||
height: 0.6,
|
||||
});
|
||||
g.set_param(framing::ID, framing::ANGLE, -1.5);
|
||||
g.set_param(framing::ID, framing::KEYSTONE_V, 42.0);
|
||||
g.set_param(framing::ID, framing::KEYSTONE_H, -17.0);
|
||||
g.rotate_quarters(1);
|
||||
|
||||
let mut sidecar = Sidecar::new();
|
||||
@@ -2157,6 +2194,12 @@ mod tests {
|
||||
assert_eq!(restored.crop(), g.crop());
|
||||
assert_eq!(restored.param(framing::ID, framing::ANGLE), Some(-1.5));
|
||||
assert_eq!(restored.param(framing::ID, framing::ROTATION), Some(1.0));
|
||||
// TRACES: FR-DEV-20
|
||||
assert_eq!(restored.param(framing::ID, framing::KEYSTONE_V), Some(42.0));
|
||||
assert_eq!(
|
||||
restored.param(framing::ID, framing::KEYSTONE_H),
|
||||
Some(-17.0)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2828,6 +2871,37 @@ mod tests {
|
||||
assert_eq!(back.flag, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_label_survives_the_round_trip_and_an_unknown_one_reads_as_none() {
|
||||
// TRACES: FR-CAT-5
|
||||
let mut v = version_of(&EditGraph::default_chain());
|
||||
v.label = 3;
|
||||
let mut sidecar = Sidecar::new();
|
||||
sidecar.put(v);
|
||||
let text = sidecar.to_text();
|
||||
assert!(text.contains("label = 3"), "{text}");
|
||||
let parsed = Sidecar::parse(&text).expect("valid");
|
||||
assert_eq!(parsed.default_version().expect("a version").label, 3);
|
||||
|
||||
let odd = "drsc 1\n\n[version u1]\nname = Default\nrevision = 1\nmodified = 0\n\
|
||||
label = 9\n";
|
||||
let parsed = Sidecar::parse(odd).expect("valid");
|
||||
assert_eq!(parsed.default_version().expect("a version").label, 0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unlabelled_device_cannot_clear_another_devices_label() {
|
||||
// TRACES: FR-CAT-5
|
||||
let mut local = version_of(&EditGraph::default_chain());
|
||||
local.label = 0;
|
||||
local.revision = 9;
|
||||
let mut remote = version_of(&EditGraph::default_chain());
|
||||
remote.label = 1;
|
||||
remote.revision = 1;
|
||||
local.merge(&remote, None);
|
||||
assert_eq!(local.label, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unrated_image_writes_no_judgement_lines() {
|
||||
// The non-default rule applied to judgement: a library that has never
|
||||
@@ -2838,6 +2912,7 @@ mod tests {
|
||||
let text = sidecar.to_text();
|
||||
assert!(!text.contains("rating"), "{text}");
|
||||
assert!(!text.contains("flag"), "{text}");
|
||||
assert!(!text.contains("label"), "{text}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1,220 +0,0 @@
|
||||
//! TRACES: FR-DEV-6
|
||||
//! The presets a first run starts with.
|
||||
//!
|
||||
//! # Why any at all
|
||||
//!
|
||||
//! A preset sheet that opens on "No presets yet" teaches the photographer that
|
||||
//! the feature is homework. These exist so the first thing the sheet does is
|
||||
//! demonstrate what a preset *is* — and so that applying one to a selection,
|
||||
//! which is the action worth discovering, is available before anybody has
|
||||
//! saved anything.
|
||||
//!
|
||||
//! # Why these are ours and not Adobe's
|
||||
//!
|
||||
//! Lightroom ships a large bundled set, and importing one of *those* files is
|
||||
//! what [`crate::preset_import`] is for — a photographer's own library,
|
||||
//! carried across. Redistributing Adobe's inside this application would be
|
||||
//! shipping their creative work under a licence that does not permit it, which
|
||||
//! is a reason on its own; and their numbers are calibrated against their tone
|
||||
//! curve rather than ours, so the look would not survive the trip even if the
|
||||
//! licence allowed it.
|
||||
//!
|
||||
//! So these are written against this pipeline, in its units, and they are
|
||||
//! deliberately mild. A starter preset is a starting point — a photographer
|
||||
//! who wanted the full effect can push the sliders, where one who is handed a
|
||||
//! caricature learns to distrust the list.
|
||||
//!
|
||||
//! # Why they live in the core rather than in the interface
|
||||
//!
|
||||
//! Because they name operations, and nothing in `ui/` may
|
||||
//! (`ui_names_no_operation.rs`, ARCH §4.3a). That test is right to object: a
|
||||
//! preset called "Punch" *is* a statement about contrast, clarity and
|
||||
//! vibrance, which makes it a statement in the pipeline's vocabulary rather
|
||||
//! than a fact about any interface. The frontend asks for the set and stores
|
||||
//! it; it never learns what is in it.
|
||||
//!
|
||||
//! # Why they are seeded rather than merged
|
||||
//!
|
||||
//! Written once, on the first run that finds no library at all, and never
|
||||
//! again. Re-adding them on every start would resurrect one the photographer
|
||||
//! deleted on purpose, and updating them in place would silently rewrite an
|
||||
//! edit they had adjusted and kept under the same name. After the first run
|
||||
//! these are ordinary presets: renameable, editable, deletable, and gone for
|
||||
//! good when deleted.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use crate::{Preset, PresetLibrary};
|
||||
|
||||
/// One starter preset: a name and the parameters that differ from default.
|
||||
struct Starter {
|
||||
name: &'static str,
|
||||
params: &'static [(&'static str, &'static str, f32)],
|
||||
}
|
||||
|
||||
/// The set. Small on purpose — six a photographer might actually reach for
|
||||
/// beats forty they have to scroll past.
|
||||
const STARTERS: &[Starter] = &[
|
||||
Starter {
|
||||
name: "Punch",
|
||||
params: &[
|
||||
("contrast", "contrast", 18.0),
|
||||
("clarity", "amount", 12.0),
|
||||
("vibrance", "vibrance", 18.0),
|
||||
("blacks_whites", "blacks", -8.0),
|
||||
],
|
||||
},
|
||||
Starter {
|
||||
name: "Soft portrait",
|
||||
params: &[
|
||||
("contrast", "contrast", -8.0),
|
||||
("highlights_shadows", "highlights", -20.0),
|
||||
("highlights_shadows", "shadows", 15.0),
|
||||
("clarity", "amount", -10.0),
|
||||
("vibrance", "vibrance", 10.0),
|
||||
("saturation", "saturation", -5.0),
|
||||
],
|
||||
},
|
||||
Starter {
|
||||
name: "Recover the sky",
|
||||
params: &[
|
||||
// The most common single fix in landscape work: a bright sky and a
|
||||
// dark foreground, both pulled back toward the middle.
|
||||
("highlights_shadows", "highlights", -55.0),
|
||||
("highlights_shadows", "shadows", 35.0),
|
||||
("blacks_whites", "whites", -10.0),
|
||||
],
|
||||
},
|
||||
Starter {
|
||||
name: "Lift the shadows",
|
||||
params: &[
|
||||
("highlights_shadows", "shadows", 40.0),
|
||||
("blacks_whites", "blacks", 12.0),
|
||||
("contrast", "contrast", -5.0),
|
||||
],
|
||||
},
|
||||
Starter {
|
||||
name: "Crisp detail",
|
||||
params: &[
|
||||
("texture", "amount", 20.0),
|
||||
("clarity", "amount", 10.0),
|
||||
("capture_sharpen", "amount", 35.0),
|
||||
],
|
||||
},
|
||||
Starter {
|
||||
name: "Muted",
|
||||
params: &[
|
||||
("saturation", "saturation", -30.0),
|
||||
("vibrance", "vibrance", 10.0),
|
||||
("contrast", "contrast", -10.0),
|
||||
("highlights_shadows", "shadows", 12.0),
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
/// The starter library, for a device that has never had one.
|
||||
pub fn library() -> PresetLibrary {
|
||||
let mut library = PresetLibrary::default();
|
||||
for starter in STARTERS {
|
||||
let params: BTreeMap<(String, String), f32> = starter
|
||||
.params
|
||||
.iter()
|
||||
.map(|(op, param, value)| ((op.to_string(), param.to_string()), *value))
|
||||
.collect();
|
||||
// The name is a literal in this file, so a refusal would be a bug here
|
||||
// rather than bad input — but it still must not take the whole set
|
||||
// down, since the alternative to five presets is not six, it is none.
|
||||
if let Err(e) = library.insert(starter.name, Preset::from_params(params)) {
|
||||
log::warn!(
|
||||
"starter preset {:?} is unusable ({e:?}); skipping",
|
||||
starter.name
|
||||
);
|
||||
}
|
||||
}
|
||||
library
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::{EditGraph, Scope};
|
||||
|
||||
#[test]
|
||||
fn every_starter_names_parameters_this_build_actually_has() {
|
||||
// The same guard the importer's table has, for the same reason: a
|
||||
// renamed parameter must break the build rather than ship a preset
|
||||
// that quietly does nothing.
|
||||
let graph = EditGraph::default_chain();
|
||||
let capabilities = graph.capabilities();
|
||||
for starter in STARTERS {
|
||||
for (op, param, _) in starter.params {
|
||||
let capability = capabilities
|
||||
.iter()
|
||||
.find(|c| c.id.0 == *op)
|
||||
.unwrap_or_else(|| panic!("{:?}: no operation {op:?}", starter.name));
|
||||
assert!(
|
||||
capability.params.iter().any(|p| p.id.0 == *param),
|
||||
"{:?}: operation {op:?} has no parameter {param:?}",
|
||||
starter.name
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_starter_actually_changes_something() {
|
||||
// A preset that applies to nothing is worse than one fewer preset: it
|
||||
// teaches the photographer that the list does not work.
|
||||
for (name, preset) in library().iter() {
|
||||
assert!(!preset.is_empty(), "{name} carries nothing");
|
||||
|
||||
let mut graph = EditGraph::default_chain();
|
||||
preset.apply(&mut graph, Scope::adjustments());
|
||||
assert_ne!(
|
||||
Preset::capture(&graph),
|
||||
Preset::default(),
|
||||
"{name} left the graph at its defaults"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_starter_carries_a_crop() {
|
||||
// These are looks, not compositions. One that re-framed every image it
|
||||
// was applied to would be the exact accident `Scope`'s default exists
|
||||
// to prevent.
|
||||
for (name, preset) in library().iter() {
|
||||
assert!(!preset.touches_framing(), "{name} carries framing");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_names_are_distinct() {
|
||||
assert_eq!(library().len(), STARTERS.len());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_values_stay_inside_what_the_controls_accept() {
|
||||
// Clamping happens on apply, so an out-of-range literal here would be
|
||||
// silently trimmed and the preset would not be the one written.
|
||||
let graph = EditGraph::default_chain();
|
||||
for starter in STARTERS {
|
||||
for (op, param, value) in starter.params {
|
||||
let mut applied = EditGraph::default_chain();
|
||||
let capability = graph
|
||||
.capabilities()
|
||||
.into_iter()
|
||||
.find(|c| c.id.0 == *op)
|
||||
.unwrap();
|
||||
let descriptor = capability.params.iter().find(|p| p.id.0 == *param).unwrap();
|
||||
applied.set_param(capability.id, descriptor.id, *value);
|
||||
assert_eq!(
|
||||
applied.param(capability.id, descriptor.id),
|
||||
Some(*value),
|
||||
"{}: {op}.{param} = {value} was clamped",
|
||||
starter.name
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -235,7 +235,9 @@ mod tests {
|
||||
lut: vec![[0.5, 0.5, 0.5]; 8],
|
||||
density_max: 2.0,
|
||||
lut_size: 2,
|
||||
grain_particles: [0.0; 3],
|
||||
grain_particles: [[0.0; 3]; crate::ops::film_sim::FORMAT_COUNT],
|
||||
push_stations: vec![0.0],
|
||||
paper: None,
|
||||
grain_density_max: [2.0; 3],
|
||||
grain_uniformity: 1.0,
|
||||
},
|
||||
|
||||
@@ -1344,6 +1344,28 @@ fn a_correction_painted_onto_a_subject_survives_a_round_trip() {
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// An intersecting part is written under its own word and reads back as an
|
||||
/// intersection. A build from before intersection read that word as a union
|
||||
/// (see `an_unknown_join_adds_the_part`), which keeps the part visible and
|
||||
/// fixable rather than silently cutting the mask down.
|
||||
#[test]
|
||||
fn an_intersecting_part_survives_a_round_trip() {
|
||||
let mut graph = EditGraph::default_chain();
|
||||
graph.masks_mut().push(corrected("m1", Join::Intersect));
|
||||
|
||||
let mut sidecar = Sidecar::new();
|
||||
sidecar.put(Version::from_graph("default", "Default", &graph));
|
||||
let text = sidecar.to_text();
|
||||
assert!(text.contains("join = intersect"), "{text}");
|
||||
|
||||
let restored = round_trip(&graph);
|
||||
let layer = &restored.masks().layers()[0];
|
||||
assert_eq!(layer.parts().len(), 2);
|
||||
assert_eq!(layer.parts()[1].join, Join::Intersect);
|
||||
assert_eq!(layer.parts()[0].join, Join::Union, "the base is untouched");
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-19a
|
||||
/// A part left out of the build comes back left out, and the file says so
|
||||
/// under a word that cannot be confused with the layer's own switch.
|
||||
|
||||
@@ -44,7 +44,7 @@
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
use dr_pipeline::Preset;
|
||||
use dr_pipeline::{Preset, Reach};
|
||||
use quick_xml::events::Event;
|
||||
use quick_xml::XmlVersion;
|
||||
|
||||
@@ -300,7 +300,11 @@ pub fn read_xmp(text: &str) -> Result<Import, ImportError> {
|
||||
|
||||
Ok(Import {
|
||||
name,
|
||||
preset: Preset::from_params(params),
|
||||
// A look, because that is what a Lightroom preset is: it changes the
|
||||
// settings it was saved with and leaves every other one where the
|
||||
// photograph had it. Applied as a whole edit instead, a preset
|
||||
// holding only a grade would reset the exposure it was put on top of.
|
||||
preset: Preset::from_params(params).with_reach(Reach::Named),
|
||||
skipped,
|
||||
})
|
||||
}
|
||||
@@ -472,7 +476,10 @@ mod tests {
|
||||
// second kind of thing with a second apply path.
|
||||
let import = read_xmp(ATTRIBUTE_FORM).unwrap();
|
||||
let mut graph = EditGraph::default_chain();
|
||||
import.preset.apply(&mut graph, Scope::adjustments());
|
||||
import
|
||||
.preset
|
||||
.apply(&mut graph, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(
|
||||
graph.param(
|
||||
dr_pipeline::ops::exposure::ID,
|
||||
@@ -482,6 +489,26 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// TRACES: FR-DEV-6
|
||||
/// Lightroom's own rule: a preset changes what it was saved with and
|
||||
/// nothing else, so a grade lands on top of the photograph's correction.
|
||||
#[test]
|
||||
fn an_imported_preset_leaves_what_it_does_not_set_alone() {
|
||||
use dr_pipeline::ops::dehaze;
|
||||
|
||||
let import = read_xmp(ATTRIBUTE_FORM).unwrap();
|
||||
assert_eq!(import.preset.reach(), Reach::Named);
|
||||
assert!(value(&import, "dehaze", "amount").is_none());
|
||||
|
||||
let mut graph = EditGraph::default_chain();
|
||||
graph.set_param(dehaze::ID, dehaze::AMOUNT, 30.0);
|
||||
import
|
||||
.preset
|
||||
.apply(&mut graph, Scope::adjustments())
|
||||
.expect_no_film();
|
||||
assert_eq!(graph.param(dehaze::ID, dehaze::AMOUNT), Some(30.0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_value_that_is_not_a_number_costs_that_setting_and_not_the_file() {
|
||||
let text =
|
||||
|
||||
@@ -22,6 +22,11 @@ pub struct LoginFlow {
|
||||
pub login_url: String,
|
||||
#[serde(rename = "poll")]
|
||||
pub poll: PollInfo,
|
||||
/// The server the flow was started against — the address the user typed,
|
||||
/// already normalised. Not part of the response: [`begin`] fills it in so
|
||||
/// [`poll`] can put it in the credentials instead of the server's answer.
|
||||
#[serde(skip)]
|
||||
pub server: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
@@ -66,9 +71,57 @@ pub async fn begin(
|
||||
});
|
||||
}
|
||||
|
||||
resp.json::<LoginFlow>()
|
||||
let mut flow = resp
|
||||
.json::<LoginFlow>()
|
||||
.await
|
||||
.map_err(|e| RemoteError::Protocol(e.to_string()))
|
||||
.map_err(|e| RemoteError::Protocol(e.to_string()))?;
|
||||
|
||||
// Both URLs are the server's to choose, and neither may be trusted as
|
||||
// sent. The login URL is handed to the operating system to open, where a
|
||||
// `file:` or UNC path is a program launch rather than a web page; the poll
|
||||
// endpoint is where the app password comes back from.
|
||||
flow.login_url = upgraded("login URL", &flow.login_url)?;
|
||||
flow.poll.endpoint = upgraded("poll endpoint", &flow.poll.endpoint)?;
|
||||
flow.server = server.trim_end_matches('/').to_string();
|
||||
Ok(flow)
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
/// A URL the server sent, upgraded to HTTPS, or refused.
|
||||
///
|
||||
/// `http` is upgraded rather than refused: a Nextcloud behind a TLS-terminating
|
||||
/// proxy without `overwriteprotocol` builds every absolute URL it returns with
|
||||
/// `http`, and the same path over `https` is the one that works. Any other
|
||||
/// scheme is refused, because the only thing it could be for is reaching
|
||||
/// something that is not this server.
|
||||
///
|
||||
/// The host is not checked. A server reached by its LAN address may answer with
|
||||
/// its public name, and nothing here is safer for refusing that: the account
|
||||
/// is stored under the address the user typed (see [`poll`]), not under
|
||||
/// anything the server said.
|
||||
pub(crate) fn upgraded(what: &str, url: &str) -> Result<String, RemoteError> {
|
||||
let mut parsed = url::Url::parse(url).map_err(|e| {
|
||||
RemoteError::Protocol(format!(
|
||||
"the server sent a {what} that is not a URL ({e}): {url}"
|
||||
))
|
||||
})?;
|
||||
match parsed.scheme() {
|
||||
"https" => {}
|
||||
"http" => parsed.set_scheme("https").map_err(|()| {
|
||||
RemoteError::Protocol(format!("{what} cannot be upgraded to https: {url}"))
|
||||
})?,
|
||||
other => {
|
||||
return Err(RemoteError::Protocol(format!(
|
||||
"the server sent a {what} using {other}:, and only https is accepted: {url}"
|
||||
)))
|
||||
}
|
||||
}
|
||||
if parsed.host_str().is_none_or(str::is_empty) {
|
||||
return Err(RemoteError::Protocol(format!(
|
||||
"the server sent a {what} with no host: {url}"
|
||||
)));
|
||||
}
|
||||
Ok(parsed.into())
|
||||
}
|
||||
|
||||
/// Poll until the user finishes authenticating in the browser.
|
||||
@@ -98,10 +151,11 @@ pub async fn poll(
|
||||
|
||||
match resp.status().as_u16() {
|
||||
200 => {
|
||||
return resp
|
||||
let creds = resp
|
||||
.json::<AppCredentials>()
|
||||
.await
|
||||
.map_err(|e| RemoteError::Protocol(e.to_string()))
|
||||
.map_err(|e| RemoteError::Protocol(e.to_string()))?;
|
||||
return Ok(under_typed_server(creds, &flow.server));
|
||||
}
|
||||
// Still waiting for the user.
|
||||
404 => tokio::time::sleep(POLL_INTERVAL).await,
|
||||
@@ -117,6 +171,26 @@ pub async fn poll(
|
||||
Err(RemoteError::AuthFailed)
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
/// Credentials filed under the address the user typed, not the one the server
|
||||
/// reports.
|
||||
///
|
||||
/// That report is the server's idea of its own URL, and behind a proxy
|
||||
/// without `overwriteprotocol` it says `http://` — which, stored, would send
|
||||
/// the app password in the clear on every request from then on. The typed
|
||||
/// address has just carried the whole flow, so it is known to reach the
|
||||
/// server.
|
||||
fn under_typed_server(mut creds: AppCredentials, server: &str) -> AppCredentials {
|
||||
if creds.server.trim_end_matches('/') != server {
|
||||
log::info!(
|
||||
"server reports itself as {}; keeping {server}",
|
||||
creds.server
|
||||
);
|
||||
}
|
||||
creds.server = server.to_string();
|
||||
creds
|
||||
}
|
||||
|
||||
/// Percent-encode a form value.
|
||||
fn urlencode(s: &str) -> String {
|
||||
s.bytes()
|
||||
@@ -167,6 +241,49 @@ mod tests {
|
||||
assert!(flow.login_url.contains("/login/v2/flow/"));
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn server_urls_are_upgraded_to_https_or_refused() {
|
||||
assert_eq!(
|
||||
upgraded("login URL", "http://cloud.example/login/v2/flow/xyz").unwrap(),
|
||||
"https://cloud.example/login/v2/flow/xyz"
|
||||
);
|
||||
// A non-default port stays, and only the scheme changes.
|
||||
assert_eq!(
|
||||
upgraded("poll endpoint", "http://cloud.example:8443/login/v2/poll").unwrap(),
|
||||
"https://cloud.example:8443/login/v2/poll"
|
||||
);
|
||||
assert_eq!(
|
||||
upgraded("login URL", "https://cloud.example/x").unwrap(),
|
||||
"https://cloud.example/x"
|
||||
);
|
||||
// What `rundll32 url.dll,FileProtocolHandler` would run, and what
|
||||
// `xdg-open` would hand to whatever claims it.
|
||||
for hostile in [
|
||||
"file:///C:/Windows/System32/calc.exe",
|
||||
"\\\\evil\\share\\x.exe",
|
||||
"C:\\x.exe",
|
||||
"javascript:alert(1)",
|
||||
"-v",
|
||||
"",
|
||||
] {
|
||||
assert!(upgraded("login URL", hostile).is_err(), "{hostile:?}");
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn credentials_keep_the_typed_server_not_the_reported_one() {
|
||||
let reported = AppCredentials {
|
||||
server: "http://cloud.example".into(),
|
||||
login_name: "duncan".into(),
|
||||
app_password: "secret-token".into(),
|
||||
};
|
||||
let c = under_typed_server(reported, "https://cloud.example");
|
||||
assert_eq!(c.server, "https://cloud.example");
|
||||
assert_eq!(c.app_password, "secret-token");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn form_values_are_encoded() {
|
||||
assert_eq!(urlencode("abc123"), "abc123");
|
||||
|
||||
@@ -376,6 +376,33 @@ impl RemoteBackend for NextcloudBackend {
|
||||
Ok(body)
|
||||
}
|
||||
|
||||
async fn get_reporting(
|
||||
&self,
|
||||
id: &RemoteId,
|
||||
progress: &(dyn Fn(u64, Option<u64>) + Send + Sync),
|
||||
) -> Result<Vec<u8>, RemoteError> {
|
||||
let url = self.url_for_id(id)?;
|
||||
let mut resp = self
|
||||
.client
|
||||
.get(&url)
|
||||
.basic_auth(&self.login, Some(&self.password))
|
||||
.send()
|
||||
.await
|
||||
.map_err(map_send_error)?;
|
||||
map_status(resp.status(), &url)?;
|
||||
|
||||
// Read chunk by chunk rather than with `bytes()`, which is the same
|
||||
// transfer with nothing to say until it ends.
|
||||
let declared = resp.content_length();
|
||||
let mut body = Vec::with_capacity(declared.unwrap_or(0) as usize);
|
||||
progress(0, declared);
|
||||
while let Some(chunk) = resp.chunk().await.map_err(map_send_error)? {
|
||||
body.extend_from_slice(&chunk);
|
||||
progress(body.len() as u64, declared);
|
||||
}
|
||||
Ok(body)
|
||||
}
|
||||
|
||||
async fn put(
|
||||
&self,
|
||||
path: &RemotePath,
|
||||
@@ -723,6 +750,15 @@ pub fn http_client(user_agent: &str) -> Result<reqwest::Client, RemoteError> {
|
||||
// for months. [`EXTRA_ROOTS`] carries those, and is additive — the
|
||||
// webpki-roots set is still installed alongside.
|
||||
.tls_certs_only(extra_roots())
|
||||
// TRACES: NFR-SEC-3
|
||||
// Refuse `http` here, below every URL this crate builds, rather than
|
||||
// trusting each place a URL comes from. `normalise_endpoint` upgrades
|
||||
// what the user types, but the login flow's poll endpoint, the account
|
||||
// an older build saved and a redirect all arrive from somewhere else,
|
||||
// and any of them naming `http://` would send the app password in
|
||||
// Basic auth in the clear. reqwest checks this before connecting and
|
||||
// again on every redirect, so a refused request never opens a socket.
|
||||
.https_only(true)
|
||||
// A request that hangs forever is indistinguishable from a worker that
|
||||
// died, and cost a long time to tell apart once. These turn that into
|
||||
// an error the UI can show.
|
||||
@@ -780,8 +816,16 @@ fn map_send_error(e: reqwest::Error) -> RemoteError {
|
||||
let mut detail = e.to_string();
|
||||
let mut src: Option<&dyn std::error::Error> = std::error::Error::source(&e);
|
||||
let mut tls = false;
|
||||
// A URL the client refused to send — `http` under `https_only`, on the
|
||||
// first request or a redirect. Nothing left the process, so this is the
|
||||
// account's configuration, not the network: reported as `Network` it
|
||||
// would put the app into offline mode over a connection that is fine.
|
||||
let mut refused = e.is_builder();
|
||||
while let Some(s) = src {
|
||||
let text = s.to_string();
|
||||
if text.contains("URL scheme is not allowed") {
|
||||
refused = true;
|
||||
}
|
||||
// rustls surfaces every verification failure through this wording:
|
||||
// UnknownIssuer, Expired, NotValidForName, BadSignature.
|
||||
if text.contains("invalid peer certificate")
|
||||
@@ -795,7 +839,9 @@ fn map_send_error(e: reqwest::Error) -> RemoteError {
|
||||
src = std::error::Error::source(s);
|
||||
}
|
||||
|
||||
if tls {
|
||||
if refused {
|
||||
RemoteError::Configuration(detail)
|
||||
} else if tls {
|
||||
RemoteError::Tls(detail)
|
||||
} else {
|
||||
RemoteError::Network(detail)
|
||||
@@ -1013,6 +1059,35 @@ mod tests {
|
||||
assert!(c.is_ok(), "client must build without a backend");
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[tokio::test]
|
||||
async fn plain_http_is_refused_before_a_connection_opens() {
|
||||
// A listener that would take the connection if one were made. The
|
||||
// app password travels in a header, so a request that reached the
|
||||
// socket has already leaked it; failing on the response is too late.
|
||||
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
listener.set_nonblocking(true).unwrap();
|
||||
let url = format!("http://{}/remote.php/dav/", listener.local_addr().unwrap());
|
||||
|
||||
let err = http_client("test")
|
||||
.unwrap()
|
||||
.get(&url)
|
||||
.basic_auth("duncan", Some("app-password"))
|
||||
.send()
|
||||
.await
|
||||
.expect_err("http must be refused");
|
||||
|
||||
assert!(
|
||||
matches!(map_send_error(err), RemoteError::Configuration(_)),
|
||||
"a refused scheme is configuration, not the network"
|
||||
);
|
||||
assert_eq!(
|
||||
listener.accept().err().map(|e| e.kind()),
|
||||
Some(std::io::ErrorKind::WouldBlock),
|
||||
"nothing may have connected"
|
||||
);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn delta_is_unsupported_and_says_why() {
|
||||
// Verified absent in the server; the engine must fall back rather
|
||||
|
||||
@@ -96,6 +96,18 @@ impl BackendProvider for NextcloudProvider {
|
||||
}
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
/// An `http://` account written before sign-in kept the address the user
|
||||
/// typed: it was stored as the server reported itself, and behind a proxy
|
||||
/// without `overwriteprotocol` that is `http`. The client refuses to send
|
||||
/// to it now, so it is upgraded here rather than left to fail. The
|
||||
/// namespace ignores the scheme, so the catalog stays where it is.
|
||||
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
|
||||
stored
|
||||
.strip_prefix("http://")
|
||||
.map(|rest| format!("https://{rest}"))
|
||||
}
|
||||
|
||||
fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError> {
|
||||
let creds = Self::credentials(conn)?;
|
||||
Ok(Box::new(NextcloudBackend::new(
|
||||
@@ -132,6 +144,18 @@ mod tests {
|
||||
assert!(p.normalise_endpoint(" ").is_err());
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn a_stored_http_endpoint_is_upgraded_and_nothing_else_is_touched() {
|
||||
let p = NextcloudProvider;
|
||||
assert_eq!(
|
||||
p.upgrade_endpoint("http://cloud.example/nextcloud")
|
||||
.as_deref(),
|
||||
Some("https://cloud.example/nextcloud")
|
||||
);
|
||||
assert_eq!(p.upgrade_endpoint("https://cloud.example"), None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_account_keeps_the_dav_user_id_apart_from_the_login() {
|
||||
// A login can be an email address while the user id is something
|
||||
|
||||
+223
-1
@@ -29,7 +29,7 @@ use dr_plat::{SecretError, SecretRef, SecretStore};
|
||||
use dr_types::{Format, FormatFilter};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::RemoteError;
|
||||
use crate::{BackendRegistry, RemoteError};
|
||||
|
||||
/// The connector every account had before there was a choice.
|
||||
///
|
||||
@@ -510,6 +510,95 @@ impl AccountStore {
|
||||
written
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
/// Rewrite the endpoints an older build stored in a form this one would
|
||||
/// not, asking each account's connector
|
||||
/// ([`upgrade_endpoint`](crate::BackendProvider::upgrade_endpoint)).
|
||||
///
|
||||
/// Per account and best effort: one that cannot be moved — its keyring
|
||||
/// locked, say — is logged and left as it was, and is tried again on the
|
||||
/// next launch, rather than stopping the others or the launch. Returns
|
||||
/// the accounts it rewrote.
|
||||
pub fn upgrade_endpoints(&self, registry: &BackendRegistry) -> Vec<Account> {
|
||||
let mut upgraded = Vec::new();
|
||||
for account in self.list() {
|
||||
let Ok(provider) = registry.for_account(&account) else {
|
||||
continue;
|
||||
};
|
||||
let Some(endpoint) = provider.upgrade_endpoint(&account.endpoint) else {
|
||||
continue;
|
||||
};
|
||||
if endpoint == account.endpoint {
|
||||
continue;
|
||||
}
|
||||
match self.move_endpoint(&account, &endpoint) {
|
||||
Ok(moved) => {
|
||||
log::info!("account {} moved to {endpoint}", account.describe());
|
||||
upgraded.push(moved);
|
||||
}
|
||||
Err(e) => log::warn!(
|
||||
"account {} could not be moved to {endpoint}: {e}",
|
||||
account.describe()
|
||||
),
|
||||
}
|
||||
}
|
||||
upgraded
|
||||
}
|
||||
|
||||
/// Move an account to a new endpoint, taking its credential with it.
|
||||
///
|
||||
/// The endpoint is half of two keys, and both have to be dealt with. The
|
||||
/// credential is filed under it ([`Account::secret_ref`]), so rewriting
|
||||
/// the record alone would strand the app password under the old key and
|
||||
/// sign the user out. And it feeds [`Account::namespace`], so a rewrite
|
||||
/// that changed the namespace would abandon the catalog and everything
|
||||
/// beside it; that is refused outright rather than left to the caller.
|
||||
///
|
||||
/// Ordered so an interruption at any step leaves something that works:
|
||||
/// the credential is copied before the record names the new key, and the
|
||||
/// old copy is deleted only once nothing names the old one.
|
||||
fn move_endpoint(&self, account: &Account, endpoint: &str) -> Result<Account, AccountError> {
|
||||
let mut moved = account.clone();
|
||||
moved.endpoint = endpoint.to_string();
|
||||
if moved.namespace() != account.namespace() {
|
||||
return Err(AccountError::WouldMoveData {
|
||||
from: account.namespace(),
|
||||
to: moved.namespace(),
|
||||
});
|
||||
}
|
||||
|
||||
let mut config = self.read_config();
|
||||
// Already there — the user signed in again at the new address. That
|
||||
// record and its credential are the newer, so the old one just goes.
|
||||
let duplicate = config.sessions.iter().any(|a| a.is_same_as(&moved));
|
||||
|
||||
let old_ref = account.secret_ref();
|
||||
let secret = match self.secrets.retrieve(&old_ref) {
|
||||
Ok(s) => Some(s),
|
||||
Err(SecretError::NotFound) => None,
|
||||
Err(e) => return Err(e.into()),
|
||||
};
|
||||
if let (Some(s), false) = (&secret, duplicate) {
|
||||
self.secrets.store(&moved.secret_ref(), s)?;
|
||||
}
|
||||
|
||||
if duplicate {
|
||||
config.sessions.retain(|a| !a.is_same_as(account));
|
||||
} else {
|
||||
// In place, not removed and pushed: the last record is the one
|
||||
// the next launch resumes.
|
||||
for a in config.sessions.iter_mut().filter(|a| a.is_same_as(account)) {
|
||||
*a = moved.clone();
|
||||
}
|
||||
}
|
||||
self.write_config(&config)?;
|
||||
|
||||
if secret.is_some() {
|
||||
self.secrets.delete(&old_ref)?;
|
||||
}
|
||||
Ok(moved)
|
||||
}
|
||||
|
||||
fn read_config(&self) -> ConfigFile {
|
||||
std::fs::read_to_string(&self.config_path)
|
||||
.ok()
|
||||
@@ -545,6 +634,11 @@ pub enum AccountError {
|
||||
|
||||
#[error(transparent)]
|
||||
Remote(#[from] RemoteError),
|
||||
|
||||
/// A change that would give an account a different local data directory,
|
||||
/// leaving its catalog and caches behind under the old one.
|
||||
#[error("moving the account would leave its local data behind ({from} → {to})")]
|
||||
WouldMoveData { from: String, to: String },
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -574,6 +668,134 @@ mod tests {
|
||||
d
|
||||
}
|
||||
|
||||
/// A connector that upgrades `http://` endpoints the way Nextcloud's does,
|
||||
/// and every other one not at all.
|
||||
struct Upgrading(&'static str);
|
||||
impl crate::BackendProvider for Upgrading {
|
||||
fn id(&self) -> &'static str {
|
||||
self.0
|
||||
}
|
||||
fn display_name(&self) -> &'static str {
|
||||
"U"
|
||||
}
|
||||
fn endpoint_label(&self) -> &'static str {
|
||||
"Server"
|
||||
}
|
||||
fn endpoint_placeholder(&self) -> &'static str {
|
||||
""
|
||||
}
|
||||
fn sign_in(&self) -> crate::SignIn {
|
||||
crate::SignIn::Browser
|
||||
}
|
||||
fn normalise_endpoint(&self, i: &str) -> Result<String, String> {
|
||||
Ok(i.into())
|
||||
}
|
||||
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
|
||||
stored
|
||||
.strip_prefix("http://")
|
||||
.map(|rest| format!("https://{rest}"))
|
||||
}
|
||||
fn connect(&self, _: &Connection) -> Result<Box<dyn crate::RemoteBackend>, RemoteError> {
|
||||
Err(RemoteError::Unsupported("stub"))
|
||||
}
|
||||
}
|
||||
|
||||
fn upgrading(id: &'static str) -> BackendRegistry {
|
||||
let mut r = BackendRegistry::new();
|
||||
r.register(std::sync::Arc::new(Upgrading(id)));
|
||||
r
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn an_http_account_is_upgraded_and_keeps_its_credential_and_data() {
|
||||
let dir = tmpdir("upgrade");
|
||||
let store = store_in(&dir);
|
||||
let old =
|
||||
Account::new(LEGACY_BACKEND, "http://cloud.example").with_login("duncan", "duncan");
|
||||
store.save(&folder(), None).unwrap();
|
||||
store
|
||||
.save(&old, Some(&Secret::new("secret-token")))
|
||||
.unwrap();
|
||||
|
||||
let moved = store.upgrade_endpoints(&upgrading(LEGACY_BACKEND));
|
||||
assert_eq!(moved.len(), 1);
|
||||
|
||||
let now = store.current().expect("still the account resumed");
|
||||
assert_eq!(now.endpoint, "https://cloud.example");
|
||||
assert_eq!(
|
||||
now.namespace(),
|
||||
old.namespace(),
|
||||
"the catalog directory must not move"
|
||||
);
|
||||
assert_eq!(
|
||||
store
|
||||
.connection(&now, true)
|
||||
.unwrap()
|
||||
.require_secret()
|
||||
.unwrap()
|
||||
.expose(),
|
||||
"secret-token",
|
||||
"the credential moves with the account"
|
||||
);
|
||||
assert!(
|
||||
store.connection(&old, true).is_err(),
|
||||
"nothing is left under the old key"
|
||||
);
|
||||
assert_eq!(store.list().len(), 2, "the folder library is untouched");
|
||||
|
||||
// And a second launch has nothing to do.
|
||||
assert!(store
|
||||
.upgrade_endpoints(&upgrading(LEGACY_BACKEND))
|
||||
.is_empty());
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn a_move_that_would_change_the_data_directory_is_refused() {
|
||||
// A scheme change keeps the namespace, which is what makes the upgrade
|
||||
// safe; a host change does not, and would give the library a new,
|
||||
// empty data directory. The account is left exactly as it was.
|
||||
let dir = tmpdir("upgrade-refused");
|
||||
let store = store_in(&dir);
|
||||
let old = nextcloud();
|
||||
store
|
||||
.save(&old, Some(&Secret::new("secret-token")))
|
||||
.unwrap();
|
||||
|
||||
assert!(matches!(
|
||||
store.move_endpoint(&old, "https://elsewhere.example"),
|
||||
Err(AccountError::WouldMoveData { .. })
|
||||
));
|
||||
assert_eq!(store.current().unwrap().endpoint, old.endpoint);
|
||||
assert!(store.connection(&old, true).is_ok());
|
||||
}
|
||||
|
||||
/// TRACES: NFR-SEC-3
|
||||
#[test]
|
||||
fn an_upgrade_onto_an_existing_sign_in_keeps_the_newer_one() {
|
||||
let dir = tmpdir("upgrade-duplicate");
|
||||
let store = store_in(&dir);
|
||||
let old =
|
||||
Account::new(LEGACY_BACKEND, "http://cloud.example").with_login("duncan", "duncan");
|
||||
let new =
|
||||
Account::new(LEGACY_BACKEND, "https://cloud.example").with_login("duncan", "duncan");
|
||||
store.save(&old, Some(&Secret::new("stale"))).unwrap();
|
||||
store.save(&new, Some(&Secret::new("fresh"))).unwrap();
|
||||
|
||||
store.upgrade_endpoints(&upgrading(LEGACY_BACKEND));
|
||||
assert_eq!(store.list().len(), 1);
|
||||
assert_eq!(
|
||||
store
|
||||
.connection(&new, true)
|
||||
.unwrap()
|
||||
.require_secret()
|
||||
.unwrap()
|
||||
.expose(),
|
||||
"fresh"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_saved_account_survives_reopening() {
|
||||
let dir = tmpdir("survives");
|
||||
|
||||
@@ -98,6 +98,26 @@ pub trait RemoteBackend: Send + Sync {
|
||||
/// either way; [`Capabilities::range_reads`] says whether it was cheap.
|
||||
async fn get(&self, id: &RemoteId, range: Option<Range<u64>>) -> Result<Vec<u8>, RemoteError>;
|
||||
|
||||
/// Fetch a whole object, saying how much of it has arrived as it arrives.
|
||||
///
|
||||
/// `progress` is called with the bytes received so far and the length the
|
||||
/// server declared, if it declared one. For the one transfer a person
|
||||
/// watches: an original opened in develop is tens of megabytes, and a
|
||||
/// view that can only say "downloading" for that long reads as stuck.
|
||||
///
|
||||
/// The default fetches with [`get`](Self::get) and reports once, at the
|
||||
/// end — right for a backend whose `get` is a local read, where there is
|
||||
/// no wait to report on.
|
||||
async fn get_reporting(
|
||||
&self,
|
||||
id: &RemoteId,
|
||||
progress: &(dyn Fn(u64, Option<u64>) + Send + Sync),
|
||||
) -> Result<Vec<u8>, RemoteError> {
|
||||
let body = self.get(id, None).await?;
|
||||
progress(body.len() as u64, Some(body.len() as u64));
|
||||
Ok(body)
|
||||
}
|
||||
|
||||
/// Upload, optionally guarded by a precondition.
|
||||
///
|
||||
/// Backends handle chunking internally based on body size — chunked
|
||||
|
||||
@@ -83,6 +83,20 @@ pub trait BackendProvider: Send + Sync {
|
||||
/// wrong rather than naming a type.
|
||||
fn normalise_endpoint(&self, input: &str) -> Result<String, String>;
|
||||
|
||||
/// The form a *stored* endpoint should take now, where an older build
|
||||
/// wrote one this build would not.
|
||||
///
|
||||
/// Not [`normalise_endpoint`](Self::normalise_endpoint) run again: that
|
||||
/// judges what a person typed, and may touch the world to do it — a folder
|
||||
/// is canonicalised and must exist — so rerunning it on every launch would
|
||||
/// fail a library whose disk is unplugged, or rename one whose path now
|
||||
/// resolves differently. This is a pure rewrite of the string, and `None`
|
||||
/// means leave it alone, which is the answer for almost every connector.
|
||||
fn upgrade_endpoint(&self, stored: &str) -> Option<String> {
|
||||
let _ = stored;
|
||||
None
|
||||
}
|
||||
|
||||
/// Build an account from a normalised endpoint alone.
|
||||
///
|
||||
/// Only meaningful for [`SignIn::EndpointOnly`]; a browser flow produces
|
||||
|
||||
@@ -115,6 +115,10 @@ pub enum PlaceScope {
|
||||
#[serde(default)]
|
||||
pub struct StoredFilter {
|
||||
pub min_rating: u8,
|
||||
/// TRACES: FR-UI-5
|
||||
/// The top of a star range. A record from before ranges existed has none,
|
||||
/// which reads as no ceiling — what it meant when it was written.
|
||||
pub max_rating: Option<u8>,
|
||||
pub unjudged: bool,
|
||||
pub flag: Option<FlagState>,
|
||||
pub local_only: bool,
|
||||
@@ -131,6 +135,10 @@ pub struct StoredFilter {
|
||||
/// Travels: it is a narrowing like `local_only`, and a record without it
|
||||
/// — from a build before it existed — reads as off.
|
||||
pub eyes_open: bool,
|
||||
/// TRACES: FR-CAT-6
|
||||
/// The colour label the grid was narrowed to. A record without it reads
|
||||
/// as none, as every term added after the first does.
|
||||
pub label: Option<crate::ColourLabel>,
|
||||
}
|
||||
|
||||
/// Where the photographer was, at the moment they were there.
|
||||
|
||||
@@ -727,6 +727,18 @@ pub struct ExportSettings {
|
||||
/// [`Self::destination`], where empty means "ask each time" because there
|
||||
/// is no sensible folder to assume on a filesystem.
|
||||
pub remote_destination: String,
|
||||
|
||||
/// TRACES: FR-EXP-10
|
||||
/// The album exports go to, by its catalog uuid. Empty until one is
|
||||
/// chosen, and an export with none is refused and says so.
|
||||
///
|
||||
/// This replaced [`Self::target`] and the two destination fields as what
|
||||
/// the export sheet chooses: a destination is now a named album in the
|
||||
/// catalog, which records what was exported into it. The three older
|
||||
/// fields stay so a settings file written before albums still reads, and
|
||||
/// so the first album can be made from the folder they named; the batch
|
||||
/// fills them from the album when it runs.
|
||||
pub album: String,
|
||||
}
|
||||
|
||||
impl Default for ExportSettings {
|
||||
@@ -748,6 +760,7 @@ impl Default for ExportSettings {
|
||||
target: ExportTarget::default(),
|
||||
destination: String::new(),
|
||||
remote_destination: String::new(),
|
||||
album: String::new(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -332,6 +332,29 @@ else
|
||||
echo " assets: no models found (face indexing and the scene tab will be off on the device)"
|
||||
fi
|
||||
|
||||
# The manual: the rendered page and its pictures, read in place by
|
||||
# ManualActivity's WebView as file:///android_asset/manual/index.html. Not
|
||||
# unpacked like the models: a WebView reads an asset straight out of the APK,
|
||||
# and relative links to media/ resolve inside the same asset tree, so the page
|
||||
# costs no first-launch copy and no second copy on /data.
|
||||
#
|
||||
# About 27 MB, stored below like the models — a GIF or PNG is already
|
||||
# compressed, and deflating it again buys nothing. The pictures are LFS
|
||||
# objects, and unlike a missing model a pointer would not fail loudly: it
|
||||
# ships as a manual full of broken images. So it stops the build here.
|
||||
rm -rf "${OUT}/staging/assets/manual"
|
||||
mkdir -p "${OUT}/staging/assets/manual/media"
|
||||
cp "${REPO}/docs/manual/index.html" "${OUT}/staging/assets/manual/"
|
||||
for f in "${REPO}/docs/manual/media"/*; do
|
||||
if head -c 40 "${f}" | grep -q '^version https://git-lfs'; then
|
||||
echo "error: $(basename "${f}") is an LFS pointer, not a picture." >&2
|
||||
echo " run: git lfs pull --include='docs/manual/media/**'" >&2
|
||||
exit 1
|
||||
fi
|
||||
cp "${f}" "${OUT}/staging/assets/manual/media/"
|
||||
done
|
||||
echo " manual: index.html and $(ls "${OUT}/staging/assets/manual/media" | wc -l) picture(s), $(du -sh "${OUT}/staging/assets/manual" | cut -f1)"
|
||||
|
||||
# -0 "" stores the .so without compression so Android can mmap it directly
|
||||
# (extractNativeLibs=false territory); for a 37 MB library that also keeps
|
||||
# install times sane.
|
||||
|
||||
@@ -69,6 +69,23 @@ for dir in face scene inpaint; do
|
||||
done
|
||||
echo "==> staged $(ls "${STAGE}/models" | wc -l) model file(s)"
|
||||
|
||||
# The manual: the rendered page and its pictures, beside the executable where
|
||||
# dr_ui::manual looks on Windows (dr_plat::system_data_dirs is the exe's own
|
||||
# directory there). The pictures are LFS objects; a pointer is ~130 bytes of
|
||||
# text that every browser draws as a broken image, so refuse it here rather
|
||||
# than ship a manual with no pictures in it.
|
||||
mkdir -p "${STAGE}/manual/media"
|
||||
cp "${REPO}/docs/manual/index.html" "${STAGE}/manual/"
|
||||
for f in "${REPO}/docs/manual/media"/*; do
|
||||
if head -c 40 "${f}" | grep -q '^version https://git-lfs'; then
|
||||
echo "error: $(basename "${f}") is an LFS pointer, not a picture." >&2
|
||||
echo " run: git lfs pull --include='docs/manual/media/**'" >&2
|
||||
exit 1
|
||||
fi
|
||||
cp "${f}" "${STAGE}/manual/media/"
|
||||
done
|
||||
echo "==> staged the manual and $(ls "${STAGE}/manual/media" | wc -l) picture(s)"
|
||||
|
||||
# One installer in the output directory, the one just built. The directory
|
||||
# is cached between CI runs, so after a version bump a glob over it would find
|
||||
# two and the smoke test would hand Wine both names as one path.
|
||||
|
||||
+9
-7
@@ -7,8 +7,8 @@ second.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| [The manual](manual/README.md) | Every feature, pictured from the application itself — opening a library, rating and filing, developing, local masks, repair, film, panoramas, export |
|
||||
| [How it is driven](gestures.md) | Every gesture and shortcut, by screen. Generated from the code, so it cannot describe one the application does not have |
|
||||
| [The manual](manual/README.md) | Every feature, pictured from the application itself — opening a library, rating, labelling and filing, duplicate originals, developing, local masks, repair, film, presets, panoramas, export and albums. The application carries it and opens it from Help and from Settings |
|
||||
| [How it is driven](gestures.md) | Every gesture and shortcut, by screen. Generated from the code, so it cannot describe one the application does not have. The same list is the in-app help sheet: `Help` or `F1` in the grid, `?` or `F1` in develop |
|
||||
|
||||
The [top-level README](../README.md) says what DarkRoom is, how to get it on
|
||||
each platform, and what is still missing.
|
||||
@@ -74,11 +74,13 @@ recollection.
|
||||
|
||||
## Conventions
|
||||
|
||||
Two files here are generated and must not be edited by hand:
|
||||
`gestures.md` and `dev/traceability.md`. Both come from
|
||||
`cargo run -p traceability` and the pre-commit hook keeps them in step with
|
||||
the tree. The manual's pictures are recorded by
|
||||
[`tools/manual`](../tools/manual/README.md) and live in LFS.
|
||||
Three files here are generated and must not be edited by hand:
|
||||
`gestures.md`, `dev/traceability.md` and `manual/index.html`, the page the
|
||||
packages install, rendered from `manual/README.md`. All three come from
|
||||
`cargo run -p traceability`, the pre-commit hook keeps them in step with the
|
||||
tree, and CI fails when one is not what the tree generates. The manual's
|
||||
pictures are recorded by [`tools/manual`](../tools/manual/README.md) and live
|
||||
in LFS.
|
||||
|
||||
A design document links to the requirements it satisfies and to the code
|
||||
that satisfies them. When the code moves, the link moves with it; a document
|
||||
|
||||
+50
-11
@@ -11,9 +11,10 @@ and records the decisions and constraints behind the design.
|
||||
## 1. Overview
|
||||
|
||||
DarkRoom is a Rust application with a Slint interface. On Linux it renders through wgpu to Vulkan.
|
||||
On Android it renders through Skia to OpenGL — not by preference but because wgpu's Vulkan
|
||||
swapchain cannot pre-rotate, which tears a portrait window on a landscape-mounted panel
|
||||
([technical-debt.md TD-1](technical-debt.md)). The compute passes are wgpu on both. The design is organised around four ideas, each of which the rest of this
|
||||
On Android it renders through Skia, also on wgpu's Vulkan swapchain, which a patched wgpu-hal and
|
||||
Skia renderer pre-rotate for a landscape-mounted panel ([technical-debt.md TD-1](technical-debt.md),
|
||||
[third_party/](../../third_party/README.md)). The compute passes are wgpu on both, on the same
|
||||
device the compositor draws with. The design is organised around four ideas, each of which the rest of this
|
||||
document elaborates:
|
||||
|
||||
1. **Pixels stay on the GPU.** From decode to display, image data never round-trips through the
|
||||
@@ -51,7 +52,7 @@ darkroom/
|
||||
│ ├── dr-types SourceRef, ImageId, VersionId, ParamValue — shared vocabulary
|
||||
│ ├── dr-catalog SQLite index, scan, query, metadata
|
||||
│ ├── dr-sidecar the authoritative edit store (§6.12)
|
||||
│ ├── dr-decode RawDecoder trait, rawler impl, embedded-preview extraction
|
||||
│ ├── dr-decode Decoder trait (§3.2), rawler impl, embedded-preview extraction
|
||||
│ ├── dr-pipeline Operation trait, descriptors, edit graph, registry
|
||||
│ ├── dr-gpu wgpu device, tile scheduler, WGSL shaders, mask rasteriser
|
||||
│ ├── dr-colour lcms2 bindings, camera profiles, working-space transforms
|
||||
@@ -131,6 +132,15 @@ Four separate entry points because the caller's needs differ sharply by phase. C
|
||||
`RawImage` carries CFA-pattern sensor data plus black/white levels and camera colour matrices — it
|
||||
is *not* demosaiced. Demosaic is a GPU pipeline stage (§5.2).
|
||||
|
||||
> **As built (FR-RAW-2, 0.15.0).** The trait is `dr_decode::Decoder`, and it takes bytes rather
|
||||
> than a reader: `header_bytes` says how much of a file metadata needs, `metadata` and
|
||||
> `orientation` read it, `locate_preview` says where the embedded preview sits so the caller's
|
||||
> storage can fetch that range, and `preview` and `decode` take the bytes fetched. The split this
|
||||
> section argues for survives; what moved is who reads the file, which is the storage layer
|
||||
> (§3.1's `read_range`), not the decoder. `dr_decode::Rawler` is the one implementation, and only
|
||||
> the places that start a job name `dr_decode::default()`; everything below them takes a
|
||||
> `&dyn Decoder`.
|
||||
|
||||
### 3.3 Operation and descriptors
|
||||
|
||||
The develop pipeline is a sequence of operations with a uniform interface. Polymorphism is by
|
||||
@@ -337,6 +347,8 @@ RawImage (sensor data, CPU)
|
||||
│ upload
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ hot/dead photosites │ repaired on the mosaic (FR-RAW-3)
|
||||
├─────────────────────┤
|
||||
│ black/white levels │ integer normalise
|
||||
├─────────────────────┤
|
||||
│ demosaic │ Bayer or Markesteijn (X-Trans, FR-RAW-5)
|
||||
@@ -369,8 +381,29 @@ RawImage (sensor data, CPU)
|
||||
|
||||
Working precision is f16 in a linear wide-gamut space, quantising once at the output transform.
|
||||
|
||||
**A hot or dead photosite is repaired before the demosaic, not after.** Past it, one photosite of
|
||||
nonsense is a coloured cross three pixels wide that no later stage can tell from detail. The pass
|
||||
(`shaders/hot_pixels.wgsl`, run by `Demosaicer::run` into a second buffer) replaces a photosite that
|
||||
stands apart from every same-colour photosite in its 5×5 window *and* from each of its eight
|
||||
immediate neighbours with the nearest value that neighbourhood vouches for. The second test is what
|
||||
keeps a star: real light arrives through a lens and lights a patch, so its neighbours are lit too. A
|
||||
6×6 sensor-anchored colour tile serves Bayer and X-Trans alike, every path that demosaics gets it,
|
||||
and there is no setting.
|
||||
|
||||
**A mask layer runs inside this chain, not after it.** Its settings are offsets to the global ones
|
||||
(`mask::offset_onto`), and each operation a layer touches is composed at the operation's own place:
|
||||
the global fragment and each layer's combined fragment read the same input, and the pixel moves by
|
||||
each layer's weighted difference, `c_g + Σ wᵢ(cᵢ − c_g)`. At full weight that is the combined
|
||||
setting exactly and at zero the global result exactly, so global contrast −30 under a layer at −20
|
||||
is contrast −50 where contrast runs, never −30 now and −20 again later — which is what the layer
|
||||
chain did before 0.18.1, and how the shadows of a night shot went magenta. A photograph with no
|
||||
layers composes to the same shader byte for byte. The film is the one exception, because it is a
|
||||
rendering rather than an adjustment and cross-fading two developments is not what a region of a
|
||||
pushed negative looks like: an operation that `blends_settings` has its uniforms averaged by mask
|
||||
weight instead, and runs once (FR-DEV-3f).
|
||||
|
||||
There is a second reduction that does not hang off the bottom of this chain. The raw histogram
|
||||
(FR-CULL-3) taps the demosaiced scene-linear texture directly — the box four rows from the top —
|
||||
(FR-CULL-3) taps the demosaiced scene-linear texture directly — the demosaic box's output —
|
||||
because what it measures is the file rather than the render. See §5.5.
|
||||
|
||||
### 5.3 Tiling and scheduling
|
||||
@@ -583,6 +616,13 @@ does not silently become uploadable by existing.
|
||||
The UI executor never blocks — this is the mechanism behind R4 and NFR-P9, which the requirements
|
||||
state as outcomes without saying how.
|
||||
|
||||
**In code:** `ui/dr-ui/src/executors.rs`. `Executor` names the five and states each count with its
|
||||
reason (`Executor::threads`); `executors::spawn(executor, role, f)` starts a worker named
|
||||
`<executor>:<role>` and marks it with its executor. `run` marks its own thread as the UI executor
|
||||
before the window exists, and `executors::assert_not_ui` — called by `net_runtime`'s `block_on` —
|
||||
fails a debug or test build that blocks there. The counts are not yet enforced: a job still gets a
|
||||
thread of its own, and pooling by executor is where NFR-ARCH-2's priority classes will live.
|
||||
|
||||
### 7.2 Cancellation
|
||||
|
||||
Cooperative, with tokens threaded through every long operation. Observed within 100 ms
|
||||
@@ -1125,12 +1165,11 @@ At 4K the shader finishes in 0.28 ms and then 7.15 ms is spent moving pixels thr
|
||||
26× overhead that scales with area, which is why an uncapped window resize falls off a cliff. The
|
||||
constraint is not a stylistic preference; it is the dominant cost in the frame.
|
||||
|
||||
**One exception, on Android only, and it is debt rather than a revision.** The develop view there
|
||||
reads the frame back rather than handing over a texture, because zero-copy requires Slint to draw
|
||||
with wgpu and wgpu's Android swapchain tears a portrait window. The reasoning, the measurements
|
||||
that forced it and what would remove it are in [technical-debt.md TD-1](technical-debt.md). The
|
||||
constraint above still governs every other path, including the desktop develop view and the export
|
||||
pipeline, and the Android exception is expected to be temporary.
|
||||
**No exceptions since 0.15.0.** Android read the develop frame back until then, because wgpu's
|
||||
Vulkan swapchain could not pre-rotate and a portrait window tore on the tablet. Two local patches
|
||||
removed the need ([technical-debt.md TD-1](technical-debt.md)), so the frame reaches the compositor
|
||||
as a texture on both platforms. The constraint governs every display path. The export pipeline
|
||||
reads pixels back by design, because a file is its output.
|
||||
|
||||
### 6.2 Tiling from day one
|
||||
|
||||
|
||||
@@ -58,8 +58,13 @@ Two of those rows carry a qualifier, and the qualifiers are the point.
|
||||
|
||||
**NFR-P1 — catalog open under 2 s.** The measured span is the four things the
|
||||
library view cannot paint without: `Catalog::open` (which connects, migrates and
|
||||
**backfills**, and the backfill is three passes over the images table on every
|
||||
open), `count`, the first 400-row `window`, and the monthly `timeline`. Tagged
|
||||
**backfills**, and the backfill is passes over the whole images table), `count`,
|
||||
the first 400-row `window`, and the monthly `timeline`. Since 0.17.0 the
|
||||
backfill runs on the first open of a catalog in a process and is skipped by
|
||||
later ones while nothing has changed ([catalog.md §2](catalog.md)), so
|
||||
`catalog_open_ms` includes it and `catalog_open_warm_ms`, the second open in
|
||||
the same process, does not — which is what a library reopened in the same
|
||||
session costs. Tagged
|
||||
`TRACES: NFR-P1` in [`tools/bench/src/catalog_open.rs`](../../tools/bench/src/catalog_open.rs),
|
||||
because a build that breaks it fails this gate.
|
||||
|
||||
|
||||
+118
-15
@@ -129,6 +129,37 @@ CREATE INDEX members_image ON collection_members(image_id);
|
||||
partial index over the non-NULL subset is both smaller and what FR-CAT-9's reconnection-by-hash
|
||||
and FR-CAT-11's duplicate detection actually query.
|
||||
|
||||
**Made on first use, not by a migration.** A new `user_version` makes every older build refuse this
|
||||
catalog's snapshot at sync (`sync::remote_is_mergeable` compares it and nothing else), and a tablet
|
||||
a release behind would stop merging collections, keywords and people for a feature it does not
|
||||
have. So what later releases added without needing old rows rewritten is created with `IF NOT
|
||||
EXISTS` where it is first used, and an older build that meets it ignores it:
|
||||
|
||||
- `dedup_probes` (FR-CAT-11a, §3.5);
|
||||
- the albums (FR-EXP-10, `core/dr-catalog/src/albums.rs`): `albums`, `album_exports` — one row per
|
||||
file written into an album, keyed on the file name, since two crops of one photograph are two
|
||||
files — and `album_folders`, this device's folder for each (§8.2);
|
||||
- `keywords_term_version (keyword, version_id)`, made by `keywords::list`, which counts each word's
|
||||
photographs on every selection change and without it read a `keywords` row per assignment to
|
||||
learn its version;
|
||||
- `faces_box (image_id, model_id, x, y, w, h)`, made by the merge's `match_faces`, which reads every
|
||||
local face's box and model and without it opened each ~8 KB `faces` row to do so.
|
||||
|
||||
A failure to make one of the indexes — a read-only or busy catalog — is logged and the query runs
|
||||
without it, as it did before.
|
||||
|
||||
**Opening does not repeat the backfill.** `schema::backfill` repairs what a write left owing — an
|
||||
image a scan inserted without its default version, the RAW/JPEG pairing, a merged version's uuid, a
|
||||
keyword assignment whose word has no term — and it used to run in every `Catalog::open`. Every
|
||||
worker opens its own connection, so a develop landing paid it five times (~80 ms of CPU on the
|
||||
reference catalog) to learn that nothing had changed. `core/dr-catalog/src/backfilled.rs` now
|
||||
records, per path and per process, a stamp read before each backfill: the schema version, the
|
||||
file's device and inode, and the newest image, version and keyword assignment by content as well as
|
||||
id — because none of those tables is `AUTOINCREMENT`, a freed newest id is handed out again, and the
|
||||
row that takes it is exactly one the backfill is owed. An open whose stamp matches skips it (~1 ms);
|
||||
the first open in a process, a migration, a pull and a replaced file always run it. Kept in memory
|
||||
rather than in the catalog, so nothing about it travels in the sync snapshot.
|
||||
|
||||
---
|
||||
|
||||
## 3. Incremental scan
|
||||
@@ -249,6 +280,13 @@ what you actually have.
|
||||
only when something needs it: import duplicate detection (FR-CAT-11), or reconnecting a moved source
|
||||
(FR-CAT-9). Never during a routine scan.
|
||||
|
||||
Consolidating the duplicates a library already holds (FR-CAT-11a, 0.16.0) does not wait for it. It
|
||||
proves a group the same by `content_hash` where every copy has one, and otherwise by a digest of
|
||||
each copy's first and last megabyte, read by range through the backend and kept in `dedup_probes`
|
||||
keyed on the size and mtime it was taken at, so a second review reads nothing.
|
||||
`dedup_probes` is created on first use rather than by a migration, because a schema bump would make
|
||||
an older build refuse this catalog's snapshot at sync (`core/dr-catalog/src/duplicates.rs`).
|
||||
|
||||
---
|
||||
|
||||
## 4. The library view
|
||||
@@ -413,10 +451,33 @@ pub enum JobKind {
|
||||
}
|
||||
```
|
||||
|
||||
**Thumbnails are not queued (decided 2026-09-26, #73).** `Thumbnail` is kept only so its number
|
||||
stays taken, and is listed in `JobKind::RETIRED`. Up to 0.16.0 every remote scan enqueued one per
|
||||
photograph, and nothing claimed the kind — no `JobHandler` was ever registered for it, on desktop or
|
||||
Android (the same `dr-ui`), and the catalog snapshot carries `jobs` but the merge never reads them
|
||||
(§8.2). The reference catalog held 23,582 such rows, about 1 MB with its indexes. Thumbnails are
|
||||
made another way, and by the right source of truth:
|
||||
|
||||
- the grid asks a worker for the cells it is drawing, which serves them from the thumbnail store or
|
||||
range-fetches the embedded preview (§7.2);
|
||||
- the thumbnail sweep's work list is *what the store does not hold* (`thumbnails_outstanding`).
|
||||
|
||||
The store is shared between devices (§7.3), so it is the only thing that knows another device already
|
||||
made a thumbnail; a per-device queue row cannot. A queue row would be a second, staler record of the
|
||||
same debt. Metadata is owed the same way — `metadata_state < 2` is the sweep's work list — so the local
|
||||
walk no longer enqueues `ExtractMetadata` either.
|
||||
|
||||
The rows already queued are dropped by `jobs::drop_retired`, from `runner::recover` at every catalog
|
||||
open, rather than by a migration: a schema bump would make a device still on an older build refuse the
|
||||
synced snapshot, and "every open" rather than "once" because an older build sharing the catalog
|
||||
queues them again on its next scan. With the rows gone it is one probe of the `(kind, subject_id)`
|
||||
index. `every_queued_kind_has_a_consumer` (dr-catalog) holds the rule: it reads the shipping sources
|
||||
and fails if any kind is enqueued that no handler or claim names.
|
||||
|
||||
### 6.2 Coalescing is the point
|
||||
|
||||
`UNIQUE(kind, subject_id)` on `jobs` means enqueueing is idempotent: an image touched five times
|
||||
during a scan has one thumbnail job, not five. Enqueue is
|
||||
has one job of a kind, not five. Enqueue is
|
||||
`INSERT … ON CONFLICT DO UPDATE SET priority = max(priority, excluded.priority)`, so a re-request at
|
||||
higher priority promotes the existing row rather than duplicating it.
|
||||
|
||||
@@ -436,7 +497,8 @@ priority governs the whole app:
|
||||
|
||||
Visible-cell work is enqueued by the grid as it scrolls, at `Interactive`. The effect is that a
|
||||
freshly scanned library fills in *where the user is looking* first, and grinds through the rest
|
||||
behind them.
|
||||
behind them. (As built, thumbnails and metadata get this ordering without the queue — the grid
|
||||
requests its visible cells directly and the sweeps take what is left; see §6.1.)
|
||||
|
||||
### 6.4 Durability and failure
|
||||
|
||||
@@ -448,8 +510,11 @@ Failures increment `attempts` and set `not_before` to an exponential backoff. Af
|
||||
count the job is marked failed and attached to its image as a typed error (NFR-ARCH-4) — one
|
||||
corrupt file does not stall the queue, and the user can see which files failed and why.
|
||||
|
||||
**A job runner never touches the UI executor**, and `Interactive` work runs on the decode pool with
|
||||
the I/O pool behind it (ARCH §7.1).
|
||||
**A job runner never touches the UI executor**, and `Interactive` work runs on the decode executor
|
||||
with the I/O executor behind it (ARCH §7.1). Those are named rather than pooled today: thumbnails
|
||||
on demand start as `decode:thumbs` and the sweeps as `decode:thumb-sweep` and `decode:metadata`,
|
||||
through `dr_ui::executors::spawn` and each on a thread of its own, and the thread counts §7.1 gives
|
||||
are a budget nothing yet enforces (NFR-ARCH-1).
|
||||
|
||||
---
|
||||
|
||||
@@ -655,21 +720,57 @@ mechanism.
|
||||
|
||||
### 8.2 What the file sync does and does not carry
|
||||
|
||||
Only **collections and their membership** merge. The rest of a catalog describes *local* state —
|
||||
folder mtimes, cache file paths, job rows, `tier_actual` — and importing another device's version
|
||||
of those would be actively wrong. The downloaded remote is read for its collections and discarded.
|
||||
Collections were the first thing merged, and the rules in §8.4 were written for them. What else
|
||||
merges reuses those rules or keys on the same identities, and each has no other home:
|
||||
|
||||
This is what keeps §6.12 substantially intact: nothing here makes the local database authoritative
|
||||
for anything a rebuild could not recover. The catalog is still deletable. What syncs is one table
|
||||
pair that had no other home.
|
||||
- **Collections and their membership** — by uuid and revision, membership as a set union.
|
||||
- **Keywords** — the vocabulary by the same verdict, the assignments as a union.
|
||||
- **People and identity judgements** — people by uuid and revision, and the confirmed and rejected
|
||||
face assignments matched to local faces (`merge::match_faces`): by box first, and — since 0.18.0,
|
||||
only on the photographs where a remote face is left over — by embedding, a pair being accepted
|
||||
at cosine ≥ 0.7 when each is the other's best by a lead of ≥ 0.2 (#77; [faces.md §18.2](faces.md)). After every merge,
|
||||
`dedup_people` folds people of one name whose confirmed faces agree, and a face held twice in
|
||||
one photograph, through the ordinary `merged_into` redirect, which older builds already honour
|
||||
(#78; [faces.md §19](faces.md)).
|
||||
- **Albums** (FR-EXP-10, 0.17.0) — by uuid and revision with tombstones, and what went into each as
|
||||
a set union keyed on the server's file id (a content hash on a folder library). An album's
|
||||
server folder is a column of its row and travels with it; a folder on *this device* is in
|
||||
`album_folders`, which the merge never reads and the upload snapshot drops (§8.3), because a
|
||||
path or a SAF grant on one device means nothing on another.
|
||||
- **Capture metadata** — the one exception inside `images`: a date, a camera, a lens and an ISO are
|
||||
facts about the file's bytes, so a row this device has not yet read takes them from a peer that
|
||||
has (`merge_metadata`), matched by `oc:fileid`.
|
||||
|
||||
The rest of a catalog describes *local* state — folder mtimes, cache file paths, job rows,
|
||||
`tier_actual` — and importing another device's version of those would be actively wrong; the
|
||||
downloaded remote is read for the tables above and discarded.
|
||||
|
||||
This is what keeps §6.12 substantially intact. The catalog is still deletable; what a rebuild from
|
||||
sidecars cannot recover — collections and albums, which nothing in the filesystem records — is what
|
||||
the sync exists to carry.
|
||||
|
||||
### 8.3 Two hazards the implementation must handle
|
||||
|
||||
**A WAL database is not one file.** Committed transactions can sit in `catalog.sqlite-wal` with the
|
||||
main file lagging, so copying `catalog.sqlite` alone uploads a torn snapshot — internally consistent
|
||||
as of some older point, silently missing everything since. Upload therefore runs a `TRUNCATE`
|
||||
checkpoint and then SQLite's backup API, which serialises against concurrent writers rather than
|
||||
racing them. It never copies the live file.
|
||||
as of some older point, silently missing everything since. The upload therefore never copies the
|
||||
live file. It builds the snapshot in an empty file. It attaches the catalog, creates each table
|
||||
from the catalog's own schema, and fills it with `INSERT … SELECT`, all inside one transaction.
|
||||
That transaction holds a single read snapshot of the catalog, so concurrent writers are serialised
|
||||
rather than raced, as the backup API did before. The file is then checked with `quick_check` before
|
||||
it goes anywhere.
|
||||
|
||||
**The face crops stay out of the upload.** A crop is a ~5 KB JPEG on each `faces` row. On a 19k-face
|
||||
library they are 96 MB of a 158 MB catalog. The face shards carry them to other devices, once each.
|
||||
The merge reads a remote face's box and model to match it to a local one — and, where the boxes cannot decide, its embedding — never its pixels. No
|
||||
device adopts a downloaded catalog as its own: a fresh device starts empty and takes faces, crops
|
||||
included, from the shards. So the snapshot's `crop` is NULL, and a merge never writes a local
|
||||
crop. They were first stripped (2026-08) by copying the whole file with the backup API, setting
|
||||
`crop` to NULL and `VACUUM`ing. That wrote the file about three times to upload 50 MB. Since #71
|
||||
the snapshot is built without them. Its header is kept as it was (`user_version`, page size and
|
||||
the WAL flag), so every earlier build merges it unchanged. NFR-R2 backups still use the backup API
|
||||
and keep the crops, because a backup is a file the user may have to live on. `album_folders` is
|
||||
dropped from the built file for the reason §8.2 gives.
|
||||
|
||||
**Integer primary keys are not identities.** Two devices each allocate `collections.id = 1` for
|
||||
different collections, so a row-level merge keyed on the integer id would collide them. Collections
|
||||
@@ -685,6 +786,7 @@ integer ids stay local and are never compared across catalogs.
|
||||
| Deletion | Tombstone (`deleted = 1`) carrying a revision | Without it, merging against a device that still holds the collection resurrects it. With a revision, deletion competes on equal footing with a rename |
|
||||
| An image the remote has and we do not | Skip the membership row | It joins on a later merge, once a scan has catalogued the file. Not an error |
|
||||
| A remote from a newer schema | Decline before attaching | Attempting it would fail mid-transaction rather than declining cleanly |
|
||||
| People with the same name | Folded after each merge when their faces agree ([faces.md §19](faces.md)) | Names typed separately on two devices otherwise stay two people for ever |
|
||||
|
||||
Merging is idempotent: running it twice reports no changes the second time. That property is tested,
|
||||
because a merge that oscillates would upload on every sync forever.
|
||||
@@ -711,8 +813,9 @@ because if the SQLite path proves troublesome, this is the fallback with a known
|
||||
membership needs to be *stable* — for manual ordering, or for a pinned set that must not shift
|
||||
under the user — it needs materialising with an invalidation rule. Deferred until there is a
|
||||
concrete need.
|
||||
- **Multi-root capture-time collisions.** FR-CAT-11 detects duplicates on import; the same image
|
||||
catalogued under two roots is a related but distinct case, not yet specified.
|
||||
- **Multi-root capture-time collisions.** FR-CAT-11 detects duplicates on import, and FR-CAT-11a
|
||||
consolidates the copies one root holds in several folders; the same image catalogued under two
|
||||
roots is a related but distinct case, not yet specified.
|
||||
- **Timeline granularity selection.** Which bucket size the UI picks for a given zoom is a UI
|
||||
concern, but the catalog should probably suggest one from the query's date span rather than have
|
||||
the UI guess.
|
||||
|
||||
@@ -22,7 +22,7 @@ and the reason is that some of the work is done and untagged.
|
||||
| FR-DSP-1 proxy rendering | **Done.** The develop view renders at viewport resolution, not source. |
|
||||
| FR-DSP-2 tiled computation | **Absent, and §2 now says it should stay that way.** Measured: the fused pass is inside the budget everywhere. See [frame-budget.md](frame-budget.md). |
|
||||
| FR-DSP-3 interactive latency | **Measured and asserted** for the fused path — `core/dr-gpu/tests/frame_budget.rs`. Missed by one operation, clarity, for the reason recorded as TD-4. |
|
||||
| FR-DSP-4 progressive refinement | **Absent**, and §4's condition did not fire. Every render is full quality and can afford to be. |
|
||||
| FR-DSP-4 progressive refinement | **Built** (0.15.0), although §4's condition did not fire on the fused path: a half-resolution draft while a gesture moves, one sharp frame 120 ms after it stops, the histogram dimmed while it lags, and the draft faded out over 150 ms — `ui/dr-ui/src/refine.rs`. See [frame-budget.md](frame-budget.md). |
|
||||
| FR-DSP-5 zoom and pan | **Done and tagged**, against tests that fail if the behaviour is removed — `core/dr-gpu/tests/zoom_resolution.rs`. `Framing::view` shrinks the sampled region while the render target keeps its size, so zooming *raises* the resolution the pipeline works at. That is FR-DSP-5's requirement, arrived at without tiles. |
|
||||
| FR-DSP-6 colour management | **Done.** Output space is a parameter of composition. |
|
||||
| FR-DSP-7 histogram and clipping | **Done**, GPU-side, no per-frame readback. |
|
||||
|
||||
+69
-48
@@ -18,7 +18,7 @@ permission to a package rather than after.
|
||||
| Platform | Channel | State | What it constrains |
|
||||
|---|---|---|---|
|
||||
| Linux | Arch source package — [`packaging/PKGBUILD`](../../packaging/PKGBUILD) | Built, in tree | Nothing. Full filesystem access, system Vulkan, system secret daemon |
|
||||
| Linux | Flatpak — [`packaging/flatpak/`](../../packaging/flatpak/) | Manifest in tree, **library selection does not work** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths |
|
||||
| Linux | Flatpak — [`packaging/flatpak/`](../../packaging/flatpak/) | Manifest in tree; folders chosen through the FileChooser portal since 0.17.0, **never built or run here** (§4) | Portals only. No `--filesystem=`, no host mount table, no typed paths |
|
||||
| Linux | AppImage | v1 channel, **recipe not yet written** (§5) | Oldest supported glibc, and no sandbox at all |
|
||||
| Android | F-Droid | v1 channel, not yet submitted | GPLv3-clean build, reproducible, no proprietary blobs |
|
||||
| Android | Play Store | **Not v1** (§6) | Would make ARCH §6.9 binding as policy rather than as engineering |
|
||||
@@ -45,10 +45,11 @@ somewhere:
|
||||
`project_license` is GPL-3.0-or-later; those differ on purpose — see the
|
||||
comment in the file.
|
||||
- **Vulkan is a requirement, not a preference.** The develop pipeline is
|
||||
compute shaders through wgpu, and NFR-R8 — how far a CPU fallback goes — is
|
||||
still open, so today there is nothing behind it. A package that installs onto
|
||||
a machine with no working ICD produces an application that starts and cannot
|
||||
develop.
|
||||
compute shaders through wgpu, and NFR-R8 was decided on 2026-09-19 against a
|
||||
CPU pipeline: without a device the library, the grid and the judgements work
|
||||
on embedded previews, and develop and export say they are unavailable. A
|
||||
package that installs onto a machine with no working ICD produces an
|
||||
application that starts and cannot develop.
|
||||
- **A Secret Service implementation, or an honest degraded mode.** FR-NC-2 is
|
||||
explicit that the absence of a secrets daemon is a stated degraded mode and
|
||||
never a silent fall back to plaintext. Packages express this as an optional
|
||||
@@ -59,6 +60,12 @@ somewhere:
|
||||
the Flatpak manifest check the file size and refuse, because the alternative
|
||||
is a package whose face indexing fails inside the graph loader on a user's
|
||||
machine rather than on the packager's.
|
||||
- **So are the manual's pictures.** Since 0.15.0 the Arch package, the Windows
|
||||
installer and the APK carry `docs/manual/index.html` and its `media/`, where
|
||||
`dr_ui::manual` looks for them (`/usr/share/darkroom/manual`, `manual\` beside
|
||||
`darkroom.exe`, the APK's `assets/manual`), and each refuses a pointer where a
|
||||
picture should be — shipped, a pointer is a manual of broken images that
|
||||
nothing reports. The Flatpak manifest does not install it yet.
|
||||
|
||||
---
|
||||
|
||||
@@ -71,7 +78,7 @@ The Arch package and an AppImage both hand the application the same
|
||||
unrestricted process the developer runs it in, so neither can discover that a
|
||||
design assumed unrestricted access. Flatpak takes that assumption away, and
|
||||
FR-PLAT-LIN-3 exists to make the discovery happen deliberately rather than in a
|
||||
bug report. §4 is what it discovered.
|
||||
bug report. §4 is what it discovered, and what has been done about it.
|
||||
|
||||
The same argument runs the other way on Android, where SAF has been the only
|
||||
option since before the first line was written (ARCH §6.9) and `SourceRef`
|
||||
@@ -113,31 +120,55 @@ advance:
|
||||
|
||||
---
|
||||
|
||||
## 4. What does not work: choosing a library
|
||||
## 4. Choosing a library: built, not yet proved in the sandbox
|
||||
|
||||
**FR-PLAT-LIN-3 is not satisfied today, and the manifest does not pretend
|
||||
otherwise.**
|
||||
**FR-PLAT-LIN-3 was not satisfied up to 0.16.0, and is still not shown to
|
||||
be.** What changed in 0.17.0 is the code; what has not changed is that no
|
||||
Flatpak has been built here, so nothing below has been observed inside one.
|
||||
|
||||
A folder library is chosen by typing an absolute path. `dr-sync-folder`'s
|
||||
provider declares `SignIn::EndpointOnly` with the placeholder
|
||||
`/home/you/Pictures`, and `normalise_endpoint` expands `~`, requires the path
|
||||
to be absolute, and checks it with `std::fs`. Nothing in the tree calls the
|
||||
FileChooser portal — there is no `ashpd`, no `rfd`, and no toolkit file dialog
|
||||
anywhere in `ui/`, `platform/` or `core/`.
|
||||
Up to 0.16.0 a folder library was chosen by typing an absolute path, and
|
||||
nothing in the tree called the FileChooser portal. Inside a sandbox with no
|
||||
`--filesystem=`, `$HOME` still resolves to the real home *path* but that
|
||||
directory holds only the application's own `.var/app/…` tree, so a typed
|
||||
`~/Pictures` failed the `exists()` check and the launch screen said so — a
|
||||
truthful message about a situation the user could not fix from inside the
|
||||
application.
|
||||
|
||||
Inside a sandbox with no `--filesystem=`, `$HOME` still resolves to the real
|
||||
home *path* but that directory holds only the application's own
|
||||
`.var/app/…` tree. So a typed `~/Pictures` fails the `exists()` check and the
|
||||
launch screen says `No folder at /home/you/Pictures.` — a truthful message
|
||||
about a situation the user cannot fix from inside the application.
|
||||
**Every folder the desktop asks for is now chosen in the platform's
|
||||
dialogue** (FR-EXP-6): the library folder on the launch screen (`Choose
|
||||
folder…`), an import's source and second copy, a folder or file of
|
||||
Lightroom presets, and an album's folder on this device.
|
||||
`ui/dr-ui/src/folder_dialog.rs` asks through `rfd` with its `xdg-portal`
|
||||
backend — `org.freedesktop.portal.FileChooser` over D-Bus, which Flatpak
|
||||
always permits without a `--talk-name` — and the common item dialogue on
|
||||
Windows. No path is typed anywhere on the desktop any more; Android keeps
|
||||
its fields (`Pickers.local-paths`), because SAF returns document trees rather
|
||||
than paths.
|
||||
|
||||
Import is blocked one step earlier. `dr_plat::volumes()` finds a camera card by
|
||||
reading `/proc/self/mountinfo` and the `removable` flag under `/sys`. A
|
||||
What the portal hands back inside a sandbox is a path under
|
||||
`/run/user/$UID/doc/` that the document portal has exported, and the chosen
|
||||
library goes through the same `normalise_endpoint` a typed one did, which
|
||||
checks it with `std::fs`. That is the route this section used to ask for —
|
||||
`rfd` drives `ashpd` underneath, the crate it named. Two things differ from
|
||||
the plan, and both are stated rather than smoothed over:
|
||||
|
||||
- **It is not behind a platform seam.** The plan put the chooser beside
|
||||
`LocalStorage::grant` in `dr-plat`, the one place a `Path` enters the
|
||||
application. It is in `ui/`, and the path it returns reaches the folder
|
||||
connector as a string, as a typed one did.
|
||||
- **Nothing has confirmed the sandbox half.** Whether the exported path
|
||||
still resolves after a restart — a library is remembered across launches,
|
||||
so it has to — and whether the export is writable, so a sidecar can be
|
||||
written beside a photograph, are the first two things a Flatpak build has
|
||||
to check.
|
||||
|
||||
Import is still blocked one step earlier. `dr_plat::volumes()` finds a camera
|
||||
card by reading `/proc/self/mountinfo` and the `removable` flag under `/sys`. A
|
||||
sandboxed process is in its own mount namespace, so the table it reads
|
||||
describes the sandbox; a card mounted at `/run/media/…` on the host is not in
|
||||
it. `volumes()` correctly returns an empty list, which the interface presents
|
||||
as "no card found" — right for the code, wrong for the user, who is looking at
|
||||
a card.
|
||||
as "no card found" — but the import page now offers `Browse…` beside that
|
||||
message, and the dialogue it opens is the portal's, which can reach the card.
|
||||
|
||||
### The permission that would hide this, and why it is not in the manifest
|
||||
|
||||
@@ -145,41 +176,31 @@ a card.
|
||||
names as the alternative to portals. Granting it would mean the sandboxed build
|
||||
never exercises the sandbox, which removes the entire reason for shipping one
|
||||
(§2). `--filesystem=xdg-pictures` is narrower and would be tempting, but it is
|
||||
still a static grant that lets a typed path resolve — it makes the same design
|
||||
work by not testing it, only in a smaller directory.
|
||||
still a static grant that lets a path resolve without the portal — it makes
|
||||
the same design work by not testing it, only in a smaller directory.
|
||||
|
||||
So the manifest grants no filesystem access at all. The consequence is stated
|
||||
plainly: **a Flatpak built from this manifest can open photographs handed to it
|
||||
and cannot yet be pointed at a library.**
|
||||
So the manifest grants no filesystem access at all, and a Flatpak built from
|
||||
it reaches the user's photographs only through what the portal hands it.
|
||||
|
||||
### What closes it
|
||||
|
||||
Two changes, in this order:
|
||||
|
||||
1. **A portal file chooser behind a platform seam.** `ashpd`'s
|
||||
`OpenFileRequest` with `directory(true)` returns a URI the document portal
|
||||
has exported, which the sandbox can read and which stays valid across
|
||||
restarts. It resolves to a real path under `/run/user/$UID/doc/`, so
|
||||
`normalise_endpoint` accepts it as it stands — `canonicalize()` on a fuse
|
||||
path returns the path itself. The seam matters more than the crate: this
|
||||
belongs beside `LocalStorage::grant` in `dr-plat`, which is already the one
|
||||
place a `Path` enters the application, and must not become a second way for
|
||||
`ui/` to learn about paths.
|
||||
2. **Removable volumes through the same door.** There is no portal for "list
|
||||
the mounted cards". The honest answer is that under a sandbox
|
||||
`imports_supported()` should report the same `false` it reports on Android,
|
||||
for the same reason it gives there — the operation cannot be performed
|
||||
however hard the user tries — and the import flow should offer the folder
|
||||
chooser instead of a volume list.
|
||||
1. **Build it and run it.** `flatpak-builder` is not installed on the machine
|
||||
this is developed on, so the manifest has never produced a package.
|
||||
2. **Removable volumes.** There is no portal for "list the mounted cards".
|
||||
Under a sandbox `imports_supported()` should report the same `false` it
|
||||
reports on Android, for the same reason — the volume list cannot be right
|
||||
however hard the user tries — and leave `Browse…` as the way to a card.
|
||||
|
||||
**Done when:** a Flatpak built from
|
||||
[`packaging/flatpak/paris.tourolle.darkroom.yml`](../../packaging/flatpak/paris.tourolle.darkroom.yml),
|
||||
with its `finish-args` unchanged and no `flatpak override` applied, can select a
|
||||
library root, scan it, and write a sidecar back into it.
|
||||
library root, scan it, write a sidecar back into it, and open it again after a
|
||||
restart.
|
||||
|
||||
### Running a Flatpak build before then
|
||||
|
||||
For testing the rest of the application inside the sandbox, grant the access
|
||||
If the portal's path turns out not to hold across a restart, the rest of the
|
||||
application can still be tested inside the sandbox by granting the access
|
||||
per-installation rather than in the manifest, so the file that describes the
|
||||
application keeps telling the truth:
|
||||
|
||||
|
||||
+34
-2
@@ -1575,5 +1575,37 @@ It is a match, not an update in place, and that is why the per-face repairs exis
|
||||
detection: where nothing about a face but one field needs doing, `record_updates` keeps the id and
|
||||
there is nothing to judge.
|
||||
|
||||
The merge's `match_faces` still matches by overlap alone across devices. It is the same question,
|
||||
and the same answer would serve it; it is not changed here.
|
||||
Since #77 (0.18.0) the merge's `match_faces` answers it too, within a photograph's `file_id` and
|
||||
one embedder: box IoU ≥ 0.5, unique on both sides, first; then, only for photographs where a remote
|
||||
face is left over and a local face is free, embedding cosine ≥ 0.7, mutual best, with a lead of
|
||||
≥ 0.2 over the runner-up on both sides. A box match is never overruled by a low cosine (about 150
|
||||
genuine cross-device pairs of tiny faces score below 0.45). On the reference desktop/tablet pair this
|
||||
recovers 20 of 631 unmatched faces with no false matches; the rest are faces one device alone found.
|
||||
The merge also keeps one person to one face per photograph: an incoming assignment is refused when
|
||||
another local face already holds that person, unless it is a remote confirmation over a local
|
||||
suggestion, which moves the suggestion. Refusals are counted in `faces_one_per_photograph`.
|
||||
|
||||
The threshold differs from `SAME_FACE_COSINE` (0.45) above on purpose: re-detection additionally
|
||||
requires the boxes to overlap, while the merge's embedding route exists for boxes that don't.
|
||||
|
||||
## 19. Deduplicating people · 2026-09-26
|
||||
|
||||
`dr_catalog::dedup_people::run` runs after every successful sync merge (`sync::merge_remote`, on the
|
||||
sync worker), in one transaction, and logs one `dedup:` line (#78).
|
||||
|
||||
**People.** Named people with the same name, trimmed and case-folded, merge into the one with the
|
||||
most confirmed faces (ties go to the smaller uuid) when every shared embedder's confirmed-face
|
||||
centroids agree at cosine ≥ 0.7 (distance < 0.3). Each side needs at least two confirmed faces to
|
||||
compare; a namesake holding no faces merges outright; a face confirmed as one and rejected as the
|
||||
other keeps them apart; unnamed and set-aside people are never touched. On the reference library the
|
||||
same-person centroid median is 0.91, and different named people have a 99.9th percentile of 0.41.
|
||||
|
||||
**Faces.** Two faces in the same image and embedder with IoU ≥ 0.5 and cosine ≥ 0.7 are one: the
|
||||
job keeps the stronger detector's face (`FaceDetector::outranks`), then the confirmed one, then the
|
||||
lower id, and it takes both faces' assignment and rejections.
|
||||
|
||||
**Propagation.** The merge is `faces::merge_people`, whose `merged_into` redirect a 0.17.0 peer
|
||||
already honours, so an older device never resurrects the duplicate. The job also follows redirects
|
||||
left by earlier manual merges, moving this device's own assignments onto the person kept, and
|
||||
breaks a mutual redirect at the smaller uuid, which every device computes alike. A merge now also
|
||||
carries the merged-away person's rejections to the person kept.
|
||||
|
||||
@@ -264,6 +264,14 @@ base, computed at reduced resolution and correct at any moment the user stops.
|
||||
"Render coarse while dragging, sharpen when it settles" would paper over the same
|
||||
34 ms with a visible swap. Fix the stage.
|
||||
|
||||
> **Since, 0.15.0.** The stage was fixed (§ The reduced base, below), and the
|
||||
> draft was built anyway, in the form this section would accept: the develop
|
||||
> view renders at half resolution while a gesture moves and once at full
|
||||
> resolution 120 ms after it stops (`ui/dr-ui/src/refine.rs`), the histogram
|
||||
> dims while it describes an older frame, and the last draft fades out over
|
||||
> 150 ms rather than being swapped. It is not a mask over a slow stage; it
|
||||
> spares a drag the full-resolution frames it does not need.
|
||||
|
||||
---
|
||||
|
||||
## Which GPU, on a machine with more than one
|
||||
@@ -317,9 +325,12 @@ without measuring them:
|
||||
Stated because §7 of [display-and-extension.md](display-and-extension.md) asks
|
||||
for it, and because each of these could move the numbers.
|
||||
|
||||
- **Local adjustments.** The mask stack is a separate chain per layer and is not
|
||||
in any row above. `render_masked` takes them and the fused shader addresses
|
||||
them per layer, so a heavily masked edit costs more than `all`.
|
||||
- **Local adjustments.** Not in any row above. Since 0.18.1 a layer is no longer
|
||||
a separate chain after the global one: each operation a layer touches runs a
|
||||
second fragment at its own place in the chain, blended by the layer's mask
|
||||
([architecture.md §5.2](architecture.md)), and `render_masked` binds the mask
|
||||
array the fused shader samples. A heavily masked edit therefore costs more
|
||||
than `all`, by roughly one fragment per touched operation per layer.
|
||||
- **Spot repairs.** These add detail passes, and their cost is per spot.
|
||||
- **Lens corrections.** Not part of `EditGraph::default_chain` — they are built
|
||||
from a matched profile — so the `point` row does not include the warp chain.
|
||||
@@ -431,3 +442,76 @@ difference between a quarter-scale and a half-scale base to 0.03 stops of peak
|
||||
excursion and 2% of frame reach. But it is a change in reach rather than only
|
||||
in cost, it is what a tile scheduler would be handed, and it is worth knowing
|
||||
that the number moved rather than discovering it later as a seam.
|
||||
|
||||
---
|
||||
|
||||
## The fit view, again — 2026-09-25
|
||||
|
||||
**Status:** Measured in the commits named, not re-run for this file.
|
||||
|
||||
Two changes to the fused path made the `fit` rows above cheaper again,
|
||||
each with its before and after in its commit message. Both were measured
|
||||
on the laptop RTX 3050 with its clocks held at 420/810 MHz by the power
|
||||
cap, on the synthetic 60 MP source of `examples/frame_budget.rs`, median of
|
||||
five alternated runs; both leave the rgba8 output bit-identical.
|
||||
|
||||
**The source gather is read once per framing** (`1dc7b45`). At fit every
|
||||
output pixel reads one texel on a stride through a source three or four
|
||||
times its width, and that gather was most of the fused pass. The pass now
|
||||
keeps a render-sized `rgba16float` cache of it, keyed on the framing, and
|
||||
reads it back while only the adjustments move:
|
||||
|
||||
| scene | before | after |
|
||||
|---|---:|---:|
|
||||
| neutral, 2560 × 1600 fit | 10.62 ms | 3.88 ms |
|
||||
| neutral, 3840 × 2160 fit | 21.05 ms | 7.11 ms |
|
||||
| clarity, 3840 × 2160 fit | 42.20 ms | 27.88 ms |
|
||||
| neutral, 2560 × 1600 1:1 (control) | 3.83 ms | 3.84 ms |
|
||||
|
||||
An interpolated read (straightening, lens warps, CA) is not cached, and the
|
||||
cache is written on the second frame with a given key, so a crop or zoom
|
||||
drag pays nothing for it.
|
||||
|
||||
**A detail pass that changes nothing is dropped** (`d430ec9`). Capture
|
||||
sharpening at a scale too coarse to draw its radius emits an empty pass,
|
||||
which cost a full read and write when another neighbourhood operation
|
||||
followed it: sharpen with clarity at 2560 × 1600 fit went from 18.66 ms to
|
||||
14.16 ms, and at 3840 × 2160 from 37.93 ms to 27.88 ms.
|
||||
|
||||
## Dehaze in two passes — 2026-09-26
|
||||
|
||||
**Status:** Measured in the commit named, not re-run for this file.
|
||||
|
||||
**Dehaze erodes each axis in one pass and recovers in the second**
|
||||
(`bee5c58`, #74). It was five passes — a run and a span erosion along x,
|
||||
the same along y, and the recovery — and cost 22.9 ms of a 2560 × 1600
|
||||
frame on the laptop RTX 3050, 54.1 ms at 3840 × 2160, with the memory
|
||||
clock held at 810 MHz by the power cap. At those clocks a detail pass costs
|
||||
what it reads and writes rather than what it taps: a pass with an empty
|
||||
body, one render-sized `rgba16float` read and write, measured 4.0 ms, and
|
||||
each dehaze pass 4.4–4.6 ms, so the taps were about 2 ms of the 22 and the
|
||||
four hand-offs between passes were the rest. Each axis now takes the
|
||||
minimum over its whole window directly, and the recovery rides in the y
|
||||
pass, which already holds the veil and the pixel's own colour: 36 texture
|
||||
reads a pixel in place of 12, nearly all cache hits, and two passes in
|
||||
place of five.
|
||||
|
||||
The same synthetic 60 MP source, only a detail parameter moving so the
|
||||
fused pass is reused, 30 frames a scene after six of warm-up, five runs of
|
||||
each binary alternated, median of the per-run p50:
|
||||
|
||||
| scene | before | after |
|
||||
|---|---:|---:|
|
||||
| dehaze, 2560 × 1600 fit | 22.88 ms | 9.06 ms |
|
||||
| dehaze, 2560 × 1600 1:1 | 23.41 ms | 9.52 ms |
|
||||
| dehaze, 3840 × 2160 fit | 54.09 ms | 28.12 ms |
|
||||
| all five detail operations, 2560 × 1600 fit | 53.11 ms | 39.97 ms |
|
||||
| all five detail operations, 2560 × 1600 1:1 | 67.48 ms | 56.42 ms |
|
||||
| every operation with film, 2560 × 1600 fit | 57.59 ms | 44.19 ms |
|
||||
| every operation with film, 2560 × 1600 1:1 | 71.83 ms | 57.93 ms |
|
||||
|
||||
The five detail operations are noise reduction, sharpening, clarity,
|
||||
texture and dehaze; the scenes without dehaze moved within ±2%. The
|
||||
picture is the same bits: a minimum is exact in any order, the window is
|
||||
the one the split passes covered, and the rgba8 output hashed identically
|
||||
before and after in all 64 scene, view and size combinations measured.
|
||||
|
||||
@@ -269,7 +269,7 @@ The set operations are already expressible in fixed-function blending over
|
||||
|------|-------------------------------|--------|-------|
|
||||
| Union | `One`, `One`, `Max` | `max(dst, src)` | M1 |
|
||||
| Subtract | `Zero`, `OneMinusSrc`, `Add` | `dst · (1 − src)` | M1 |
|
||||
| Intersect | `Zero`, `Src`, `Add` | `dst · src` | M2 |
|
||||
| Intersect | `Zero`, `Src`, `Add` | `dst · src` | M2 (built 2026-09-24) |
|
||||
|
||||
The middle row is `brush_erase`, already constructed in
|
||||
[`MaskPass::new`](../../core/dr-gpu/src/mask.rs). The other two are the same
|
||||
@@ -292,6 +292,16 @@ rendering as it did. `an_erase_stroke_holes_its_own_part_and_not_the_mask` in
|
||||
[`local_adjustments.rs`](../../core/dr-gpu/tests/local_adjustments.rs) is the test
|
||||
that holds this in place.
|
||||
|
||||
Intersection, as built, is exactly the table's row: a third pipeline beside
|
||||
the other two in `MaskPass::new`, `Join::apply` stating the three on the CPU,
|
||||
and `a_part_unioned_subtracted_and_intersected_gives_the_three_fields` and
|
||||
`the_joins_match_their_definition` in `local_adjustments.rs` holding the GPU
|
||||
to it. It is a product, not a minimum: the two agree wherever either side is
|
||||
fully in or out, and between two soft edges the product is the softer reading,
|
||||
which is the right way for an overlap of two partial selections to be wrong.
|
||||
Its blend is `Add`, not `Max`, so it adds nothing to the Android question
|
||||
below.
|
||||
|
||||
`Max` blending on `r8unorm` is core WGPU and universally supported on the
|
||||
desktop backends; **verify it on the Android adapter before M2 lands**, since
|
||||
that is the platform where a blend mode is most likely to be quietly emulated
|
||||
@@ -689,6 +699,14 @@ nothing, and every report of it came back as "the masks do not work".
|
||||
per-part distance fields (§5.5), the part list with its join chips, folding
|
||||
two layers.
|
||||
|
||||
Done (2026-09-24): `Intersect` — the join, its blend state, the sidecar word
|
||||
`intersect`, the part row's chip cycling + / − / ∩, and an "∩ Intersect"
|
||||
button beside Add and Subtract (§7.1). The part list with its chips and a
|
||||
per-part eye was already there from M1. Outstanding: joining a model,
|
||||
gradient or range part from the panel (the buttons still join a painted
|
||||
part, so an intersection is painted where the mask should survive), per-part
|
||||
distance fields, and folding two layers.
|
||||
|
||||
**M3 — Push, and cling.** The warp pass (§5.3) and edge-aware deposit (§5.6).
|
||||
Both are refinements of a tool that already works, which is the right place
|
||||
for the two riskiest pieces.
|
||||
|
||||
+188
-77
@@ -28,6 +28,24 @@ Android memory pressure and lost-root recovery (FR-PLAT-AND-5, FR-PLAT-AND-2), i
|
||||
recorded where it appears rather than deleted, because a requirement that is *half* met is the
|
||||
one most likely to be reported as closed.
|
||||
|
||||
**Swept again on 2026-09-26, for 0.16.0.** Progressive refinement (FR-DSP-4) and the panorama
|
||||
(§3.11) were built and are struck below; consolidating duplicate originals (FR-CAT-11a) now sits
|
||||
beside re-import detection; the accessibility and localisation counts in §6 had not been read
|
||||
against the tree since 2026-08-30 and are replaced; and §4a records what the develop and keyboard
|
||||
work of 0.15.0 and 0.16.0 left open.
|
||||
|
||||
**And again for 0.17.0.** Folders are chosen through the platform's dialogue now, so FR-PLAT-LIN-3
|
||||
in §5 is rewritten around what the Flatpak has still not shown; albums brought the first SAF code,
|
||||
which FR-PLAT-AND-1's entry now describes, along with why its new tag overstates it.
|
||||
|
||||
**And for 0.18.2.** FR-CULL-13's write-path half is held by a test now, and §2 records the evidence
|
||||
half it leaves; NFR-ARCH-1's executors are named and guarded but not pooled, recorded in §4. Three
|
||||
things closed without having had an entry: a mask layer's film settings, which the panel offered
|
||||
and which moved nothing until 0.18.2 (FR-DEV-3f); hot and dead photosites, repaired on the mosaic
|
||||
before the demosaic, for Bayer and X-Trans alike and with no setting (FR-RAW-3; the defect-map
|
||||
reader in `dr-decode` is still not wired in, and a CR2 carries no map for it to read); and the
|
||||
tablet's scrollers, which §4a now describes.
|
||||
|
||||
---
|
||||
|
||||
## 1. Plugins — post-v1 since 2026-09-19
|
||||
@@ -80,8 +98,8 @@ somebody reads the matrix.
|
||||
|
||||
## 2. Culling — the stated differentiator, half built
|
||||
|
||||
[D11](requirements.md) names culling "the core differentiator". FR-CULL-1, -2, -3, -4 and -8 through
|
||||
-12 are built. Three are not.
|
||||
[D11](requirements.md) names culling "the core differentiator". FR-CULL-1 through -5 and -8 through
|
||||
-12 are built, and FR-CULL-13 is half built. Two are not built at all.
|
||||
|
||||
**FR-CULL-3 — Raw-truth overlays. Built, all three bullets.** Focus peaking is
|
||||
`core/dr-gpu/src/focus.rs` and `ui/dr-ui/src/peaking.rs`; the raw histogram and the raw clipping
|
||||
@@ -115,8 +133,24 @@ labels — that comment was a forward reference and is now simply wrong, rather
|
||||
And `dr_catalog::bursts::choose_representative` is written and tested but bound to no gesture, so
|
||||
today the only override is expanding the burst.
|
||||
|
||||
`core/dr-catalog/src/dedup.rs` remains a different thing: re-import detection under FR-CAT-11,
|
||||
matching a file against one already catalogued, not two photographs against each other.
|
||||
Neither is deduplication, which is about one file held twice rather than two frames that look
|
||||
alike. `core/dr-catalog/src/dedup.rs` is re-import detection under FR-CAT-11, matching a file on
|
||||
a card against one already catalogued. Its other half, FR-CAT-11a, is built since 0.16.0:
|
||||
`core/dr-catalog/src/duplicates.rs` groups the copies a library already holds (same root, camera,
|
||||
capture instant and size), `ui/dr-ui/src/duplicates.rs` proves each group the same by digest and
|
||||
compares their edits, and a group is folded onto one survivor with the others trashed in one
|
||||
transaction, from the Duplicate originals review in the sidebar and in Settings.
|
||||
|
||||
**FR-CULL-13 — Evidence, never verdicts. The write-path half.** Every write of a rating, flag,
|
||||
colour label or trash membership in the shipped code is enumerated by
|
||||
`tools/traceability/src/verdicts.rs`, which parses the tree with `syn` and holds each site to a
|
||||
reviewed list with its reason: a key, click or tap, a function writing for callers that are checked
|
||||
in turn, or a verdict carried from elsewhere (a sidecar or `.xmp` pull, the sync merge, the catalog
|
||||
mirrored to a file, duplicates consolidation). An unlisted write, or a listed one that is gone,
|
||||
fails `cargo test -p traceability`. What is not built is the evidence itself: chips for clipping,
|
||||
focus, burst membership and face counts on the grid cell and in FR-CULL-4's mode, shown as absent
|
||||
rather than zero, with a filter per signal. Eye state is the one signal shown today, on People's
|
||||
face cells and as a library filter.
|
||||
|
||||
**FR-CULL-6 — Compare and survey.** Absent. No side-by-side view, no synchronised zoom or pan.
|
||||
This is the one of the four with no adjacent machinery at all, and it is also the one that most
|
||||
@@ -145,7 +179,7 @@ place.
|
||||
|
||||
---
|
||||
|
||||
## 4. The render path — FR-DSP-2, FR-DSP-4, NFR-RES-2
|
||||
## 4. The render path — FR-DSP-2, FR-DSP-4, NFR-RES-2, NFR-ARCH-1
|
||||
|
||||
**FR-DSP-2 — Tiled computation. Unbuilt, and under challenge.** [architecture.md §6.2](architecture.md)
|
||||
calls for tiling "from day one" on the grounds that retrofitting it is a rewrite. It was not built,
|
||||
@@ -166,10 +200,15 @@ Two measurements say it costs more than it saves on the interactive path. Neithe
|
||||
about the export path or about a device under memory pressure, which is where the case for it
|
||||
actually lives — and that is spike S6, which has not run.
|
||||
|
||||
**FR-DSP-4 — Progressive refinement.** Unbuilt. FR-DSP-1's proxy rendering and TD-4's
|
||||
quarter-resolution base are adjacent and are not it: both are fixed choices about what resolution to
|
||||
compute at, where FR-DSP-4 asks for a first frame that is deliberately cheap and a second that
|
||||
replaces it. Nothing tracks a "this frame is provisional" state.
|
||||
**FR-DSP-4 — Progressive refinement. Built in 0.15.0.** While a gesture moves the canvas renders
|
||||
a half-resolution draft, and the sharp frame lands once, 120 ms after the last movement: the
|
||||
decision is `ui/dr-ui/src/refine.rs`, a debounce whose every draft re-arms the settle timer, driven
|
||||
through simulated timelines in its tests. The draft flag reaches the interface (`canvas-draft`,
|
||||
`Levels.provisional`), so the histogram dims while it describes a frame older than the one on
|
||||
screen, and the last draft is kept and faded out over 150 ms when the sharp frame arrives, so
|
||||
refinement is not a swap. Draft frames themselves date from 2026-08-09; what this entry missed,
|
||||
and 0.15.0 finished, was a settle that waited for the gesture to stop, a provisional state the
|
||||
interface could see, and a refinement that was not a jarring swap.
|
||||
|
||||
**NFR-RES-2 — Images larger than GPU memory.** Half answered. NFR-R8's "decide explicitly" was
|
||||
decided on 2026-09-19: there is no CPU render pipeline, the degraded mode is the viewer on
|
||||
@@ -179,30 +218,83 @@ and no spill. Spike S6 — a tiled pipeline on a
|
||||
mid-range Android device with an image larger than available GPU memory — is the one that would
|
||||
settle both this and FR-DSP-2, and there is no evidence it has run.
|
||||
|
||||
**NFR-ARCH-1 — Named executors. Named and guarded, not bounded.** `dr_ui::executors` names the
|
||||
five executors of [architecture.md §7.1](architecture.md) with their thread counts, every
|
||||
long-lived worker in `dr-ui` and the Android entry point starts through its `spawn` as
|
||||
`<executor>:<role>`, and `net_runtime`'s `block_on` fails a debug or test build on the UI thread.
|
||||
The counts are a budget and not yet a limit: each job still gets a thread of its own, and a pool
|
||||
sized from `Executor::threads` is where NFR-ARCH-2's priority classes would live. The guard covers
|
||||
`block_on` only — not a synchronous file read or a catalog query on the UI thread, which the library
|
||||
and People screens make on a click by design ([catalog.md §1](catalog.md)). The two mask workers in
|
||||
`masks_ui.rs` still call `std::thread::spawn`, and the threads the core crates start are outside the
|
||||
module.
|
||||
|
||||
---
|
||||
|
||||
## 4a. Develop, masks and the keyboard — what the 0.15.0 and 0.16.0 work left open
|
||||
|
||||
Most of what these two releases built closed a clause outright, and is listed here only so that
|
||||
nobody looks for it below: perspective correction (FR-DEV-20, Vertical and Horizontal in Compose),
|
||||
the crop that says when it orphans a mask (FR-DEV-17), intersection as a third mask join
|
||||
(FR-DEV-19a), the keyboard vocabulary for develop (FR-DEV-16, met and held by `gestures-check` in
|
||||
both directions), colour labels set, shown and filtered (NFR-A11Y-3's first example), the RAW
|
||||
decoder behind a trait (FR-RAW-2), and duplicate originals (FR-CAT-11a, §2 above). What they left:
|
||||
|
||||
- **FR-UI-5's pointer half.** Keys cover navigation, rating and common adjustments in every view,
|
||||
and judging works in develop on the open photograph; but no develop slider takes the scroll
|
||||
wheel, which the clause names. Its status note in the register says so rather than rounding up.
|
||||
- **FR-RAW-2's second decoder.** The trait is built and a stub decoder is proved to reach the
|
||||
scan, the preview ladder and export without a caller changing; LibRaw itself (D2) is not
|
||||
built, and S7 is what would say when it is needed.
|
||||
- **FR-DEV-19's M2 remainder** ([mask-editing.md §11](mask-editing.md)). The panel's Add, Subtract
|
||||
and Intersect buttons join a painted part only, so an intersection with a gradient or a range is
|
||||
painted where the mask should survive; per-part distance fields and folding two layers are
|
||||
unbuilt. From M1, the brush has no ring drawn on the photograph and a layer's whole stroke
|
||||
history is redrawn per dab.
|
||||
- **FR-DEV-17's blind spot, by design.** A layer that adds a range or a region selection is never
|
||||
reported, because the check has no label map to measure it with (`dr_pipeline::orphan`); a
|
||||
false alarm on the common path would teach the notice to be dismissed unread.
|
||||
|
||||
The desktop's scrollbars (the develop column, the grid, the sidebar, Settings, the film list and
|
||||
the help sheet) are drawn only where `dr_plat::is_touch_first()` is false. On Android the same
|
||||
scrollers show instead a 3 px position cue (#69): `Scrolling.cue` in `widgets.slint`, the thumb
|
||||
alone, drawn while the viewport moves and faded 500 ms after, with no touch target, so a flick that
|
||||
starts on it scrolls the list (`tests/scroll_cue.rs`). A desktop build shows the cue when
|
||||
`DR_SCROLL_CUE=1` is set, for checking it without a device. Not yet checked on the tablet.
|
||||
|
||||
---
|
||||
|
||||
## 5. Android beyond running, and Flatpak
|
||||
|
||||
The Android app is not a stub — it builds an APK, runs the whole application, unpacks bundled face
|
||||
models, and has been measured on a tablet ([faces.md §12.1](faces.md),
|
||||
[technical-debt.md TD-1](technical-debt.md)). What is missing is the platform contract around it.
|
||||
models, has been measured on a tablet ([faces.md §12.1](faces.md)), and since 0.15.0 draws the
|
||||
develop view zero-copy as the desktop does ([technical-debt.md TD-1](technical-debt.md), paid off).
|
||||
It carries the manual and opens it in a WebView. What is missing is the platform contract around it.
|
||||
|
||||
**FR-PLAT-AND-1 is untagged, and what it was tagged for was intent rather than code.**
|
||||
The requirement demands that library access be obtained *exclusively* through the Storage Access
|
||||
Framework. There is no SAF code: no `ACTION_OPEN_DOCUMENT_TREE`, no `takePersistableUriPermission`,
|
||||
no `DocumentsContract`. Its two tags rested on a `SourceRef::Document` variant constructed only
|
||||
inside `#[cfg(test)]` — `LocalStorage::open` refuses it, and the test that proves so is named
|
||||
`a_reference_of_the_wrong_kind_is_refused_rather_than_guessed_at` — and on
|
||||
`dr_plat::imports_supported`, which *returns false on Android* and whose own documentation says it
|
||||
"stops being false when a SAF implementation lands". The second tag documented the absence of the
|
||||
thing it was counted as evidence for. Both have been removed; this is the "plumbing a future feature
|
||||
would use" case [CONTRIBUTING.md](../../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both
|
||||
warn about. Android reaches a library through a Nextcloud account or a folder, over paths, like the
|
||||
desktop.
|
||||
**FR-PLAT-AND-1 — SAF is built for export folders, not for the library.** The requirement demands
|
||||
that library access be obtained *exclusively* through the Storage Access Framework. Until 0.17.0
|
||||
there was no SAF code at all, and the requirement's two tags rested on a `SourceRef::Document`
|
||||
variant constructed only inside `#[cfg(test)]` and on `dr_plat::imports_supported`, which *returns
|
||||
false on Android* and whose own documentation says it "stops being false when a SAF implementation
|
||||
lands". Both were removed as intent rather than code — the "plumbing a future feature would use"
|
||||
case [CONTRIBUTING.md](../../CONTRIBUTING.md) and [code-health.md CH-4](code-health.md) both warn
|
||||
about.
|
||||
|
||||
0.17.0 brought the first real SAF code, for albums (FR-EXP-10): `FolderPicker.java` starts
|
||||
`ACTION_OPEN_DOCUMENT_TREE` from a translucent activity of its own (the main activity is
|
||||
`NativeActivity`, whose results are not ours) and takes a persistable grant; `Saf.java` writes each
|
||||
export through `DocumentsContract`; `ui/dr-ui/src/saf.rs` is the JNI bridge. They shipped tagged
|
||||
`TRACES: FR-PLAT-AND-1`, which made the matrix count the requirement as covered, and that overstated
|
||||
it: the mechanism is the one the requirement names, but its subject is the library, and Android
|
||||
still reaches a library through a Nextcloud account or a folder, over paths, like the desktop. The
|
||||
tags now say FR-EXP-10 alone (0.17.1), so FR-PLAT-AND-1 reads as uncovered again until the library
|
||||
itself is reached through SAF.
|
||||
|
||||
That has a consequence for the rest of the cluster: **FR-PLAT-AND-2** — detecting the loss of a
|
||||
granted tree permission and marking images offline rather than deleting rows — cannot be built until
|
||||
there is a permission to lose. It is listed here as unbuilt, but it is blocked, not skipped.
|
||||
granted tree permission and marking images offline rather than deleting rows — is still blocked for
|
||||
the library, because there is no library grant to lose. An album's folder has a grant now, and
|
||||
nothing checks for its loss: an export into a folder whose grant has gone fails with whatever the
|
||||
write raises, rather than the album saying beforehand that it needs a folder again.
|
||||
|
||||
**FR-PLAT-AND-4 — half built.** The runner is done (`core/dr-catalog/src/runner.rs`): the
|
||||
queue that `jobs.rs` always had is now claimed from, completed, failed and recovered after a
|
||||
@@ -211,11 +303,13 @@ the platform half — a foreground `Service`, `FOREGROUND_SERVICE` and `POST_NOT
|
||||
manifest, and a stated Doze behaviour. The build step that blocked it is no longer a blocker: the
|
||||
APK now compiles its own Java.
|
||||
|
||||
Note also that **no handler is registered**, deliberately. The only enqueue site reachable in the
|
||||
shipping app produces remote thumbnail jobs already served by the async grid worker, and
|
||||
`walk::scan_root` — which holds the other two enqueue sites — has no caller outside an example.
|
||||
Wiring the sweep to claim from the queue is the honest next step and is an async rewrite of
|
||||
`library.rs`.
|
||||
Note also that **no handler is registered**, deliberately — and, since #73, nothing enqueues
|
||||
either. The remote scan's per-photograph `Thumbnail` jobs, and `walk::scan_root`'s `Thumbnail` and
|
||||
`ExtractMetadata` jobs, duplicated debts the thumbnail store and `metadata_state` already record,
|
||||
and were never claimed; they were removed and `Thumbnail` retired ([catalog.md §6.1](catalog.md)).
|
||||
Feeding FR-PLAT-AND-4's platform half is therefore a matter of choosing a kind whose work has no
|
||||
better source of truth, registering its handler, and enqueuing it in the same change —
|
||||
`every_queued_kind_has_a_consumer` refuses the enqueue without the handler.
|
||||
|
||||
**FR-PLAT-AND-5 — built.** A tiered eviction registry drives GPU caches, then proxies, then
|
||||
thumbnails, from `MainEvent::LowMemory` and `MainEvent::Stop`.
|
||||
@@ -226,14 +320,20 @@ Intent read over JNI, and an `ExportProvider` rooted at `getFilesDir()` rather t
|
||||
been exercised on a device** — the tests read the manifest and the Java through `include_str!`,
|
||||
which catches a deleted filter but not a class loader that cannot find the class.
|
||||
|
||||
**FR-PLAT-LIN-3 — packaged, not satisfied.** There is a Flatpak manifest now, granting no
|
||||
filesystem permission of any kind, plus AppStream metainfo and `docs/distribution.md`. The
|
||||
requirement is still not met, and cannot be met by packaging: a folder library is chosen by typing
|
||||
an absolute path, nothing in the tree calls the FileChooser portal, and inside the sandbox `$HOME`
|
||||
holds only `.var/app/...`. `dr_plat::volumes()` reads `/proc/self/mountinfo`, so a card mounted on
|
||||
the host is invisible to a sandboxed process as well. The fix is an `ashpd` directory picker beside
|
||||
`LocalStorage::grant`, not a change to the manifest. No Flatpak has been built here —
|
||||
`flatpak-builder` is not installed — so the permission set is reasoned, not observed.
|
||||
**FR-PLAT-LIN-3 — packaged and wired, not yet proved.** There is a Flatpak manifest, granting no
|
||||
filesystem permission of any kind, plus AppStream metainfo and `docs/distribution.md`. Until 0.17.0
|
||||
the requirement could not be met by packaging: a folder library was chosen by typing an absolute
|
||||
path, nothing called the FileChooser portal, and inside the sandbox `$HOME` holds only
|
||||
`.var/app/...`. Since 0.17.0 every folder the desktop asks for — the library, an import's source
|
||||
and second copy, Lightroom presets, an album's folder — is chosen in the platform's dialogue
|
||||
(`ui/dr-ui/src/folder_dialog.rs`, `rfd` over the XDG portal), which is what a sandbox needs. Two
|
||||
things stop this entry being struck. No Flatpak has been built here — `flatpak-builder` is not
|
||||
installed — so the portal path has never been seen to open a library, reopen it after a restart,
|
||||
or take a sidecar inside the sandbox; [distribution.md §4](distribution.md) says what has to be
|
||||
checked. And
|
||||
`dr_plat::volumes()` still reads `/proc/self/mountinfo`, so a card mounted on the host is invisible
|
||||
to a sandboxed process; the import page's `Browse…` reaches one through the portal instead. The
|
||||
chooser landed in `ui/` rather than behind the `dr-plat` seam distribution.md had proposed.
|
||||
|
||||
**NFR-COMPAT-2 — distribution channels. Stated, which is all this requirement asks.** The paragraph
|
||||
above cites [distribution.md](distribution.md) and it is the same document that answers this: §1
|
||||
@@ -267,38 +367,44 @@ fixes users will need — is unaddressed, and there is no update mechanism of an
|
||||
|
||||
## 6. Accessibility and internationalisation — the hard half is done and the easy half is not
|
||||
|
||||
**NFR-A11Y-1 — Localisation.** `@tr(` appears **zero** times across 14,482 lines of Slint. That
|
||||
number overstates the problem, because the part that is genuinely architectural was got right:
|
||||
`LocalizedKey` keeps display strings out of `core/` entirely, every operation publishes a key rather
|
||||
than a label, and `labels::resolve` is the single point where a key becomes text. What that single
|
||||
point does, however, is a hardcoded English `match` in Rust source — so changing a translation
|
||||
requires a recompile, which is the one thing the requirement explicitly forbids. There is no message
|
||||
catalogue in any format, no locale-resolution rule, and no decision recorded about RTL.
|
||||
**NFR-A11Y-1 — Localisation.** One screen of twenty-eight Slint files is converted: the launch
|
||||
screen wraps its strings in `@tr(`, and `ui/dr-ui/build.rs` records the mechanism — Slint's bundled
|
||||
translations, a `.po` per language under `ui/dr-ui/lang/`, extracted with `slint-tr-extractor` —
|
||||
and why bundling rather than gettext (Android has no path a `.mo` could sit at). No `.po` exists
|
||||
yet, so every build is the original English, and the rest of the interface (about 23,600 lines of
|
||||
markup) is unwrapped. The part that is genuinely architectural was got right: `LocalizedKey` keeps
|
||||
display strings out of `core/` entirely, every operation publishes a key rather than a label, and
|
||||
`labels.rs` is the single point where a key becomes text — but that point is a hardcoded English
|
||||
`match`, and `build.rs` says the Rust-side mechanism is not built. Two things the requirement asks
|
||||
remain against it even when the conversion is finished: a bundled translation is compiled in, so
|
||||
changing one still needs a rebuild, which the clause forbids; and there is no locale-resolution rule
|
||||
and no decision recorded about RTL.
|
||||
|
||||
The work left is therefore smaller than it looks and entirely mechanical: a catalogue format, a load
|
||||
path behind `resolve`, and `@tr(` around the Slint literals. The design it needs already exists.
|
||||
**NFR-A11Y-2 — Accessibility. Tagged, and half tested.** `accessible-*` properties now appear on
|
||||
some 127 lines across eleven Slint files — the shared controls in `widgets.slint` and
|
||||
`controls.slint`, every slider, the rail, the grid's cells (named by file), the rating stars, the
|
||||
labels and the duplicates review — and `ui/dr-ui/tests/ui_controls_are_accessible.rs` fails when a
|
||||
shared control stops declaring its role or a slider is added without a name. The manual's recording
|
||||
scripts find every control they press through the same names (`dr_ui::automation`), so a control a
|
||||
screen reader cannot name is also one the manual cannot record. What is unchecked is stated in the
|
||||
test: whether the words are good, and whether Android exposes any of it — spike S13, which has not
|
||||
run. The requirement's other two clauses are debt with entries of their own: platform font scaling
|
||||
([TD-7](technical-debt.md)) and WCAG AA contrast ([TD-6](technical-debt.md)).
|
||||
|
||||
**NFR-A11Y-2 — Accessibility.** `accessible-*` appears five times in the whole interface, all five
|
||||
on one control — the parameter slider in `adjust.slint` — and nothing is set from the Rust side at
|
||||
all. Everything else in eighteen Slint files is unnamed to AT-SPI and TalkBack. The requirement's own
|
||||
caveat, that Slint's Android accessibility needs verifying, is spike S13, which has not run.
|
||||
**NFR-A11Y-3 — Colour-independent status.** Built, and now asserted. The clipping readout pairs a
|
||||
marker that appears or disappears with a figure in words; the rating strip is a solid star against
|
||||
an outline in an achromatic palette; the pick/reject mark is a tick against a cross; the
|
||||
focus-peaking colour chips say "Red" and "Cyan" rather than showing swatches; and colour labels —
|
||||
the requirement's first named example — now have an interface built with a shape from the start:
|
||||
every mark carries its label's initial (R, Y, G, B, P) on its colour, and every chip, picker and the
|
||||
develop bar name the label in words.
|
||||
|
||||
**NFR-A11Y-3 — Colour-independent status.** Built where a control exists, and now tagged: the
|
||||
clipping readout pairs a marker that appears or disappears with a figure in words, the rating strip
|
||||
is a solid star against an outline in an achromatic palette, the pick/reject mark is a tick against
|
||||
a cross, and the focus-peaking colour chips say "Red" and "Cyan" rather than showing swatches. Each
|
||||
of those already carried the reasoning in a comment naming this requirement and simply had no
|
||||
`TRACES` line.
|
||||
|
||||
Two caveats, because the tag now says more than the evidence does. **Only the clipping clause has a
|
||||
test** — `a_clipping_figure_distinguishes_none_from_nearly_none`, which pins `<0.1%` apart from `0%`
|
||||
so the figure cannot contradict the lit marker beside it. The three Slint components are
|
||||
inspected-and-argued, not asserted, and nothing would fail if a future edit made a star differ only
|
||||
in tint. **And the requirement's first named example has no interface at all**: catalog colour
|
||||
labels are a nullable `label INTEGER` column on the versions table and are set and shown nowhere, so
|
||||
the clause about them is untestable rather than satisfied. That clause closes when the label UI is
|
||||
built, not before, and it should be built with a shape from the outset — which is the same argument
|
||||
as below, for doing this alongside NFR-A11Y-2 rather than after it.
|
||||
The clipping clause is pinned by `a_clipping_figure_distinguishes_none_from_nearly_none`. The other
|
||||
four are pinned by `ui/dr-ui/tests/status_is_not_colour_alone.rs`, which reads the markup and fails
|
||||
if a star, a flag or a label comes to differ in tint alone — if the glyph a state selects stops
|
||||
depending on the state, if two states select the same drawing, if two labels share a letter, or if
|
||||
the peaking chips stop being words. It is a structural check, not a perceptual one: it cannot say
|
||||
whether the letters are legible at 14px on a given screen, which is still a matter of looking.
|
||||
|
||||
---
|
||||
|
||||
@@ -440,7 +546,7 @@ requirement text that asks for it:
|
||||
|---|---|---|
|
||||
| S6 | FR-DSP-2, NFR-RES-2 — tiling and images larger than GPU memory | Nothing; needs a device and a large image |
|
||||
| S9 | R1's tolerance threshold, and therefore R1 | Nothing; the threshold is defined *by* running it |
|
||||
| S10 | Whether SAF at 10k files meets NFR-P1/P3 | §5 — there is no SAF code to measure |
|
||||
| S10 | Whether SAF at 10k files meets NFR-P1/P3 | §5 — SAF reaches album folders only, never a library to enumerate |
|
||||
| S11 | NFR-COMPAT-2, and whether Play makes SAF binding | Nothing |
|
||||
| S13 | NFR-A11Y-2 on Android | §6 — there is almost nothing to test with |
|
||||
|
||||
@@ -455,18 +561,23 @@ whether something *should* be built — which is the opposite of the order §9 a
|
||||
|
||||
---
|
||||
|
||||
## 11. Merging — specified 2026-09-19, nothing built
|
||||
## 11. Merging — the panorama built, three clauses open
|
||||
|
||||
§3.11 was written on 2026-09-19 under D18, undeferring the panorama from §7 and leaving HDR merge
|
||||
and focus stacking there with their data model decided. Eleven `FR-MRG` clauses and two `NFR-MRG`
|
||||
figures entered the register at once with no code behind any of them, which is why the coverage
|
||||
figure fell from 83.0% to 77.2% on the same day — a specification, not a regression.
|
||||
and focus stacking there with their data model decided. Its clauses entered the register with no
|
||||
code behind them, which is why the coverage figure fell from 83.0% to 77.2% on that day — and the
|
||||
panorama was then built in the same week: alignment, projections, the chunked composite written as
|
||||
a DNG beside its sources, the auto-crop and the model's border fill.
|
||||
[panorama.md](panorama.md) §11 and §13 are where it stands.
|
||||
|
||||
[panorama.md](panorama.md) is the design, and its §10 is the order of work. Nothing starts before
|
||||
**S15**: whether rawler reads back a linear DNG the application writes, whether XFeat loads under
|
||||
tract at a fixed shape, whether the working-space texture can be tapped where FR-MRG-2 needs it,
|
||||
and what a chunked blend of a 100 MP composite costs on the tablet. The first two are a day each
|
||||
and either can change the design, which is the reason they come first.
|
||||
Three clauses carry no tag:
|
||||
|
||||
- **FR-MRG-9 — the tablet.** Android runs the same code, but the stated ceiling on frame count and
|
||||
source resolution, refused with a message rather than an out-of-memory kill, is not set.
|
||||
- **NFR-MRG-1 — merge latency.** Measured on the desktop and inside its 60 s (panorama.md §11);
|
||||
the tablet's figure is S15.4's open half, and nothing asserts either per commit.
|
||||
- **NFR-MRG-2 — reproducible merges.** Nothing checks that the same sources and settings give a
|
||||
byte-identical composite.
|
||||
|
||||
## 12. D12, which governed all of the above
|
||||
|
||||
|
||||
+174
-8
@@ -179,6 +179,17 @@ card into the library. Both are needed.
|
||||
original filename) and by content hash, offering skip or import-as-new. Camera filenames wrap at
|
||||
`IMG_9999`, so filename alone is insufficient. Existing catalog duplicates are detectable on demand.
|
||||
|
||||
**FR-CAT-11a — Consolidating catalog duplicates.** Files already in the library more than once
|
||||
(same root, camera, capture time and size) shall be listed on demand as groups, from the library
|
||||
and from Settings. Before anything moves, each group shall be proved the same file by a stored full
|
||||
digest or by a digest of the first and last megabyte of every copy, and its copies' develop edits
|
||||
compared; a group that differs in either is left out and the review says why. One copy survives —
|
||||
not under a backup-looking folder, then camera-named, then oldest, overridable per group — and the
|
||||
others' collections, keywords, highest rating, agreed flag and label, and faces are merged onto it,
|
||||
with disagreements reported rather than decided silently. Each group is merged and its other copies
|
||||
moved to the trash (FR-CAT-15) in one transaction: wholly consolidated or untouched. Nothing is
|
||||
deleted.
|
||||
|
||||
**FR-CAT-12 — Versions (virtual copies).** An image may carry multiple named `Version`s, each with
|
||||
an independent edit graph, without duplicating source data. Versions are creatable, nameable,
|
||||
deletable, and independently exportable; one is the default.
|
||||
@@ -243,15 +254,32 @@ read metadata, and `import.rs` fetches exactly that range through `Storage::read
|
||||
calling `dr_decode::metadata`. The decoder states its requirement and the storage layer satisfies
|
||||
it; a decoder holding its own `SourceRef` would have had to implement the range policy itself.
|
||||
|
||||
What is genuinely not built is the trait. There is one decoder, reached through free functions, so
|
||||
"without changing callers" is a claim nothing yet tests. The clause stands as written and is
|
||||
outstanding work, not a satisfied one.
|
||||
*Status (2026-09-24).* The trait is built: `dr_decode::Decoder`, over bytes — `header_bytes`,
|
||||
`metadata`, `orientation`, `locate_preview`, `preview` and `decode` — with `dr_decode::Rawler` as
|
||||
its one implementation, delegating to the free functions that were there before. The catalog scan,
|
||||
the thumbnail ladder, import, the viewer, export, merge and repairs take a `&dyn Decoder`; only the
|
||||
places that start a job name `dr_decode::default()`. "Without changing callers" is tested by
|
||||
`dr-ui`'s `decoder_seam` tests, which hand a stub decoder for a container no real decoder reads to
|
||||
the scan, the ladder and export, and fail if any of them reaches past the trait. Nothing in the
|
||||
trait takes a path or a `SourceRef`. The second decoder itself (LibRaw, D2) is not built; S7 (#48)
|
||||
is what would say when it is needed.
|
||||
|
||||
**FR-RAW-3 — Sensor data handling.** Correctly apply per-camera black/white levels, CFA pattern
|
||||
identification, and camera-native colour matrices. Demosaic quality shall be selectable, with at
|
||||
least a fast method for preview and a high-quality method for export (FR-EXP-9 requires export to
|
||||
use the latter).
|
||||
|
||||
*Status (2026-09-27), defective photosites.* Hot and dead photosites are repaired on the mosaic,
|
||||
before the demosaic, where each is still one wrong value rather than a coloured cross three pixels
|
||||
wide (`core/dr-gpu/src/shaders/hot_pixels.wgsl`). A photosite is repaired only where it stands apart
|
||||
from every same-colour photosite in its 5×5 window and from each of its eight immediate neighbours,
|
||||
which leaves stars and glints alone, and it takes the value of its brightest (or, when dead,
|
||||
darkest) same-colour neighbour, so nothing is invented. A 6×6 sensor-anchored colour tile serves
|
||||
Bayer and X-Trans alike; export and every other path that demosaics get the repair, and there is no
|
||||
setting. `core/dr-gpu/tests/hot_pixels.rs` renders a frame with and without a defect and compares
|
||||
the finished pixels. The DNG defect map `dr_decode::defects` reads is not used: few files carry
|
||||
one, and a CR2 none.
|
||||
|
||||
**FR-RAW-4 — Robustness.** A malformed or hostile RAW file shall not crash the application or
|
||||
compromise the process. Decode failures are reported per-file and do not abort a batch.
|
||||
|
||||
@@ -298,6 +326,13 @@ once, at the final export or display stage.
|
||||
- Crop, straighten, rotate, flip
|
||||
- Local adjustments: linear gradient, radial gradient, and brush masks
|
||||
|
||||
*Resolved 2026-09-26:* a mask layer's settings are **offsets to the photograph's**, applied at each
|
||||
operation's own place in the chain — global contrast −30 under a layer at −20 is −50 inside the
|
||||
mask, applied once. A moved switch or choice replaces the global one, and an offset that brings an
|
||||
operation back to neutral undoes the global setting inside the mask. Layers used to run as a second
|
||||
chain after every global operation, which compounded the two edits in ways neither slider showed
|
||||
([architecture.md §5.2](architecture.md); `core/dr-gpu/tests/local_adjustments.rs`).
|
||||
|
||||
**FR-DEV-3a — Self-describing operations.** Every processing operation shall declare its own
|
||||
parameters through a descriptor, so that adding an operation requires no changes to frontend code.
|
||||
An operation declares *what* its parameters are; the frontend decides *how* to present them.
|
||||
@@ -417,6 +452,15 @@ reference implementation.
|
||||
parameter — `core/dr-pipeline/src/sidecar.rs` records why an index was rejected (installing a
|
||||
profile would silently change which film every existing photograph was developed on).
|
||||
|
||||
*Resolved 2026-09-26:* the film's settings — exposure, push, print exposure, format — are
|
||||
**per-pixel**, evaluated by the shader against tables that hold none of them, so a mask layer can
|
||||
hold its own. A layer's settings are offsets to the photograph's, and where layers overlap a pixel
|
||||
takes the weighted average of what each asks for, the photograph's setting taking whatever weight
|
||||
the layers leave (`operation::local_settings_block`). The stock and its paper stay
|
||||
photograph-wide: a layer has no picker. The print is split at the paper's log exposure, so print
|
||||
exposure is an addition between two lookups and exact at any setting; push interpolates the
|
||||
stock's measured processes. Before this a layer offered the film's sliders and they moved nothing.
|
||||
|
||||
**FR-DEV-3g — AI denoise.** Learned denoising operating in the raw domain, ideally jointly with
|
||||
demosaic.
|
||||
|
||||
@@ -485,6 +529,13 @@ so history survives a restart. Named snapshots of an edit state.
|
||||
**FR-DEV-6 — Presets.** Save, apply, and manage named presets covering a subset of the edit
|
||||
graph. Copy/paste settings between images. Batch-apply to a selection.
|
||||
|
||||
A preset may name a film stock, which travels with the film node's own parameters. The
|
||||
application ships a read-only collection of presets, updated with each release and never
|
||||
written into the user's library; a user preset saved under a shipped name overrides it
|
||||
until deleted or renamed. Shipped and imported presets change only the operations they
|
||||
name, so a look applied to a corrected photograph keeps the correction; a copy or a saved
|
||||
edit replaces everything in scope.
|
||||
|
||||
**FR-DEV-7 — Before/after.** Compare current edit state against the unedited original or against
|
||||
a chosen history state.
|
||||
|
||||
@@ -544,9 +595,17 @@ decade; a mask that cannot be corrected is the reason an edit leaves this applic
|
||||
editor.
|
||||
|
||||
**FR-DEV-19a — Mask composition.** A layer's mask is an ordered list of parts, each naming a source
|
||||
and how it joins the mask before it — added to it, or taken out of it. A layer of one part is
|
||||
exactly the layer that existed before this, and reads and writes the same sidecar. The parts of a
|
||||
layer merge under FR-NC-9 as the layer does, and each carries its own edge treatment.
|
||||
and how it joins the mask before it — added to it, taken out of it, or intersected with it, keeping
|
||||
only where both agree. A layer of one part is exactly the layer that existed before this, and reads
|
||||
and writes the same sidecar. The parts of a layer merge under FR-NC-9 as the layer does, and each
|
||||
carries its own edge treatment.
|
||||
|
||||
*Intersection added 2026-09-24* (issue #9, `mask-editing.md` M2). Union and subtraction were built
|
||||
first; the selections that most need composing — the sky that is also bright, the subject that is
|
||||
also skin — are intersections, and spelling one as a subtraction needs a part that selects the
|
||||
complement. Intersection is the product of the two coverages, which is the minimum wherever either
|
||||
side is fully in or out. A sidecar written before it never names it; a build from before it reads
|
||||
the word as a union, which keeps the part visible rather than dropping it.
|
||||
|
||||
**FR-DEV-19b — Hand correction.** A part may be painted, with add and erase strokes, at a radius,
|
||||
hardness and flow the photographer sets. Strokes are stored as normalised source coordinates and
|
||||
@@ -569,6 +628,17 @@ which knows nothing of a layer's shaping and nothing at all about a gradient, a
|
||||
so choosing a subject or a category produced a layer whose extent was invisible, and every control
|
||||
in FR-DEV-19a and FR-DEV-19b acted on something the photographer could not see.
|
||||
|
||||
**FR-DEV-17 — A crop that orphans a mask says so.** When a crop is let go — the end of the drag,
|
||||
not any frame of it — and it has left a mask layer's coverage entirely or mostly outside the frame,
|
||||
the photographer is told how many layers, and which, and offered to keep the crop or take it back.
|
||||
The crop is applied either way and is never refused; taking it back is the ordinary undo, so the
|
||||
crop and its warning are one step in the history. A crop that strands nothing, and a layer already
|
||||
outside the frame before the crop moved, produce no notice at all.
|
||||
Mask geometry is stored in source coordinates, so a tighter crop does not destroy a layer: it makes
|
||||
it invisible, and nothing announces it. That silent loss is the whole case for cropping last; a
|
||||
notice at the moment it happens turns it into a decision, and lets the crop stay where the histogram
|
||||
argues it belongs — early.
|
||||
|
||||
**FR-DEV-16 — A keyboard vocabulary for develop.** Every develop gesture that can be reached from
|
||||
the keyboard is bound, tagged beside its implementation, and generated into the gesture book
|
||||
(FR-UI-4): stepping through the folder, fit and 1:1, holding the original, undo and redo, copying
|
||||
@@ -580,6 +650,28 @@ Editing rhythm depends on the hands staying put: reaching for a menu breaks the
|
||||
edit the same way it breaks the pace of a cull. A binding nobody can discover is the same as no
|
||||
binding, which is why the generated book is part of the requirement rather than documentation of it.
|
||||
|
||||
*Status (2026-09-24).* Met. Every binding named above is bound in develop's key handler and tagged
|
||||
beside it, and develop also answers zoom in, out, fit and 1:1, panning a magnified view, rating,
|
||||
flagging and labelling the open photograph, nudging the control last moved (the framing sliders
|
||||
included), turning a mask part's join, keeping a crop that hid a mask, and going back to the grid.
|
||||
"Cannot describe a binding the application does not have" is now checked rather than argued:
|
||||
`traces gestures-check` reads the chord every handler compares against (`Keys.chord`, in
|
||||
`ui/dr-ui/ui/keys.slint`) and fails when a bound key has no tag in its place or a tagged key has no
|
||||
handler (`tools/traceability/src/keymap.rs`).
|
||||
|
||||
**FR-DEV-20 — Perspective correction.** Framing shall include perspective correction — a vertical
|
||||
and a horizontal keystone — applied before the crop. It runs in the coordinate chain after the
|
||||
crop and the straightening and before the stored orientation and the lens warp, so that "vertical"
|
||||
means vertical in the photograph as it is shown and the lens is still corrected on the full frame
|
||||
it projected. It is part of framing and carries its `Compose` attribute, so a settings paste
|
||||
withholds it by default for the same reason it withholds the crop. The correction never exposes
|
||||
undefined area on its own; where it is combined with a straightening angle, the crop that avoids
|
||||
the empty corners (`max_inscribed_crop`) accounts for it, and the crop is refitted when the gesture
|
||||
ends.
|
||||
Converging verticals are the commonest geometric fault in architectural and interior work, and
|
||||
correcting them reframes the image — which is why the order relative to the crop is not a
|
||||
preference, and why the correction belongs to composition and not to the colour operations.
|
||||
|
||||
### 3.4 Display and interaction
|
||||
|
||||
**FR-DSP-1 — Proxy-resolution rendering.** The develop view renders at the resolution actually
|
||||
@@ -609,6 +701,12 @@ asynchronously, and the proxy result remains on screen until it is ready.
|
||||
quality or resolution, refining to full quality when interaction settles. Refinement is visually
|
||||
smooth, not a jarring swap.
|
||||
|
||||
*Status (2026-09-26).* Met in 0.15.0. A gesture renders half-resolution drafts, and the sharp frame
|
||||
lands once, 120 ms after the last movement rather than on a timer counted from the first
|
||||
(`ui/dr-ui/src/refine.rs`). While a draft is up the histogram's display reading is dimmed, since it
|
||||
describes the last settled frame; the last draft is kept over the sharp one and faded out in
|
||||
150 ms, so the refinement is not a swap.
|
||||
|
||||
**FR-DSP-5 — Zoom and pan.** Fit, 1:1, and arbitrary zoom levels. At 1:1 and above, the pipeline
|
||||
operates on the visible crop at full source resolution.
|
||||
|
||||
@@ -695,6 +793,15 @@ rating, and common adjustments. Neither is required for any operation to be reac
|
||||
> wherever they can be set, and on the roll's cells, so stepping along a set shows what has been
|
||||
> judged.
|
||||
|
||||
*Status (2026-09-24).* The keyboard half and the amendment are met; the pointer half is not yet.
|
||||
Keys cover navigation, rating and common adjustments in the grid, develop, the collections sidebar,
|
||||
People and the export and copy sheets, each one in the gesture book and held to its handler by the
|
||||
gestures gate. Rating and flagging work in develop on the open photograph without advancing, with
|
||||
stars and Pick/Reject in the top bar, and the roll's cells carry each frame's flag and stars; pick
|
||||
and reject also gained a pointer and touch route in the grid (Flag in the selection bar), where they
|
||||
had been keyboard-only. Outstanding: scroll-wheel adjustment on numeric controls — the develop
|
||||
sliders do not take the wheel.
|
||||
|
||||
**FR-UI-6 — Shared component library.** Touch and desktop presentations are variants of shared
|
||||
components, not parallel implementations. A new operation (FR-DEV-3c) becomes usable on both
|
||||
without frontend work.
|
||||
@@ -787,8 +894,16 @@ full-size TIFF alongside a 2048px sRGB JPEG.
|
||||
|
||||
**FR-EXP-6 — Naming and destination.** Output filenames are generated from a template supporting at
|
||||
minimum: original filename, sequence number, capture date, export dimensions, and preset name.
|
||||
Collision policy (overwrite / skip / auto-increment) is configurable. Destinations include a local
|
||||
path and a Nextcloud remote path (FR-NC-7).
|
||||
Collision policy (overwrite / skip / auto-increment) is configurable. The destination is an album
|
||||
(FR-EXP-10), whose folder is on this device or on the library's server (FR-NC-7) — never inside the
|
||||
library itself, where a scan would catalogue the exported files as photographs.
|
||||
|
||||
A folder is chosen by pointing, never by typing a path: the platform's own dialogue on the desktop
|
||||
(the XDG desktop portal on Linux, which reaches the user's files from inside the Flatpak; the common
|
||||
item dialogue on Windows), the Storage Access Framework's tree picker on Android, and an in-app
|
||||
browser for a folder on the server. Every one of them can make a new folder as well as open one.
|
||||
The same applies to the other folders the application asks for on the desktop — the library folder,
|
||||
an import's source and second copy, and presets brought across from Lightroom.
|
||||
|
||||
**FR-EXP-7 — Batch export.** Export a selection with one or more presets, running in the background
|
||||
with progress and cancellation. Uses all available cores and the GPU. A failure on one image is
|
||||
@@ -801,6 +916,20 @@ strip GPS and personal metadata. Copyright and contact fields are settable per-p
|
||||
regardless of what the display was showing — including the high-quality demosaic (FR-RAW-3), never
|
||||
the fast preview method.
|
||||
|
||||
**FR-EXP-10 — Albums.** An album is a named export destination listed in the library sidebar
|
||||
beneath the collections. Its folder holds only the exported files; the catalog records, for each
|
||||
file written there, the image it was rendered from, so that selecting the album shows the originals
|
||||
behind its files and re-exporting after an edit is one selection away. The export sheet chooses an
|
||||
album by name, and an export with no album chosen is refused and says so.
|
||||
|
||||
Albums sync between devices as collections do (FR-CAT-7): by uuid and revision, deletions as
|
||||
tombstones, exports as a set union keyed on the server's file id. A folder on the server syncs with
|
||||
the album; a folder on the device does not, because a path or a SAF grant on one device means
|
||||
nothing on another — an album made elsewhere arrives with no folder here until one is chosen.
|
||||
Deleting an album never deletes the files in its folder. The album tables are created on first use
|
||||
rather than by a schema migration, so a device on an older build keeps merging the rest of the
|
||||
catalog.
|
||||
|
||||
### 3.7 Nextcloud integration
|
||||
|
||||
Mechanics below are verified against Nextcloud 34 documentation and server/desktop-client source.
|
||||
@@ -1085,6 +1214,12 @@ protocol is unavailable, FR-DSP-8's stated fallback applies.
|
||||
portals and credential storage uses the Secret Service portal, both verified to satisfy FR-NC-2 and
|
||||
FR-CAT-1 within the sandbox.
|
||||
|
||||
*Status (2026-09-26).* Not verified, because no Flatpak has been built. Since 0.17.0 every folder the
|
||||
desktop asks for is chosen through the FileChooser portal (FR-EXP-6), so the filesystem half has the
|
||||
code it needs; whether the path the portal returns opens, persists and takes a sidecar inside the
|
||||
sandbox is unobserved ([distribution.md §4](distribution.md)). Credentials reach the session's secret
|
||||
daemon through a talk hole rather than the Secret Service portal, for the reason the manifest gives.
|
||||
|
||||
#### Windows
|
||||
|
||||
Specified in [windows.md](windows.md); a stated channel under NFR-COMPAT-2, not a v1 one.
|
||||
@@ -1201,6 +1336,21 @@ only from an input event. The evidence for a frame is visible in FR-CULL-4's mod
|
||||
cell without opening it, and a filter on any one signal returns exactly the set whose chips show
|
||||
it.
|
||||
|
||||
*Status (2026-09-26).* The write-path half is met; the evidence half is not. `cargo test -p
|
||||
traceability` enumerates every write of a rating, flag, colour label or trash membership in the
|
||||
shipped code — the catalog setters, any SQL that assigns those columns, the sidecar's judgement
|
||||
amendment and the fields that carry one — and holds each to a hand-written list
|
||||
(`tools/traceability/src/verdicts.rs`). Each listed write is a key, click or tap (a Slint `on_*`
|
||||
callback, checked structurally), a function writing for its caller, whose callers are then checked
|
||||
in turn, or a verdict carried from elsewhere: a sidecar or `.xmp` pull, the sync merge of two
|
||||
devices' sidecars, the catalog mirrored out to a file, and duplicates consolidation, which moves
|
||||
the copies' own verdicts onto the survivor and invents none. A new writer, including one in an
|
||||
evidence producer, fails the test until it is listed with a reason; `traces verdicts` prints the
|
||||
list. Choosing a burst's representative writes the grouping, not a verdict, and takes a press.
|
||||
Outstanding: evidence chips (clipping, focus, burst membership, face counts) on the grid cell and in
|
||||
FR-CULL-4's mode, shown as absent rather than zero, and a filter per signal. Eye state alone is
|
||||
shown today, on People's face cells and as a library filter.
|
||||
|
||||
### 3.9.1 People
|
||||
|
||||
Face recognition was deferred in §7 through the 2026-08-08 calibration. It is undeferred here in a
|
||||
@@ -2037,6 +2187,19 @@ pool, I/O pool, network — with stated thread counts and the invariant that **n
|
||||
occurs on the UI executor**. This is the mechanism behind R4 and NFR-P9, which currently assert an
|
||||
outcome with no stated means.
|
||||
|
||||
*Status (2026-09-26).* Named and guarded; not yet bounded. `dr_ui::executors` defines the five
|
||||
executors with the thread counts of architecture.md §7.1 and the reason for each, and every
|
||||
long-lived worker in `dr-ui` and the Android entry point is started through its `spawn`, which
|
||||
names the thread `<executor>:<role>` (`net:sync`, `decode:thumbs`). `run` marks its thread as the
|
||||
UI executor, and `net_runtime`'s `block_on` asserts in debug and test builds that it is not called
|
||||
there; `executors`' tests show the panic on a thread marked as the UI one and the same call passing
|
||||
on a worker. Outstanding: the counts are a stated budget, not a limit — each job still gets a
|
||||
thread of its own, and a pool sized from `Executor::threads` is the change NFR-ARCH-2's priorities
|
||||
need; the guard covers `block_on` only, not a synchronous file read or a catalog query on the UI
|
||||
thread, several of which the library and People screens make on a click by design (catalog.md
|
||||
§1); the two mask workers in `masks_ui.rs` still use `std::thread::spawn`; and threads the core
|
||||
crates start (the inference engine's reaper and probe, already named) are outside the module.
|
||||
|
||||
**NFR-ARCH-2 — Scheduler priority.** The tiling scheduler assigns priority classes, with
|
||||
visible-tile work **strictly preempting** background export and thumbnail work. Without this,
|
||||
NFR-P5's slider latency fails during a batch export — the common case, not an edge case.
|
||||
@@ -2107,6 +2270,9 @@ public channel — and it is the position until those weights are replaced.
|
||||
| Android | Signed release APK, sideloaded (`docker/android/assemble-apk.sh`, the release key of 2026-09-11) — **not** Play, **not** F-Droid | CI |
|
||||
| Windows | NSIS per-user installer cross-built from Linux (`docker/windows/`, FR-PLAT-WIN-3) | CI |
|
||||
|
||||
*Status (2026-09-26).* The Flatpak row is the decision, not yet the state: no workflow under
|
||||
`.gitea/workflows/` builds it, and none has been built by hand (FR-PLAT-LIN-3).
|
||||
|
||||
Two consequences the channel decision has on the requirements above it. ARCH §6.9's SAF-only
|
||||
storage model was written because Play would reject anything else; a sideloaded build *could*
|
||||
request broader permissions, and FR-PLAT-AND-1 keeps SAF anyway, because the design is right on
|
||||
|
||||
+69
-2
@@ -85,6 +85,9 @@ pub trait RemoteBackend: Send + Sync {
|
||||
// transfer
|
||||
async fn get(&self, id: &RemoteId, range: Option<Range<u64>>)
|
||||
-> Result<Vec<u8>, RemoteError>;
|
||||
async fn get_reporting(&self, id: &RemoteId, // defaulted
|
||||
progress: &(dyn Fn(u64, Option<u64>) + Send + Sync))
|
||||
-> Result<Vec<u8>, RemoteError>;
|
||||
async fn put(&self, path: &RemotePath, body: Vec<u8>, precond: Option<Precondition>)
|
||||
-> Result<Validator, RemoteError>;
|
||||
async fn put_many(&self, items: Vec<(RemotePath, Vec<u8>)>) // defaulted
|
||||
@@ -111,6 +114,16 @@ Rules that are not obvious from the signatures:
|
||||
- **`get` takes an optional range, and it is a hint.** A backend without cheap
|
||||
ranges may return the whole object; the caller slices. Correctness holds
|
||||
either way and `Capabilities::range_reads` says whether it was cheap.
|
||||
- **`get_reporting` is for the one transfer somebody is watching.** An original
|
||||
opened in develop is tens of megabytes, and "downloading" alone for that long
|
||||
reads as stuck. It reports the bytes received and the length the server
|
||||
declared, if it declared one. The default is `get` whole and one report at the
|
||||
end, which is right for a backend whose read is local; Nextcloud overrides it
|
||||
to read the body chunk by chunk. The develop view reads the figures by path
|
||||
from the in-flight registry in `ui/dr-ui/src/library/thumbnails_fetch.rs`,
|
||||
because a step along the roll usually lands on a frame the prefetcher is
|
||||
already fetching, and falls back on the catalog's file length when no length
|
||||
was declared.
|
||||
- **Chunked upload is not in the trait.** It is an implementation detail of
|
||||
`put`, chosen by body size. Exposing it would leak one server's protocol.
|
||||
- **`move_to` must preserve identity where the backend has stable ids.** This
|
||||
@@ -118,7 +131,9 @@ Rules that are not obvious from the signatures:
|
||||
allocates a new id, orphaning the thumbnail shard and turning a restore into a
|
||||
full re-download.
|
||||
- **`create_dir` makes parents and succeeds if the directory exists.** Callers
|
||||
use it to guarantee a destination, not to claim they created one.
|
||||
use it to guarantee a destination, not to claim they created one. It is also
|
||||
the server browser's `New folder` (§5.3), which then lists the parent again
|
||||
rather than inserting the name it asked for: the server may have normalised it.
|
||||
|
||||
### 3.2 `Capabilities` — what is cheap
|
||||
|
||||
@@ -195,6 +210,7 @@ pub trait BackendProvider: Send + Sync {
|
||||
fn endpoint_placeholder(&self) -> &'static str;
|
||||
fn sign_in(&self) -> SignIn;
|
||||
fn normalise_endpoint(&self, input: &str) -> Result<String, String>;
|
||||
fn upgrade_endpoint(&self, stored: &str) -> Option<String>; // defaulted: None
|
||||
fn account_for(&self, endpoint: &str) -> Result<Account, RemoteError>; // defaulted
|
||||
fn connect(&self, conn: &Connection) -> Result<Box<dyn RemoteBackend>, RemoteError>;
|
||||
}
|
||||
@@ -215,6 +231,18 @@ pub enum SignIn {
|
||||
Nextcloud provider upgrades `http://` to `https://` here (`NFR-SEC-3`); the
|
||||
folder provider canonicalises the path, so two spellings of one directory do
|
||||
not become two accounts indexing the same photographs.
|
||||
- **`upgrade_endpoint` is for accounts already saved,** and is not a second
|
||||
call to `normalise_endpoint`: the folder provider's needs the path to exist,
|
||||
so running it on every launch would fail a library on an unplugged disk.
|
||||
`AccountStore::upgrade_endpoints` runs it at launch; only Nextcloud answers,
|
||||
rewriting an `http://` account an older build saved to `https://`
|
||||
(#65). The keyring entry is filed under the endpoint, so the secret is
|
||||
copied across before the record changes, and a move that would change
|
||||
`namespace()` is refused rather than performed. Below both, the Nextcloud
|
||||
client sets `https_only`, so no URL it sends — a typed one, the login flow's
|
||||
poll endpoint, a redirect — can carry the app password in the clear, and
|
||||
the login flow keeps the address the user typed rather than the server's
|
||||
idea of its own URL.
|
||||
- **`connect` is synchronous and cheap.** It validates configuration and builds
|
||||
a client; it does not talk to the remote. Workers call it per task.
|
||||
- **`SignIn` is a shape, not a method.** It would be tidier to expose
|
||||
@@ -281,6 +309,7 @@ for a second backend:
|
||||
| `bulk_upload` | yes, `POST /remote.php/dav/bulk` |
|
||||
| `conditional_write` | yes, `If-Match` |
|
||||
| `server_previews` | `CommonFormatsOnly` — stock Nextcloud renders no RAW |
|
||||
| `get_reporting` | overridden: the body is read chunk by chunk, against `Content-Length` |
|
||||
| sign-in | `SignIn::Browser`, Login Flow v2, system browser, app password |
|
||||
|
||||
Also kept: the `oc:permissions` probe on a refused `PUT`, which is what
|
||||
@@ -302,7 +331,8 @@ which makes it the route that works on a machine with no secrets daemon at all.
|
||||
| `bulk_upload` | no |
|
||||
| `conditional_write` | yes, with a documented residual race |
|
||||
| `server_previews` | `None` |
|
||||
| sign-in | `SignIn::EndpointOnly` |
|
||||
| `get_reporting` | the default: a local read, reported once when it is done |
|
||||
| sign-in | `SignIn::EndpointOnly`, the folder chosen in the platform's dialogue (§5.3) |
|
||||
|
||||
**Why `LocalEtags` and not `PropagatingEtags`.** A POSIX directory's mtime
|
||||
changes when its own entry list changes and at no other time — not when a
|
||||
@@ -364,6 +394,43 @@ behind a network banner. An endpoint that is not a directory at all maps to
|
||||
`RemoteError::Configuration`: nothing was unreachable and no credential was
|
||||
wrong, so neither of the other two would send the user anywhere useful.
|
||||
|
||||
### 5.3 Choosing folders, and the folders exports go to
|
||||
|
||||
**Folders are pointed at, never typed** (FR-EXP-6). On the desktop,
|
||||
`ui/dr-ui/src/folder_dialog.rs` asks the platform: the XDG desktop portal's
|
||||
FileChooser on Linux, through `rfd`, and the common item dialogue on Windows.
|
||||
That is how the folder connector's endpoint is chosen on the launch screen, and
|
||||
the path it returns still goes through `normalise_endpoint` as a typed one did.
|
||||
A folder on the server is chosen in the in-app browser, which lists with `list`
|
||||
and makes a folder with `create_dir`; the launch screen and the album sheet
|
||||
share it through `ui/dr-ui/src/remote_folders.rs`, which keeps both round trips
|
||||
off the interface thread. Android has no filesystem dialogue, so its library
|
||||
folder is still typed, and an album's folder there comes from the Storage
|
||||
Access Framework's tree picker.
|
||||
|
||||
**An album is where exports go** (FR-EXP-10), and never inside the library: a
|
||||
JPEG written into the tree a scan catalogues comes back as a photograph beside
|
||||
the RAW it was made from. Its folder is one of two kinds
|
||||
(`dr_catalog::albums::Place`):
|
||||
|
||||
- **On the server,** relative to the *account* root, not the library root. The
|
||||
browser refuses a folder inside the library and says why; on a server whose
|
||||
whole account is the library, every folder is inside it, and it says that
|
||||
instead. Exports reach it through the outbox like any upload, and a queued
|
||||
file's `.dest` record gains a third line saying its folder is relative to
|
||||
the account — a third line rather than a leading slash, because a record
|
||||
written before albums may carry a stray slash and has to keep the meaning it
|
||||
was written with.
|
||||
- **On this device,** a path, or on Android a SAF tree URI with a persisted
|
||||
grant, written through `DocumentsContract` (`ui/dr-ui/src/saf.rs`). A
|
||||
provider renames on a collision by itself, so the album records the name it
|
||||
was given rather than the one asked for.
|
||||
|
||||
A server folder lives on the album row and syncs with it. A device folder lives
|
||||
in `album_folders`, which the merge never reads and the upload snapshot drops,
|
||||
because a path or a grant on one device means nothing on another: an album made
|
||||
on the desktop arrives on the tablet with no folder until one is chosen there.
|
||||
|
||||
---
|
||||
|
||||
## 6. Virtual filesystems
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user