Compare commits

..
15 Commits
Author SHA1 Message Date
dtourolle 48c5e74fa8 Release 0.18.1
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m46s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 49s
Build and test / Android (aarch64) (push) Successful in 17m27s
Build and test / android-image (push) Successful in 3s
🐳 Android image / Build and push (push) Successful in 3s
Build and test / Desktop (Linux) (push) Failing after 25m32s
Build and test / windows-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / Layer separation (push) Successful in 29s
Build and test / Windows (x86_64, cross) (push) Successful in 37m19s
Build and test / Publish the release (push) Skipped
2026-09-26 21:29:26 -04:00
dtourolle fc54523093 Apply a mask layer's settings as offsets to the global ones
A local adjustment ran as a second chain after every global operation,
then blended by the mask. So global contrast -30 with -20 on a face was
contrast -30, the rest of the chain, and contrast -20 again on the
result, rather than -50 where contrast runs. The two edits compounded in
ways neither slider showed; a flattening applied to an already
flattened picture is how the shadows of a night shot went magenta.

A layer's setting is now an offset from its default, added to the
global setting (clamped to the parameter's range; a moved switch or
choice replaces it) and run at that operation's own place in the chain.
At each operation the global fragment and each touching layer's
combined fragment read the same input colour, and the pixel moves by
each layer's weighted difference: c_g + sum w_i (c_i - c_g). At full
weight that is the combined setting exactly, at zero the global result
exactly, and no setting is applied twice. An offset that brings an
operation back to neutral emits an empty version, which undoes the
global setting inside the mask.

Blending the colours rather than the uniforms is deliberate: the tone
curve and colour mixer emit code only for the channels and bands that
are touched, so the global and combined versions of one operation need
not share a uniform set.

A photograph with no masks compiles to the same shader byte for byte.

Test: global -30 with a whole-frame layer at -20 renders within one
count of global -50.
2026-09-26 20:43:02 -04:00
dtourolle 8392cf772e Flatten contrast toward grey instead of scaling shadows by a ratio
Reducing contrast turned every black in a night photograph pink. The
fragment lifted each pixel's luminance to its target by multiplying the
colour by target/luma. For a pixel at 0.001 on the way to 0.09 that is
a gain of ninety, and in the deepest shadows the channels are sensor
noise: after white balance the red and blue noise sits above the green,
their multipliers being nearly twice its, so ninety times that noise is
magenta.

Flattening now mixes the colour toward middle grey, which gives the
same luminance and adds the lift as a neutral. A black goes to grey and
its noise stays the size it was.

The same fragment clamped luma/0.36 into the curve's 0..1 domain, which
scaled every tone above twice middle grey down to 0.36 in either
direction: contrast +10 took a 230 grey to 162. Those tones are now left
where they are, which is continuous with the curve's top (value 1,
slope 0).
2026-09-26 20:43:02 -04:00
dtourolle 515d4eb59e Release 0.18.0
Benchmarks / CPU and I/O (per commit) (push) Successful in 5m56s
Benchmarks / Frame budget (on demand) (push) Skipped
Traceability / Requirement traces (push) Successful in 45s
Build and test / Android (aarch64) (push) Successful in 17m8s
Build and test / android-image (push) Successful in 2s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / Desktop (Linux) (push) Successful in 48m1s
Build and test / windows-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / Layer separation (push) Successful in 36s
Build and test / Windows (x86_64, cross) (push) Successful in 21m35s
Build and test / Publish the release (push) Successful in 1m12s
2026-09-26 19:13:06 -04:00
dtourolle b2f3936a53 Say how synced faces and same-name people merge since #77 and #78
catalog.md §8.2 still said confirmed and rejected faces were matched to
local faces "by box", and that the merge reads only a remote face's box
and model; faces.md said `match_faces` "still matches by overlap alone
across devices". Since #77 the match falls back to embeddings, on the
photographs where a box leaves a remote face over, and since #78
`dedup_people` folds people of one name whose faces agree after every
sync. Both now say so, with the thresholds and the reason a less
decisive pair stays unmatched, taken from the code's own documentation.
2026-09-26 18:30:07 -04:00
dtourolle ec7a8c07ee Add a dedup_people example to measure the job on a catalog copy
`dedup_people COPY.sqlite` prints the listed people and faces before
and after, what the first run merged and kept apart and why, and the
time of three runs. The second and third runs are the cost the job
adds to every sync. `--peer PEER_COPY.sqlite` then plays two sync
round trips. The peer merges with the previous release's code path and
no job, this side merges back through sync::merge_remote, and the named
people each side lists are compared after every step.

On the reference pair: the desktop merges Claudine, Jessie x2, Mathias
and Noemi (80 -> 75 named). The tablet also merges its empty second
Ian (80 -> 74). The first run takes 1.2 s on the desktop, which is
building faces_box; the merge builds it first in practice. Later runs
take 8-15 ms.
2026-09-26 17:50:39 -04:00
dtourolle 78df4211b0 Run the people deduplication after every catalog merge (#78)
A merge is where two devices' people meet: the same name typed on each,
or a redirect one of them made. So dedup_people::run now follows every
successful sync::merge_remote. It runs on the sync worker, never the
UI thread, before the snapshot is pushed, so what it folds reaches the
server on the same pass.

It runs in its own transaction, and a failure is logged, not
returned. What the merge took is committed and valid either way, and
the next pass tries again. Once a catalog is clean it costs 8-15 ms on
the reference library (19k faces, 26k people rows). On copies of the
two real catalogs, a round trip with a peer running the previous merge
converges on 75 listed named people on both sides and stays there
over a second round. merge_remote with the job takes 75-90 ms there.
2026-09-26 17:50:38 -04:00
dtourolle c78b798cf0 Fold people of one name whose faces agree, and faces held twice (#78)
Seven names are two or three live people on both devices: Ian (756
confirmed faces, and a second Ian with none), Jessie three times,
Claudine, Mathias, Noemi, Pascal and PJ. Each was typed on its own
device and carried across by sync, which keys people on their uuid and
so keeps both. Each half of a person shows half their photographs.

dedup_people::run, in one transaction:

- Same-name people (trimmed, case-folded as the Identity screen folds
  them) merge into the one with the most confirmed faces, ties to the
  smaller uuid, through faces::merge_people_within, so confirmations,
  rejections and the survivor's name are kept. A person holding no
  faces at all merges: there is nothing to compare or to carry. Anyone
  else needs >= 2 confirmed faces per shared embedder on both sides and
  centroids at cosine >= 0.7 in each. A face confirmed as one and
  rejected as the other keeps them apart. Unnamed and set-aside people
  are never merged by name.
- Faces held twice (one image, one embedder, IoU >= 0.5, cosine >= 0.7)
  keep the stronger detector's row (FaceDetector::outranks), then the
  confirmed one, then the older. The survivor takes the confirmed
  assignment and both rows' rejections. A pair confirmed as two
  different people is left and counted.
- Judgements still on a merged-away person move to the person at the
  end of its redirects, and a redirect cycle (two devices merging one
  pair in opposite directions) is broken at the smaller uuid.

Measured on copies of the desktop catalog and the tablet's server
snapshot, w600k_mbf, confirmed faces only:
- Centroids of differently named people: 2,699 pairs, median 0.02,
  99.9th percentile 0.41. One pair reaches 0.70 (0.700 desktop, 0.705
  tablet), "Michelle Casanonve" and "Michelle Casanova", one person
  typed two ways. Next is 0.62/0.64, "Boris Jost" and "Boris". The
  highest pair that is plainly two people is 0.43/0.44.
- One person split in random halves: minimum 0.69, median 0.91 over 72
  people. Four faces against twenty-two reach 0.7 in 97% of draws.
  One face against twenty of somebody else's reached 0.74 in 3,000
  draws, and two faces reached 0.61, hence the two-face minimum.
- Pascal (22 and 4 confirmed) is at 0.57 and PJ (14 and 7) at 0.50,
  under 0.7 on both devices, so both pairs stay apart and are logged.
  The desktop's second Ian holds 4 suggestions and no confirmations,
  at 0.38 against Ian's centroid, and stays apart. On the tablet it
  holds nothing and merges.

Why a merge made here survives a peer on 0.17.0: the merged-away
person stays as a merged_into redirect with a bumped revision, which
the catalog merge has always taken on revision. The peer hides the
duplicate and never sends it back as a live person. Its own
confirmations of that person stay on the redirect, because a merge
never overwrites a local confirmation. The manual merge has always
left them there too. They follow the redirect when the peer runs this
job. A test syncs two catalog files through the previous merge code
and back, and the people converge and stay converged.

Once a catalog is clean the job reads 80 redirects, the named people,
and the face boxes from the covering faces_box index. That is ~10 ms
on the reference library. There is no schema change. The index is
created IF NOT EXISTS, as the merge already does.
2026-09-26 17:50:28 -04:00
dtourolle 6450f54199 Carry rejections when two people are merged
merge_people moved a person's faces onto the target and left the
"not this person" rejections on the redirect. A rejection there binds
nothing: once Annie is Anna, the grouping pass is free to suggest the
face the user pushed away from Annie as Anna, which is the behaviour
rejections exist to prevent. The Identity screen's merge has done this
since it was written, and the deduplication job for #78 merges through
the same function, so it would have done it for every same-name pair.

The source's rejections now move to the target (INSERT OR IGNORE, so
one the target already holds is not doubled). Where the two halves
disagree about one face, confirmed as one and rejected as the other,
the confirmation stands, as `confirm` already rules for one face; and a
moved rejection withdraws a suggestion of the same face, as `reject`
already does. Confirmations and names are unchanged.

The body is split into merge_people_within, taking the caller's
transaction, so a job that merges several pairs commits once
(unchecked_transaction cannot nest). merge_people keeps its signature
and its one transaction.
2026-09-26 17:41:55 -04:00
dtourolle 1479e45637 Keep one person to one face per photograph in the catalog merge
The grouping pass never puts two faces of one photograph in the same
group (the cannot-link in dr_face::cluster). The merge did not check
this. When the two devices disagree about which face in a frame is a
person, merge_people_within applied the remote's confirmation, or the
anchor of a set-aside group, to face X. This device already held the
same person on face Y of the same photograph, so the person ended up on
both faces.

The reference library has 80 such person/photograph pairs on the
desktop and 89 on the tablet: 79/88 unnamed set-aside groups and one
named person confirmed on two faces. There are no duplicate faces (no
pair of faces in one image and embedder with IoU >= 0.5).

An incoming assignment is now refused when another local face of the
same photograph already holds that person. The one exception is an
incoming confirmation against a local suggestion: the suggestion is
withdrawn and the confirmation is applied. Two confirmations stay as
this device has them, the same rule as a local confirmation outranking
a remote one. A face that already holds the person is not a rival to
itself, so a steady-state pass is unaffected. On the reference pair
this refuses 0 assignments and writes the same 8,954 as before; it only
changes what a future disagreement does. The refusals are counted in
MergeReport::faces_one_per_photograph.

Existing pairs are left alone. They are two different faces (cosine
0.31 for the named one), not one face twice, so there is nothing to
fuse, and which face is the wrong one is not the merge's to guess.
2026-09-26 16:54:05 -04:00
dtourolle b5b30e3750 Match synced faces by embedding where the boxes cannot (#77)
On the reference library, 631 of the faces in the tablet's snapshot
match no desktop face by box (IoU >= 0.5), so a name on them stays on
one device. Twenty of those are the same face with the box drawn
somewhere else. Whole photographs sit at IoU 0-0.48 with cosines of
0.72-0.96 between the two devices' vectors, and ten of them already
carry the same person on both sides. The merge never read the
1 KB embedding every face row carries.

match_faces now runs two passes. The box pass is unchanged except that
a pair must now be unique on both sides: a remote face with two
overlapping local faces, or a local face overlapped by two remote ones,
is no longer settled by whichever overlap is larger. Only for the
photographs where a remote face is left over, and a local face is still
free, does it read vectors: one json_each statement per side, keyed by
row id. That was 357 photographs on the reference library, not all
19 MB of vectors. A left-over face pairs with the local face it
resembles most when:
- the cosine is >= 0.7,
- the two are each other's best,
- each leads its runner-up by >= 0.2, and
- a box has not already claimed the local face.
Anything less decisive stays unmatched, so a new face stays new.

Why the threshold is safe, measured on both catalogs (w600k_mbf):
- Of 169,548 pairs of different faces in one photograph, 4 reach 0.7
  (lookalikes in one frame) and the maximum is 0.82.
- At 0.6 the rule would claim two pairs that carry different people on
  the two devices. At 0.7 it claims 20, none contradicted and 10
  corroborated, each leading its runner-up by more than 0.5.
- It only compares faces the boxes left unmatched on both sides: 737
  such pairs, so about 0.02 false pairs expected.
- A low cosine never overrules a box. About 150 box-matched pairs fall
  below 0.45, because two detectors cut the same tiny face differently.
  73 of them carry the same person on both devices.
- Faces are compared only within one file_id and one embedder, because
  the same person in another photograph reaches cosine 1.0.
- `Embedding::cosine` refuses a comparison across models.

Before -> after on the reference pair: matched by box 18,348 -> 18,348,
by embedding 0 -> 20, unmatched 631 -> 611, ambiguous 0 -> 0. The
report counts the embedding matches. No schema change.
2026-09-26 16:54:05 -04:00
dtourolle 884b681c21 Time a merge with another device's catalog in catalog_bench
The bench merged the catalog with a copy of itself. Every face in that
merge matches its own box, so the pass never reaches the faces the two
devices disagree about. On the reference library that is 631 of the
tablet's 19,052 faces, and it is the work #77 adds to.

`--remote PEER.sqlite` now also times `merge_remote_catalog` against a
copy of the peer file, and prints the first pass's report so two builds
can be checked for agreement. The first run writes what the peer
brought. The runs after it are the steady state, so compare builds from
two fresh copies of one catalog.
2026-09-26 16:54:05 -04:00
dtourolle 23abfd1827 Ship four presets for a bluer sky
Blue sky, Deep blue sky, Polariser, and Blue sky with golden land, in a
Skies section after Essentials. Each darkens the colour mixer's azure and
blue bands and adds chroma to them — what a polarising filter does to a
clear sky — and brings the highlights down with it, so a white cloud does
not read as a cut-out against the deeper blue. The stronger ones nudge
azure towards blue and add dehaze.

They are looks and work only on the hues a sky occupies, so an overcast
frame is left nearly alone: there is no blue for them to deepen, and
tinting grey cloud blue would be worse than doing nothing. Tuned by eye on
the demo library's alpine and Manhattan frames, with an overcast Étretat
frame as the control.
2026-09-26 16:46:46 -04:00
dtourolle 94ea2569ee Tag the SAF export path FR-EXP-10 only, not FR-PLAT-AND-1
saf.rs and the export path's SAF branch shipped tagged FR-PLAT-AND-1,
and the matrix counted the requirement as covered. Its subject is the
library — reached through SAF grants — and Android still reaches a
library over a Nextcloud account or a folder path. What the SAF code
does is give an album a folder on the tablet, which is FR-EXP-10.

outstanding.md said the figure overstated it and should be read with
this one subtracted; it now says the tags were narrowed, and coverage
reads 161 of 192.
2026-09-26 16:19:55 -04:00
dtourolle 7a56d16df1 Count the whole library in "All photographs" whatever is scoped (#76)
The sidebar's "All photographs" row was bound to library-total, which
is the scope's count: the header's "412 images" and the scrollbar's
size. Under a collection, and now under an album, the row read as the
album's size — 4 where the library holds 70.

It reads a separate library-whole-total now: the same number as the
scope's when nothing is scoped (no second count), and otherwise the
unscoped count under the same filter, read only when the view's facts
move, so scrolling inside an album does not recount the library.
2026-09-26 16:19:54 -04:00
30 changed files with 2908 additions and 315 deletions
Generated
+25 -25
View File
@@ -1265,7 +1265,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "darkroom-android"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"android_logger",
"dr-plat",
@@ -1278,7 +1278,7 @@ dependencies = [
[[package]]
name = "darkroom-desktop"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"anyhow",
"dr-plat",
@@ -1454,7 +1454,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]]
name = "dr-bench"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"anyhow",
"dr-catalog",
@@ -1471,7 +1471,7 @@ dependencies = [
[[package]]
name = "dr-catalog"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-face",
"dr-plat",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]]
name = "dr-decode"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-types",
"env_logger",
@@ -1500,7 +1500,7 @@ dependencies = [
[[package]]
name = "dr-export"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-decode",
"dr-gpu",
@@ -1519,7 +1519,7 @@ dependencies = [
[[package]]
name = "dr-face"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1532,7 +1532,7 @@ dependencies = [
[[package]]
name = "dr-film"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"log",
"serde",
@@ -1541,7 +1541,7 @@ dependencies = [
[[package]]
name = "dr-gpu"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"bytemuck",
"dr-decode",
@@ -1559,7 +1559,7 @@ dependencies = [
[[package]]
name = "dr-inference-engine"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"env_logger",
"libloading",
@@ -1574,7 +1574,7 @@ dependencies = [
[[package]]
name = "dr-ingest"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-plat",
"dr-types",
@@ -1586,7 +1586,7 @@ dependencies = [
[[package]]
name = "dr-lens"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"lensfun",
"log",
@@ -1594,7 +1594,7 @@ dependencies = [
[[package]]
name = "dr-pano"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-decode",
"dr-inference-engine",
@@ -1608,7 +1608,7 @@ dependencies = [
[[package]]
name = "dr-pipeline"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-types",
"log",
@@ -1617,7 +1617,7 @@ dependencies = [
[[package]]
name = "dr-plat"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"android-native-keyring-store",
"dr-types",
@@ -1633,7 +1633,7 @@ dependencies = [
[[package]]
name = "dr-preset-xmp"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-pipeline",
"log",
@@ -1643,7 +1643,7 @@ dependencies = [
[[package]]
name = "dr-segment"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1656,7 +1656,7 @@ dependencies = [
[[package]]
name = "dr-sync"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"async-trait",
"dr-plat",
@@ -1670,7 +1670,7 @@ dependencies = [
[[package]]
name = "dr-sync-folder"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"async-trait",
"dr-sync",
@@ -1682,7 +1682,7 @@ dependencies = [
[[package]]
name = "dr-sync-nextcloud"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"async-trait",
"dr-decode",
@@ -1704,7 +1704,7 @@ dependencies = [
[[package]]
name = "dr-thumbs"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-types",
"jpeg-encoder",
@@ -1716,7 +1716,7 @@ dependencies = [
[[package]]
name = "dr-types"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"serde",
"serde_json",
@@ -1725,7 +1725,7 @@ dependencies = [
[[package]]
name = "dr-ui"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"anyhow",
"async-trait",
@@ -1773,7 +1773,7 @@ dependencies = [
[[package]]
name = "dr-xmp"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"dr-types",
"log",
@@ -7109,7 +7109,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "traceability"
version = "0.17.0"
version = "0.18.1"
dependencies = [
"anyhow",
"pulldown-cmark",
+1 -1
View File
@@ -32,7 +32,7 @@ members = [
exclude = ["third_party"]
[workspace.package]
version = "0.17.0"
version = "0.18.1"
edition = "2021"
rust-version = "1.92"
license = "GPL-3.0-or-later"
+1 -1
View File
@@ -93,7 +93,7 @@ controls, its place in the chain and its tests.
## Where it stands
**0.17.0**, twenty-five tagged releases in. 192 numbered requirements in
**0.18.1**, twenty-seven tagged releases in. 192 numbered requirements in
scope, 84% of them claimed by code and [traced to it](docs/dev/traceability.md);
the rest are written down rather than merely absent.
+30 -2
View File
@@ -1,6 +1,6 @@
//! What the catalog's routine reads cost on a real library, off the GUI.
//!
//! cargo run --release -p dr-catalog --example catalog_bench -- CATALOG.sqlite [FACES_DIR]
//! cargo run --release -p dr-catalog --example catalog_bench -- CATALOG.sqlite [FACES_DIR] [--remote PEER.sqlite]
//!
//! Times `Catalog::open` — which every worker thread pays, including the
//! develop view's fetch of each original and each neighbour it prefetches —
@@ -13,6 +13,13 @@
//! count), whose `library` module is private; their SQL is spelled here as
//! it is spelled there, and has to be kept in step by hand.
//!
//! `--remote` also times a merge with another device's catalog — the
//! server snapshot — which is the pass where the two disagree: faces one
//! side found and the other did not, boxes that moved. The merge with a copy
//! of itself matches every face by its box and never reaches that work. The
//! first of its runs writes what the peer brought; the rest are the steady
//! state, so compare two builds from two fresh copies of one catalog.
//!
//! The figures are for reading side by side before and after a change; they
//! are not a gate. Compare the `cpu` column when the machine is busy. The
//! answers are printed too, so two builds can be checked for agreeing.
@@ -23,7 +30,15 @@ use std::time::{Duration, Instant};
use dr_catalog::{keywords, rating, schema, Catalog};
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
let mut args: Vec<String> = std::env::args().skip(1).collect();
let peer = args.iter().position(|a| a == "--remote").map(|at| {
let path = args.get(at + 1).map(PathBuf::from).unwrap_or_else(|| {
eprintln!("--remote needs a catalog");
std::process::exit(2);
});
args.drain(at..at + 2);
path
});
let Some(path) = args.first().map(PathBuf::from) else {
eprintln!("usage: catalog_bench CATALOG.sqlite");
std::process::exit(2);
@@ -117,6 +132,19 @@ fn main() {
let _ = std::fs::remove_file(&scratch);
let _ = std::fs::remove_file(&remote);
if let Some(peer) = &peer {
// A copy, so nothing the merge does to its input reaches the file
// the caller named.
std::fs::copy(peer, &remote).unwrap();
let mut first = None;
time("merge_remote_catalog (--remote)", 5, || {
let report = catalog.merge_remote_catalog(&remote).unwrap();
first.get_or_insert(report);
});
println!(" first pass: {first:?}");
let _ = std::fs::remove_file(&remote);
}
// The face half of a sync pass, against a copy of the face store: both
// directions in the steady state, where nothing is new either way.
if let Some(faces) = args.get(1).map(PathBuf::from) {
+141
View File
@@ -0,0 +1,141 @@
//! Run the people and face deduplication (#78) on a copy of a real catalog.
//!
//! cargo run --release -p dr-catalog --example dedup_people -- COPY.sqlite [--peer PEER_COPY.sqlite]
//!
//! It writes: run it against a *copy* (`sqlite3 catalog.sqlite ".backup
//! copy.sqlite"`), never the library's own file. Prints the live people and
//! faces before and after, what the first run merged and kept apart, and
//! how long the first and a second run took -- the second is the cost the
//! job adds to every sync once a catalog is clean.
//!
//! `--peer` then plays a sync round trip with another device's catalog (a
//! copy of the server snapshot, which it also writes): the peer merges this
//! one as the previous release would, with no job after it, then this one
//! merges the peer back through `sync::merge_remote`, twice. The named
//! people each side lists are printed after each step; they should agree.
use std::path::PathBuf;
use std::time::Instant;
use dr_catalog::{dedup_people, merge, schema, sync};
use rusqlite::Connection;
fn open(path: &std::path::Path) -> Connection {
let conn = Connection::open(path).expect("open the catalog copy");
schema::configure(&conn).expect("configure");
schema::migrate(&conn).expect("migrate");
conn
}
/// The named people a device lists, as `name (uuid prefix)`, sorted.
fn named(conn: &Connection) -> Vec<String> {
let mut v: Vec<String> = conn
.prepare(
"SELECT name, substr(uuid, 1, 8) FROM people
WHERE merged_into IS NULL AND trim(name) <> ''",
)
.unwrap()
.query_map([], |r| {
Ok(format!(
"{} ({})",
r.get::<_, String>(0)?,
r.get::<_, String>(1)?
))
})
.unwrap()
.collect::<Result<_, _>>()
.unwrap();
v.sort();
v
}
fn main() {
let mut args: Vec<String> = std::env::args().skip(1).collect();
let peer = args.iter().position(|a| a == "--peer").map(|at| {
let p = PathBuf::from(&args[at + 1]);
args.drain(at..at + 2);
p
});
let Some(path) = args.first().map(PathBuf::from) else {
eprintln!("usage: dedup_people COPY.sqlite [--peer PEER_COPY.sqlite]");
std::process::exit(2);
};
let conn = open(&path);
let counts = |label: &str| {
let q = |sql: &str| -> i64 { conn.query_row(sql, [], |r| r.get(0)).unwrap() };
println!(
"{label}: {} people listed ({} named), {} redirects, {} faces, {} confirmed",
q("SELECT COUNT(*) FROM people WHERE merged_into IS NULL"),
q("SELECT COUNT(*) FROM people WHERE merged_into IS NULL AND trim(name) <> ''"),
q("SELECT COUNT(*) FROM people WHERE merged_into IS NOT NULL"),
q("SELECT COUNT(*) FROM faces"),
q("SELECT COUNT(*) FROM face_person WHERE confirmed = 1"),
);
};
counts("before");
for pass in ["first", "second", "third"] {
let started = Instant::now();
let report = dedup_people::run(&conn).expect("dedup");
let took = started.elapsed();
println!("{pass} run: {took:?}, changed: {}", report.changed());
if pass == "first" {
println!(" merged: {:?}", report.merged);
for k in &report.kept_apart {
println!(
" kept apart: {:?} ({}) from {}: {:?}",
k.name, k.uuid, k.survivor, k.why
);
}
println!(
" redirects followed {}, cycles broken {}, faces fused {}, faces confirmed apart {}",
report.redirects_followed,
report.cycles_broken,
report.faces_fused,
report.faces_confirmed_apart
);
}
}
counts("after");
let Some(peer_path) = peer else { return };
let peer = open(&peer_path);
let show = |step: &str| {
let (ours, theirs) = (named(&conn), named(&peer));
println!(
"{step}: this device lists {} named, the peer {}; {}",
ours.len(),
theirs.len(),
if ours == theirs {
"the same".to_string()
} else {
format!("differ:\n here {ours:?}\n peer {theirs:?}")
}
);
};
show("before the round trip");
for round in 1..=2 {
peer.execute(
"ATTACH DATABASE ?1 AS remote_cat",
[path.to_string_lossy().as_ref()],
)
.unwrap();
let theirs = merge::merge_all(&peer).expect("the peer's merge");
peer.execute("DETACH DATABASE remote_cat", []).unwrap();
println!(
"round {round}: the peer took {} people updated, {} inserted",
theirs.people_updated, theirs.people_inserted
);
show(&format!("round {round}, after the peer's merge"));
let started = Instant::now();
let ours = sync::merge_remote(&conn, &peer_path).expect("our merge");
println!(
"round {round}: merge_remote with the job took {:?}; {} people updated, {} inserted",
started.elapsed(),
ours.people_updated,
ours.people_inserted
);
show(&format!("round {round}, after ours"));
}
}
File diff suppressed because it is too large Load Diff
+109 -7
View File
@@ -1029,7 +1029,8 @@ fn people_where(conn: &Connection, in_use: bool) -> Result<Vec<Person>, CatalogE
///
/// Confirmations survive the move: a face the user confirmed as the source
/// person is now a confirmed face of the target, which is what the user meant
/// by saying they are the same person.
/// by saying they are the same person. So do rejections — see
/// [`merge_people_within`].
pub fn merge_people(
conn: &Connection,
target: PersonId,
@@ -1039,7 +1040,53 @@ pub fn merge_people(
return Ok(0);
}
let tx = conn.unchecked_transaction()?;
let moved = merge_people_within(&tx, target, source)?;
tx.commit()?;
Ok(moved)
}
/// [`merge_people`] inside a transaction the caller holds, so a job that
/// merges several pairs commits once (`crate::dedup_people`).
///
/// **Rejections move with the faces.** "This face is not Annie" is a
/// judgement about the person, and once Annie is Anna it is one about Anna.
/// Left on the redirect it binds nothing, and the next grouping pass
/// suggests the face the user pushed away to the person it now belongs to. Where the
/// two halves disagree about one face — confirmed as one, rejected as the
/// other — the confirmation stands, which is the rule [`confirm`] applies
/// to one face; and a moved rejection takes a suggestion of the same face
/// with it, the rule [`reject`] applies.
pub(crate) fn merge_people_within(
tx: &Connection,
target: PersonId,
source: PersonId,
) -> Result<u64, CatalogError> {
if target == source {
return Ok(0);
}
let moved = move_judgements(tx, target, source)?;
tx.execute(
"UPDATE people SET merged_into = ?1, revision = revision + 1, modified = ?3
WHERE id = ?2",
rusqlite::params![target.0 as i64, source.0 as i64, now_secs()],
)?;
Ok(moved)
}
/// The half of [`merge_people_within`] that moves faces and rejections,
/// without touching either person's row.
///
/// Also what follows a redirect that arrived by sync
/// (`crate::dedup_people`): the other device merged the people, and this
/// one still holds judgements on the person merged away. Bumping the
/// person's revision there would be an edit of this device's own, sent back
/// on every pass, so the row is left as the merge wrote it.
pub(crate) fn move_judgements(
tx: &Connection,
target: PersonId,
source: PersonId,
) -> Result<u64, CatalogError> {
let (t, s) = (target.0 as i64, source.0 as i64);
// A face already assigned to the target must not gain a second row —
// `face_person` is keyed by face. Where both hold the same face, the
// target's row wins and the source's is dropped.
@@ -1047,18 +1094,31 @@ pub fn merge_people(
"DELETE FROM face_person
WHERE person_id = ?2
AND face_id IN (SELECT face_id FROM face_person WHERE person_id = ?1)",
rusqlite::params![target.0 as i64, source.0 as i64],
rusqlite::params![t, s],
)?;
let moved = tx.execute(
"UPDATE face_person SET person_id = ?1 WHERE person_id = ?2",
rusqlite::params![target.0 as i64, source.0 as i64],
rusqlite::params![t, s],
)?;
tx.execute(
"UPDATE people SET merged_into = ?1, revision = revision + 1, modified = ?3
WHERE id = ?2",
rusqlite::params![target.0 as i64, source.0 as i64, now_secs()],
"INSERT OR IGNORE INTO face_person_rejected (face_id, person_id)
SELECT face_id, ?1 FROM face_person_rejected WHERE person_id = ?2",
rusqlite::params![t, s],
)?;
tx.execute("DELETE FROM face_person_rejected WHERE person_id = ?1", [s])?;
tx.execute(
"DELETE FROM face_person_rejected
WHERE person_id = ?1
AND face_id IN (SELECT face_id FROM face_person
WHERE person_id = ?1 AND confirmed = 1)",
[t],
)?;
tx.execute(
"DELETE FROM face_person
WHERE person_id = ?1 AND confirmed = 0
AND face_id IN (SELECT face_id FROM face_person_rejected WHERE person_id = ?1)",
[t],
)?;
tx.commit()?;
Ok(moved as u64)
}
@@ -2518,6 +2578,48 @@ mod tests {
assert_eq!(people(&c).unwrap().len(), 1);
}
/// A rejection left on the redirect bound nothing: the next grouping
/// pass suggested the face to the merged person, whom the user had told
/// it was somebody else.
#[test]
fn merging_moves_the_rejections_too() {
let c = db();
let ids: Vec<FaceId> = (1..=3)
.map(|n| {
let img = image(&c, n);
record_detections(&c, img, "w600k_mbf", 1024, &[face(n as u8)]).unwrap()[0]
})
.collect();
let anna = create_person(&c, "Anna").unwrap();
let annie = create_person(&c, "Annie").unwrap();
// Rejected as Annie, and nothing said about Anna.
reject(&c, ids[0], annie).unwrap();
// Rejected as Annie, suggested as Anna: the rejection now covers it.
suggest(&c, ids[1], anna, 0.8).unwrap();
reject(&c, ids[1], annie).unwrap();
// Rejected as Annie, confirmed as Anna: the confirmation stands.
confirm(&c, ids[2], anna).unwrap();
reject(&c, ids[2], annie).unwrap();
merge_people(&c, anna, annie).unwrap();
let rejected: Vec<(i64, i64)> = c
.prepare("SELECT face_id, person_id FROM face_person_rejected ORDER BY face_id")
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.collect::<Result<_, _>>()
.unwrap();
let anna_id = anna.0 as i64;
assert_eq!(
rejected,
[(ids[0].0 as i64, anna_id), (ids[1].0 as i64, anna_id)]
);
assert_eq!(for_image(&c, ImageId(2)).unwrap()[0].person, None);
let kept = &for_image(&c, ImageId(3)).unwrap()[0];
assert_eq!((kept.person, kept.confirmed), (Some(anna), true));
}
#[test]
fn merging_does_not_duplicate_a_face_both_people_hold() {
let c = db();
+1
View File
@@ -42,6 +42,7 @@ pub mod bursts;
pub mod cache;
pub mod collections;
pub mod dedup;
pub mod dedup_people;
pub mod duplicates;
pub mod error;
pub mod face_shard;
+533 -49
View File
@@ -113,6 +113,12 @@ pub struct MergeReport {
pub faces_kept_local: usize,
/// "Not this person" judgements taken from the remote.
pub faces_rejected: usize,
/// Remote faces placed on a local one by their embedding, where the
/// boxes disagreed or were ambiguous (see `match_faces`).
pub faces_matched_by_embedding: usize,
/// Assignments from the remote refused because this device already has
/// that person on another face of the same photograph.
pub faces_one_per_photograph: usize,
/// Redundant identities for one word, retired by
/// [`crate::keywords::fuse_duplicates`].
@@ -973,15 +979,17 @@ fn attached_has_table(conn: &Connection, schema: &str, table: &str) -> Result<bo
/// # Faces have no cross-device identity, so one is derived
///
/// `faces.id` is a local row id and means nothing in another catalog; there is
/// no uuid to fall back on. What both devices *do* agree on is `oc:fileid` and
/// the box, so a remote face is matched to the local face on the same
/// photograph whose box overlaps it most, above a floor of 0.5 IoU.
/// no uuid to fall back on. What both devices *do* agree on is `oc:fileid`,
/// the box, and the embedding, so a remote face is matched to the local face
/// on the same photograph whose box overlaps it, above a floor of 0.5 IoU —
/// or, where no box or two boxes do, whose vector it decisively resembles
/// ([`match_faces`]).
///
/// That is not a new rule: it is the one
/// [`crate::faces::record_detections`] already uses to carry a confirmation
/// across a re-index, and it is loose on purpose — the question is "is this the
/// same face in the frame", not "is this the same rectangle", and a device
/// running a newer detector is entitled to have moved the box a little.
/// running a newer detector is entitled to have moved the box.
fn merge_people_within(tx: &Connection, report: &mut MergeReport) -> Result<(), CatalogError> {
// A remote written before faces existed has none of these tables, and one
// written before V10 has no `ignored`. Both are ordinary — `remote_is_
@@ -1076,7 +1084,14 @@ fn merge_people_within(tx: &Connection, report: &mut MergeReport) -> Result<(),
}
// ---- match the remote's faces onto this device's ----------------------
let face_map = match_faces(tx)?;
let matched = match_faces(tx)?;
report.faces_matched_by_embedding += matched.by_embedding;
let FaceMatch {
map: face_map,
on_file,
file_of,
..
} = matched;
if face_map.is_empty() {
return Ok(());
}
@@ -1122,6 +1137,8 @@ fn merge_people_within(tx: &Connection, report: &mut MergeReport) -> Result<(),
probability = excluded.probability,
confirmed = excluded.confirmed",
)?;
let mut withdraw =
tx.prepare_cached("DELETE FROM face_person WHERE face_id = ?1 AND confirmed = 0")?;
for (remote_face, person, probability, confirmed) in incoming {
let Some(&local_face) = face_map.get(&remote_face) else {
@@ -1148,6 +1165,37 @@ fn merge_people_within(tx: &Connection, report: &mut MergeReport) -> Result<(),
continue;
}
// One person, one face per photograph -- the cannot-link the
// grouping pass already keeps (`dr_face::cluster`), which the
// merge did not. When the two devices disagree about *which*
// face in a frame is somebody, taking the remote's answer
// beside this device's own puts the person on both. The
// reference library holds 80 such pairs (79 set-aside
// strangers, one named person), the same on both devices. The
// face this device already gave the person keeps them, unless
// the remote's is a confirmation and this device's only a
// suggestion.
//
// A face that already holds the person adds nothing beside it,
// whatever else the photograph holds.
let adds_person = current.is_none_or(|(held_person, ..)| held_person != person);
let rival = file_of
.get(&local_face)
.filter(|_| adds_person)
.and_then(|file| {
on_file[file].iter().copied().find(|&other| {
other != local_face && held.get(&other).is_some_and(|h| h.0 == person)
})
});
if let Some(rival) = rival {
if !confirmed || held[&rival].2 == 1 {
report.faces_one_per_photograph += 1;
continue;
}
withdraw.execute([rival])?;
held.remove(&rival);
}
// Written only when it differs. Rewriting a row with the values it
// already holds dirtied a page per face, every pass, for nothing;
// the report still counts it, as it always has.
@@ -1225,7 +1273,45 @@ fn remote_has_column(tx: &Connection, table: &str, column: &str) -> Result<bool,
Ok(stmt.exists(rusqlite::params![table, column])?)
}
/// Remote face row id to local face row id, by photograph and box overlap.
/// The cosine above which two vectors from one embedder, on one
/// photograph, on two devices, are taken to be the same face when the boxes
/// do not say so.
///
/// Measured on the reference library against the tablet's snapshot
/// (2026-09-26, #77), both `w600k_mbf`: of the 169,548 pairs of *different*
/// faces in one photograph, four reach 0.7 and none 0.83 -- lookalikes in
/// one frame, a parent and child. Of the 18,348 pairs the boxes match, 94%
/// are above 0.9; the tail below is one face cut by two detectors, which is
/// why this never overrules a box that matches on its own. At 0.6 two of
/// the pairs it would claim carry different people on the two devices; at
/// 0.7 none of the twenty it claims does, and ten carry the same person on
/// both. Stricter than [`crate::faces::SAME_FACE_COSINE`] because that one
/// is only asked about boxes that overlap, and this one is asked about boxes
/// that do not.
const SAME_FACE_ACROSS_DEVICES: f32 = 0.7;
/// How far a face's best counterpart must lead its second best, on both
/// sides, for the embedding to decide. A face that two others resemble
/// almost equally is exactly the one a merge must not guess at; every pair
/// the rule claims on the reference library leads by more than 0.5.
const DECISIVE_MARGIN: f32 = 0.2;
/// What [`match_faces`] found.
#[derive(Default)]
struct FaceMatch {
/// Remote face row id to local face row id; one-to-one.
map: std::collections::HashMap<i64, i64>,
/// Every local face on a synced photograph, by the photograph's
/// cross-device id -- what "another face in the same photograph" means
/// to the merge.
on_file: std::collections::HashMap<i64, Vec<i64>>,
/// The inverse of `on_file`.
file_of: std::collections::HashMap<i64, i64>,
/// Pairs the boxes could not settle and the embeddings did.
by_embedding: usize,
}
/// Remote face row id to local face row id, by photograph, box and vector.
///
/// See [`merge_people_within`] for why a face has no shared identity and this
/// has to be derived. Faces are compared within an *embedder*
@@ -1235,25 +1321,42 @@ fn remote_has_column(tx: &Connection, table: &str, column: &str) -> Result<bool,
/// rectangle — the same judgement `faces::record_detections` makes when it
/// carries a confirmation across a re-detection. Keying on the exact id was
/// what let a detector change strand every name on the device that made it.
fn match_faces(tx: &Connection) -> Result<std::collections::HashMap<i64, i64>, CatalogError> {
///
/// # Two passes, and the second is rare
///
/// **By box.** A remote face and a local one on the same photograph are the
/// same face when their boxes overlap by at least 0.5 IoU and neither has
/// another such candidate. That settles 18,348 of the reference library's
/// 19,052 remote faces, reads no vector, and is the whole of a steady-state
/// pass.
///
/// **By embedding**, only on the photographs where a remote face is left
/// over -- no box overlapped it, or two did. Their vectors are read (a few
/// hundred photographs, not the library's 19 MB of them) and a remote face is
/// paired with the local face it resembles most when the cosine is at least
/// [`SAME_FACE_ACROSS_DEVICES`], the pair is each other's best, each leads
/// its runner-up by [`DECISIVE_MARGIN`], and the local face was not already
/// claimed by a box. That is the face whose box one device drew somewhere
/// else -- twenty on the reference library, boxes at IoU 0 with cosines of
/// 0.72 to 0.96 -- and the face between two overlapping boxes. Anything less
/// decisive stays unmatched, which is what a new face is: a name that fails
/// to cross can be given again, a name put on the wrong face is a false
/// merge the user has to find.
fn match_faces(tx: &Connection) -> Result<FaceMatch, CatalogError> {
use std::collections::HashMap;
/// Loose on purpose — "the same face in the frame", not "the same
/// rectangle". The figure `record_detections` uses for the same job.
const MIN_IOU: f32 = 0.5;
type Boxed = (i64, f32, f32, f32, f32);
type Key = (i64, String);
ensure_face_box_index(tx);
// Local faces, grouped by the photograph's cross-device id.
let mut local: std::collections::HashMap<(i64, String), Vec<Boxed>> =
std::collections::HashMap::new();
{
let mut stmt = tx.prepare(
"SELECT f.id, r.file_id, f.model_id, f.x, f.y, f.w, f.h
FROM main.faces f
JOIN main.remote r ON r.image_id = f.image_id
WHERE r.file_id IS NOT NULL",
)?;
let read_boxes = |sql: &str| -> Result<HashMap<Key, Vec<Boxed>>, CatalogError> {
let mut out: HashMap<Key, Vec<Boxed>> = HashMap::new();
let mut stmt = tx.prepare(sql)?;
let rows = stmt.query_map([], |r| {
Ok((
r.get::<_, i64>(1)?,
@@ -1270,50 +1373,187 @@ fn match_faces(tx: &Connection) -> Result<std::collections::HashMap<i64, i64>, C
for row in rows {
let (file_id, model, boxed) = row?;
let embedder = crate::faces::embedder_of(&model).to_string();
local.entry((file_id, embedder)).or_default().push(boxed);
out.entry((file_id, embedder)).or_default().push(boxed);
}
Ok(out)
};
// Local faces, grouped by the photograph's cross-device id.
let local = read_boxes(
"SELECT f.id, r.file_id, f.model_id, f.x, f.y, f.w, f.h
FROM main.faces f
JOIN main.remote r ON r.image_id = f.image_id
WHERE r.file_id IS NOT NULL",
)?;
if local.is_empty() {
return Ok(FaceMatch::default());
}
let mut out = FaceMatch::default();
for ((file_id, _), faces) in &local {
let on = out.on_file.entry(*file_id).or_default();
for &(id, ..) in faces {
on.push(id);
out.file_of.insert(id, *file_id);
}
}
if local.is_empty() {
return Ok(Default::default());
}
let mut map = std::collections::HashMap::new();
let mut stmt = tx.prepare(
let remote = read_boxes(
"SELECT f.id, r.file_id, f.model_id, f.x, f.y, f.w, f.h
FROM remote_cat.faces f
JOIN remote_cat.remote r ON r.image_id = f.image_id
WHERE r.file_id IS NOT NULL",
)?;
let rows = stmt.query_map([], |r| {
Ok((
r.get::<_, i64>(0)?,
r.get::<_, i64>(1)?,
r.get::<_, String>(2)?,
(
r.get::<_, f64>(3)? as f32,
r.get::<_, f64>(4)? as f32,
r.get::<_, f64>(5)? as f32,
r.get::<_, f64>(6)? as f32,
),
))
})?;
for row in rows {
let (remote_id, file_id, model, rbox) = row?;
let embedder = crate::faces::embedder_of(&model).to_string();
let Some(candidates) = local.get(&(file_id, embedder)) else {
// ---- by box -----------------------------------------------------------
// Per group, which local face (by index) each remote face took.
let mut left_over: Vec<(&Key, Vec<Option<usize>>)> = Vec::new();
for (key, theirs) in &remote {
let Some(ours) = local.get(key) else {
continue;
};
let best = candidates
let overlaps: Vec<Vec<bool>> = theirs
.iter()
.map(|&(id, x, y, w, h)| (id, iou(rbox, (x, y, w, h))))
.filter(|&(_, score)| score >= MIN_IOU)
.max_by(|a, b| a.1.total_cmp(&b.1));
if let Some((local_id, _)) = best {
map.insert(remote_id, local_id);
.map(|&(_, x, y, w, h)| {
ours.iter()
.map(|&(_, lx, ly, lw, lh)| iou((x, y, w, h), (lx, ly, lw, lh)) >= MIN_IOU)
.collect()
})
.collect();
let mut taken: Vec<Option<usize>> = vec![None; theirs.len()];
for (i, row) in overlaps.iter().enumerate() {
let mut hits = row.iter().enumerate().filter(|(_, &hit)| hit);
let (Some((j, _)), None) = (hits.next(), hits.next()) else {
continue;
};
if overlaps.iter().filter(|other| other[j]).count() == 1 {
taken[i] = Some(j);
out.map.insert(theirs[i].0, ours[j].0);
}
}
// Worth reading vectors for only where a remote face is still
// unplaced and a local face is still free to be its counterpart.
let free = ours.len() > taken.iter().flatten().count();
if free && taken.iter().any(Option::is_none) {
left_over.push((key, taken));
}
}
Ok(map)
if left_over.is_empty() {
return Ok(out);
}
// ---- by embedding, for what the boxes left --------------------------
let wanted = |side: &HashMap<Key, Vec<Boxed>>| -> String {
let ids: Vec<String> = left_over
.iter()
.flat_map(|(key, _)| side[*key].iter().map(|b| b.0.to_string()))
.collect();
format!("[{}]", ids.join(","))
};
// One statement per side, keyed by row id, for the faces of those
// photographs only.
let read_vectors = |schema: &str, ids: String| -> Result<HashMap<i64, Vec<u8>>, CatalogError> {
let mut stmt = tx.prepare(&format!(
"SELECT f.id, f.embedding
FROM json_each(?1) j
JOIN {schema}.faces f ON f.id = j.value"
))?;
let rows = stmt.query_map([ids], |r| Ok((r.get(0)?, r.get(1)?)))?;
Ok(rows.collect::<Result<_, _>>()?)
};
let our_vectors = read_vectors("main", wanted(&local))?;
let their_vectors = read_vectors("remote_cat", wanted(&remote))?;
for (key, taken) in left_over {
let model = dr_face::ModelId::new(key.1.as_str());
let decode =
|vectors: &HashMap<i64, Vec<u8>>, faces: &[Boxed]| -> Vec<Option<dr_face::Embedding>> {
faces
.iter()
.map(|b| {
let blob = vectors.get(&b.0)?;
dr_face::Embedding::from_f16_bytes(model.clone(), blob)
})
.collect()
};
let (theirs, ours) = (&remote[key], &local[key]);
let pairs = pair_by_embedding(
&decode(&their_vectors, theirs),
&decode(&our_vectors, ours),
&taken,
);
for (i, j) in pairs {
out.map.insert(theirs[i].0, ours[j].0);
out.by_embedding += 1;
}
}
Ok(out)
}
/// The pairs the embeddings decide, as `(remote index, local index)`, for the
/// remote faces the boxes left unplaced (`taken[i] == None`).
///
/// Both sides are compared in full -- a local face a box already claimed can
/// still be a remote face's best resemblance, and then that remote face is
/// not placed elsewhere, because its best counterpart is spoken for and its
/// second best is not decisive. Vectors that are missing or of another
/// embedder compare as nothing ([`dr_face::Embedding::cosine`]).
fn pair_by_embedding(
theirs: &[Option<dr_face::Embedding>],
ours: &[Option<dr_face::Embedding>],
taken: &[Option<usize>],
) -> Vec<(usize, usize)> {
let cos: Vec<Vec<f32>> = theirs
.iter()
.map(|t| {
ours.iter()
.map(|o| match (t, o) {
(Some(t), Some(o)) => t.cosine(o).unwrap_or(f32::NEG_INFINITY),
_ => f32::NEG_INFINITY,
})
.collect()
})
.collect();
/// The index of the largest value, and by how much it leads the next.
fn best(values: impl Iterator<Item = f32>) -> Option<(usize, f32, f32)> {
let mut first: Option<(usize, f32)> = None;
let mut second = f32::NEG_INFINITY;
for (at, v) in values.enumerate() {
match first {
Some((_, top)) if v <= top => second = second.max(v),
_ => {
if let Some((_, top)) = first {
second = top;
}
first = Some((at, v));
}
}
}
first.map(|(at, top)| (at, top, second))
}
let decisive =
|top: f32, second: f32| top >= SAME_FACE_ACROSS_DEVICES && top - second >= DECISIVE_MARGIN;
let claimed: std::collections::HashSet<usize> = taken.iter().flatten().copied().collect();
let mut pairs = Vec::new();
for (i, row) in cos.iter().enumerate() {
if taken[i].is_some() {
continue;
}
let Some((j, top, second)) = best(row.iter().copied()) else {
continue;
};
if claimed.contains(&j) || !decisive(top, second) {
continue;
}
let Some((back, top, second)) = best(cos.iter().map(|row| row[j])) else {
continue;
};
if back == i && decisive(top, second) {
pairs.push((i, j));
}
}
pairs
}
/// The index the local half of [`match_faces`] is read from: every column
@@ -1332,7 +1572,7 @@ fn match_faces(tx: &Connection) -> Result<std::collections::HashMap<i64, i64>, C
/// extra index is invisible to them. The first merge after an upgrade pays
/// for building it, once. A failure is logged and the merge goes on reading
/// rows, as it did before.
fn ensure_face_box_index(tx: &Connection) {
pub(crate) fn ensure_face_box_index(tx: &Connection) {
if let Err(e) = tx.execute_batch(
"CREATE INDEX IF NOT EXISTS main.faces_box ON faces(image_id, model_id, x, y, w, h);",
) {
@@ -1341,7 +1581,7 @@ fn ensure_face_box_index(tx: &Connection) {
}
/// Intersection over union of two `(x, y, w, h)` boxes.
fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
pub(crate) fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
let x0 = a.0.max(b.0);
let y0 = a.1.max(b.1);
let x1 = (a.0 + a.2).min(b.0 + b.2);
@@ -2541,4 +2781,248 @@ mod tests {
.unwrap();
assert_eq!(people, 1);
}
// ── faces the boxes cannot place, and their vectors ───────────────────
/// A unit vector in the embedder's space, the same for the same seed.
/// Two seeds are near-orthogonal, as two strangers' faces are.
fn vector(seed: u32) -> Vec<f32> {
let mut s = seed.wrapping_mul(2_654_435_761).wrapping_add(1);
let mut v: Vec<f32> = (0..dr_face::EMBEDDING_DIM)
.map(|_| {
s = s.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
(s >> 8) as f32 / (1u32 << 23) as f32 - 0.5
})
.collect();
let norm = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter_mut().for_each(|x| *x /= norm);
v
}
/// Store `v` as `face`'s embedding, as the embedder would.
fn embed(c: &Connection, db: &str, face: i64, v: &[f32]) {
let e = dr_face::Embedding {
model: dr_face::ModelId::new("w600k_mbf"),
v: Box::new(v.try_into().unwrap()),
};
c.execute(
&format!("UPDATE {db}.faces SET embedding = ?2 WHERE id = ?1"),
rusqlite::params![face, e.to_f16_bytes()],
)
.unwrap();
}
/// Anna confirmed on the remote's face 42.
fn anna_on(c: &Connection, remote: i64) {
add_person(c, "remote_cat", 3, "u-anna", "Anna", false);
assign(c, "remote_cat", remote, 3, true);
}
/// The case #77 was opened for: one device drew the box somewhere else —
/// on the reference library, whole photographs whose boxes sit at IoU 0
/// with cosines above 0.9 — and the name stayed behind. The vector says
/// it is the same face.
#[test]
fn a_shifted_box_with_the_same_embedding_matches() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
}
let local = add_face(&c, "main", 7, 1, 0.10);
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
embed(&c, "main", local, &vector(1));
embed(&c, "remote_cat", remote, &vector(1));
anna_on(&c, remote);
let report = merge_all(&c).unwrap();
assert_eq!(report.faces_matched_by_embedding, 1);
assert_eq!(person_of(&c, local), Some(("Anna".to_string(), true)));
}
/// Two faces close enough that both boxes overlap the remote's by more
/// than half: the box cannot say which, and must not guess. The vector
/// can.
#[test]
fn two_overlapping_faces_are_told_apart_by_embedding() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
}
let front = add_face(&c, "main", 7, 1, 0.30);
let behind = add_face(&c, "main", 8, 1, 0.36);
let remote = add_face(&c, "remote_cat", 42, 1, 0.33);
embed(&c, "main", front, &vector(1));
embed(&c, "main", behind, &vector(2));
embed(&c, "remote_cat", remote, &vector(2));
anna_on(&c, remote);
merge_all(&c).unwrap();
assert_eq!(person_of(&c, behind), Some(("Anna".to_string(), true)));
assert_eq!(person_of(&c, front), None);
}
/// The same two overlapping boxes, and vectors that do not decide: the
/// face stays unmatched rather than going to the larger overlap.
#[test]
fn an_ambiguous_box_with_no_decisive_vector_stays_unmatched() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
}
let front = add_face(&c, "main", 7, 1, 0.30);
let behind = add_face(&c, "main", 8, 1, 0.35);
let remote = add_face(&c, "remote_cat", 42, 1, 0.33);
embed(&c, "main", front, &vector(1));
embed(&c, "main", behind, &vector(2));
// Equally like both: 0.71 each, no margin.
let between: Vec<f32> = vector(1)
.iter()
.zip(vector(2))
.map(|(a, b)| (a + b) / 2f32.sqrt())
.collect();
embed(&c, "remote_cat", remote, &between);
anna_on(&c, remote);
merge_all(&c).unwrap();
assert_eq!(person_of(&c, front), None);
assert_eq!(person_of(&c, behind), None);
}
/// The same person in another photograph has the same vector — the
/// worst lookalike there is — and is not the same face. Nor is a face
/// in the right photograph that neither box nor vector ties to it: that
/// is a face this device found and the other did not, and it stays new.
#[test]
fn a_similar_embedding_in_a_different_photograph_never_matches() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
add_synced_image(&c, db, 2, 6000);
}
let elsewhere = add_face(&c, "main", 7, 2, 0.10);
let stranger = add_face(&c, "main", 8, 1, 0.10);
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
embed(&c, "main", elsewhere, &vector(1));
embed(&c, "main", stranger, &vector(2));
embed(&c, "remote_cat", remote, &vector(1));
anna_on(&c, remote);
let report = merge_all(&c).unwrap();
assert_eq!(report.faces_matched_by_embedding, 0);
assert_eq!(person_of(&c, elsewhere), None, "matched across photographs");
assert_eq!(person_of(&c, stranger), None, "a new face was matched");
}
/// Vectors from two embedders live in two spaces; a cosine between
/// them is a number that means nothing.
#[test]
fn different_embedders_never_compare() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
}
let local = add_face(&c, "main", 7, 1, 0.10);
c.execute(
"UPDATE main.faces SET model_id = 'scrfd_10g+other_embedder' WHERE id = ?1",
[local],
)
.unwrap();
let remote = add_face(&c, "remote_cat", 42, 1, 0.60);
embed(&c, "main", local, &vector(1));
embed(&c, "remote_cat", remote, &vector(1));
anna_on(&c, remote);
merge_all(&c).unwrap();
assert_eq!(person_of(&c, local), None, "matched across embedders");
}
/// A local face its box already placed is not handed to a second remote
/// face because that one resembles it: one face, one counterpart.
#[test]
fn a_face_the_box_placed_is_not_taken_again_by_a_vector() {
let c = two_catalogs();
for db in ["main", "remote_cat"] {
add_synced_image(&c, db, 1, 5000);
}
let placed = add_face(&c, "main", 7, 1, 0.10);
let free = add_face(&c, "main", 8, 1, 0.70);
let by_box = add_face(&c, "remote_cat", 41, 1, 0.10);
let remote = add_face(&c, "remote_cat", 42, 1, 0.40);
embed(&c, "main", placed, &vector(1));
embed(&c, "main", free, &vector(3));
embed(&c, "remote_cat", by_box, &vector(2));
embed(&c, "remote_cat", remote, &vector(1));
anna_on(&c, remote);
merge_all(&c).unwrap();
assert_eq!(person_of(&c, placed), None);
assert_eq!(person_of(&c, free), None);
}
// ── one person, one face per photograph ───────────────────────────────
/// Two faces far apart in one photograph, one each side's remote
/// counterpart can be matched to by box: (local 7, local 8, remote 42
/// over 8).
fn two_faces_one_photograph(c: &Connection) -> (i64, i64, i64) {
for db in ["main", "remote_cat"] {
add_synced_image(c, db, 1, 5000);
}
let here = add_face(c, "main", 7, 1, 0.10);
let there = add_face(c, "main", 8, 1, 0.60);
let remote = add_face(c, "remote_cat", 42, 1, 0.60);
(here, there, remote)
}
/// The devices disagree about which stranger in a crowd a set-aside
/// group holds. Taking the remote's anchor beside this device's own put
/// one person on two faces of one frame.
#[test]
fn a_set_aside_anchor_does_not_land_beside_this_devices_own() {
let c = two_catalogs();
let (here, there, remote) = two_faces_one_photograph(&c);
add_person(&c, "main", 1, "u-stranger", "", true);
add_person(&c, "remote_cat", 3, "u-stranger", "", true);
assign(&c, "main", here, 1, false);
assign(&c, "remote_cat", remote, 3, false);
let report = merge_all(&c).unwrap();
assert_eq!(report.faces_one_per_photograph, 1);
assert_eq!(person_of(&c, here), Some((String::new(), false)));
assert_eq!(person_of(&c, there), None);
}
/// A confirmation from the other device outranks a suggestion here for
/// the same person on another face, which gives the person up.
#[test]
fn a_remote_confirmation_moves_a_local_suggestion_off_the_other_face() {
let c = two_catalogs();
let (here, there, remote) = two_faces_one_photograph(&c);
add_person(&c, "main", 1, "u-anna", "Anna", false);
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
assign(&c, "main", here, 1, false);
assign(&c, "remote_cat", remote, 3, true);
merge_all(&c).unwrap();
assert_eq!(person_of(&c, there), Some(("Anna".to_string(), true)));
assert_eq!(person_of(&c, here), None);
}
/// Two confirmations of one person on two faces of one photograph is a
/// disagreement no merge can settle; this device's stands, and the
/// second is not added beside it.
#[test]
fn a_remote_confirmation_does_not_double_a_local_one() {
let c = two_catalogs();
let (here, there, remote) = two_faces_one_photograph(&c);
add_person(&c, "main", 1, "u-anna", "Anna", false);
add_person(&c, "remote_cat", 3, "u-anna", "Anna", false);
assign(&c, "main", here, 1, true);
assign(&c, "remote_cat", remote, 3, true);
let report = merge_all(&c).unwrap();
assert_eq!(report.faces_one_per_photograph, 1);
assert_eq!(person_of(&c, here), Some(("Anna".to_string(), true)));
assert_eq!(person_of(&c, there), None);
}
}
+12
View File
@@ -344,6 +344,18 @@ pub fn merge_remote(conn: &Connection, remote: &Path) -> Result<MergeReport, Cat
log::warn!("failed to detach remote catalog: {e}");
}
// After every merge, because a merge is where two devices' people meet:
// the same name typed on each, or a redirect one of them made. Its own
// transaction, and a failure is logged rather than returned -- what the
// merge took is committed and valid whether or not the duplicates were
// folded, and the next pass tries again. Runs on the sync worker, never
// the UI thread, and costs ~10 ms when there is nothing to do.
if result.is_ok() {
if let Err(e) = crate::dedup_people::run(conn) {
log::warn!("dedup after the catalog merge: {e}");
}
}
result
}
+79
View File
@@ -0,0 +1,79 @@
//! Contrast, on a device, at the two ends of the tonal range.
//!
//! The descriptor tests check that the fragment says the right words; these
//! check what those words do to a pixel. Both failures here passed every
//! descriptor test for months, because each is a property of the arithmetic
//! at the extremes rather than of the shape of the code.
use dr_gpu::{AdjustPass, DemosaicedImage, GpuContext};
use dr_pipeline::descriptor::ParamId;
use dr_pipeline::operation::compose;
use dr_pipeline::ops;
const SIZE: u32 = 8;
fn ctx() -> Option<GpuContext> {
pollster::block_on(GpuContext::new_headless()).ok()
}
/// A flat frame of one sRGB colour, contrast set to `amount`, rendered and
/// read back as the colour of one pixel.
fn render(ctx: &GpuContext, rgb: [u8; 3], amount: f32) -> [u8; 3] {
let data: Vec<u8> = (0..SIZE * SIZE)
.flat_map(|_| [rgb[0], rgb[1], rgb[2], 255])
.collect();
let source = DemosaicedImage::from_rgba8(ctx, &data, SIZE, SIZE).expect("upload");
let mut chain = ops::chain();
let op = chain
.iter_mut()
.find(|o| o.descriptor().id.0 == "contrast")
.expect("contrast is in the chain");
op.set_param(ParamId("contrast"), amount);
let shader = compose(&chain);
let mut adjust = AdjustPass::new(ctx);
adjust.render(&source, &shader, SIZE, SIZE).expect("render");
let pixels = adjust.export_pixels().expect("readback").0;
[pixels[0], pixels[1], pixels[2]]
}
/// **The pink-blacks bug.** A near-black pixel whose red and blue sit a count
/// above its green — what white-balanced sensor noise in a night shadow looks
/// like — must come out grey when contrast is reduced, not magenta.
///
/// The ratio form lifted it by a gain of well over a hundred, and a hundred
/// times a one-count cast is a saturated colour.
#[test]
fn reducing_contrast_lifts_a_black_to_grey_not_to_magenta() {
let Some(ctx) = ctx() else {
eprintln!("no GPU adapter; skipping");
return;
};
let [r, g, b] = render(&ctx, [4, 1, 4], -50.0);
let spread = r.max(g).max(b) - r.min(g).min(b);
assert!(
g > 40,
"a black at half contrast should be lifted toward grey, got ({r}, {g}, {b})"
);
assert!(
spread <= 6,
"the lift must be neutral: ({r}, {g}, {b}) has a cast of {spread}"
);
}
/// **The pinned highlights.** A light tone, above twice middle grey, must not
/// be pulled down to the top of the curve by the smallest positive contrast.
#[test]
fn a_little_contrast_leaves_a_highlight_where_it_was() {
let Some(ctx) = ctx() else {
eprintln!("no GPU adapter; skipping");
return;
};
let before = render(&ctx, [230, 230, 230], 0.0)[1];
let after = render(&ctx, [230, 230, 230], 10.0)[1];
assert!(
after >= before.saturating_sub(2),
"contrast +10 took a highlight from {before} to {after}"
);
}
+73
View File
@@ -1135,3 +1135,76 @@ fn two_shown_masks_are_drawn_each_in_its_own_colour() {
"between them, alpha shows black: ({r}, {g}, {b})"
);
}
/// Render a mid-grey-and-shadows frame through `chain` and `stack`.
fn render_chain(
ctx: &GpuContext,
chain: &[Box<dyn dr_pipeline::operation::Operation>],
stack: &MaskStack,
field: Option<&LabelField>,
) -> Vec<u8> {
// A ramp, so both ends of the tonal range are in the comparison.
let data: Vec<u8> = (0..SIZE * SIZE)
.flat_map(|i| {
let v = ((i % SIZE) * 255 / (SIZE - 1)) as u8;
[v, v / 2, v, 255]
})
.collect();
let source = DemosaicedImage::from_rgba8(ctx, &data, SIZE, SIZE).expect("upload");
let shader = compose_full(
chain,
&Framing::new(),
ColourSpace::Srgb,
stack,
&SpotSet::new(),
&[],
);
let mut masks = MaskPass::new(ctx).expect("mask pass");
let array = masks
.render(stack, field, None, None, SIZE, SIZE)
.expect("rasterise");
let mut adjust = AdjustPass::new(ctx);
adjust
.render_masked(&source, &shader, SIZE, SIZE, Some(array))
.expect("render");
adjust.export_pixels().expect("readback").0
}
fn contrast_chain(v: f32) -> Vec<Box<dyn dr_pipeline::operation::Operation>> {
let mut chain = ops::chain();
chain
.iter_mut()
.find(|o| o.descriptor().id.0 == "contrast")
.expect("contrast")
.set_param(ParamId("contrast"), v);
chain
}
/// A layer's setting is an offset to the global one, applied once: global
/// −30 with a whole-frame layer at −20 is exactly global −50 — not −30 and
/// then −20 again on the result, which is what a layer used to do.
#[test]
fn a_whole_frame_layer_adds_its_setting_to_the_global_one() {
let Some(ctx) = ctx() else {
eprintln!("no GPU adapter; skipping");
return;
};
let mut layer = MaskLayer::new("m1", whole_frame());
layer.set_param("contrast", ParamId("contrast"), -20.0);
let mut stack = MaskStack::new();
stack.push(layer);
let field = split_field(&ctx);
let offset = render_chain(&ctx, &contrast_chain(-30.0), &stack, Some(&field));
let direct = render_chain(&ctx, &contrast_chain(-50.0), &MaskStack::new(), None);
let worst = offset
.iter()
.zip(&direct)
.map(|(a, b)| a.abs_diff(*b))
.max()
.unwrap();
assert!(
worst <= 1,
"layer offset differs from the summed setting by {worst}"
);
}
+62 -28
View File
@@ -35,41 +35,62 @@ helpers: [luminance, apply_tone_gain]
define:
contrast_curve: |
// A symmetric S-curve on a 0..1 perceptual position.
// The steepening S, on a 0..1 perceptual position.
//
// `amount` above zero steepens, below zero flattens. The smoothstep form is
// used for the steepening direction because it has zero gradient at both
// ends, so the curve cannot invert however hard it is pushed — the failure
// that makes naive gain-about-a-pivot unusable past moderate settings.
// Blends toward a smoothstep, which has zero gradient at both ends, so the
// curve cannot invert however hard it is pushed — the failure that makes
// naive gain-about-a-pivot unusable past moderate settings. Only the
// positive direction comes here: flattening is not a curve at all (see
// the fragment).
fn contrast_curve(x: f32, amount: f32) -> f32 {
let clamped = clamp(x, 0.0, 1.0);
if (amount >= 0.0) {
// Blend toward a smoothstep, which is the S.
let s = clamped * clamped * (3.0 - 2.0 * clamped);
return mix(clamped, s, amount);
}
// Flattening: pull toward the mid-point. At amount = -1 every tone
// collapses to 0.5, which is the meaningful limit of 'no contrast'.
return mix(clamped, 0.5, -amount);
let s = clamped * clamped * (3.0 - 2.0 * clamped);
return mix(clamped, s, amount);
}
wgsl: |
let luma = luminance(c);
if (luma > 0.0001) {
// Work on luminance and rescale the colour by the ratio, rather than
// curving each channel independently. Per-channel contrast shifts hue
// wherever the channels differ — the classic symptom being skies going
// cyan as contrast rises.
if (amount < 0.0) {
// **Flattening mixes toward middle grey; it does not scale.**
//
// MIDDLE_GREY is 0.18: the linear value the eye reads as mid-tone. The
// curve operates on luma/(2*0.18) so that middle grey lands at the
// curve's own 0.5 pivot.
let pos = clamp(luma / 0.36, 0.0, 1.0);
let curved = contrast_curve(pos, amount);
// Not `target`: that is a WGSL reserved keyword, and using it produces a
// parse error in generated code rather than anywhere a reader would look.
let curved_luma = curved * 0.36;
c = apply_tone_gain(c, curved_luma / luma);
// Every tone moves the same fraction of the way to 0.18, which at -1
// collapses the picture to grey — the meaningful limit of 'no contrast'.
// In luminance this is exactly what the ratio form below would compute,
// but the ratio form reaches it by multiplying: a pixel at 0.001 has to
// be lifted to 0.09, a gain of ninety, and in the deepest shadows the
// channels are sensor noise, not a colour. After white balance the red
// and blue noise sits above the green (their multipliers are nearly
// twice its), so ninety times that noise is magenta — every black in the
// frame turned pink. Mixing adds the lift as a neutral, so a black goes
// to grey and its noise stays the size it was.
//
// The grey is (1, 1, 1) scaled, because this runs after white balance
// in the camera's space, where that is what neutral is.
c = mix(c, vec3<f32>(0.18), -amount);
} else {
let luma = luminance(c);
// Only up to twice middle grey, which is the curve's whole domain. Above
// it the curve's value is 1 and its slope 0, so leaving those tones
// alone is the continuous continuation — where scaling them to the
// curve's top, as this once did through a clamp, pinned every highlight
// in the photograph to 0.36 at the smallest touch of the slider.
if (luma > 0.0001 && luma < 0.36) {
// Work on luminance and rescale the colour by the ratio, rather than
// curving each channel independently. Per-channel contrast shifts hue
// wherever the channels differ — the classic symptom being skies going
// cyan as contrast rises. Safe here where it was not for flattening:
// the S only ever pulls a shadow down, so the gain is at most one
// below the pivot and noise is never amplified.
//
// MIDDLE_GREY is 0.18: the linear value the eye reads as mid-tone.
// The curve operates on luma/(2*0.18) so that middle grey lands at
// the curve's own 0.5 pivot.
let pos = luma / 0.36;
let curved = contrast_curve(pos, amount);
// Not `target`: that is a WGSL reserved keyword, and using it produces a
// parse error in generated code rather than anywhere a reader would look.
let curved_luma = curved * 0.36;
c = apply_tone_gain(c, curved_luma / luma);
}
}
c = max(c, vec3<f32>(0.0));
@@ -104,6 +125,19 @@ tests:
propagate through everything downstream.
expect_wgsl: ["luma > 0.0001"]
- name: flattening_mixes_toward_grey_rather_than_scaling
why: |
Lifting a shadow by a luminance ratio multiplies its noise by the same
ratio — ninety at the bottom of a night photograph — and after white
balance that noise is magenta. A mix adds the lift as a neutral.
expect_wgsl: ["mix(c, vec3<f32>(0.18), -amount)"]
- name: highlights_are_not_pinned_to_the_top_of_the_curve
why: |
The curve covers 0..0.36. A clamp into that range scaled every brighter
pixel down to 0.36; tones above it are left as they are.
expect_wgsl: ["luma < 0.36"]
- name: the_curve_cannot_invert
why: |
A gain-about-a-pivot form produces a non-monotonic curve past moderate
+52
View File
@@ -0,0 +1,52 @@
drpl 1
# Skies: a bluer, deeper sky without touching the rest of the picture.
#
# Written against this pipeline, not derived from anybody's preset. Each
# works the colour mixer's azure and blue bands — the hues a clear sky
# occupies, 210° and 240° — darkening them and adding chroma, which is what
# a polarising filter does to a sky and why it reads as "more blue" rather
# than "more saturated". A grey sky has no hue for the bands to find, so on
# an overcast frame these do little, by construction: they cannot invent a
# sky, and a preset that tinted grey clouds blue would be one nobody trusted.
#
# Highlights come down with the sky in the stronger ones, because a darker
# blue beside a clipped white cloud looks like a mask edge.
[preset Blue sky]
colour_mixer.azure_lum = -20
colour_mixer.azure_sat = 25
colour_mixer.blue_lum = -15
colour_mixer.blue_sat = 20
highlights_shadows.highlights = -15
[preset Deep blue sky]
colour_mixer.azure_hue = 10
colour_mixer.azure_lum = -30
colour_mixer.azure_sat = 35
colour_mixer.blue_lum = -25
colour_mixer.blue_sat = 30
colour_mixer.cyan_sat = 10
highlights_shadows.highlights = -30
[preset Polariser]
colour_mixer.azure_hue = 10
colour_mixer.azure_lum = -35
colour_mixer.azure_sat = 40
colour_mixer.blue_lum = -30
colour_mixer.blue_sat = 35
colour_mixer.cyan_lum = -10
colour_mixer.cyan_sat = 15
dehaze.amount = 20
highlights_shadows.highlights = -35
vibrance.vibrance = 10
[preset Blue sky, golden land]
colour_mixer.azure_lum = -20
colour_mixer.azure_sat = 25
colour_mixer.blue_lum = -15
colour_mixer.blue_sat = 20
colour_mixer.orange_sat = 12
colour_mixer.yellow_hue = -10
colour_mixer.yellow_sat = 15
highlights_shadows.highlights = -20
+1
View File
@@ -62,6 +62,7 @@ const SECTIONS: &[(&str, &str, &str)] = &[
"Essentials",
include_str!("../presets/essentials.drpl"),
),
("skies", "Skies", include_str!("../presets/skies.drpl")),
(
"colour_film",
"Colour film",
+291 -82
View File
@@ -73,7 +73,7 @@ use std::fmt::Write as _;
use std::sync::Arc;
use crate::coverage::Coverage;
use crate::descriptor::{Attribute, OpDescriptor, ParamId};
use crate::descriptor::{Attribute, OpDescriptor, ParamId, ParamKind};
use crate::operation::Operation;
use crate::ops;
@@ -1368,17 +1368,19 @@ pub struct MaskLayer {
/// This layer's adjustments.
///
/// A full chain, the same one [`crate::EditGraph`] holds. That is the
/// whole reason local adjustments need no per-operation support: the
/// composer already knows how to turn a chain into WGSL, and a mask layer
/// is a chain that happens to be multiplied by a mask afterwards.
/// whole reason local adjustments need no per-operation support: each
/// setting here is an offset from its default, added to the global chain's
/// setting and run where that operation runs, weighted by the mask (see
/// [`offset_onto`]).
pub ops: Vec<Box<dyn Operation>>,
}
/// The chain a mask layer holds: every point operation, and neither the
/// neighbourhood ones nor the optical corrections.
///
/// A layer's adjustments are fused into the colour dispatch and multiplied by
/// the mask afterwards, which is exactly why a layer needs no per-operation
/// A layer's adjustments are fused into the colour dispatch, each beside the
/// global operation it offsets and weighted by the mask, which is exactly why
/// a layer needs no per-operation
/// support — the composer already knows how to turn a chain into WGSL. A
/// neighbourhood operation cannot go through that path at all: it runs as its
/// own dispatch in [`crate::detail`], after the fused pass and after the masks
@@ -1642,8 +1644,8 @@ impl MaskLayer {
/// The operations in this layer's chain that reach the shader.
///
/// Neighbourhood operations are excluded, and not as an oversight. A
/// layer's chain is *fused into the point-operation pass* and multiplied
/// by the mask afterwards; the detail stage runs once, over the whole
/// layer's chain is *fused into the point-operation pass*, weighted by the
/// mask at each operation; the detail stage runs once, over the whole
/// frame, after that pass has finished (see [`crate::detail`]). There is
/// nowhere in that arrangement for a sharpening confined to one mask to
/// happen, so a detail operation in a layer would contribute an empty
@@ -1975,7 +1977,9 @@ impl MaskStack {
Some(self.layers.remove(i))
}
/// Reorder, since later layers composite over earlier ones.
/// Reorder. Layers add their changes, so order no longer decides the
/// picture — but it is the order the panel lists them in and the order
/// their slots are assigned.
pub fn move_to(&mut self, id: &str, index: usize) {
let Some(from) = self.layers.iter().position(|l| l.id == id) else {
return;
@@ -2041,25 +2045,83 @@ impl MaskStack {
}
}
/// One layer's contribution to the generated shader.
/// The layers' contribution to the generated shader.
///
/// Not a block of its own any more. A layer's adjustments are *offsets to the
/// global ones*, applied at each operation's own place in the chain, so what
/// this hands back is pieces the composer threads through its loop over the
/// global operations: the weights, sampled once before the first operation,
/// and one [`LocalOp`] per layer per operation the layer moved.
pub(crate) struct LayerShader {
pub uniform_fields: String,
pub uniform_values: Vec<f32>,
pub body: String,
/// Each layer's shaped mask, `mask_w{slot}`, sampled once ahead of the
/// operations that read it. Empty when no layer changes a pixel.
pub weights: String,
/// Every layer's version of every operation it moved, in layer order and
/// then chain order — see [`LocalOp`].
pub ops: Vec<LocalOp>,
pub helpers: Vec<crate::operation::Helper>,
/// TRACES: FR-DEV-19c
/// The block that draws one layer's mask over the finished picture, empty
/// when nothing is being revealed.
///
/// Kept apart from `body` because it belongs at the other end of the
/// shader. Everything in `body` runs on scene-referred colour in the
/// working space, where a flat tint would then be pushed through the base
/// curve and the camera matrix and arrive as some other colour, and a
/// Kept apart from the rest because it belongs at the other end of the
/// shader. Everything else runs on scene-referred colour in the working
/// space, where a flat tint would then be pushed through the base curve
/// and the camera matrix and arrive as some other colour, and a
/// white-on-black alpha would arrive as neither. This runs after the
/// output transform, so what is written is what is seen.
pub reveal: String,
}
/// One layer's version of one operation: the global settings with the layer's
/// offsets added, as a fragment reading this layer's own uniforms.
///
/// The composer runs it beside the global fragment on the same input colour,
/// and moves the pixel toward its result by the layer's weight — see
/// `operation::local_block`.
pub(crate) struct LocalOp {
pub op: &'static str,
pub slot: usize,
/// Empty when the offsets cancel the global setting back to neutral. That
/// is still an entry, because it still means something: inside the mask
/// this operation does nothing at all.
pub fragment: String,
}
/// A layer's settings for one operation, applied as offsets to the global
/// operation's.
///
/// **This is what a local adjustment means**, and the reason it is not a
/// second chain run over the finished picture. A photographer who sets
/// contrast −30 on the whole frame and −20 on a face means −50 on the face,
/// at the place contrast sits in the chain — not −30, then everything after
/// contrast, then −20 applied again to the result. Stacked that way the two
/// edits compound in ways neither slider shows, and a flattening applied to an
/// already-flattened picture is how a shadow's noise ended up magenta.
///
/// Per parameter: one the layer left at its default takes the global value; a
/// moved scalar adds its distance from default to the global value, clamped to
/// the parameter's range; a moved switch or choice replaces it, since there is
/// no such thing as half a variant.
fn offset_onto(dst: &mut dyn Operation, local: &dyn Operation, global: Option<&dyn Operation>) {
let desc = local.descriptor();
for p in &desc.params {
let here = local.param(p.id);
let base = global.map_or(p.default, |g| g.param(p.id));
let value = if here == p.default {
base
} else {
match p.kind {
ParamKind::Scalar { .. } => p.clamp(base + (here - p.default)),
ParamKind::Bool | ParamKind::Enum { .. } => here,
}
};
dst.set_param(p.id, value);
}
}
/// Emit the WGSL for every layer that renders, and for the mask being looked
/// at.
///
@@ -2068,11 +2130,18 @@ pub(crate) struct LayerShader {
/// an adjustment on it, which is why the two are one sequence and why every
/// other half of the pipeline has to be given the same `reveal` for the slots
/// to mean the same thing.
pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal>) -> LayerShader {
///
/// `global` is the chain the layers are offsets to.
pub(crate) fn compose_layers_revealing(
stack: &MaskStack,
reveal: Option<&Reveal>,
global: &[Box<dyn Operation>],
) -> LayerShader {
let mut out = LayerShader {
uniform_fields: String::new(),
uniform_values: Vec::new(),
body: String::new(),
weights: String::new(),
ops: Vec::new(),
helpers: Vec::new(),
reveal: String::new(),
};
@@ -2104,13 +2173,14 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
);
out.uniform_values.extend_from_slice(&layer.uniforms());
let _ = writeln!(
out.body,
"\n // ======== mask {slot}: {} ({}) ========",
layer.display_name(),
layer.base().source.kind()
);
let _ = writeln!(out.body, " {{");
// A layer only being looked at moves no pixel, so it needs a slot for
// the reveal and no weight.
if layer.active_ops().next().is_none() {
continue;
}
// The weight, once per pixel, ahead of every operation that reads it.
//
// **`uv_src`, not `gid.xy`.** The mask array is rasterised in *source*
// space, and `uv_src` is the source position this output pixel came
// from — after the crop, the zoom, the pan, the straightening and the
@@ -2123,33 +2193,49 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
// place. A second copy here would be a second thing to keep in step
// with `Framing::wgsl_prologue`, and the failure would be a mask that
// is subtly wrong only when straightened.
let _ = writeln!(out.body, " var m = sample_mask(uv_src, {slot});");
let w = format!("mask_w{slot}");
let _ = writeln!(
out.body,
" m = select(m, 1.0 - m, u.{prefix}_invert > 0.5);"
out.weights,
"\n // ======== mask {slot}: {} ({}) ========",
layer.display_name(),
layer.base().source.kind()
);
let _ = writeln!(out.weights, " var {w} = sample_mask(uv_src, {slot});");
let _ = writeln!(
out.weights,
" {w} = select({w}, 1.0 - {w}, u.{prefix}_invert > 0.5);"
);
let _ = writeln!(
out.body,
" m = clamp(m * u.{prefix}_opacity, 0.0, 1.0);"
out.weights,
" {w} = clamp({w} * u.{prefix}_opacity, 0.0, 1.0);"
);
// Skipping the work where the mask is empty is most of the point of a
// local adjustment: a mask covering a tenth of the frame should cost
// about a tenth of the shader. Safe as non-uniform control flow —
// nothing inside samples with derivatives or synchronises.
let _ = writeln!(out.body, " if (m > 0.0) {{");
// `masked` is the outer-scope carrier: op fragments write to a `c`
// they expect to own, so the inner block shadows `c` and copies the
// result back out. Assigning the outer `c` from inside is not possible
// precisely because it is shadowed.
let _ = writeln!(out.body, " var masked = c;");
let _ = writeln!(out.body, " {{");
let _ = writeln!(out.body, " var c = masked;");
for op in layer.active_ops() {
let id = op.descriptor().id.0;
// A fresh chain to hold the combined settings: the layer's own ops
// are its offsets and must stay that way.
let mut combined = layer_chain();
for (dst, local) in combined.iter_mut().zip(&layer.ops) {
let local = local.as_ref();
if !local.is_active() || local.detail().is_some() {
continue;
}
let id = local.descriptor().id.0;
let g = global
.iter()
.map(|o| o.as_ref())
.find(|o| o.descriptor().id.0 == id);
offset_onto(dst.as_mut(), local, g);
if !dst.is_active() {
out.ops.push(LocalOp {
op: id,
slot,
fragment: String::new(),
});
continue;
}
let op_prefix = format!("{prefix}_{}", crate::operation::sanitise(id));
let op_uniforms = op.uniforms();
let op_uniforms = dst.uniforms();
if !op_uniforms.is_empty() {
let _ = writeln!(out.uniform_fields, " // mask {slot}: {id}");
}
@@ -2158,13 +2244,13 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
out.uniform_values.push(u.value);
}
for h in op.helpers() {
for h in dst.helpers() {
if !out.helpers.iter().any(|e| e.name == h.name) {
out.helpers.push(*h);
}
}
let mut fragment = op.wgsl_body();
let mut fragment = dst.wgsl_body();
for u in &op_uniforms {
fragment = crate::operation::rewrite_uniform(
&fragment,
@@ -2172,20 +2258,12 @@ pub(crate) fn compose_layers_revealing(stack: &MaskStack, reveal: Option<&Reveal
&format!("u.{op_prefix}_{}", u.name),
);
}
let _ = writeln!(out.body, " // ---- {id} ----");
let _ = writeln!(out.body, " {{");
for line in fragment.lines() {
let _ = writeln!(out.body, " {line}");
}
let _ = writeln!(out.body, " }}");
out.ops.push(LocalOp {
op: id,
slot,
fragment,
});
}
let _ = writeln!(out.body, " masked = c;");
let _ = writeln!(out.body, " }}");
let _ = writeln!(out.body, " c = mix(c, masked, m);");
let _ = writeln!(out.body, " }}");
let _ = writeln!(out.body, " }}");
}
out
@@ -2317,7 +2395,10 @@ mod tests {
let mut stack = MaskStack::new();
stack.push(layer);
assert!(stack.is_neutral());
assert_eq!(compose_layers_revealing(&stack, None).body, "");
assert_eq!(
compose_layers_revealing(&stack, None, &ops::chain()).weights,
""
);
}
#[test]
@@ -2403,11 +2484,11 @@ mod tests {
stack.push(lit_layer("m1", 1.0));
stack.push(lit_layer("m2", -1.0));
let shader = compose_layers_revealing(&stack, None);
assert!(shader.body.contains("sample_mask(uv_src, 0)"));
assert!(shader.body.contains("sample_mask(uv_src, 1)"));
assert!(shader.body.contains("u.mask0_opacity"));
assert!(shader.body.contains("u.mask1_opacity"));
let shader = compose_layers_revealing(&stack, None, &ops::chain());
assert!(shader.weights.contains("sample_mask(uv_src, 0)"));
assert!(shader.weights.contains("sample_mask(uv_src, 1)"));
assert!(shader.weights.contains("u.mask0_opacity"));
assert!(shader.weights.contains("u.mask1_opacity"));
}
/// The slot a layer renders through must follow `active()`, not the raw
@@ -2420,12 +2501,12 @@ mod tests {
stack.push(off);
stack.push(lit_layer("m2", -1.0));
let shader = compose_layers_revealing(&stack, None);
let shader = compose_layers_revealing(&stack, None, &ops::chain());
assert!(
shader.body.contains("sample_mask(uv_src, 0)"),
shader.weights.contains("sample_mask(uv_src, 0)"),
"the one active layer must use slot 0, not slot 1"
);
assert!(!shader.body.contains("sample_mask(uv_src, 1)"));
assert!(!shader.weights.contains("sample_mask(uv_src, 1)"));
}
/// TRACES: FR-DEV-19c
@@ -2445,7 +2526,7 @@ mod tests {
stack.push(MaskLayer::new("m2", MaskSource::brush()));
let reveal = Reveal::one("m2", RevealStyle::Alpha);
let shader = compose_layers_revealing(&stack, Some(&reveal));
let shader = compose_layers_revealing(&stack, Some(&reveal), &ops::chain());
assert_eq!(
stack.rendered_count(Some(&reveal)),
@@ -2483,7 +2564,7 @@ mod tests {
],
style: RevealStyle::Tint,
};
let shader = compose_layers_revealing(&stack, Some(&reveal));
let shader = compose_layers_revealing(&stack, Some(&reveal), &ops::chain());
let sky = shader
.reveal
@@ -2509,7 +2590,9 @@ mod tests {
fn nothing_is_revealed_unless_it_was_asked_for() {
let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0));
assert!(compose_layers_revealing(&stack, None).reveal.is_empty());
assert!(compose_layers_revealing(&stack, None, &ops::chain())
.reveal
.is_empty());
}
/// A reveal aimed at a layer that is not in the stack is not a slot, and
@@ -2520,9 +2603,11 @@ mod tests {
stack.push(lit_layer("m1", 1.0));
let reveal = Reveal::one("gone", RevealStyle::Tint);
assert_eq!(stack.rendered_count(Some(&reveal)), 1);
assert!(compose_layers_revealing(&stack, Some(&reveal))
.reveal
.is_empty());
assert!(
compose_layers_revealing(&stack, Some(&reveal), &ops::chain())
.reveal
.is_empty()
);
}
#[test]
@@ -2531,7 +2616,7 @@ mod tests {
stack.push(lit_layer("m1", 1.0));
stack.push(lit_layer("m2", -1.0));
let shader = compose_layers_revealing(&stack, None);
let shader = compose_layers_revealing(&stack, None, &ops::chain());
assert!(shader.uniform_fields.contains("mask0_exposure_"));
assert!(shader.uniform_fields.contains("mask1_exposure_"));
assert_eq!(
@@ -2545,16 +2630,140 @@ mod tests {
);
}
/// The whole shader for `stack` over a global chain with `global`
/// applied to it.
fn composed_with(stack: &MaskStack, global: impl FnOnce(&mut [Box<dyn Operation>])) -> String {
let mut chain = ops::chain();
global(&mut chain);
crate::operation::compose_full(
&chain,
&crate::Framing::new(),
dr_types::ColourSpace::Srgb,
stack,
&crate::spot::SpotSet::new(),
&[],
)
.source
}
fn set(chain: &mut [Box<dyn Operation>], op: &str, param: &'static str, v: f32) {
chain
.iter_mut()
.find(|o| o.descriptor().id.0 == op)
.expect("op in chain")
.set_param(ParamId(param), v);
}
fn contrast_layer(v: f32) -> MaskLayer {
let mut layer = MaskLayer::new("m1", regions(&[1]));
layer.set_param("contrast", ParamId("contrast"), v);
layer
}
#[test]
fn the_inner_block_shadows_c_and_copies_back() {
fn the_layer_version_shadows_c_and_blends_by_its_difference() {
let mut stack = MaskStack::new();
stack.push(lit_layer("m1", 1.0));
let body = compose_layers_revealing(&stack, None).body;
let src = composed_with(&stack, |_| {});
assert!(body.contains("var masked = c;"));
assert!(body.contains("var c = masked;"));
assert!(body.contains("masked = c;"));
assert!(body.contains("c = mix(c, masked, m);"));
assert!(src.contains("var c = local_in;"));
assert!(src.contains("local_sum = local_sum + mask_w0 * (c - local_global);"));
assert!(src.contains("c = max(local_sum, vec3<f32>(0.0));"));
}
/// **The bug this shape exists for.** A layer's contrast is added to the
/// global contrast, not run a second time on top of it.
#[test]
fn a_layer_setting_is_an_offset_to_the_global_one() {
let mut stack = MaskStack::new();
stack.push(contrast_layer(-20.0));
let mut chain = ops::chain();
set(&mut chain, "contrast", "contrast", -30.0);
let shader = compose_layers_revealing(&stack, None, &chain);
let at = shader
.uniform_fields
.lines()
.filter(|l| l.trim_start().starts_with("mask"))
.position(|l| l.contains("mask0_contrast_amount"))
.expect("the layer carries its own contrast");
assert_eq!(
shader.uniform_values[at], -0.5,
"global -30 and local -20 is -50 inside the mask"
);
}
/// Where the operation runs is where the layer's version of it runs —
/// between the global operations either side, not after all of them.
#[test]
fn a_layer_runs_at_its_operations_place_in_the_chain() {
let mut stack = MaskStack::new();
stack.push(contrast_layer(-20.0));
let src = composed_with(&stack, |c| set(c, "saturation", "saturation", 20.0));
let contrast = src.find("// ---- contrast ----").expect("contrast block");
let blend = src.find("mask_w0 * (c - local_global)").expect("blend");
let saturation = src
.find("// ---- saturation ----")
.expect("saturation block");
assert!(contrast < blend && blend < saturation);
}
/// **No setting is applied twice.** The layer's version of an operation
/// starts from the colour the operation was handed, not from the global
/// result, and reads only its own combined setting — so global −30 and
/// local −20 is one contrast of −50 inside the mask, never −30 and then
/// −50 again, and the operation does not run a second time after the
/// chain as it once did.
#[test]
fn a_setting_is_applied_once_not_stacked() {
let mut stack = MaskStack::new();
stack.push(contrast_layer(-20.0));
let src = composed_with(&stack, |c| set(c, "contrast", "contrast", -30.0));
assert_eq!(
src.matches("// ---- contrast ----").count(),
1,
"contrast runs at one place in the chain"
);
let version = &src[src.find("if (mask_w0 > 0.0)").expect("layer version")..];
let version = &version[..version.find("local_sum = local_sum").unwrap()];
assert!(
version.contains("var c = local_in;"),
"starts from the operation's input"
);
assert!(version.contains("u.mask0_contrast_amount"));
assert!(
!version.contains("u.contrast_amount"),
"the global setting is already inside the combined one"
);
}
/// An offset that cancels the global setting is not nothing: inside the
/// mask the operation is back at neutral, so the layer's version is empty
/// and the blend pulls toward the colour the operation was handed.
#[test]
fn an_offset_back_to_neutral_undoes_the_global_setting() {
let mut stack = MaskStack::new();
stack.push(contrast_layer(30.0));
let mut chain = ops::chain();
set(&mut chain, "contrast", "contrast", -30.0);
let shader = compose_layers_revealing(&stack, None, &chain);
let local: Vec<_> = shader.ops.iter().filter(|l| l.op == "contrast").collect();
assert_eq!(local.len(), 1);
assert!(local[0].fragment.is_empty());
}
/// A global chain with nothing moved still hands a layer's operation a
/// place to run: the global side of the blend is simply empty.
#[test]
fn an_operation_only_a_layer_moved_still_runs_in_its_place() {
let mut stack = MaskStack::new();
stack.push(contrast_layer(-20.0));
let src = composed_with(&stack, |_| {});
assert!(src.contains("// ---- contrast ----"));
assert!(!src.contains("(local only)"));
}
#[test]
+125 -37
View File
@@ -597,10 +597,11 @@ pub fn compose_with_framing(
/// TRACES: FR-DEV-3
/// Compose the global chain, the framing, and the local adjustments.
///
/// Mask layers are emitted **after** every global operation and before the
/// conversion out of camera space, so a local exposure acts on the tones the
/// global chain settled on — which is what a photographer means by "and then
/// lift the shadows on her face".
/// A mask layer's settings are **offsets to the global ones**, applied at each
/// operation's own place in the chain: global contrast −30 and a face at −20
/// is contrast −50 on the face, run where contrast runs. They once ran as a
/// second chain after every global operation, which compounded the two edits
/// in ways neither slider showed — see `mask::offset_onto`.
///
/// The fused-dispatch property survives: three global adjustments and two
/// masked ones are still one shader, one read and one write. The masks
@@ -858,48 +859,76 @@ fn compose_inner(
}
}
for op in &active {
// TRACES: FR-DEV-3
// The local adjustments. Composed first because they are threaded through
// the loop below rather than appended after it: a layer's settings are
// offsets to the global ones, applied at each operation's own place in
// the chain (see `mask::offset_onto` for why). The weights are sampled
// here, once per pixel, ahead of every operation that reads them.
let layers = crate::mask::compose_layers_revealing(masks, reveal, ops);
body.push_str(&layers.weights);
// Every point operation that is active globally *or* in some layer. One
// that only a layer moved still runs here, at its place in the chain,
// with nothing on the global side of its blend.
for op in ops
.iter()
.map(|o| o.as_ref())
.filter(|o| o.detail().is_none())
{
let id = op.descriptor().id.0;
let local: Vec<&crate::mask::LocalOp> = layers.ops.iter().filter(|l| l.op == id).collect();
if !op.is_active() && local.is_empty() {
continue;
}
let prefix = sanitise(id);
// Each op's uniforms are prefixed, so two operations may both declare
// a field called `amount` without colliding.
let op_uniforms = op.uniforms();
if !op_uniforms.is_empty() {
let _ = writeln!(uniform_fields, " // {id}");
}
for u in &op_uniforms {
let _ = writeln!(uniform_fields, " {prefix}_{}: f32,", u.name);
uniform_values.push(u.value);
}
let mut fragment = String::new();
if op.is_active() {
// Each op's uniforms are prefixed, so two operations may both
// declare a field called `amount` without colliding.
let op_uniforms = op.uniforms();
if !op_uniforms.is_empty() {
let _ = writeln!(uniform_fields, " // {id}");
}
for u in &op_uniforms {
let _ = writeln!(uniform_fields, " {prefix}_{}: f32,", u.name);
uniform_values.push(u.value);
}
for h in op.helpers() {
if !helpers.iter().any(|existing| existing.name == h.name) {
helpers.push(*h);
for h in op.helpers() {
if !helpers.iter().any(|existing| existing.name == h.name) {
helpers.push(*h);
}
}
// Rewrite bare uniform names to their prefixed struct fields, so a
// fragment is written without knowing about any other operation.
fragment = op.wgsl_body();
for u in &op_uniforms {
fragment = rewrite_uniform(&fragment, u.name, &format!("u.{prefix}_{}", u.name));
}
}
// Rewrite bare uniform names to their prefixed struct fields, so a
// fragment is written without knowing about any other operation.
let mut fragment = op.wgsl_body();
for u in &op_uniforms {
fragment = rewrite_uniform(&fragment, u.name, &format!("u.{prefix}_{}", u.name));
}
let _ = writeln!(body, "\n // ---- {id} ----");
let _ = writeln!(body, " {{");
for line in fragment.lines() {
let _ = writeln!(body, " {line}");
}
let _ = writeln!(body, " }}");
body.push_str(&local_block(&fragment, &local));
}
// A layer's operation the global chain does not hold at all. Not a case
// any editor produces — both chains come from `ops::chain` — but a layer
// must not lose an edit because a caller composed a shorter chain.
let mut orphans: Vec<&'static str> = Vec::new();
for l in &layers.ops {
if !orphans.contains(&l.op) && !ops.iter().any(|o| o.descriptor().id.0 == l.op) {
orphans.push(l.op);
}
}
for id in orphans {
let local: Vec<&crate::mask::LocalOp> = layers.ops.iter().filter(|l| l.op == id).collect();
let _ = writeln!(body, "\n // ---- {id} (local only) ----");
body.push_str(&local_block("", &local));
}
// The local adjustments, after every global one: a masked exposure should
// act on the tones the global chain arrived at, not on the ones it started
// from. Their uniforms follow the global ops' in the block for the same
// reason those follow framing's — slot order is emission order, and
// nothing addresses a slot by number.
let layers = crate::mask::compose_layers_revealing(masks, reveal);
// TRACES: FR-DEV-19c
// Held apart from the body, because it belongs after the output transform
// rather than among the operations — see `mask::LayerShader::reveal`.
@@ -908,7 +937,6 @@ fn compose_inner(
let reveal_block = layers.reveal.clone();
uniform_fields.push_str(&layers.uniform_fields);
uniform_values.extend_from_slice(&layers.uniform_values);
body.push_str(&layers.body);
for h in &layers.helpers {
if !helpers.iter().any(|existing| existing.name == h.name) {
helpers.push(*h);
@@ -1255,6 +1283,66 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
}
}
/// One operation's block: its global fragment, and each layer's version of it
/// blended in by that layer's weight.
///
/// Every version reads the same input — the colour as it arrived at this
/// operation — and the pixel moves from the global result by each layer's
/// difference from it: `c_g + Σ w_i (c_i − c_g)`. At full weight that is the
/// layer's combined setting exactly, at zero it is the global result exactly,
/// and two overlapping layers add their changes rather than one repainting
/// the other.
///
/// With no layer touching the operation this is the block the composer always
/// emitted, byte for byte: a photograph with no masks compiles to the shader
/// it did before layers were offsets.
fn local_block(global: &str, local: &[&crate::mask::LocalOp]) -> String {
let mut out = String::new();
let _ = writeln!(out, " {{");
if local.is_empty() {
for line in global.lines() {
let _ = writeln!(out, " {line}");
}
let _ = writeln!(out, " }}");
return out;
}
let _ = writeln!(out, " let local_in = c;");
let _ = writeln!(out, " {{");
for line in global.lines() {
let _ = writeln!(out, " {line}");
}
let _ = writeln!(out, " }}");
let _ = writeln!(out, " let local_global = c;");
let _ = writeln!(out, " var local_sum = c;");
for l in local {
let w = format!("mask_w{}", l.slot);
// Skipping where the mask is empty is most of the point of a local
// adjustment: a mask covering a tenth of the frame should cost about a
// tenth of the extra work. Safe as non-uniform control flow — nothing
// inside samples with derivatives or synchronises.
let _ = writeln!(out, " if ({w} > 0.0) {{");
// Fragments write to a `c` they expect to own, so the layer's version
// gets one of its own, shadowing the outer one and starting from what
// this operation was handed.
let _ = writeln!(out, " var c = local_in;");
let _ = writeln!(out, " {{");
for line in l.fragment.lines() {
let _ = writeln!(out, " {line}");
}
let _ = writeln!(out, " }}");
let _ = writeln!(
out,
" local_sum = local_sum + {w} * (c - local_global);"
);
let _ = writeln!(out, " }}");
}
// Two layers pulling the same way can overshoot below zero, and a
// negative component poisons every operation after this one.
let _ = writeln!(out, " c = max(local_sum, vec3<f32>(0.0));");
let _ = writeln!(out, " }}");
out
}
/// The WGSL converting linear sRGB into the output space's primaries.
///
/// A constant matrix rather than a uniform: the space is chosen when the
+8 -2
View File
@@ -723,7 +723,12 @@ merges reuses those rules or keys on the same identities, and each has no other
- **Collections and their membership** — by uuid and revision, membership as a set union.
- **Keywords** — the vocabulary by the same verdict, the assignments as a union.
- **People and identity judgements** — people by uuid and revision, and the confirmed and rejected
face assignments matched to local faces by box (`merge::match_faces`).
face assignments matched to local faces (`merge::match_faces`): by box first, and — since 0.18.0,
only on the photographs where a remote face is left over — by embedding, a pair being accepted
at cosine ≥ 0.7 when each is the other's best by a lead of ≥ 0.2 (#77; [faces.md §18.2](faces.md)). After every merge,
`dedup_people` folds people of one name whose confirmed faces agree, and a face held twice in
one photograph, through the ordinary `merged_into` redirect, which older builds already honour
(#78; [faces.md §19](faces.md)).
- **Albums** (FR-EXP-10, 0.17.0) — by uuid and revision with tombstones, and what went into each as
a set union keyed on the server's file id (a content hash on a folder library). An album's
server folder is a column of its row and travels with it; a folder on *this device* is in
@@ -754,7 +759,7 @@ it goes anywhere.
**The face crops stay out of the upload.** A crop is a ~5 KB JPEG on each `faces` row. On a 19k-face
library they are 96 MB of a 158 MB catalog. The face shards carry them to other devices, once each.
The merge reads a remote face's box and model to match it to a local one, never its pixels. No
The merge reads a remote face's box and model to match it to a local one — and, where the boxes cannot decide, its embedding — never its pixels. No
device adopts a downloaded catalog as its own: a fresh device starts empty and takes faces, crops
included, from the shards. So the snapshot's `crop` is NULL, and a merge never writes a local
crop. They were first stripped (2026-08) by copying the whole file with the backup API, setting
@@ -778,6 +783,7 @@ integer ids stay local and are never compared across catalogs.
| Deletion | Tombstone (`deleted = 1`) carrying a revision | Without it, merging against a device that still holds the collection resurrects it. With a revision, deletion competes on equal footing with a rename |
| An image the remote has and we do not | Skip the membership row | It joins on a later merge, once a scan has catalogued the file. Not an error |
| A remote from a newer schema | Decline before attaching | Attempting it would fail mid-transaction rather than declining cleanly |
| People with the same name | Folded after each merge when their faces agree ([faces.md §19](faces.md)) | Names typed separately on two devices otherwise stay two people for ever |
Merging is idempotent: running it twice reports no changes the second time. That property is tested,
because a merge that oscillates would upload on every sync forever.
+34 -2
View File
@@ -1575,5 +1575,37 @@ It is a match, not an update in place, and that is why the per-face repairs exis
detection: where nothing about a face but one field needs doing, `record_updates` keeps the id and
there is nothing to judge.
The merge's `match_faces` still matches by overlap alone across devices. It is the same question,
and the same answer would serve it; it is not changed here.
Since #77 (0.18.0) the merge's `match_faces` answers it too, within a photograph's `file_id` and
one embedder: box IoU ≥ 0.5, unique on both sides, first; then, only for photographs where a remote
face is left over and a local face is free, embedding cosine ≥ 0.7, mutual best, with a lead of
≥ 0.2 over the runner-up on both sides. A box match is never overruled by a low cosine (about 150
genuine cross-device pairs of tiny faces score below 0.45). On the reference desktop/tablet pair this
recovers 20 of 631 unmatched faces with no false matches; the rest are faces one device alone found.
The merge also keeps one person to one face per photograph: an incoming assignment is refused when
another local face already holds that person, unless it is a remote confirmation over a local
suggestion, which moves the suggestion. Refusals are counted in `faces_one_per_photograph`.
The threshold differs from `SAME_FACE_COSINE` (0.45) above on purpose: re-detection additionally
requires the boxes to overlap, while the merge's embedding route exists for boxes that don't.
## 19. Deduplicating people · 2026-09-26
`dr_catalog::dedup_people::run` runs after every successful sync merge (`sync::merge_remote`, on the
sync worker), in one transaction, and logs one `dedup:` line (#78).
**People.** Named people with the same name, trimmed and case-folded, merge into the one with the
most confirmed faces (ties go to the smaller uuid) when every shared embedder's confirmed-face
centroids agree at cosine ≥ 0.7 (distance < 0.3). Each side needs at least two confirmed faces to
compare; a namesake holding no faces merges outright; a face confirmed as one and rejected as the
other keeps them apart; unnamed and set-aside people are never touched. On the reference library the
same-person centroid median is 0.91, and different named people have a 99.9th percentile of 0.41.
**Faces.** Two faces in the same image and embedder with IoU ≥ 0.5 and cosine ≥ 0.7 are one: the
job keeps the stronger detector's face (`FaceDetector::outranks`), then the confirmed one, then the
lower id, and it takes both faces' assignment and rejections.
**Propagation.** The merge is `faces::merge_people`, whose `merged_into` redirect a 0.17.0 peer
already honours, so an older device never resurrects the duplicate. The job also follows redirects
left by earlier manual merges, moving this device's own assignments onto the person kept, and
breaks a mutual redirect at the smaller uuid, which every device computes alike. A merge now also
carries the merged-away person's rejections to the person kept.
+6 -6
View File
@@ -250,12 +250,12 @@ about.
0.17.0 brought the first real SAF code, for albums (FR-EXP-10): `FolderPicker.java` starts
`ACTION_OPEN_DOCUMENT_TREE` from a translucent activity of its own (the main activity is
`NativeActivity`, whose results are not ours) and takes a persistable grant; `Saf.java` writes each
export through `DocumentsContract`; `ui/dr-ui/src/saf.rs` is the JNI bridge. `saf.rs` and the export
path now carry `TRACES: FR-PLAT-AND-1`, and the matrix counts the requirement as covered. **That
overstates it.** The mechanism is the one the requirement names, but its subject is the library, and
Android still reaches a library through a Nextcloud account or a folder, over paths, like the
desktop. Either the tags narrow to FR-EXP-10 or the requirement is met for the library too; until
one of those, read the coverage figure with this one subtracted.
export through `DocumentsContract`; `ui/dr-ui/src/saf.rs` is the JNI bridge. They shipped tagged
`TRACES: FR-PLAT-AND-1`, which made the matrix count the requirement as covered, and that overstated
it: the mechanism is the one the requirement names, but its subject is the library, and Android
still reaches a library through a Nextcloud account or a folder, over paths, like the desktop. The
tags now say FR-EXP-10 alone (0.17.1), so FR-PLAT-AND-1 reads as uncovered again until the library
itself is reached through SAF.
That has a consequence for the rest of the cluster: **FR-PLAT-AND-2** — detecting the loss of a
granted tree permission and marking images offline rather than deleting rows — is still blocked for
File diff suppressed because one or more lines are too long
+26 -26
View File
@@ -521,7 +521,7 @@ The right match confidence is a property of your library, not of the model. "Wha
Touch has no ctrl, so without a mode there is no way to select a second photograph — the first tap would open it. The hold is the fast way in and the button is the one that can be found.
<sub>`ui/dr-ui/ui/library.slint:1701`</sub>
<sub>`ui/dr-ui/ui/library.slint:1706`</sub>
### Add or remove one photograph
@@ -531,7 +531,7 @@ Touch has no ctrl, so without a mode there is no way to select a second photogra
While selecting, a tap never opens. That is the whole point of the mode: one meaning per gesture at a time. Press Done to get tap-to-open back.
<sub>`ui/dr-ui/ui/library.slint:1711`</sub>
<sub>`ui/dr-ui/ui/library.slint:1716`</sub>
### Leave selecting
@@ -540,7 +540,7 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
- **Keyboard** — `Escape`, or `Back`; an open sheet closes first
- **See it** — [in the manual](manual/README.md#selecting-several)
<sub>`ui/dr-ui/ui/library.slint:1720`</sub>
<sub>`ui/dr-ui/ui/library.slint:1725`</sub>
### Pick a photograph up to drag it
@@ -550,7 +550,7 @@ While selecting, a tap never opens. That is the whole point of the mode: one mea
A finger on a photograph might be starting a scroll, and for the first half-second the grid assumes it is. Holding says otherwise, and the ring is the grid saying it heard — from there the drag cannot be lost to a scroll. A mouse never waits: the cursor is precise enough that a sideways drag is unambiguous from the first pixel.
<sub>`ui/dr-ui/ui/library.slint:1751`</sub>
<sub>`ui/dr-ui/ui/library.slint:1756`</sub>
### Select a range
@@ -561,7 +561,7 @@ A finger on a photograph might be starting a scroll, and for the first half-seco
This replaced a double tap, which had no visible state and could take forty photographs by accident. The run is resolved by the catalog rather than by what is on screen, so the grid can scroll between the two taps — the ranges that hurt on a tablet are longer than a screenful, which is exactly where a finger sweep runs out.
<sub>`ui/dr-ui/ui/library.slint:1817`</sub>
<sub>`ui/dr-ui/ui/library.slint:1822`</sub>
### Take the blinks out of a burst
@@ -571,7 +571,7 @@ This replaced a double tap, which had no visible state and could take forty phot
Face indexing reads each face's eyes. The chip drops frames where the chosen people are caught blinking, and leaves sunglasses and eyes it could not read alone.
<sub>`ui/dr-ui/ui/library.slint:2535`</sub>
<sub>`ui/dr-ui/ui/library.slint:2540`</sub>
### Find photographs with two people in them
@@ -581,7 +581,7 @@ Face indexing reads each face's eyes. The chip drops frames where the chosen peo
"Any of them" is a union and "all of them" is an intersection. The tray is where both terms and the choice between them live, because a filter belongs on the filter bar.
<sub>`ui/dr-ui/ui/library.slint:2565`</sub>
<sub>`ui/dr-ui/ui/library.slint:2570`</sub>
### Show only photographs with one colour label
@@ -591,7 +591,7 @@ Face indexing reads each face's eyes. The chip drops frames where the chosen peo
Each chip is the label's mark and its name, so the one you want is found by reading it; tap the lit chip again to show every label.
<sub>`ui/dr-ui/ui/library.slint:2689`</sub>
<sub>`ui/dr-ui/ui/library.slint:2694`</sub>
### Export the selection as the last export was
@@ -602,7 +602,7 @@ Each chip is the label's mark and its name, so the one you want is found by read
Lightroom's and darktable's chords. Every export runs on the saved defaults, so the plain chord opens them beside an Export button and the shifted one skips straight to exporting.
<sub>`ui/dr-ui/ui/library.slint:3159`</sub>
<sub>`ui/dr-ui/ui/library.slint:3164`</sub>
### Paste copied settings onto the selection
@@ -611,7 +611,7 @@ Lightroom's and darktable's chords. Every export runs on the saved defaults, so
- **Keyboard** — `Ctrl+V`
- **See it** — [in the manual](manual/README.md#copying-settings)
<sub>`ui/dr-ui/ui/library.slint:3183`</sub>
<sub>`ui/dr-ui/ui/library.slint:3188`</sub>
### Keyword the selection
@@ -621,7 +621,7 @@ Lightroom's and darktable's chords. Every export runs on the saved defaults, so
Lightroom's keywording chord. The sheet opens with its field ready for typing, so the keys that judge in the grid are out of the way until it closes.
<sub>`ui/dr-ui/ui/library.slint:3212`</sub>
<sub>`ui/dr-ui/ui/library.slint:3217`</sub>
### Show only photographs with some number of stars
@@ -632,7 +632,7 @@ Lightroom's keywording chord. The sheet opens with its field ready for typing, s
The chips say "this many or more". A range with a ceiling — the twos and threes still to be decided — is the keyboard's alone, and the bar says so in words while it holds.
<sub>`ui/dr-ui/ui/library.slint:3246`</sub>
<sub>`ui/dr-ui/ui/library.slint:3251`</sub>
### Give photographs a colour label
@@ -643,7 +643,7 @@ The chips say "this many or more". A range with a ceiling — the twos and three
Lightroom's keys, so hands that learned them there need not learn them again. Purple has no key there either, and is on the bar. Every mark carries its label's initial, so the label is read without telling the colours apart.
<sub>`ui/dr-ui/ui/library.slint:3296`</sub>
<sub>`ui/dr-ui/ui/library.slint:3301`</sub>
### Pick or reject a photograph
@@ -653,7 +653,7 @@ Lightroom's keys, so hands that learned them there need not learn them again. Pu
The keys every culling tool uses, so muscle memory built elsewhere works here.
<sub>`ui/dr-ui/ui/library.slint:3320`</sub>
<sub>`ui/dr-ui/ui/library.slint:3325`</sub>
### Move photographs to the trash
@@ -663,7 +663,7 @@ The keys every culling tool uses, so muscle memory built elsewhere works here.
The bin acts on one photograph, so a stray click cannot trash a selection; the key acts on the selection because that is what every file manager's Delete does. Both are undone from the trash view.
<sub>`ui/dr-ui/ui/library.slint:3347`</sub>
<sub>`ui/dr-ui/ui/library.slint:3352`</sub>
### Open this list
@@ -671,7 +671,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
- **Pointer** — Press Help in the header, and Done to put it away
- **Keyboard** — `F1`, and `Escape` to put it away
<sub>`ui/dr-ui/ui/library.slint:3372`</sub>
<sub>`ui/dr-ui/ui/library.slint:3377`</sub>
### Rename the collection the grid is showing
@@ -679,7 +679,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
- **Pointer** — Double-click it in the sidebar
- **Keyboard** — `F2`
<sub>`ui/dr-ui/ui/library.slint:3380`</sub>
<sub>`ui/dr-ui/ui/library.slint:3385`</sub>
### Move through the grid
@@ -689,7 +689,7 @@ The bin acts on one photograph, so a stray click cannot trash a selection; the k
The cursor selects what it lands on, so walking and judging are one hand's work.
<sub>`ui/dr-ui/ui/library.slint:3400`</sub>
<sub>`ui/dr-ui/ui/library.slint:3405`</sub>
### Resize the thumbnails
@@ -700,7 +700,7 @@ The cursor selects what it lands on, so walking and judging are one hand's work.
There is no wheel on a tablet, so without the pinch the cell size could only be changed by a control a finger cannot reach.
<sub>`ui/dr-ui/ui/library.slint:3531`</sub>
<sub>`ui/dr-ui/ui/library.slint:3536`</sub>
### File photographs in a collection
@@ -710,7 +710,7 @@ There is no wheel on a tablet, so without the pinch the cell size could only be
The selection is what the drag carries, which is why selecting several is worth the mode: forty photographs file in one gesture.
<sub>`ui/dr-ui/ui/library.slint:3730`</sub>
<sub>`ui/dr-ui/ui/library.slint:3735`</sub>
### Open a photograph
@@ -721,7 +721,7 @@ The selection is what the drag carries, which is why selecting several is worth
A tap opens; a tap that *moved* does not. Travel is what separates a deliberate tap from a hand brushing past, and it is the only thing that does: the two are the same length. An earlier version required the finger to dwell 120 ms instead, and that rejected ordinary taps — a real tap is often quicker than a brush.
<sub>`ui/dr-ui/ui/library.slint:4035`</sub>
<sub>`ui/dr-ui/ui/library.slint:4040`</sub>
### Rate a photograph without opening it
@@ -732,7 +732,7 @@ A tap opens; a tap that *moved* does not. Travel is what separates a deliberate
A star has to take the press without it also reaching the cell, or every rating throws the user into develop.
<sub>`ui/dr-ui/ui/library.slint:4158`</sub>
<sub>`ui/dr-ui/ui/library.slint:4163`</sub>
### Choose the frame a folded burst shows
@@ -742,7 +742,7 @@ A star has to take the press without it also reaching the cell, or every rating
A folded burst draws its earliest frame, which is a fact about the clock and not a judgement about the photograph — nothing in this application ranks a frame (FR-CULL-5). But the point of a burst is that one of the twelve is better than the other eleven, and the photographer is the only one who knows which. So the choice is offered on the frames themselves, while they are open and side by side, which is the one moment the alternatives are on screen to be compared.
<sub>`ui/dr-ui/ui/library.slint:4291`</sub>
<sub>`ui/dr-ui/ui/library.slint:4296`</sub>
### Drop the selection but keep selecting
@@ -753,7 +753,7 @@ A folded burst draws its earliest frame, which is a fact about the clock and not
Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the next selection can start straight away.
<sub>`ui/dr-ui/ui/library.slint:4982`</sub>
<sub>`ui/dr-ui/ui/library.slint:4987`</sub>
### Select everything the grid is showing
@@ -764,7 +764,7 @@ Distinct from Done, which leaves the mode entirely. Clearing keeps it, so the ne
A scoped grid of two hundred frames is two hundred taps otherwise, and "all of them, except those three" is a far more common shape than the taps it took to say it.
<sub>`ui/dr-ui/ui/library.slint:5001`</sub>
<sub>`ui/dr-ui/ui/library.slint:5006`</sub>
### Take photographs out of a collection
@@ -774,7 +774,7 @@ A scoped grid of two hundred frames is two hundred taps otherwise, and "all of t
The badge on a cell says a photograph is filed in three collections and never which. This is the sheet that names them, and the only way out of one the grid is not currently scoped to.
<sub>`ui/dr-ui/ui/library.slint:5182`</sub>
<sub>`ui/dr-ui/ui/library.slint:5187`</sub>
## Settings
+1 -1
View File
@@ -278,7 +278,7 @@ apply elsewhere, and imports Lightroom presets — `Folder…` for a folder of
them, `.xmp file…` for one.
The sheet lists your own presets first, then the ones DarkRoom ships —
Essentials, and colour, cinema and black-and-white film, one measured stock
Essentials, Skies, and colour, cinema and black-and-white film, one measured stock
each. A shipped preset is a look: it changes what it names and leaves the
photograph's own corrections alone, as an imported Lightroom preset does.
Saving under a shipped preset's name makes your version the one that name
+1 -1
View File
@@ -334,7 +334,7 @@ keeps the current state under a name. <code>Presets…</code> saves the settings
apply elsewhere, and imports Lightroom presets — <code>Folder…</code> for a folder of
them, <code>.xmp file…</code> for one.</p>
<p>The sheet lists your own presets first, then the ones DarkRoom ships —
Essentials, and colour, cinema and black-and-white film, one measured stock
Essentials, Skies, and colour, cinema and black-and-white film, one measured stock
each. A shipped preset is a look: it changes what it names and leaves the
photograph's own corrections alone, as an imported Lightroom preset does.
Saving under a shipped preset's name makes your version the one that name
+1 -1
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom
pkgver=0.17.0
pkgver=0.18.1
# Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1
+1 -1
View File
@@ -213,7 +213,7 @@ pub fn place(
if destination.trim().is_empty() {
return Err("No export folder is set. Choose an album to export to.".into());
}
// TRACES: FR-EXP-10 | FR-PLAT-AND-1
// TRACES: FR-EXP-10
// A SAF tree on Android: written through the provider, which may
// rename on a collision, so the name it reports is the one kept.
#[cfg(target_os = "android")]
+15
View File
@@ -356,6 +356,21 @@ pub(super) fn load_window(window: &AppWindow, ctl: &Rc<LibraryController>) {
// signal that something was deleted out from under the view.
let was = window.global::<Library>().get_library_total().max(0) as usize;
window.global::<Library>().set_library_total(total as i32);
// TRACES: FR-CAT-7 | FR-EXP-10
// The sidebar's "All photographs" is the whole library whatever is
// scoped. The same number as `total` when nothing is — no second count
// then — and otherwise read only when the facts above moved, so a
// scroll inside an album does not recount the library.
if scope.is_none() && !trash {
window
.global::<Library>()
.set_library_whole_total(total as i32);
} else if describes_something_new {
let whole = library::total_images_scoped(catalog, None, &filter).unwrap_or(0);
window
.global::<Library>()
.set_library_whole_total(whole as i32);
}
let shrank = total < was;
// Clamp so a scrub to the very end still fills the window rather than
+5 -1
View File
@@ -1,6 +1,10 @@
//! TRACES: FR-EXP-10 | FR-PLAT-AND-1
//! TRACES: FR-EXP-10
//! Android's Storage Access Framework, for an album's folder on the device.
//!
//! Export folders only. FR-PLAT-AND-1 asks for the *library* to be reached
//! through SAF, and it still is not — a library on the tablet is a server —
//! so this module does not claim it.
//!
//! The folder is chosen in the system's own picker, which can make a new
//! folder too, and comes back as a tree URI with a persisted grant. Exports
//! are then written into it through `DocumentsContract` — a tree URI is not a
+1 -1
View File
@@ -1404,7 +1404,7 @@ in property <bool> panel-visible: true;
height: 100%;
rows: Collections.collection-rows;
selected-id: root.collection-selected;
total-images: Library.library-total;
total-images: Library.library-whole-total;
error: Collections.collection-error;
// TRACES: FR-NC-6a
+5
View File
@@ -500,6 +500,11 @@ export struct LibraryCell {
export global Library {
in-out property <[LibraryCell]> library-cells;
in property <int> library-total: 0;
/// The whole library under the current filter, whatever the grid is
/// narrowed to — what the sidebar's "All photographs" row counts.
/// `library-total` is the scope's, which under a collection or an album
/// made that row read as the album's size.
in property <int> library-whole-total: 0;
// Narrows the grid to images whose RAW is stored locally — the ones that
// can actually be opened while offline. Off by default: the catalog is the
// library, and hiding most of it the moment a connection drops would read