Compare commits

...
10 Commits
Author SHA1 Message Date
dtourolle 8a897bbc01 Release 0.13.4
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m22s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 29s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m22s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
2026-09-20 14:03:58 +02:00
dtourolle a03e082fe2 Point the manual's white balance scene at "pick", and record it again
The scene clicked 40px to the right of "pick", on "reset", so the
recording showed a neutral group being reset and a click on the wall
that panned. Re-recorded with the picker fixed: the word lights, the
sample moves temperature and tint, and Before shows what it corrected.
2026-09-20 13:41:17 +02:00
dtourolle 4576499c3b Refuse a clipped highlight as a neutral
Sampling the overcast sky on a Canon 6D frame set tint to -100 and
temperature to -15 for a patch the canvas showed as pure white. A clipped
photosite is sensor white, not a colour: every channel stopped counting,
so what the tap hands back is the as-shot multipliers themselves, which
are strongly magenta, and the solver dutifully drove green to its stop.
The display shader already fades such a pixel to a neutral of the same
brightness before any operation runs, so the picker was balancing against
something the photographer could not see.

The probe now refuses a sample with any channel at or above the onset the
shader fades from, the way the solver already refuses black. The threshold
is one constant, CLIP_ONSET, formatted into the shader and read by the
probe, so the two cannot drift apart.
2026-09-20 13:41:17 +02:00
dtourolle 2f47087223 Measure the white balance probe in camera RGB, where the gains multiply
Pressing "pick" and clicking a near-neutral wall on a Canon 6D frame set
tint to -77 and turned the whole photograph green. The white balance
operation runs first in the chain, on camera RGB, before the body's base
curve and colour matrix; the probe was read off a display render after
all three, and the solve treated that sRGB triple as if the gains
multiplied it directly. On a JPEG the two spaces coincide, which is why
the existing tests passed while the picker was broken on every raw file.

The probe now reads the camera-space tap a merge stitches from, composed
under the edit's own framing so a fraction of the canvas is a fraction of
the probe, and puts the as-shot balance on itself - exactly the value the
operation's gains are about to multiply. No operations run in the tap, so
nothing has to be stripped and restored, and the display target is left
alone, so a sample that found nothing usable no longer needs a redraw.

A raw-frame test with the 6D's matrix and a typical as-shot balance
samples a warm grey and asserts the rendered pixel comes back neutral; it
fails on the previous probe.
2026-09-20 13:41:11 +02:00
dtourolle 764ad55ead Stand each person in the grouping pass by at most 100 references
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m23s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 55s
Build and test / Layer separation (push) Successful in 27s
Traceability / Requirement traces (push) Failing after 54s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Every face the user has ruled on entered the pass as an anchor, and the
scan is exhaustive by design (`dr_face::neighbours`), so a person with
750 confirmed faces cost 750 comparisons against every other face in
the library — and the cost of a library grew with how well it was
named. Most of those comparisons said nothing new: thirty frames from
one afternoon are one point of view, not thirty, and a face that
matches one of them matches the rest.

Each person now enters through at most 100 of their anchored faces
(`dr_face::references`). Eligible are those whose raw embedding is at
least 15 long — one above the gallery floor, since a reference speaks
for someone rather than merely being admitted — with an unmeasured
length admitted as it is everywhere else. From those, the set spanning
the greatest volume is chosen greedily: the longest vector first, then
at each step the face with the largest component orthogonal to the
chosen so far. That is pivoted Gram–Schmidt, and the product of the
residuals it picks is the Gram determinant, so the greedy step is the
exact greedy on the objective. A near-duplicate of a chosen face has
no residual and is passed over; the one profile shot among two hundred
frontal frames is taken early; faces inside the span of the chosen add
no volume and are not taken to fill the cap.

The faces not chosen keep their confirmations and are not touched by
the pass — they stay in the anchor map, so it never releases them —
they are simply not compared. A person none of whose faces is long
enough is still stood for, by their longest, rather than losing their
anchor and having their next face filed as a stranger. Under the cap
nothing changes: every eligible face stands, and the short ones stay
in as the probes they were.

At the reference library's 3,851 confirmations the scan shrinks by
about a fifth; at 15,000 it is a fifth of what it was.
2026-09-20 13:29:16 +02:00
dtourolle 301e6f3828 Release 0.13.3
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m21s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 26s
Traceability / Requirement traces (push) Failing after 39s
🐳 Android image / Build and push (push) Successful in 1s
Build and test / android-image (push) Successful in 1s
🐳 Windows image / Build and push (push) Successful in 1s
Build and test / windows-image (push) Successful in 1s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m8s
2026-09-20 13:01:41 +02:00
dtourolle a437363bd6 Schema V20: put the mis-spelled run markers right
The markers the previous commit stops writing are already in the
catalogs — 2 on the desktop, 429 on the tablet — and in the shards
both have exchanged. Renaming them to the faces' own id with a fresh
time is what makes the export send each image again, under an entry
newer than the empty one `held_model` would otherwise pick. Where the
old write had inserted its marker beside the right one, the wrong one
goes and the right one is refreshed for the same reason: its entry in
the shards is older than the empty one.

Images V14 left with faces and no marker are not touched. That state is
the quality pass's cue, and the fixed write marks them correctly when
it reaches them.

Checked against copies of both real catalogs: the desktop renames 2,
the tablet deletes 429, both in under 200 ms.
2026-09-20 13:00:59 +02:00
dtourolle 065872bec5 Keep a run marker under the detector that found the faces
`record_updates` — the write behind the quality, eye and crop passes —
re-marked the image as indexed under the pipeline the pass ran as, and
left the faces it had updated under the id of the detector that found
them. On a desktop set to Thorough that put `scrfd_10g+w600k_mbf` over
faces spelled `w600k_mbf`; on the tablet, `scrfd_10g_i8+w600k_mbf` over
faces it had adopted from the desktop's thorough pass.

Every reader takes the marker and the faces to agree. `marker_under`
reads the marker as the detector having examined the image, so the
upgrade repair never revisits it. The shard store keys each face by
its pipeline id, so `export_to_shards` selects an image's faces by the
marker's id, finds none, and sends an entry that says the thorough
detector looked and found nothing — over photographs with named faces
on them. The desktop's shard index holds 54 such entries beside real
faces; the tablet's eye pass over the faces it had adopted made 430
more, and both devices have exchanged them. `held_model` takes the
newest entry for an image, which is the empty one. Nothing has been
lost yet only because the two spellings of the thorough detector rank
equal and neither side adopts the other's; a third device, or either
one after a reinstall, would adopt "nothing here" for 484 images. And
the desktop's eye pass is 4,739 images from doing the same to every
face from before V14 — which are the ones that only exist on the
desktop, and would then never reach anywhere.

The marker now takes the id the faces carry; the pass's own id is used
only when it dropped the last of them and there is no detector left to
name. A stale marker under another spelling of the same embedder is
removed in the same transaction, so one embedder has one marker.
2026-09-20 13:00:58 +02:00
dtourolle 0ed38ada28 Adopt a peer's unmeasured faces instead of refusing them
The tablet showed a fraction of each person: 681 of the desktop's 3,851
confirmations, and none of Ian's 746, Catherine's 626 or my own 480.
Every face that existed on both devices agreed on who it was, and the
people rows were identical — the merge was fine. The missing 3,170
confirmations were on faces the tablet did not hold at all: the
desktop's 16,080 faces from the original detector, on 4,310 images,
detected before schema V14 kept the quality reading.

Those faces were in shards the tablet had already downloaded, in
August's export. `import_from_shards` looked at them on every sync pass
and declined each one, because a face without a quality reading was
"work this device cannot finish": adopting it would write the run
marker, and the marker was what stopped an image being looked at again.
That was true when it was written and has not been since the quality
repair existed — that pass lists its work by `f.quality IS NULL`, not by
the marker, exactly as the eye pass does, and faces without an eye
reading were already adopted on that reasoning.

The refusal had no exit. V14 had deleted the markers of every image
holding such faces so the quality pass would find them, and
`export_to_shards` walks the markers, so the desktop never re-exported
them either; the unmeasured August copies were the only ones there
would ever be. The tablet's answer was to queue all 17,727 images for a
re-detection of its own, a fetch of the whole library, while holding
the faces on disk.

Adopt them. The receiving device's quality pass measures them when it
reaches them, and the desktop's confirmations match onto them by box
overlap on the next catalog merge. The test that asserted the refusal
now asserts the adoption and that the image is still owed to the pass.
2026-09-20 12:58:41 +02:00
dtourolle 695d5ec304 Correct four claims in the README against the tree
Benchmarks / CPU and I/O (per commit) (push) Failing after 6m26s
Benchmarks / Frame budget (on demand) (push) Skipped
Build and test / Desktop (Linux) (push) Failing after 46s
Build and test / Layer separation (push) Successful in 28s
Traceability / Requirement traces (push) Failing after 40s
🐳 Android image / Build and push (push) Successful in 2s
Build and test / android-image (push) Successful in 2s
🐳 Windows image / Build and push (push) Successful in 2s
Build and test / windows-image (push) Successful in 2s
Build and test / Android (aarch64) (push) Failing after 2m21s
Build and test / Windows (x86_64, cross) (push) Failing after 3m5s
Eighteen declared operations, not fifteen; JPEG XL is an export format;
the grid does not filter by keyword, only the catalog's query can; and a
panorama's provenance is a sidecar beside the composite, not a history
step in it.
2026-09-20 11:57:17 +02:00
20 changed files with 1010 additions and 188 deletions
Generated
+25 -25
View File
@@ -1221,7 +1221,7 @@ checksum = "f27ae1dd37df86211c42e150270f82743308803d90a6f6e6651cd730d5e1732f"
[[package]]
name = "darkroom-android"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"android_logger",
"dr-plat",
@@ -1234,7 +1234,7 @@ dependencies = [
[[package]]
name = "darkroom-desktop"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"anyhow",
"dr-plat",
@@ -1408,7 +1408,7 @@ checksum = "d8b14ccef22fc6f5a8f4d7d768562a182c04ce9a3b3157b91390b52ddfdf1a76"
[[package]]
name = "dr-bench"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"anyhow",
"dr-catalog",
@@ -1425,7 +1425,7 @@ dependencies = [
[[package]]
name = "dr-catalog"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-face",
"dr-plat",
@@ -1440,7 +1440,7 @@ dependencies = [
[[package]]
name = "dr-decode"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-types",
"env_logger",
@@ -1454,7 +1454,7 @@ dependencies = [
[[package]]
name = "dr-export"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-decode",
"dr-gpu",
@@ -1473,7 +1473,7 @@ dependencies = [
[[package]]
name = "dr-face"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1486,7 +1486,7 @@ dependencies = [
[[package]]
name = "dr-film"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"log",
"serde",
@@ -1495,7 +1495,7 @@ dependencies = [
[[package]]
name = "dr-gpu"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"bytemuck",
"dr-decode",
@@ -1513,7 +1513,7 @@ dependencies = [
[[package]]
name = "dr-inference-engine"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"libloading",
"log",
@@ -1527,7 +1527,7 @@ dependencies = [
[[package]]
name = "dr-ingest"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-plat",
"dr-types",
@@ -1539,7 +1539,7 @@ dependencies = [
[[package]]
name = "dr-lens"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"lensfun",
"log",
@@ -1547,7 +1547,7 @@ dependencies = [
[[package]]
name = "dr-pano"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-decode",
"dr-inference-engine",
@@ -1561,7 +1561,7 @@ dependencies = [
[[package]]
name = "dr-pipeline"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-types",
"log",
@@ -1570,7 +1570,7 @@ dependencies = [
[[package]]
name = "dr-plat"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"android-native-keyring-store",
"dr-types",
@@ -1586,7 +1586,7 @@ dependencies = [
[[package]]
name = "dr-preset-xmp"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-pipeline",
"log",
@@ -1596,7 +1596,7 @@ dependencies = [
[[package]]
name = "dr-segment"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-inference-engine",
"env_logger",
@@ -1609,7 +1609,7 @@ dependencies = [
[[package]]
name = "dr-sync"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"async-trait",
"dr-plat",
@@ -1623,7 +1623,7 @@ dependencies = [
[[package]]
name = "dr-sync-folder"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"async-trait",
"dr-sync",
@@ -1635,7 +1635,7 @@ dependencies = [
[[package]]
name = "dr-sync-nextcloud"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"async-trait",
"dr-decode",
@@ -1657,7 +1657,7 @@ dependencies = [
[[package]]
name = "dr-thumbs"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-types",
"jpeg-encoder",
@@ -1669,7 +1669,7 @@ dependencies = [
[[package]]
name = "dr-types"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"serde",
"serde_json",
@@ -1678,7 +1678,7 @@ dependencies = [
[[package]]
name = "dr-ui"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"anyhow",
"async-trait",
@@ -1720,7 +1720,7 @@ dependencies = [
[[package]]
name = "dr-xmp"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"dr-types",
"log",
@@ -7021,7 +7021,7 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "traceability"
version = "0.13.2"
version = "0.13.4"
dependencies = [
"anyhow",
"serde",
+1 -1
View File
@@ -29,7 +29,7 @@ members = [
]
[workspace.package]
version = "0.13.2"
version = "0.13.4"
edition = "2021"
rust-version = "1.92"
license = "GPL-3.0-or-later"
+6 -6
View File
@@ -16,12 +16,12 @@ is still missing.
or one a Nextcloud client keeps in virtual-files mode, where a placeholder
is treated as the photograph rather than as a one-byte file — or at a
Nextcloud account directly. The grid is virtualised, ordered by capture
time with a timeline beside it, and filtered by rating, flag, keyword,
person and whether the file is here. Ratings, keywords, collections and a
time with a timeline beside it, and filtered by rating, flag, person and
whether the file is here. Ratings, keywords, collections and a
trash that survives a crash mid-operation. Card ingest. Bursts fold. Face
detection and identity, with the index syncing between devices.
**Developing.** Fifteen declared operations fused into one compute
**Developing.** Eighteen declared operations fused into one compute
dispatch, plus the neighbourhood work that cannot be: clarity, texture,
capture sharpening, noise reduction, lens correction, spectral film
simulation. Crop and straighten, spot repair, and local adjustments over
@@ -33,11 +33,11 @@ judging what is recoverable. Named presets; XMP sidecars other editors read.
**Panoramas.** Select the frames, align, choose a projection, fill the
ragged border rather than crop it, and the composite lands beside its
sources as a DNG with the merge as the first step in its history.
sources as a DNG, with a sidecar recording what it was merged from.
[![Twelve hand-held frames aligned on a cylinder](docs/manual/media/panorama-aligned.png)](docs/manual/README.md#merging-a-panorama)
**Export.** JPEG, PNG, AVIF, 8- and 16-bit TIFF, with resize, output
**Export.** JPEG, PNG, AVIF, JPEG XL, 8- and 16-bit TIFF, with resize, output
sharpening, a naming template and a colour space — to a folder here or back
into the library.
@@ -76,7 +76,7 @@ controls, its place in the chain and its tests.
## Where it stands
**0.13.2**, sixteen tagged releases in. 184 numbered requirements in
**0.13.4**, eighteen tagged releases in. 184 numbered requirements in
scope, 84% of them claimed by code and [traced to it](docs/traceability.md);
the rest are written down rather than merely absent.
+31 -23
View File
@@ -820,20 +820,22 @@ pub fn import_from_shards(
let Some((faces, edge)) = store.get_image(file_id as u64, &held)? else {
continue;
};
// A peer that embedded before the quality was kept has done work this
// device cannot finish: the number exists only at embedding time, and
// adopting the faces would write the run marker that keeps them from
// ever being measured (schema V14). Left for this device's own pass —
// or for the peer's, whose re-export replaces these.
// A face the peer embedded before its quality was kept (schema V14)
// is adopted with the reading missing, exactly as one without an eye
// reading is. The measuring passes find their work by the NULL
// column, not by the run marker (`dr_ui::repairs`, `faces_needing`),
// so adopting costs the reading nothing and this device's own pass
// fills it.
//
// A missing *eye* reading is not the same case and is adopted. The
// measuring pass finds those by the NULL, not by the marker, so
// adopting the faces costs the reading nothing (schema V16) — and a
// peer that has no eye models may be the only one that has done the
// detection at all.
if faces.iter().any(|f| f.quality.is_none()) {
continue;
}
// This used to refuse such faces, on the reasoning that the marker
// would stop them ever being measured — true before the quality
// repair existed, and wrong after. What it cost: V14 had dropped the
// markers of every image holding such faces, so the peer never
// re-exported them, and the only copies in the shards were the
// unmeasured ones. A tablet holding shards with 4,310 of the
// desktop's images and 3,170 of its confirmations declined every one
// of them, showed a fraction of each person, and queued the whole
// library for a re-detection of its own instead.
let local: Vec<crate::faces::DetectedFace> = faces
.into_iter()
.map(|f| crate::faces::DetectedFace {
@@ -1365,11 +1367,12 @@ mod catalog_round_trip {
);
}
/// A face a peer embedded without measuring it is work this device
/// cannot finish, and adopting it would write the marker that stops it
/// ever being measured. The image stays outstanding instead.
/// A face a peer embedded without measuring it is adopted all the same,
/// and left on this device's quality pass by its missing reading. Refusing
/// it was what stranded every confirmation the desktop had made on faces
/// from before V14: the tablet held the shards and would not use them.
#[test]
fn a_peers_unmeasured_faces_are_left_for_this_device_to_index() {
fn a_peers_unmeasured_faces_are_adopted_and_left_for_the_quality_pass() {
let b = device(&[(90, 5001), (91, 5002)]);
let mut store = FaceShardStore::open(&tempdir("unmeasured")).unwrap();
let shared = |file_id: u64, quality: Option<f32>| SharedFace {
@@ -1395,13 +1398,18 @@ mod catalog_round_trip {
.put_image(5002, "w600k_mbf", 2560, &[shared(5002, Some(19.0))])
.unwrap();
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 1);
assert_eq!(import_from_shards(&b, &store, "w600k_mbf").unwrap(), 2);
let cov = faces::coverage(&b, "w600k_mbf").unwrap();
assert_eq!(cov.indexed, 1);
assert_eq!(cov.outstanding(), 1, "the unmeasured image was adopted");
assert!(faces::for_image(&b, dr_types::ImageId(90))
.unwrap()
.is_empty());
assert_eq!(cov.indexed, 2);
assert_eq!(cov.outstanding(), 0, "the unmeasured image was refused");
let got = faces::for_image(&b, dr_types::ImageId(90)).unwrap();
assert_eq!(got.len(), 1);
assert_eq!(got[0].quality, None, "a reading was invented");
// Still owed to the measuring pass, which lists by the column.
assert_eq!(
faces::count_needing(&b, "w600k_mbf", "f.quality IS NULL").unwrap(),
1
);
}
#[test]
+110 -5
View File
@@ -556,6 +556,19 @@ impl FaceUpdate {
/// bookkeeping: `face_shard::export_to_shards` re-exports an image whose
/// marker is newer than the store's copy, which is how what was written
/// here reaches the other devices.
///
/// It is re-written under the pipeline id the **faces carry**, not the one
/// this pass ran as. `model_id` names the pass only through its embedder;
/// the detector half of a marker is a statement about who drew the boxes,
/// and this pass drew none. Every reader takes the two to agree: the export
/// selects an image's faces by the marker's id, `marker_under` takes a
/// marker as proof the detector has been over the image, and the shard
/// store keys each face by it. When the marker was written as
/// `scrfd_10g+w600k_mbf` over faces still spelled `w600k_mbf`, the export
/// found no faces under it and sent the other devices an entry saying the
/// thorough detector had looked and found nothing — over photographs with
/// named faces on them. With no faces left, the pass's own id is the only
/// one there is, and the marker says so.
pub fn record_updates(
conn: &Connection,
image_id: ImageId,
@@ -602,13 +615,25 @@ pub fn record_updates(
for f in dropped {
tx.execute("DELETE FROM faces WHERE id = ?1", [f.0 as i64])?;
}
let remaining: i64 = tx.query_row(
let (remaining, found_by): (i64, Option<String>) = tx.query_row(
&format!(
"SELECT COUNT(*) FROM faces WHERE image_id = ?1 AND {} = ?2",
"SELECT COUNT(*), MIN(model_id) FROM faces WHERE image_id = ?1 AND {} = ?2",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, embedder_of(model_id)],
|r| r.get(0),
|r| Ok((r.get(0)?, r.get(1)?)),
)?;
let marker = found_by.as_deref().unwrap_or(model_id);
// One marker per embedder: a stale one under another spelling would
// keep saying that detector had been here, which is the claim the
// faces' own id is now making in its place.
tx.execute(
&format!(
"DELETE FROM face_index
WHERE image_id = ?1 AND model_id != ?2 AND {} = ?3",
embedder_sql("model_id")
),
rusqlite::params![image_id.0 as i64, marker, embedder_of(model_id)],
)?;
tx.execute(
"INSERT INTO face_index (image_id, model_id, indexed_at, faces_found, source_edge)
@@ -619,7 +644,7 @@ pub fn record_updates(
source_edge = excluded.source_edge",
rusqlite::params![
image_id.0 as i64,
model_id,
marker,
now_secs(),
remaining,
source_edge as i64,
@@ -1557,7 +1582,7 @@ fn iou(a: (f32, f32, f32, f32), b: (f32, f32, f32, f32)) -> f32 {
}
}
fn now_secs() -> i64 {
pub(crate) fn now_secs() -> i64 {
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_secs() as i64)
@@ -1890,6 +1915,86 @@ mod tests {
assert!(at >= marked_at, "the marker was not refreshed");
}
/// The marker a per-face pass leaves names the detector that drew the
/// boxes, whatever pipeline the pass itself ran as. A marker under the
/// pass's id over faces spelled another way is one the export finds no
/// faces under — and it sent every other device "nothing here".
#[test]
fn an_update_keeps_the_marker_under_the_detector_that_found_the_faces() {
let c = db();
let img = image(&c, 1);
let ids = record_detections(
&c,
img,
"w600k_mbf",
1024,
&[DetectedFace {
quality: None,
..face(1)
}],
)
.unwrap();
// The state V14 leaves: the faces, and no marker at all.
c.execute("DELETE FROM face_index", []).unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
embedding: Some((vec![9; 1024], 21.5)),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let markers: Vec<(String, i64)> = c
.prepare("SELECT model_id, faces_found FROM face_index")
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(markers, vec![("w600k_mbf".to_string(), 1)]);
// A marker already there under the pass's own id is replaced, not
// kept beside the right one.
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (1, 'scrfd_10g+w600k_mbf', 0, 0, 6000)",
[],
)
.unwrap();
record_updates(
&c,
img,
"scrfd_10g+w600k_mbf",
6000,
&[FaceUpdate {
crop: Some(vec![1, 2, 3]),
..FaceUpdate::for_face(ids[0])
}],
&[],
)
.unwrap();
let n: i64 = c
.query_row("SELECT COUNT(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 1, "a second marker survived");
// With every face dropped there is no detector left to name, and
// the pass's own id records that it looked.
record_updates(&c, img, "scrfd_10g+w600k_mbf", 6000, &[], &ids).unwrap();
let marker: (String, i64) = c
.query_row("SELECT model_id, faces_found FROM face_index", [], |r| {
Ok((r.get(0)?, r.get(1)?))
})
.unwrap();
assert_eq!(marker, ("scrfd_10g+w600k_mbf".to_string(), 0));
}
/// Re-detection is coalesced per image, so it must replace rather than
/// append — otherwise every re-index doubles the library's face count.
#[test]
+174 -1
View File
@@ -15,7 +15,7 @@ use rusqlite::Connection;
use crate::error::CatalogError;
/// Schema version this build writes and understands.
pub const SCHEMA_VERSION: i64 = 19;
pub const SCHEMA_VERSION: i64 = 20;
/// Apply migrations up to [`SCHEMA_VERSION`].
///
@@ -188,9 +188,98 @@ pub fn migrate(conn: &Connection) -> Result<i64, CatalogError> {
tx.commit()?;
}
if from < 20 {
let tx = conn.unchecked_transaction()?;
v20_markers_name_the_detector_that_found_the_faces(&tx)?;
tx.pragma_update(None, "user_version", 20)?;
tx.commit()?;
}
Ok(from)
}
// V20 -- TRACES: FR-CAT-7
//
// Run markers that named the wrong detector, put right.
//
// `faces::record_updates` -- the write behind the quality, eye and crop
// passes -- re-marked an image under the pipeline the pass ran as, while
// the faces it had updated kept the id of the detector that found them.
// A marker of `scrfd_10g+w600k_mbf` over faces spelled `w600k_mbf` reads,
// to every consumer, as the thorough detector having examined the image:
// the upgrade repair skips it, and `face_shard::export_to_shards` selects
// its faces by the marker's id, finds none, and tells every other device
// that the thorough detector found nothing there. The desktop's shard index
// held 54 such entries over photographs with named faces, and the tablet's
// eye pass over faces it had adopted from the desktop had made 430 more.
//
// The write is fixed to keep the marker under the faces' own id. This puts
// the markers already written right, with a fresh time so the export sends
// each image again under an entry newer than the empty one -- which is what
// `held_model` orders by. Where the right marker is still there beside the
// wrong one (the old write inserted rather than replaced), the wrong one
// goes and the right one is refreshed for the same reason: its entry in
// the shards is older than the empty one, and a device that has neither
// would take the empty one. An image V14 left with faces and no marker at
// all is not touched: that state is the quality pass's cue, and the fixed
// write marks it correctly when the pass reaches it.
//
// Restated in Rust rather than SQL because the embedder half of a pipeline
// id is `faces::embedder_sql`, which this must agree with.
fn v20_markers_name_the_detector_that_found_the_faces(tx: &Connection) -> Result<(), CatalogError> {
let fi = crate::faces::embedder_sql("face_index.model_id");
let f = crate::faces::embedder_sql("f.model_id");
// A marker is wrong when the image holds faces of its embedder under
// another id. First the wrong ones that sit beside a right one -- the
// update below would collide with it -- then the rest are renamed.
let wrong = format!(
"EXISTS (SELECT 1 FROM faces f
WHERE f.image_id = face_index.image_id
AND {f} = {fi}
AND f.model_id != face_index.model_id)"
);
let found_by = format!(
"(SELECT MIN(f.model_id) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi})"
);
let now = crate::faces::now_secs();
tx.execute(
&format!(
"UPDATE face_index
SET indexed_at = ?1
WHERE model_id = {found_by}
AND EXISTS (SELECT 1 FROM face_index w
WHERE w.image_id = face_index.image_id
AND w.model_id != face_index.model_id
AND {} = {fi})",
crate::faces::embedder_sql("w.model_id")
),
[now],
)?;
tx.execute(
&format!(
"DELETE FROM face_index
WHERE {wrong}
AND EXISTS (SELECT 1 FROM face_index o
WHERE o.image_id = face_index.image_id
AND o.model_id = {found_by})"
),
[],
)?;
tx.execute(
&format!(
"UPDATE face_index
SET model_id = {found_by},
faces_found = (SELECT COUNT(*) FROM faces f
WHERE f.image_id = face_index.image_id AND {f} = {fi}),
indexed_at = ?1
WHERE {wrong}"
),
[now],
)?;
Ok(())
}
/// The seven columns V16 adds to `faces`, in the order the readers name them.
///
/// Named once because three places have to agree on them: this migration,
@@ -1854,6 +1943,90 @@ mod tests {
assert_eq!(faces, 2);
}
#[test]
fn v20_renames_markers_to_the_detector_that_found_the_faces() {
let c = mem();
c.pragma_update(None, "user_version", 0).unwrap();
migrate(&c).unwrap();
c.execute(
"INSERT INTO roots(id, kind, label) VALUES (1, 'local', 'test')",
[],
)
.unwrap();
c.execute(
"INSERT INTO images(id, root_id, source_ref, added_at)
VALUES (1,1,'a',0),(2,1,'b',0),(3,1,'c',0),(4,1,'d',0),(5,1,'e',0)",
[],
)
.unwrap();
// 1: the desktop's case -- old faces, re-marked as thorough.
// 2: the tablet's case -- adopted thorough faces, re-marked int8,
// and the right marker still beside it (refreshed, so it is
// exported again over the empty entry).
// 3: right already. 4: examined and empty. 5: V14's state, faces
// and no marker.
for (image, model) in [
(1, "scrfd_10g+w600k_mbf"),
(2, "scrfd_10g_i8+w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(4, "scrfd_10g+w600k_mbf"),
] {
c.execute(
"INSERT INTO face_index(image_id, model_id, indexed_at, faces_found, source_edge)
VALUES (?1, ?2, 100, 0, 6000)",
rusqlite::params![image, model],
)
.unwrap();
}
for (image, model) in [
(1, "w600k_mbf"),
(1, "w600k_mbf"),
(2, "scrfd_10g+w600k_mbf"),
(3, "scrfd_10g+w600k_mbf"),
(5, "w600k_mbf"),
] {
c.execute(
"INSERT INTO faces
(image_id, x, y, w, h, landmarks, detector_confidence, embedding,
crop_px, model_id, detected_at)
VALUES (?1, 0.1, 0.1, 0.2, 0.2, X'00', 0.9, X'00', 180.0, ?2, 0)",
rusqlite::params![image, model],
)
.unwrap();
}
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let markers: Vec<(i64, String, i64, bool)> = c
.prepare(
"SELECT image_id, model_id, faces_found, indexed_at > 100
FROM face_index ORDER BY image_id, model_id",
)
.unwrap()
.query_map([], |r| Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?)))
.unwrap()
.map(Result::unwrap)
.collect();
assert_eq!(
markers,
vec![
(1, "w600k_mbf".to_string(), 2, true),
(2, "scrfd_10g+w600k_mbf".to_string(), 0, true),
(3, "scrfd_10g+w600k_mbf".to_string(), 0, false),
(4, "scrfd_10g+w600k_mbf".to_string(), 0, false),
]
);
// Re-enterable: nothing left to rename.
c.pragma_update(None, "user_version", 19).unwrap();
migrate(&c).unwrap();
let n: i64 = c
.query_row("SELECT count(*) FROM face_index", [], |r| r.get(0))
.unwrap();
assert_eq!(n, 4);
}
#[test]
fn job_uniqueness_coalesces_rather_than_duplicating() {
let c = mem();
+1
View File
@@ -50,6 +50,7 @@ pub mod eyes;
pub mod landmarks;
pub mod naming;
pub mod neighbours;
pub mod references;
/// Smallest long edge a face crop may be sampled from.
///
+232
View File
@@ -0,0 +1,232 @@
//! TRACES: FR-CULL-10 | NFR-P9
//! Which of a person's faces stand for them in a grouping pass.
//!
//! # Why not all of them
//!
//! Every face the user has ruled on enters [`crate::cluster`] as an anchor,
//! and the pass compares every face against every other
//! ([`crate::neighbours`] is exhaustive by design). So a person with 750
//! confirmed faces costs 750 comparisons against each of the library's other
//! faces, and the cost of naming a library well grows with how well it is
//! named: a fully confirmed library of 25,000 faces spends almost the whole
//! scan re-comparing faces whose identity is already settled against each
//! other.
//!
//! Most of those comparisons say nothing new. A person's confirmed faces are
//! heavily redundant — thirty frames from one afternoon are one point of
//! view, not thirty — and a new face that matches one of them matches the
//! others too. What a new face needs to be measured against is the person's
//! *range*: the angles, ages and lights they have been photographed in, each
//! represented once.
//!
//! # The choice: the most diverse of the good ones
//!
//! Two rules, in order.
//!
//! **Good enough to vouch.** Only faces whose raw embedding was at least
//! [`MIN_REFERENCE_QUALITY`] long are eligible — a stricter floor than the
//! gallery's ([`crate::embedding::MIN_GALLERY_QUALITY`]), because a reference
//! is asked to speak *for* a person rather than merely be admitted to the
//! comparison. A face whose length was never recorded is admitted, as it is
//! everywhere else: a rule that cannot be checked admits rather than excludes.
//!
//! **As far apart as possible.** From the eligible pool, up to
//! [`MAX_REFERENCES`] faces are chosen to maximise the volume they span —
//! the determinant of their Gram matrix — greedily: start from the longest
//! vector, and at each step add the face with the largest component
//! orthogonal to everything chosen so far. That is Gram–Schmidt with a
//! pivot, and the product of the squared residuals it picks *is* the
//! determinant, so the greedy step is the exact greedy on the objective.
//! The effect is that a near-duplicate of a chosen face has almost no
//! residual and is passed over, while the one profile shot among two
//! hundred frontal frames is taken early.
//!
//! What is not chosen still belongs to the person. Those faces keep their
//! confirmations and are not touched by the pass; they are simply not
//! compared, which is the whole saving.
/// The most faces that stand for one person.
///
/// A hundred is far more points of view than a person has. What it bounds
/// is the cost: with every person at the cap, a scan against the named part
/// of a library is `people × 100` comparisons per face rather than
/// `confirmations`, and the two part company as soon as a library is used.
pub const MAX_REFERENCES: usize = 100;
/// The shortest raw embedding that may stand for a person.
///
/// One above the gallery floor: a reference vouches for someone, and the
/// margin keeps the faces that only just cleared the gallery — the ones
/// nearest the middle of the sphere — out of the set that speaks for a
/// person.
pub const MIN_REFERENCE_QUALITY: f32 = 15.0;
/// Whether a face of this quality may stand for a person.
///
/// `None` is "never measured" and is admitted, as in
/// [`crate::embedding::in_gallery`].
pub fn eligible(quality: Option<f32>) -> bool {
quality.is_none_or(|q| q >= MIN_REFERENCE_QUALITY)
}
/// Choose which of one person's faces stand for them.
///
/// `embeddings` and `quality` are one entry per face, the embeddings unit
/// length and all of one dimension. Returns the indices chosen, in the order
/// chosen — the first is the longest eligible vector, and each after it is
/// the one furthest from the span of those before. Every eligible face is
/// returned when there are `max` or fewer of them, so a person under the
/// cap loses nothing.
///
/// Deterministic: equal residuals break on the longer vector, then the lower
/// index, so two devices holding the same faces choose the same references
/// and group the same way (`cluster::clustering_is_deterministic`).
pub fn select(embeddings: &[&[f32]], quality: &[Option<f32>], max: usize) -> Vec<usize> {
debug_assert_eq!(embeddings.len(), quality.len());
let mut pool: Vec<usize> = (0..embeddings.len())
.filter(|&i| eligible(quality[i]))
.collect();
if pool.len() <= max {
return pool;
}
// Longest first, so the seed is the pool's front and a tie on residual
// resolves to the earlier position. A missing reading ranks below any
// measured one for this purpose only: it is admitted, but a face that
// was measured and found long is the better seed.
pool.sort_by(|&a, &b| {
let qa = quality[a].unwrap_or(0.0);
let qb = quality[b].unwrap_or(0.0);
qb.total_cmp(&qa).then(a.cmp(&b))
});
// Residuals: what remains of each pool vector outside the span of the
// chosen ones. Copied, since they are rewritten in place.
let mut residual: Vec<Vec<f32>> = pool.iter().map(|&i| embeddings[i].to_vec()).collect();
let mut taken = vec![false; pool.len()];
let mut chosen = Vec::with_capacity(max);
while chosen.len() < max {
// The face with the most left outside the span. The seed is the
// pool's front by construction: every unit vector has the same
// residual before anything is chosen, up to rounding, and rounding
// is not a reason to prefer one. After that `> best` and not `>=`,
// so a genuine tie keeps the earlier (longer) candidate.
let mut pick = None;
let mut best = 0.0_f32;
if chosen.is_empty() {
pick = Some(0);
best = residual[0].iter().map(|x| x * x).sum();
} else {
for (k, r) in residual.iter().enumerate() {
if taken[k] {
continue;
}
let n2: f32 = r.iter().map(|x| x * x).sum();
if n2 > best {
best = n2;
pick = Some(k);
}
}
}
// Nothing left outside the span: every remaining face is a
// combination of the chosen ones and adds no volume.
let Some(k) = pick.filter(|_| best > 1e-6) else {
break;
};
taken[k] = true;
chosen.push(pool[k]);
// Project the chosen direction out of every remaining residual.
let inv = best.sqrt().recip();
let q: Vec<f32> = residual[k].iter().map(|x| x * inv).collect();
for (j, r) in residual.iter_mut().enumerate() {
if taken[j] {
continue;
}
let d: f32 = r.iter().zip(&q).map(|(a, b)| a * b).sum();
for (x, y) in r.iter_mut().zip(&q) {
*x -= d * y;
}
}
}
chosen
}
#[cfg(test)]
mod tests {
use super::*;
fn unit(v: &[f32]) -> Vec<f32> {
let n = v.iter().map(|x| x * x).sum::<f32>().sqrt();
v.iter().map(|x| x / n).collect()
}
#[test]
fn a_person_under_the_cap_keeps_every_eligible_face() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0]), unit(&[1.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), None, Some(16.0)];
assert_eq!(select(&refs, &q, 100), vec![0, 1, 2]);
}
#[test]
fn a_short_vector_never_stands_for_a_person() {
let e = [unit(&[1.0, 0.0]), unit(&[0.0, 1.0])];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0), Some(MIN_REFERENCE_QUALITY - 0.01)];
assert_eq!(select(&refs, &q, 100), vec![0]);
}
/// Two hundred frames from one afternoon and one profile shot: the
/// profile is the second choice, not the two-hundred-and-first.
#[test]
fn the_odd_one_out_is_chosen_before_any_duplicate() {
let mut e: Vec<Vec<f32>> = Vec::new();
let mut q = Vec::new();
for i in 0..200 {
// Near-duplicates of one direction, with a little noise.
let t = (i as f32) * 1e-3;
e.push(unit(&[1.0, t, t * 0.5]));
q.push(Some(20.0 + (i % 7) as f32));
}
e.push(unit(&[0.0, 0.0, 1.0]));
q.push(Some(16.0));
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let chosen = select(&refs, &q, 3);
assert_eq!(chosen.len(), 3);
assert_eq!(
chosen[1], 200,
"the profile shot was not second: {chosen:?}"
);
// Seeded on the longest vector.
assert_eq!(q[chosen[0]], Some(26.0));
}
/// Faces inside the span of the chosen ones add no volume and are not
/// taken to fill the cap.
#[test]
fn the_cap_is_not_filled_from_inside_the_span() {
let e = [
unit(&[1.0, 0.0]),
unit(&[0.0, 1.0]),
unit(&[1.0, 1.0]),
unit(&[2.0, -1.0]),
];
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = [Some(20.0); 4];
assert_eq!(select(&refs, &q, 3).len(), 2);
}
#[test]
fn the_choice_is_deterministic() {
let e: Vec<Vec<f32>> = (0..50)
.map(|i| {
let a = (i as f32) * 0.37;
unit(&[a.cos(), a.sin(), (a * 3.0).sin(), 0.2])
})
.collect();
let refs: Vec<&[f32]> = e.iter().map(Vec::as_slice).collect();
let q = vec![Some(18.0); 50];
assert_eq!(select(&refs, &q, 5), select(&refs, &q, 5));
}
}
+11
View File
@@ -857,6 +857,17 @@ impl EditGraph {
crate::operation::compose_camera_linear(&self.warps, self.framing.baseline(), view)
}
/// TRACES: FR-DEV-3
/// The camera-space tap under this edit's own framing — crop, view,
/// rotation and all — so a fraction of what is on the canvas is a
/// fraction of what this renders. Nothing else of the edit: no
/// operation, no mask, no repair. See
/// [`crate::operation::compose_camera_probe`] for why the white balance
/// picker reads from here and not from the display.
pub fn compose_camera_probe(&self) -> ComposedShader {
crate::operation::compose_camera_probe(&self.warps, &self.framing)
}
/// TRACES: FR-DEV-19c
/// [`Self::compose_for`], with one layer's mask drawn over the picture.
///
+2 -1
View File
@@ -65,7 +65,8 @@ pub use history::{Edit, Entry as HistoryEntry, History, Step};
pub use lens::{compose_warps, ComposedWarp, LensProfile, Tca, Warp};
pub use operation::{
compose, compose_with_framing, Affects, ComposedShader, Helper, Invalidation, Operation,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, RESERVED_UNIFORM_FIELDS,
OutputMode, Uniform, BASE_CURVE_POINTS, BASE_CURVE_UNIFORM_OFFSET, CLIP_ONSET,
RESERVED_UNIFORM_FIELDS,
};
pub use preset::{LibraryParseError, NameError, Preset, PresetLibrary, Scope};
pub use sidecar::{Sidecar, Version};
+15 -9
View File
@@ -69,20 +69,26 @@ const ROUNDS: u32 = 3;
/// Below this a channel carries no ratio worth balancing.
///
/// A sample in the deep shadows, or one taken on a blown highlight where a
/// channel has already clipped to nothing, has no white balance in it: the
/// logarithms below would run away and the picker would slam a slider to its
/// stop. Refusing is the honest answer, and the caller reports that the point
/// was not usable rather than moving the photograph.
/// A sample in the deep shadows has no white balance in it: the logarithms
/// below would run away and the picker would slam a slider to its stop.
/// Refusing is the honest answer, and the caller reports that the point was
/// not usable rather than moving the photograph. The other end — a blown
/// highlight, where every channel has stopped counting — is refused by the
/// caller before the sample is taken, because only the caller can see the
/// sensor value; see [`crate::operation::CLIP_ONSET`].
const FLOOR: f32 = 1e-4;
/// TRACES: FR-DEV-3
/// Move the graph so that `sample` renders neutral.
///
/// `sample` is linear RGB, as the operation's own gains multiply it — that is,
/// measured with the sampling operation at its defaults. Returns whether the
/// graph was moved: `false` where the chain offers no white point widget, or
/// where the colour has no balance in it to correct.
/// `sample` is the linear triple the operation's own gains multiply — camera
/// RGB with the camera's as-shot balance on, *before* the body's base curve
/// and matrix, and with the sampling operation at its defaults. Not the
/// pixel on the screen: the matrix mixes the channels on the way there, so
/// a colour read after it does not answer to these gains, and a solve over
/// one lands somewhere no sample asked for. Returns whether the graph was
/// moved: `false` where the chain offers no white point widget, or where the
/// colour has no balance in it to correct.
///
/// **Absolute, not relative.** The values written depend on the colour and not
/// on where the sliders happened to be, so sampling the same wall twice lands
+41 -6
View File
@@ -431,9 +431,11 @@ pub enum OutputMode {
/// no operations, and the caller fills the reserved uniforms neutral —
/// unit white balance, identity matrix, base curve off — so what is
/// stored is the sensor's own numbers, demosaiced and undistorted. Only
/// [`compose_camera_linear`] produces it, and only
/// `AdjustPass::render_camera_linear` accepts it, so the neutral
/// uniforms cannot be forgotten by a caller that composed it by mistake.
/// [`compose_camera_probe`] produces it — for a merge through
/// [`compose_camera_linear`], and for the white balance picker under the
/// edit's own framing — and only `AdjustPass::render_camera_linear`
/// accepts it, so the neutral uniforms cannot be forgotten by a caller
/// that composed it by mistake.
///
/// Thirty-two bits rather than sixteen because the composite is written
/// back as a RAW at the sensor's scale (FR-MRG-3): a 14-bit sensor has
@@ -490,6 +492,19 @@ pub const BASE_CURVE_UNIFORM_OFFSET: usize = 16;
/// selection in [`compose_full`].
pub const BASE_CURVE_POINTS: usize = 5;
/// Where the highlight desaturation begins: the fraction of the white level
/// above which a photosite is treated as clipped.
///
/// A photosite this close to saturation has stopped counting, so its ratio
/// to its neighbours is not a colour. The generated prologue fades a pixel
/// above this toward a neutral of the same brightness before any operation
/// runs, and the white balance probe refuses to sample one: a blown sky is
/// sensor white, which the as-shot multipliers make magenta, and a solve
/// over that slams tint to its stop. One number, so the two cannot drift
/// apart — a probe that accepted what the shader had already desaturated
/// would be balancing against a pixel the photographer cannot see.
pub const CLIP_ONSET: f32 = 0.985;
/// Where an operation's own uniforms begin in the generated block.
///
/// The base fields, then framing's. Exported because `dr-gpu` writes the
@@ -614,9 +629,26 @@ pub fn compose_camera_linear(
// upright too, and `view` is a fraction of the upright frame.
framing.set_baseline(baseline);
framing.set_view(view);
compose_camera_probe(warps, &framing)
}
/// TRACES: FR-DEV-3
/// The same tap under a framing the caller chose: the white balance probe.
///
/// A neutral picked off the canvas has to be measured in the space the
/// white balance gains multiply, and that is camera RGB — the operation
/// runs before the body's matrix, and a probe read after the matrix would
/// be solving the wrong equation on any body whose matrix mixes the
/// channels, which is every body. It also has to be measured at the pixel
/// the canvas is showing, which is why this takes the edit's own framing
/// where a merge passes the file's orientation and a tile.
pub fn compose_camera_probe(
warps: &[Box<dyn crate::lens::Warp>],
framing: &Framing,
) -> ComposedShader {
compose_inner(
&[],
&framing,
framing,
ColourSpace::Srgb,
&MaskStack::new(),
&crate::spot::SpotSet::new(),
@@ -668,7 +700,7 @@ fn compose_inner(
// twice and be bound to a texture of the wrong format.
//
// `forced` is the one exception, and it is not a caller flag in the
// sense above: `compose_camera_linear` is the only function that passes
// sense above: `compose_camera_probe` is the only function that passes
// it, with an empty operation list, and the mode it forces has its own
// storage format and its own render entry on the GPU side.
let output_mode = forced.unwrap_or(
@@ -997,6 +1029,9 @@ fn compose_inner(
.to_string()
};
// Formatted with Rust's `Display` so the shader reads the same threshold
// the probe checks against; see `CLIP_ONSET`.
let clip_onset = CLIP_ONSET;
let source = format!(
"// GENERATED — do not edit.
//
@@ -1067,7 +1102,7 @@ fn main(@builtin(global_invocation_id) gid: vec3<u32>) {{
// A photosite at its white level carries no colour information — every
// channel simply stopped counting — so the balance below must not be
// allowed to tint it.
let clipped = smoothstep(0.985, 1.0, max(c.r, max(c.g, c.b)));
let clipped = smoothstep({clip_onset}, 1.0, max(c.r, max(c.g, c.b)));
c = c * u.as_shot_wb.rgb;
+10
View File
@@ -821,6 +821,16 @@ here, and it is the phone and tablet story that should decide whether it gets bu
already has, the raw vector written over the old one and its id, box and identity untouched
(`faces::record_updates`). No detector runs and no suggestion is lost — the cost is the
original fetched once more, since the length exists only at the moment of embedding.
- **A person is stood for by their references.** Every face the user has ruled on is an anchor,
and the scan is exhaustive, so a person with 750 confirmations would cost 750 comparisons
against every other face — and the cost of a library would grow with how well it was named.
Instead each person enters through at most `MAX_REFERENCES` (100) of their anchored faces,
chosen by `dr_face::references`: those whose raw embedding is at least `MIN_REFERENCE_QUALITY`
(15) long, and among them the set spanning the greatest volume — greedy max-determinant, the
longest vector first and then, at each step, the face with the largest component orthogonal to
the chosen so far. Thirty frames from one afternoon contribute one reference; the single profile
shot is taken early. The faces not chosen keep their confirmations and are not touched by the
pass; they are simply not compared.
**The algorithm.** Constrained average-link agglomeration over the probability graph, merging while
the average pairwise probability exceeds **0.9** and no cannot-link is violated. Average-link rather
Binary file not shown.
+51 -51
View File
File diff suppressed because one or more lines are too long
+1 -1
View File
@@ -4,7 +4,7 @@
# makes `makepkg -si` in this directory install what you are actually working
# on. Swap `source` for a tagged tarball when there is something to release.
pkgname=darkroom
pkgver=0.13.2
pkgver=0.13.4
# Back to 1 with the version: a new pkgver is a new archive name, so there is
# nothing for makepkg to reuse and nothing for a release number to disambiguate.
pkgrel=1
+1 -1
View File
@@ -207,7 +207,7 @@ def develop_wb():
group('colour')
rec('develop-wb')
pause(0.4)
dr.click(1542, 767); pause(0.8) # pick
dr.click(1487, 776); pause(0.8) # pick
dr.click(1000, 300); pause(1.5) # a neutral wall
hold(*BEFORE, 1.4); pause(0.8)
cut()
+181 -51
View File
@@ -4067,9 +4067,9 @@ impl DevelopSession {
let space = self.display_space;
// TRACES: FR-DEV-19c
// **The one composition that may show a mask.** Every other caller of
// the graph — `render_the_file`, the thumbnail, `sample_as_shot` —
// goes through `compose_for`, which cannot ask for a reveal, so no
// exported file can carry one.
// the graph — `render_the_file`, the thumbnail — goes through
// `compose_for`, which cannot ask for a reveal, so no exported file
// can carry one; `sample_as_shot` composes no operations at all.
let shader = self.graph.compose_revealing(space, self.reveal().as_ref());
// Rasterise the masks first: the shader addresses array slices by
@@ -4442,18 +4442,28 @@ impl DevelopSession {
true
}
/// The colour at a point with every adjustment taken off, in linear RGB.
/// The colour at a point in the space the white balance gains multiply:
/// camera RGB with the camera's own balance on, linear, nothing else.
///
/// **Measured before the chain rather than off the screen**, and that is
/// the difference between a picker that converges and one that chases
/// itself. The frame on the canvas has already been through the white
/// balance being solved for, the tone curve, the contrast and whatever
/// else is on; neutralising *that* pixel would be correcting a correction,
/// and the second sample of the same wall would land somewhere else. With
/// the adjustments stripped the value read is the colour as the file has
/// it, which is the domain `neutralise` is defined over.
/// **Measured where the operation acts, not where the photographer
/// looks.** The white balance node runs first in the chain, on camera
/// RGB, before the body's base curve and its matrix; the canvas shows
/// the pixel after all three. The probe used to be read off a display
/// render with the adjustments stripped, and the solve then treated an
/// sRGB triple as if the gains multiplied it directly. On a JPEG the two
/// spaces coincide, so it worked; on a raw file from any real body the
/// matrix mixes the channels, and a slightly blue wall on a Canon 6D
/// came back tint −77 with the whole frame green. This reads the
/// camera-space tap a merge stitches from — the sensor's numbers after
/// the lens warp — and puts the as-shot balance on itself, which is
/// exactly the value the operation's gains are about to multiply.
///
/// The framing stays on, exactly as it does for [`Self::render_original`]
/// That also means nothing has to be stripped and restored: the tap
/// runs no operations at all, and the display target is untouched, so
/// a sample that found nothing usable leaves the canvas exactly as it
/// was.
///
/// The framing is the edit's own, exactly as for [`Self::render_original`]
/// and for the same reason: `x` and `y` are fractions of what is on
/// screen, and a probe rendered without the crop and the zoom would be
/// answering about a different part of the photograph.
@@ -4462,41 +4472,23 @@ impl DevelopSession {
/// averages a small neighbourhood into each of its pixels, which is what
/// every eyedropper does deliberately: a single photosite off a noisy
/// shadow is a worse answer than the patch around it, and the photographer
/// is pointing at a grey card rather than at a pixel. It is also two
/// dispatches' worth of work on a click.
///
/// This overwrites the frame the adjust pass is holding, so the caller
/// must redraw — which the callback that samples does anyway, since the
/// picture has just changed.
/// is pointing at a grey card rather than at a pixel. It is also one
/// dispatch's worth of work on a click.
fn sample_as_shot(&mut self, x: f32, y: f32) -> Option<[f32; 3]> {
/// Long edge of the probe render. See the note above on why it is
/// small rather than large.
const PROBE_EDGE: u32 = 192;
// sRGB regardless of the display: this is a measurement, not something
// anybody looks at, and decoding it needs a transfer function known
// here. Composing for a wide-gamut panel would put the reading in a
// space the arithmetic below does not undo.
let space = dr_types::ColourSpace::Srgb;
let saved = self.graph.state();
self.strip_adjustments();
let (sw, sh) = self.demosaiced.size();
let (fw, fh) = self.graph.output_size(sw, sh);
let (w, h) = fit(fw, fh, PROBE_EDGE, PROBE_EDGE);
let shader = self.graph.compose_for(space);
let probe = self
.render_with_masks(&shader, w, h, space)
.and_then(|()| self.adjust.export_pixels().map_err(|e| e.to_string()));
// Restored whatever happened, for the reason every other suspension
// here restores: leaving the graph stripped after a failed probe would
// discard the edit silently.
let debt = self.graph.set_state(&saved);
self.pay_film_debt(&debt);
let (rgba, pw, ph) = probe
let shader = self.graph.compose_camera_probe();
let rendered = self
.adjust
.render_camera_linear(&self.demosaiced, &shader, w, h)
.map(|_| ());
let (rgba, pw, ph) = rendered
.and_then(|()| self.adjust.read_camera_linear())
.inspect_err(|e| log::warn!("could not read a neutral off the frame: {e}"))
.ok()?;
@@ -4504,18 +4496,27 @@ impl DevelopSession {
let px = ((x.clamp(0.0, 1.0) * pw as f32) as usize).min(pw.saturating_sub(1));
let py = ((y.clamp(0.0, 1.0) * ph as f32) as usize).min(ph.saturating_sub(1));
let at = (py * pw + px) * 4;
let pixel = rgba.get(at..at + 3)?;
let pixel = rgba.get(at..at + 4)?;
// The tap marks a pixel the lens correction pulled in from outside
// the frame with alpha 0. There is nothing there to balance against.
if pixel[3] < 0.5 {
return None;
}
// Nor in a clipped one. A blown sky reads as sensor white, and sensor
// white with the as-shot balance on is strongly magenta — a solve over
// it drives tint to its stop for a pixel that, on the canvas, the
// shader has already desaturated to neutral. The same threshold the
// shader fades from, so what is refused here is what it would have
// hidden there.
if pixel[..3].iter().any(|c| *c >= dr_pipeline::CLIP_ONSET) {
return None;
}
// The probe was encoded for the screen; the solve is multiplicative
// and only means anything in linear light (ARCH §5.2). Undone with
// the space's own transfer function rather than a second copy of the
// curve written out here.
let transfer = space.transfer();
Some([
transfer.decode(f32::from(pixel[0]) / 255.0),
transfer.decode(f32::from(pixel[1]) / 255.0),
transfer.decode(f32::from(pixel[2]) / 255.0),
])
// The tap is the sensor's numbers with the profile filled neutral;
// the operation multiplies them *after* the camera's own balance, so
// that goes on here and the solve sees what the gains will see.
let wb = self.demosaiced.as_shot_wb();
Some([pixel[0] * wb[0], pixel[1] * wb[1], pixel[2] * wb[2]])
}
/// TRACES: FR-PLAT-AND-5 | NFR-RES-1
@@ -6670,6 +6671,135 @@ mod tests {
);
}
/// TRACES: FR-DEV-3
/// The whole point of the picker, measured where the photographer sees
/// it: a cast grey on a *raw* frame, sampled, renders grey.
///
/// On a raw frame and not a JPEG, because that is where it was wrong. The
/// white balance gains multiply camera RGB, before the body's matrix
/// turns it into sRGB; the probe was read *after* the matrix, and the
/// solve treated the two as the same space. On a body whose matrix mixes
/// the channels as much as a Canon's does, a slightly blue wall came back
/// tint −77 and the whole frame went green. A JPEG carries an identity
/// matrix, so the same test on one passed while the picker was broken.
#[test]
fn sampling_a_cast_grey_on_a_raw_frame_renders_it_grey() {
let Some(ctx) = headless() else { return };
// A Canon EOS 6D's D65 matrix (rows summing to one, as
// `neutral_stays_neutral_through_the_colour_matrix` requires) and a
// typical as-shot balance for it.
let cam_to_srgb = [
1.9125, -1.0587, 0.1461, //
-0.2249, 1.6466, -0.4217, //
0.0099, -0.5093, 1.4994,
];
let as_shot = [1.9, 1.0, 1.7];
// What the wall should look like once the camera's own balance is on:
// a warm cast, a little over half a stop between red and blue.
let balanced = [0.30f32, 0.25, 0.20];
let sensor: Vec<u16> = (0..3)
.map(|c| (balanced[c] / as_shot[c] * 65535.0).round() as u16)
.collect();
let size = 64u32;
let raw = RawImage {
width: size,
height: size,
data: sensor.repeat((size * size) as usize),
cfa_pattern: dr_decode::CfaPattern::Rggb,
black_level: [0; 4],
white_level: 65535,
wb_coeffs: [as_shot[0], as_shot[1], as_shot[2], 0.0],
color_matrix: Some(cam_to_srgb),
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 3,
profile: None,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
x: 0,
y: 0,
width: size,
height: size,
},
};
let mut session =
DevelopSession::open(&ctx, &raw, dr_types::Orientation::NORMAL).expect("session");
let at = ((size / 2) * size + size / 2) as usize * 4;
let before = read_back(&ctx, &session.render(size, size).expect("render"));
let cast = |px: &[u8]| px.iter().max().unwrap() - px.iter().min().unwrap();
assert!(
cast(&before[at..at + 3]) > 20,
"the premise: the wall renders with a cast, {:?}",
&before[at..at + 3]
);
assert!(
session.sample_neutral(0.5, 0.5),
"a mid-grey is a usable sample"
);
let after = read_back(&ctx, &session.render(size, size).expect("render"));
let px = &after[at..at + 3];
assert!(
cast(px) <= 3,
"the sampled point should render neutral, got {px:?} with {:?}",
session
.rows()
.iter()
.filter(|r| r.value != r.default_value)
.map(|r| (r.param_label.to_string(), r.value))
.collect::<Vec<_>>()
);
}
/// TRACES: FR-DEV-3
/// A blown highlight is refused, the way black is.
///
/// Sensor white is not a colour: every channel stopped counting, so the
/// ratio between them is the as-shot multipliers and nothing about the
/// scene. Sampling the overcast sky on a Canon 6D frame drove tint to
/// -100 and temperature to -15 for a patch the canvas showed as pure
/// white, which is the picker being wrong rather than the point being a
/// poor choice. Refused, nothing moves and no step is taken.
#[test]
fn sampling_a_blown_highlight_moves_nothing() {
let Some(ctx) = headless() else { return };
let size = 16u32;
let raw = RawImage {
width: size,
height: size,
data: vec![65535; (size * size * 3) as usize],
cfa_pattern: dr_decode::CfaPattern::Rggb,
black_level: [0; 4],
white_level: 65535,
wb_coeffs: [1.9, 1.0, 1.7, 0.0],
color_matrix: None,
base_curve: dr_decode::BaseCurve::IDENTITY,
samples_per_pixel: 3,
profile: None,
make: String::new(),
model: String::new(),
crop: dr_decode::CropRect {
x: 0,
y: 0,
width: size,
height: size,
},
};
let mut session =
DevelopSession::open(&ctx, &raw, dr_types::Orientation::NORMAL).expect("session");
let steps = session.history_rows().len();
assert!(
!session.sample_neutral(0.5, 0.5),
"a clipped photosite has no balance in it"
);
assert!(session.is_neutral(), "and so nothing was corrected");
assert_eq!(session.history_rows().len(), steps);
}
/// TRACES: FR-DEV-7 | FR-DEV-5
/// A held comparison hands the edit straight back.
///
+112
View File
@@ -1069,6 +1069,79 @@ impl Population {
ids.push(f.face);
}
// Not every anchored face enters the scan: each person is stood for
// by their references (`dr_face::references`), and the rest of their
// faces stay out of the comparison. They keep their place -- they are
// in `anchors`, so the pass never releases them -- and they are the
// cost that would otherwise grow with every name the user gives.
let mut by_person: std::collections::BTreeMap<u64, Vec<usize>> =
std::collections::BTreeMap::new();
for (i, c) in candidates.iter().enumerate() {
if let Some(p) = c.confirmed_person {
by_person.entry(p).or_default().push(i);
}
}
let mut keep = vec![true; candidates.len()];
let mut anchored = 0usize;
for members in by_person.values() {
anchored += members.len();
// Only a person over the cap loses anyone: under it, `select`
// returns every eligible face, and the ineligible are kept too,
// since a short vector was a probe before and still is.
if members.len() <= dr_face::references::MAX_REFERENCES {
continue;
}
let embeddings: Vec<&[f32]> = members
.iter()
.map(|&i| candidates[i].embedding.as_slice())
.collect();
let quality: Vec<Option<f32>> =
members.iter().map(|&i| candidates[i].quality).collect();
let chosen = dr_face::references::select(
&embeddings,
&quality,
dr_face::references::MAX_REFERENCES,
);
for &i in members {
keep[i] = false;
}
for &k in &chosen {
keep[members[k]] = true;
}
// A person none of whose faces is long enough to vouch is still
// a person, and a pass they had no anchor in would file their
// next face as a stranger. The longest stand in.
if chosen.is_empty() {
let mut by_quality = members.clone();
by_quality.sort_by(|&a, &b| {
let qa = candidates[a].quality.unwrap_or(0.0);
let qb = candidates[b].quality.unwrap_or(0.0);
qb.total_cmp(&qa).then(a.cmp(&b))
});
for &i in by_quality.iter().take(dr_face::references::MAX_REFERENCES) {
keep[i] = true;
}
}
}
let left_out = keep.iter().filter(|k| !**k).count();
if left_out > 0 {
let mut i = 0;
candidates.retain(|_| {
i += 1;
keep[i - 1]
});
let mut i = 0;
ids.retain(|_| {
i += 1;
keep[i - 1]
});
log::info!(
"references: {} of {anchored} anchored face(s) stand for {} people; {left_out} left out of the scan",
anchored - left_out,
by_person.len(),
);
}
Ok(Self {
cal,
candidates,
@@ -2121,6 +2194,45 @@ mod tests {
assert_eq!(after[0].suggested_faces, 3);
}
/// A person over the reference cap is stood for by a chosen few, and the
/// rest of their faces stay where they are: not compared, not released,
/// and still theirs when the next face arrives.
#[test]
fn a_well_covered_person_is_stood_for_by_their_references() {
let many = dr_face::references::MAX_REFERENCES + 50;
let catalog = catalog_with(many + 1);
for i in 0..many {
put_face(&catalog, i as u64 + 1, 0, 1.0 - (i as f32) * 1e-3);
}
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
let him = faces::people(catalog.connection()).unwrap()[0].id;
faces::rename_person(catalog.connection(), him, "Ian").unwrap();
// Every one of his faces lies in one plane, so two references span
// them all and the selector stops there rather than filling the cap.
let pop = Population::read(&catalog, TEST_MODEL).unwrap();
let standing = pop
.candidates
.iter()
.filter(|c| c.confirmed_person.is_some())
.count();
assert!(
(2..=dr_face::references::MAX_REFERENCES).contains(&standing),
"{standing} of {many} faces entered the scan"
);
assert_eq!(pop.anchors.len(), many, "the rest lost their anchor");
put_face(&catalog, many as u64 + 1, 0, 0.98);
recluster(&catalog, TEST_MODEL, &FaceSettings::default()).unwrap();
let after = faces::people(catalog.connection()).unwrap();
assert_eq!(after.len(), 1, "a second Ian appeared: {after:?}");
assert_eq!(
after[0].suggested_faces,
(many + 1) as u64,
"faces were released"
);
}
/// `dr-types` sits below the face engine and cannot name its constant, so
/// it restates the number. This is the thing that stops the two drifting:
/// a settings page marking 0.80 as "default" while the engine had moved to
+3 -5
View File
@@ -3127,14 +3127,12 @@ pub fn run(paths: Vec<PathBuf>) -> Result<()> {
.borrow_mut()
.as_mut()
.is_some_and(|s| s.sample_neutral(x, y));
// Nothing to redraw otherwise: the probe renders to its own
// target and a sample that found nothing usable moved nothing.
if sampled {
sync_rows(&w, &rows, &session);
redraw(&w);
}
// Redrawn either way. A sample that found nothing usable still
// overwrote the frame the adjust pass was holding, and leaving the
// canvas showing a probe of the unedited image would look like the
// edit had been thrown away.
redraw(&w);
});
}